Weird Things in High Dimensions
Some weird stuff happens in high-dimensions.
1. High Dimensional Oranges Are Almost All Peel
Consider an $n$-dimensional cube of side length 1 containing a smaller $n$-dimensional cube with side length $0.8$ (“pulp”) surrounded by a $0.1$-width border (“peel”).
The volume of the pulp is $0.8^n$, which rapidly approaches 0 as $n$ increases:
| Dimensions | Pulp Volume ($0.8^n$) |
|---|---|
| 1 | 0.800 |
| 2 | 0.640 |
| 3 | 0.512 |
| 5 | 0.328 |
| 10 | 0.107 |
| 20 | 0.012 |
| 50 | 0.000014 |
Another perspective: To randomly sample a point in this cube, we select $n$ independent coordinates from $[0,1]$. The point lies in the pulp only if all coordinates fall within $(0.1, 0.9)$. This probability is $(0.8)^n$, approaching 0 as $n$ increases.
In high dimensions, the volume is almost all in the surface.
2. Unit spheres get smaller in high dimensions
todo
This is very weird and requires non-trivial math to show. Wikepdia
Good twitter post talking about this
Drake came up with a good explanation which is explained here
3. Spheres “spill out” in higher dimensions
todo
4. Randomly chosen vectors in high-dimensional spaces are almost all orthogonal
todo
5. Local minima are rare
todo