Some weird stuff happens in high-dimensions.

Reference

1. High Dimensional Oranges Are Almost All Peel

Consider an $n$-dimensional cube of side length 1 containing a smaller $n$-dimensional cube with side length $0.8$ (“pulp”) surrounded by a $0.1$-width border (“peel”).

The volume of the pulp is $0.8^n$, which rapidly approaches 0 as $n$ increases:

Dimensions Pulp Volume ($0.8^n$)
1 0.800
2 0.640
3 0.512
5 0.328
10 0.107
20 0.012
50 0.000014

Another perspective: To randomly sample a point in this cube, we select $n$ independent coordinates from $[0,1]$. The point lies in the pulp only if all coordinates fall within $(0.1, 0.9)$. This probability is $(0.8)^n$, approaching 0 as $n$ increases.

In high dimensions, the volume is almost all in the surface.

2. Unit spheres get smaller in high dimensions

todo

This is very weird and requires non-trivial math to show. Wikepdia

Good twitter post talking about this

Drake came up with a good explanation which is explained here

3. Spheres “spill out” in higher dimensions

todo

4. Randomly chosen vectors in high-dimensional spaces are almost all orthogonal

todo

5. Local minima are rare

todo