TLDR: convex functions in high dimensions are much weirder than in 1D: do you please have insights or references to share?
Let $c(x)$ be a convex function on the real line.
There is an obvious measure of curvature: the second derivative: $c''(x)$.
There is a convex function with only a bit of curvature at $t$: $x \rightarrow |x-t| / 2$.
Any convex function is equal, up to a linear term, to a convolution over these elementary functions:
$$c(t) = a + bx + \int_{t \in \mathbb R} c''(t) |x-t| dt$$
(Assuming bounded support)
- as implied by the result above, there is a bijection between the convex functions and the positive functions
I naively thought that this would nicely translate in higher dimensions replacing the second derivative with the Hessian matrix of second derivatives. Let $c(x)$ now denote a convex function over $R^d$.
The hessian matrix is the matrix of second order derivatives: $H_{i,j}c(x) = \partial^2/\partial_i\partial_j c$.
But, there is no single convex function with curvature only at $t$. For example, both $\sum |x_i|$ and $||x||$ and infinitely many others are peaked just at $t$.
worse, convex functions have this weird property that their hessian is positive, but additional constraints hold:
- in 1D: the huber function has a second derivative with finite support. It is defined as:
$$ x \rightarrow h(x) = \cases{x^2 / 2 & \text{ if x \leq 1} \\ x & \text{ if x \geq 1}} $$
expanding the huber function to higher dimensions: $x \rightarrow h(||x||)$, we instead have a function that a Hessian which is non-zero everywhere!
in fact, I'm fairly certain that convex functions must have curvature everywhere, in the sense that it is impossible for them to have curvature only inside a bounded region
All of this, taken together, has me fairly convinced that the space of convex functions in $\mathbb R^d$ is quite a bit harder than in 1D hence my questions:
- Do you know a reference or a result that applies to the Hessian of convex functions where I could learn some details about this?
- Is there an equivalent of the convolution representation in higher dimensions?
- Is there another measure of curvature that would behave better than the Hessian here?
- Is there a reference that deals with the properties of convex functions that you would recommend, something more generalist than the usual presentation of convexity that you get in optimization?
Any help or lead on this topic would be a great help. Thank you for your time.