Why Convexity?

Etymologically, the term convex originates from the latin word convexus—which translates to arched. It is not uncommon for one to wonder why convexity seems to accompany the word "optimization" in almost every context? Why are convex functions, sets, and formulations so important? Why is there a convex relaxation (such as LASSO) craze sweeping the machine learning community, both in academia and industry? T. Rockafellar’s Convex …