Convex and Non-convex functions

A convex function and a non-convex function differ primarily in the shape of their curves and in the nature of their optimization landscapes, which impacts how we can use techniques like gradient descent to find minimum points effectively.

Convex Function:

• A function f(x) is convex if, for any two points x1​ and x2​ in its domain, the line segment connecting f(x1​) and f(x2​) lies above or on the graph of f(x).

• Formally, f(x) is convex if: f(αx1+(1−α)x2)≤αf(x1)+(1−α)f(x2)f for all x1,x2x_1, x_2x1​,x2​ in the domain and α∈[0,1].

• In simple terms, a convex function has a single global minimum and no local minima, making it easier to optimize because any gradient-based algorithm like gradient descent will move toward the global minimum.

• Example: The function f(x) = x^2 is convex. Its plot is a U-shaped curve, and it has one minimum at x=0x = 0x=0.

  The exponential function f(x) = e^x is also convex, as it has a continuously increasing slope.

  Non - Convex Function:

• A function is non-convex if it does not satisfy the convexity property above. In a non-convex function, the line segment between any two points in the function's graph may lie partially below the function.

• Non-convex functions can have multiple minima (local minima) and maxima (local maxima), which makes optimization more challenging. In such functions, gradient descent might get "stuck" in a local minimum, failing to find the global minimum.

• Example: The function f(x) = x^4 - 4x^2 is non-convex. Its plot is W-shaped and has multiple local minima and maxima.

  The sine function f(x) = sin(x)is also non-convex on an interval longer than one period (e.g., x∈[0,2π]), as it oscillates and has multiple peaks and valleys.