A local maximum is a point or region that is greater (in some way) than adjacent points or regions. A local maximum is not necessary a global maximum which is greater than all other points or regions. This concept has broad applications, from the strictly mathematical to a mental concept to help think about design and education.
As The Podcast Title
I chose to name my podcast “The Local Maximum” because of the triple meaning. I study and practice machine learning and software design, for which the local maximum (as described below) is a very important concept. It also contains the word “local” which matches with my career-long interest in Location data and local search. Finally, it also contains my name, “Max”.
As a Mental Concept
Local Maximums come into play when performing any task that requires iteration and improvement. Often, these small iterations lead to better and better outcomes - but at some point this process stops working and an entirely new process needs to start taking place in order to get to a better outcome. Sometimes, this new process will lead to worse outcomes in the short run. Here are some examples:
In Design: Whether designing websites, software, physical objects (industrial design), or in architecture - the iterative process often involves taking tried-and-tested methods and applying it to a specific problem. When a local maximum is reached, an entirely new design principle is needed in order to hope to beat the incumbent design. Because the incumbent design has gone through so many iterations, the new framework is often worse to start with - and there is no guarantee that it’ll end up being better! Progress requires risk.
In training and education: This concept can be applied to mastery of a subject, whether it’s an academic subject, a musical talent, or in sports training. Certain practices could help you get better and better, but sometimes it helps to look at things a different way (or try a new technique) in order to be as successful as possible.
In machine learning: This is closer to the true mathematical model of The Local Maximum, but in any hill-climbing algorithm (where the machine starts at one solution and tweaks it to get better and better), the computer is prone to getting stuck in a local maximum. In episode 4, we go over one such example and show how we found the global maximum (best) solution.
In Mathematics
A local maximum is said to be strict if a point is larger than all nearby points.
Everything that’s said about local maxima is also true about local minima.
In order to dive into this further, we need to know what is a point, and what is nearby. This means that we need to be working in some sort of mathematical space where we have settled on our class of points, and we also agree on what “nearby” means.
We also need to know (or be able to calculate) the value at each point so that we can compare which ones are larger. Usually, values are taken to be real numbers, but any partially ordered set will do.
On a Directed Graph
A simple example where we can define a Local Maximum is with a mathematical space - one where there are a countable (and usually finite) number of points. In order to define nearness, a directed graph is defined on the space, so that for each point we know which points are “the nearby ones”. There’s no requirement for this graph to be symmetric (if A is near B, then B is near A) but in many situations this is true.
In this case, a point A is a Local Maximum if all the nearby points of A have a value less than or equal to the value at A.
This means that if you are in a hill-climbing algorithm of some sort and you reach A, you’ll never move thinking that you’ve found a solution! The ability to search for a higher local maximum (or hopefully the global maximum) can include
- Probabilistically moving downhill
- Restarting at other random points
- Using a different directed graph, often with more potential connections
In a topological space
In many mathematical spaces - called continuous spaces - there is no notion of “A is nearby B”. For example, what real numbers are nearby 0? For any non-0 number that we pick, we can find some other number that comes between it and 0.
In this case, a directed graph is not possible. Instead we need to define a topology on a space, which takes as given the notion of an open set. Intuitively, if a point is in an open set, then all of the points nearby (or surrounding it) are also in that set.
So for example, the set (0, 1) - which is all real numbers between 0 and 1 is open in the standard topology on real numbers. If I take any point inside it (say 3/4) - then all of the points surrounding 3/4 are in the set.
In a topological space, a point A is a local maximum if there exists an open set containing A (no matter how small) such that the value at A is greater than or equal to the value of any point in the set.
For example, if I’m dealing with real numbers and I want to know whether I have a local maximum at the number 2 - I look at the value of 2, and try to construct an open set around it - maybe (1.99, 2.01) - where that value is the highest. If I find some other values that are higher, it doesn’t necessarily mean that 2 isn’t a local maximum; it could mean that the open set isn’t tight enough. But if it proves impossible to come up with a set that’s tight enough to make the value of 2 highest, then it is not a Local Maximum.