Paper Explainer: Geometry and Stability of Supervised Learning Problems
Just released a new paper ! In it, my coauthors and I try to make sense of some challenges in machine learning by creating a "space of all problems". If you don't know what that means, that's okay! This post explains the big ideas for non-mathematicians. What is Supervised Learning? Suppose you've got some data on the IQ and SAT scores of a bunch of people, and the data looks like this: (Note: I made this data up. Don't believe it.) Using this data, can you use someone's IQ score to get a rough estimate for their SAT score? Sure, you could fit a trendline to the data using some good ol' linear regression. It'll look something like this: Now if you know someone's IQ (say, 110), you can predict what their SAT score might be using the trendline (in this case, about 1207). Congratulations! You just took part in supervised learning ! You used an algorithm to... take data about the relationship between two variables $x$ and $y$, and use t