Machine learning programs seek to model a given set of data in order to make predictions about new data. I’ve been learning about the subject from Christopher M. Bishop‘s book Pattern Recognition and Machine Learning.
Two important machine learning problems are regression and classification. In a regression problem the given data is assumed to arise from some unknown, underlying function. For example, perhaps we are considering experimental measurements of the velocity of a falling object with respect to time. Then the underlying function is a line whose slope is the acceleration of gravity. In general, the goal of a regression problem is to produce a “good” approximation to the unknown underlying function.
In a classification problem we are given a collection of data points, each point belonging to one of a finite number of classes. The goal is to assign each new data point to its optimal class.
For example in a calculus course each student has a midterm exam score, a final exam score and a curved letter grade for the semester. Each student’s midterm and final score is a two dimensional data point, and the student’s letter grade is the class to which the data point belongs.
If we plot the data points in the plane we see that the classes should divide the plane into distinct regions. A new data point that falls within one of these regions should be assigned the corresponding letter grade. The classification problem will be solved if we can systematically describe all of the regions.