A method for fitting a curve (not necessarily a straight line) through a set of points using some goodness-of-fit criterion. The most common type of regression is Linear Regression.
See also Least Squares Fitting, Linear Regression, Multiple Regression, Nonlinear Least Squares Fitting, Regression Coefficient
References
Kleinbaum, D. G. and Kupper, L. L. Applied Regression Analysis and Other Multivariable Methods. North Scituate, MA: Duxbury Press, 1978.