Regularization is a technique used in machine learning to prevent overfitting. It is used to introduce additional information or bias to a learning algorithm to prevent it from overfitting the training data. It can be implemented in different ways, such as adding a penalty term to the cost function, introducing a prior distribution on the parameters, or using dropout.
For example, when using linear regression, regularization can be used to prevent overfitting by adding a penalty term to the cost function. This penalty term is usually the L2 norm of the weights, which penalizes large weights and encourages the learning algorithm to find a solution with smaller weights. This regularization technique is known as Ridge Regression.