Regression models are monitored using direct loss estimation (DLE). The idea behind DLE is that the ground truth for regression models is not always available post-deployment, and when it is it may be delayed. You generally want to know if your model is drifting right away, not months from now.

It turns out we can estimate the live performance of the production model by training a secondary model on the observed residuals associated with the primary model, i.e. using the difference between actual and predicted values generated by the primary model during its training phase. Using this secondary model we can then estimate the loss (specifically the root mean square error or RMSE) of the primary model post-deployment.

By comparing the model's estimated RMSE over time with its expected value computed during the training phase you can get a sense of whether your model is experiencing any data drift or not. The G2M platform model monitoring page will also provide upper and lower bounds for your model's estimated RMSE to indicate whether model performance is within its expected range or not.

Note that the DLE approach will detect data drift but not concept drift. Concept drift is hard to detect methodically, so you should always be on the lookout for domain-specific changes which might invalidate your model, like know business or market transitions.

More details on the DLE approach can be found here.

​