What is MLOps

A model as part of a service

An ML model is a program you get from data instead of writing it by hand. Take an example that will run through the whole track: an online store wants to know in advance which customers are about to stop buying. That kind of customer loss is called churn.

A data scientist takes two years of customer history. From it, they train the model: the algorithm adjusts the model's internal numbers, its weights, so the model learns to predict which customers churned in the past. The result is saved to a file, for example model.pkl.

The service loads this file and computes an answer for each customer: "will churn with probability 0.82." Running the model on new data is called inference.

Training and inference are different processes. Training runs rarely, taking minutes or hours on large datasets. Inference runs on every request, or once a day across all customers, and has to answer in milliseconds.

Feature

The model doesn't read a whole database row. It's fed numbers that describe the customer: the number of orders in the last 30 days, the average order value, days since the last purchase. Each such number is called a feature.

Features are computed from raw data by code. From the orders table you get orders_30d = 3, avg_check = 1850, days_since_last = 12. This code matters as much as the model itself, and you'll see why later.

MLOps

A data scientist usually trains the model in a Jupyter notebook. That's a file where code is split into cells, and each cell's output appears right below it. It's convenient for experiments, but a notebook isn't a service.

MLOps is a set of practices that take a model from a notebook to production, the live environment with real users, and keep it working there. Google's MLOps paper defines it as the union of ML systems development (Dev) and their operation (Ops), by analogy with DevOps. Hence the same tools: automation, CI/CD, monitoring. But an ML service has problems a regular service doesn't.

Code, data and the model

In a regular service, behavior is determined by code. Build commit a1b2c3 and you get the same binary and the same behavior.

In an ML service, three things determine behavior:

• the code for training and the service;
• the data it was trained on;
• the model, the file produced from the code and data.

The same code on different data produces a different model. To understand a month from now why the model gave a particular answer, you need versions of all three. Git only stores the code. Data and models need separate versioned storage, covered in the next lessons.

Data drift

A regular service with no releases behaves the same as yesterday. A model with no releases can start making more mistakes: the code didn't change, the data did.

Example. The churn model was trained on last year's data. This year a competitor launched one-hour delivery, and customers who used to order weekly started ordering monthly. The model sees unfamiliar days_since_last values and gets it wrong. There are no errors in the logs, and the service returns 200 OK.

This kind of change is called drift. Data drift: the input data itself changed. For example, 80% of customers used to place more than two orders a month, and now only 50% do. The share of values across ranges is called a feature's distribution. Drift is detected by watching how distributions change. Concept drift: the relationship between the input and the correct answer changed.

CPU and response-code monitoring doesn't catch drift. You need separate metrics, covered in the monitoring lesson.

Training-serving skew

Another failure mode from the same Google paper. Features are computed by different code during training and in production, and the values diverge. This is called training-serving skew, a mismatch between training and production.

Example. During training, the data scientist computed days_since_last in Python relative to the data export date. The backend in production computes it in Go relative to the current moment and rounds differently. In production the model gets different numbers from the ones it saw during training, and it makes more mistakes. Sharing the feature code between training and production protects against skew.

The model's code is a small piece

In 2015 Google engineers published the paper "Hidden Technical Debt in Machine Learning Systems." It has a diagram: the model's own code is drawn as a small box, with everything else drawn large around it. Data collection and verification, feature computation, infrastructure, inference, monitoring, configuration. Google's MLOps paper reproduces it as Figure 1.

Most of "everything else" is familiar to a DevOps engineer: containers, Kubernetes, CI/CD, monitoring. What's new in MLOps is everything around data and models.

What to remember

A feature is a number fed into the model. It's computed from raw data by code. Inference is a trained model running on new data.

Code, data and the model determine an ML service's behavior. All three need versions, and Git only covers the code.

A model degrades with no code changes because of drift, and regular monitoring doesn't see it. Training-serving skew happens when features are computed by different code during training and in production.