Introduction to MLOps: The DevOps of Machine Learning

Every few months a data scientist on the team drops a Jupyter notebook in Slack, says "the model works, can you deploy it?", and walks away. And every time, the same thing happens. The notebook runs on their laptop and nowhere else. There's a model.pkl nobody can find. Half the imports aren't in any requirements file. And when it finally does run in production, three weeks later it quietly starts giving worse answers and nobody notices until a customer complains.

If you're a DevOps or backend engineer, this is the moment you get pulled into MLOps whether you signed up for it or not. The good news: you already know most of it. CI/CD, containers, Kubernetes, Terraform, monitoring — that's 90% of the job. This series is about the other 10%, the parts that are genuinely different when the thing you're shipping is a model instead of a web app.

This is the first article in a 10-part series. We'll take one small model and walk it all the way to a production setup on AWS — FastAPI, Docker, GitHub Actions, MLflow, Kubernetes, Terraform, Prometheus, Grafana, and drift detection. No PhD required. I'm coming at this the same way you would: as someone who has to keep the thing running at 3am, not someone tuning hyperparameters.

Let's start with the obvious question.

What is MLOps, really?

MLOps is the set of practices for taking a machine learning model from "it works on my laptop" to "it's running reliably in production and someone knows when it breaks." That's it. If you swap "machine learning model" for "web service," you've just described DevOps.

So why does it get its own name and its own set of tools? Because a model isn't code the way a REST endpoint is code. A model is code plus the data it learned from plus a training process that turned that data into a bunch of numbers. When any one of those three changes, the behavior changes — and two of them (the data and the training) live completely outside the world your Git repo normally cares about.

Here's the mental model I use. In a normal app, the same input always gives the same output until you change the code. With a model, the code can sit frozen for six months and the quality still degrades — because the real world drifted away from the data the model was trained on. That single fact is the reason MLOps exists.

The ML lifecycle (from a DevOps lens)

You already know the app lifecycle: write code, test, build, deploy, monitor, repeat. The ML lifecycle is the same loop with two extra stages bolted onto the front and one nasty feedback arrow at the end.

• Data — you collect and clean the data the model learns from. Think of this as the "source code" that you didn't write by hand.
• Training — an offline job turns that data into a model artifact. This is your "build," except it can take hours and produce a slightly different result each run.
• Packaging — you wrap the artifact in an API and a container. This part is 100% normal DevOps.
• Deployment — ship it to Kubernetes / ECS / wherever. Also normal DevOps.
• Monitoring — watch latency and errors like always, and also watch whether the predictions are still any good.
• Retraining — when quality drops, you go back to step one with fresh data. This arrow is what makes it a cycle instead of a line.

Notice that four of those six stages are things you do every day. The two new ones — training and retraining — and the extra dimension of monitoring are where the whole series lives.

Training vs inference: the one distinction to burn into your brain

If you take one idea away from this article, make it this one. Almost every design decision in MLOps comes back to keeping these two things separate.

Training is the expensive, offline, occasional job that reads a pile of data and produces a model. It needs lots of CPU or GPU, it needs the whole dataset, and it runs maybe once a day, once a week, or once when someone clicks a button. If it fails, no user notices.

Inference is the cheap, online, constant job that takes one request and returns one answer. It needs the finished model and nothing else — no dataset, no training library baggage if you can help it. It runs thousands of times a minute, and if it falls over, users absolutely notice.

Mixing these up is the number one rookie mistake. You do not want your production API importing your training code, re-reading the dataset on boot, or calling .fit() anywhere near a live request. The training job's output — and its only job — is to hand off a single file: the model artifact.

The model artifact: your new deployable

In web land, the thing you deploy is a container image. In ML land, the star of the show is the model artifact — a serialized file that contains the trained model. For scikit-learn it's usually a .joblib or .pkl. For other frameworks it might be a .pt, a .onnx, or a folder full of weights.

Treat this file the way you'd treat a build artifact: it's immutable, it's versioned, and it should be reproducible. A specific version of your training code plus a specific snapshot of the data should always give you the same artifact. When that's true, debugging a bad prediction in production stops being witchcraft and starts being "which artifact was live, and what trained it?"

Right now we'll just drop the artifact next to our code. That's fine for article one. By article five it graduates to a proper Model Registry in MLflow, because "which .joblib is in production" is not a question you want to answer by SSH-ing into a box and checking file timestamps.

Why DevOps alone isn't enough

You could take a model, wrap it in FastAPI, containerize it, and ship it with the CI/CD pipeline you already have. And honestly, that gets you surprisingly far. So where does plain DevOps run out of road?

• Your artifact isn't in Git. A trained model can be tens of megabytes to many gigabytes of binary weights. Git chokes on it, and diffs are meaningless. You need artifact and data versioning that Git was never built for.
• "It passed the tests" isn't enough. A model can pass every unit test and still be worse than the one it's replacing. You need to compare model quality — accuracy, error, whatever the metric is — as part of the pipeline, not just green checkmarks.
• It rots without any code change. This is the big one. Data drift and concept drift mean a frozen model gets worse over time on its own. Nothing in classic DevOps monitors for that, because normal services don't do it.
• Reproducibility spans three things, not one. To reproduce a deployment you need the code version and the data version and the training config. Miss one and you can't rebuild the same model.

So MLOps isn't a replacement for DevOps. It's DevOps plus artifact/data versioning, plus quality gates, plus drift monitoring, plus retraining. Everything we build in this series is a familiar tool doing one of those extra jobs.

Hands-on: train an artifact, then run inference

Enough theory. Let's make the training-vs-inference split concrete with about 30 lines of Python. We'll use scikit-learn's California housing dataset to predict median house prices — a boring, well-understood regression problem, which is exactly what we want. The model is not the point. The workflow is the point.

Project setup

Create a clean folder and a virtual environment. This same repo grows across all 10 articles, so start it right.

mkdir mlops && cd mlops
python3 -m venv .venv
source .venv/bin/activate

pip install scikit-learn joblib pandas

The training job

Here's train.py. Read the comments — the whole file exists to produce one artifact and then get out of the way.

# train.py — turns data into a model artifact. Runs offline.
import json
from datetime import datetime, timezone
from pathlib import Path

import joblib
from sklearn.datasets import fetch_california_housing
from sklearn.ensemble import RandomForestRegressor
from sklearn.metrics import mean_absolute_error, r2_score
from sklearn.model_selection import train_test_split

MODEL_DIR = Path("model")
FEATURES = ["MedInc", "HouseAge", "AveRooms", "AveBedrms",
            "Population", "AveOccup", "Latitude", "Longitude"]

data = fetch_california_housing()
X_train, X_test, y_train, y_test = train_test_split(
    data.data, data.target, test_size=0.2, random_state=42)

model = RandomForestRegressor(n_estimators=200, max_depth=12, random_state=42)
model.fit(X_train, y_train)

preds = model.predict(X_test)
mae = mean_absolute_error(y_test, preds)
r2 = r2_score(y_test, preds)
print(f"MAE: {mae:.3f}  R2: {r2:.3f}")

MODEL_DIR.mkdir(exist_ok=True)
joblib.dump(model, MODEL_DIR / "model.joblib")

metadata = {
    "model_name": "california-house-price",
    "model_version": "1.0.0",
    "features": FEATURES,
    "metrics": {"mae": round(mae, 4), "r2": round(r2, 4)},
    "trained_at": datetime.now(timezone.utc).isoformat(),
}
(MODEL_DIR / "metadata.json").write_text(json.dumps(metadata, indent=2))
print("Saved model/model.joblib")

Run it:

python train.py

On my machine that printed:

MAE: 0.346  R2: 0.792
Saved model/model.joblib

An MAE of 0.346 means the model is off by about $34,600 on average (the target is in units of $100,000), and an R² of 0.79 means it explains most of the variation in prices. Good enough. But look at what actually landed on disk — that's the important part:

model/
├── model.joblib      # the artifact — this is what we deploy
└── metadata.json     # what it is, how good it is, when it was made

The metadata.json is a small habit that pays off forever. It travels with the artifact and answers "what is this file?" without anyone having to guess:

{
  "model_name": "california-house-price",
  "model_version": "1.0.0",
  "features": ["MedInc", "HouseAge", "AveRooms", "AveBedrms",
               "Population", "AveOccup", "Latitude", "Longitude"],
  "metrics": { "mae": 0.346, "r2": 0.792 },
  "trained_at": "2026-07-08T00:00:00+00:00"
}

The inference job

Now the other half. Notice what predict.py does not do: it never imports the training code, never touches the dataset, never calls .fit(). It loads a finished file and asks it a question. This is the code that becomes our production API next article.

# predict.py — loads the artifact and runs one prediction. Online.
import json
from pathlib import Path
import joblib

model = joblib.load(Path("model") / "model.joblib")
metadata = json.loads((Path("model") / "metadata.json").read_text())

# One California block group, features in the trained order.
sample = [8.3252, 41.0, 6.9841, 1.0238, 322.0, 2.5556, 37.88, -122.23]

prediction = model.predict([sample])[0]
print(f"Model: {metadata['model_name']} v{metadata['model_version']}")
print(f"Predicted median house value: ${prediction * 100_000:,.0f}")

python predict.py

Model: california-house-price v1.0.0
Predicted median house value: $425,117

That's the entire ML lifecycle in miniature. train.py is the build. model.joblib is the artifact. predict.py is the runtime. Two files, one boundary, and everything else in this series is about making that boundary production-grade.

What's next

We've got a trained artifact and a script that runs inference on it. A script isn't a service, though — you can't put python predict.py behind a load balancer. In the next article we wrap this exact model in a FastAPI service with real request validation, automatic Swagger docs, and a project structure that's ready to containerize.

Quick recap before you go:

• MLOps is DevOps for models — same loop, plus data/artifact versioning, quality gates, and drift monitoring.
• Training is offline, expensive, and occasional. Inference is online, cheap, and constant. Keep them apart.
• The model artifact is your new deployable — version it, and give it metadata.
• Plain DevOps gets you 90% there; the last 10% is the reason this series exists.

If this was useful, the full code lives in the series repo and every article builds on the last one. See you in part two. Happy shipping 🚀