Description

A focused course, tailored for you

From Research Notebook to Audited ML Pipeline

For research engineers moving lab-grade ML work into pipelines that survive a reviewer, a regulator, and the next person on the team.

The model number is in the slide deck. Somebody asks for the exact data hash, the seed, the env lockfile, and the eval script that produced it. The half-day you planned turns into three days of git archaeology.

$199 one-time

Tailored to your situation. Access within 24 hours. 30-day money-back.

Includes a hand-built implementation playbook delivered alongside course access, generated for your specific situation.

Why this course

Research engineering produces notebooks, ad-hoc training scripts, and result tables that work because the person who wrote them remembers the magic combination of CUDA version, data split, and preprocessing flag. The handover moment exposes that. A collaborator wants to rerun the experiment. A reviewer asks for the eval script. A new hire on the team needs to retrain. A risk reviewer asks how the headline metric was produced and which slices it covers. Each ask uncovers a gap: data not versioned, environment not pinned, runs not tracked, model not documented, no calibration on the subgroups that matter, no automated regression test. The course is about closing those gaps with the lightest possible tooling so the research throughput does not collapse and the pipeline still survives audit.

What you walk away with

A data-versioning setup so every training run points to a specific dataset commit.
A pinned environment that rebuilds bit-for-bit on a colleague's machine.
A run-tracking layout that records seed, code commit, data hash, hyperparameters, and metrics for every experiment.
A model card with calibration curves and subgroup slice metrics ready for reviewer or risk-committee scrutiny.
A CI job that retrains on a held-out set and fails the build on metric regression.

The 12 modules

Module 1. The reproducibility handover

What actually goes wrong when a notebook is handed to another engineer or a reviewer. Walks through five typical failure modes: missing data version, unpinned CUDA, undocumented preprocessing, non-deterministic eval, lost seed. Sets the working repo and dataset the rest of the course builds against, so every later module produces a concrete artefact in the same project.

Module 2. Data versioning that actually scales

DVC and LakeFS compared on a real dataset. How to version raw, intermediate, and feature-engineered tables without copying terabytes. Hashing strategies for arrow, parquet, and image directories. When a git-LFS pointer is enough and when it stops being enough. Sets up the data lineage every later run will reference, including the link from a model artefact back to its exact training-data hash.

Module 3. Environment pinning that survives a year

conda-lock, pixi, and Nix on the same example. Why pip freeze breaks across platforms. How to pin CUDA and cuDNN versions deterministically. Building a Docker image whose lockfile reproduces the lab-cluster environment on a CI runner. Covers the subtle case of GPU-bound dependencies that only resolve on the target architecture and how to handle that without forking the lockfile.

Module 4. Experiment tracking with provenance

MLflow, Weights and Biases, and a flat structured-log alternative compared. The minimum metadata every run must record: code commit, data hash, environment hash, seed, hyperparameters, hardware, wall-clock. Building a run-lookup that goes from a headline metric in a slide deck back to the exact command that produced it. Avoids the trap of tracking metrics without tracking what produced them.

Module 5. Deterministic eval harnesses

How to write an evaluation script that produces the same number on the same data on the same hardware every time. Seeding numpy, torch, cuda, and dataloader workers. Pinning batch order. Handling non-deterministic GPU kernels. Building a held-out eval set that lives independently of the training pipeline and gets evaluated by a separate CI job. Covers floating-point reproducibility traps that catch even careful engineers.

Module 6. Calibration and subgroup slicing

Why a single accuracy number is not enough for any reviewer who matters. Reliability diagrams, expected calibration error, and per-subgroup metric breakdowns. Choosing the slice dimensions a risk reviewer will ask about and pre-computing them at eval time. Translating a sliced eval table into a one-page model card section the reviewer can read without rerunning anything.

Module 7. The model card template

A one-page model card with intended use, training data summary, evaluation methodology, calibration, subgroup metrics, known failure modes, and the lineage hashes that let a reviewer trace any number back to its run. Built as a markdown template that lives next to the model artefact and gets generated by the same CI job that produces the model. Designed to satisfy both a paper appendix and an EU AI Act high-risk technical documentation file.

Module 8. Inference containers from the artefact store

Wrapping a trained model in a container that loads from the artefact store rather than baking weights into the image. How to make the container's environment match the training environment without duplicating the lockfile. Building a smoke test that fails the container if the loaded model's checksum does not match the registered artefact. Covers the handover to a serving team who will not want to know about your training stack.

Module 9. CI that retrains and regresses

Setting up a CI job that pulls the held-out eval set, runs the deterministic eval harness, and fails the build if any tracked metric regresses beyond a threshold. How to choose thresholds per metric and per subgroup. Handling expected variance from non-deterministic kernels. Wiring the CI job into the same pull-request flow the code itself uses, so a reviewer sees the metric impact alongside the code diff.

Module 10. Audit trail for EU AI Act high-risk

The technical documentation file that an EU AI Act high-risk system needs and what already comes for free from modules 2 through 9. Mapping data versioning, environment pinning, run tracking, calibration, model card, and CI to the relevant articles of the Act. What still has to be written by hand. Avoids the trap of building documentation that is parallel to the working pipeline rather than generated from it.

Module 11. Reviewer and collaborator checklists

The one-page checklist you hand a new collaborator: clone, sync data, build environment, run smoke eval, run full eval, generate model card. The separate checklist for a paper reviewer or risk reviewer: where to find the data hash, the env lockfile, the run record, the calibration table, the subgroup eval, the model card. Both calibrated against the failure modes from module 1 so each line item closes a specific gap.

Module 12. Migrating an existing project

How to retrofit modules 2 through 11 onto a real existing project without a six-month freeze. Order of operations that keeps research throughput moving: track first, version second, pin third, automate fourth. The honest cost in engineering time at each step. Triage rules for which models in a backlog deserve the full treatment and which can stay as notebooks with a documented stale-after date.

How this addresses your situation

Specific modules that map to what you said you are dealing with.

A collaborator asks to rerun an experiment from six months ago and the lockfile no longer resolves. Modules 3 and 4 close that gap.

A paper reviewer asks for the eval script and the exact data split. Modules 2, 5, and 7 produce both directly from the run record.

A risk committee asks how the headline accuracy breaks down by user subgroup. Modules 6 and 7 turn that into a one-page model card section.

A serving team wants to take the model into production without learning the training stack. Modules 8 and 11 hand them a container and a checklist they can act on.

What you get with this course

Twelve text-based modules in the Art of Service learning environment.
Downloadable templates: model card markdown, reviewer checklist, EU AI Act high-risk technical documentation skeleton, CI job YAML, conda-lock and pixi starter files.
Worked examples on a sample image-classification and a sample tabular-prediction repo.
A hand-built implementation playbook applied to one of your actual repos, delivered alongside course access.
30-day money-back guarantee.

What you will have in hand by Day 1, Week 1, Month 1

Within 24 hours: course access provisioned and the hand-built implementation playbook delivered.

Weeks 1 to 2: modules 1 to 5 and the data versioning, environment pinning, and run tracking are working on your repo.

Weeks 3 to 4: modules 6 to 9 and the model card, inference container, and CI regression job are working.

Weeks 5 to 6: modules 10 to 12 and the audit trail, reviewer checklist, and retrofit plan for any remaining projects.

Before and after

Before

A notebook that works on your machine, a slide deck that quotes a model number, and an undocumented chain of preprocessing scripts that only you can rerun.

After

A pipeline a collaborator can clone and rerun, a model card a reviewer can read in five minutes, a CI job that catches regressions before merge, and an audit trail that maps directly to the technical documentation a high-risk-system reviewer will request.

What happens if you do not address this

The handover moment, the reviewer request, or the risk-committee question still arrives. Without the tooling in place, each one costs a week of reactive work and produces a one-off artefact that does not generalise. The cost compounds with every new model and every new collaborator. The skill of building reproducible pipelines is moving from nice-to-have to a hiring filter for senior research-engineering roles.

Who it is for

A research engineer, applied scientist, ML PhD or postdoc, or senior MLE who has shipped models that worked in a paper or a demo and now needs those models to survive being handed off, audited, or rerun by someone else. Often working across a lab compute cluster and a production environment, often at the boundary between an academic group and an industrial collaboration. Usually the person their team asks when the question is 'can you reproduce that result from six months ago'.

Who this is NOT for. Not for ML beginners who have not yet trained a model end to end. Not for production platform engineers running large-scale serving infrastructure who already have a mature MLOps stack. Not for purely theoretical researchers who never need to hand a pipeline to anyone else.

How it arrives

Text-based course in the Art of Service learning environment, plus downloadable templates and worked examples for every module, plus the hand-built implementation playbook delivered alongside course access.

Time investment. Around 25 to 35 hours total over four to six weeks, with most of the time spent applying each module to your actual repo rather than reading. Designed to fit alongside ongoing research work.

Why $199 is the right number

Free MLOps blog posts cover individual tools well but rarely connect them into a handover-ready pipeline or an audit trail. Cloud-vendor MLOps certifications optimise for that vendor's managed services and assume a production team. This course is written for the research-engineering boundary where the constraint is keeping research throughput moving while building the minimum reproducibility scaffolding a reviewer or regulator will accept.

FAQ

Does this assume a specific framework like PyTorch or JAX?

No. Worked examples use PyTorch because it is the most common research stack, but every module is framework-agnostic. The data-versioning, environment-pinning, run-tracking, and CI patterns transfer to JAX, scikit-learn, or anything else.

I work on a lab cluster with no internet egress. Does the tooling still work?

Yes. The conda-lock and pixi patterns produce lockfiles that resolve from a local mirror. DVC supports a local or NFS-backed remote. MLflow can run as a local tracking server. The course flags every step that needs adapting for an air-gapped or restricted-egress environment.

How specific is the implementation playbook?

It is written against one of your actual repos. You send the repo URL or a description after purchase, and the playbook names the data sources, the entry-point scripts, the existing CI, and the gaps to close in order. Not a generic checklist.

What if my work is not subject to the EU AI Act?

Module 10 is the only AI-Act-specific module. The other eleven modules are useful for any handover, paper, or audit context. The Act-specific patterns also map cleanly to NIST AI RMF and to internal model-risk-management frameworks at financial-services and healthcare collaborators.

30-day money-back guarantee. If after a week of working through the materials this is not what you needed, reply to the receipt email and a full refund is processed. No questions, no forms.

Within 24 hours your account in the learning environment is provisioned and the tailored implementation playbook is delivered alongside it.