Description

A focused course, tailored for you

The Feed Ecosystem Quality Playbook for Recommender Leaders

A working handbook for the head of feed quality and recommendations: integrity signals, creator-ecosystem health, and the ranking guardrails you defend in product review.

Your integrity composite collapses borderline-content rate, creator diversity, repeat-exposure fatigue, and report-rate per impression into one number. When it dips, the ranking team and the trust and safety team read the dip differently because they own different numerators. The weekly read gets rewritten in the product review.

$199 one-time

Tailored to your situation. Access within 24 hours. 30-day money-back.

Includes a hand-built implementation playbook delivered alongside course access, generated for your specific situation.

Why this course

Heads of feed ecosystem quality and recommendations sit at the join between ranking, integrity, creator ecosystem, and policy enforcement. The ranking team owns engagement and retention metrics. The trust and safety team owns violation rates and appeal volume. The creator-ecosystem team owns diversity and retention of the long-tail supply. Each team has its own dashboards, its own definitions, and its own weekly read. The head of feed quality is the person who has to write the one composite that the head of product, the head of legal, and the regulator-facing policy team all read the same way. The hard part is not building the composite. The hard part is defending it weekly when ranking shipped a new model, integrity rolled out a new classifier, and the creator mix shifted because three large creators left and the long tail grew. Attribution of the weekly delta is the core skill, and it is the one nobody has written down for this role.

What you walk away with

Define a single integrity composite for your feed that ranking, trust and safety, and policy read the same way.
Attribute a weekly delta in the composite to ranking change, classifier change, policy change, or creator-mix change with defensible math.
Write the weekly read that the head of product signs off on without rewriting it in the room.
Run a quarterly creator-ecosystem health audit that surfaces long-tail supply risk before it shows up in the composite.
Defend the composite in front of legal and regulator-facing policy teams when a slide is requested for a filing.

The 12 modules

Module 1. Define the integrity composite

How to pick the four to six underlying signals that go into a single composite for the For You or main feed. Borderline-content rate, repeat-exposure fatigue, downstream report-rate per impression, creator diversity index, escape-rate to long sessions. Weighting choices, why a fixed weighting beats a learned weighting for an exec metric, and the version-bump protocol when a signal definition shifts. Worked example with the spreadsheet that holds the canonical definition.

Module 2. Borderline-content rate without classifier drift

How to measure borderline-content exposure week over week when the underlying classifier is being retrained on a four-week cadence. The frozen-classifier shadow score trick, the human-labelled audit set you maintain in parallel, the size of that set, the refresh cadence, and how to read a borderline-rate move when the new classifier shipped on Tuesday. Templates for the audit-set sampling plan and the parallel-score reconciliation.

Module 3. Creator-ecosystem diversity that survives a Gini debate

Why Gini, Herfindahl, entropy, and top-k share each tell a different story about creator concentration, which one a head of product actually reads, and how to publish all four without confusing the room. How to segment by creator tenure, follower band, and content category. How to read a diversity dip when three large creators left and a thousand long-tail creators joined the same week. Worked example with the dashboard layout.

Module 4. Repeat-exposure fatigue without a control group

How to estimate repeat-exposure fatigue when an A and B holdout is not available because the feed is the product. Within-user comparison methods, lagged-exposure regression, the dose-response curve you publish quarterly, and the safety net of a small permanent holdout. The political case for that holdout when growth wants it shut down. Templates for the within-user analysis and the holdout-defence brief.

Module 5. Report-rate per impression and the appeal pipeline

How to read downstream report-rate per impression alongside appeal-overturn rate, and why the two together are the honest integrity signal that legal reads. The joint dashboard, the lag between report and appeal, the categorical breakdown by policy area, and how to detect a classifier false-positive spike before legal notices. Templates for the joint dashboard and the weekly note for trust and safety.

Module 6. Weekly delta attribution

The core skill: given a one to three point move in the composite, attribute the move to ranking change, classifier change, policy change, creator-mix change, or seasonality. The attribution waterfall, the order of operations, the fallback when two changes shipped the same day, and the language to use in the weekly read so the ranking team and trust and safety read the attribution the same way. Templates for the waterfall and the weekly read.

Module 7. Ranking-team joint ownership without rewriting in the room

How to run the weekly two-person read with the ranking lead so the head of product hears one story, not two. The pre-read protocol, the shared definitions document, the disagreement-resolution path, and the meeting hygiene that keeps the read from getting rewritten on Monday morning. Worked example of a disagreement that resolved offline and a script for the one that did not.

Module 8. Trust and safety joint ownership

How to share custody of borderline-content rate and report-rate with the trust and safety team without ending up with two contradicting numbers in the weekly. Shared sampling, shared classifier audit, the divergence note when the two teams legitimately read a week differently, and the escalation path when the divergence is policy-relevant. Templates for the shared audit and the divergence note.

Module 9. Quarterly creator-ecosystem health audit

How to run a quarterly audit that goes deeper than the weekly diversity index. Cohort retention by creator tenure band, supply-side churn analysis, long-tail entry rate, the creator-side surveys you run twice a year, and the recommendations that go to the creator-ecosystem product lead. The cadence, the brief, and the recommendations memo template. Worked example of a quarterly audit that surfaced a long-tail supply risk a quarter before it hit the composite.

Module 10. Defending the composite in front of legal and policy

How to write the slide that goes into a regulatory filing or a policy team brief without misstating the methodology. The methodological footnote, the confidence-band convention, the language to use when the regulator-facing team asks for a number that the composite does not support, and the path for adding a new signal when a filing requires it. Templates for the methodology footnote and the briefing memo.

Module 11. Model-launch readiness review for ranking changes

How to run the readiness review that a major ranking model has to pass before launch. The integrity-side checklist, the borderline-exposure forecast, the creator-ecosystem stress test, the appeal-volume forecast, and the rollback criteria. The role of the readiness review versus the experiment design, who signs off, and how to handle a model that the ranking team really wants to ship and the readiness review flags. Worked example of a readiness review that delayed a launch by two weeks.

Module 12. Building the team and the on-call rotation

How to grow the data science and ML engineering team that owns the composite. The five roles you actually need, the on-call rotation when a classifier breaks at 2 am Pacific, the relationship with ranking platform on-call, the quarterly skip-level with the head of product, and the career-track conversation with each member of the team. Templates for the on-call runbook and the quarterly skip-level brief.

How this addresses your situation

Specific modules that map to what you said you are dealing with.

When the weekly composite dips and the ranking team and trust and safety team disagree on why, modules 1, 6, 7, and 8 are the working sequence.

When a regulator-facing policy team asks for a number for a filing, modules 5, 10, and the methodology-footnote template are the working sequence.

When the creator ecosystem looks healthy on the weekly but you suspect long-tail supply risk, modules 3, 9, and the quarterly audit template are the working sequence.

When ranking wants to launch a major new model, modules 2, 4, 11 and the readiness-review checklist are the working sequence.

What you get with this course

12 written modules in the Art of Service learning environment, average 25 to 35 minutes of reading and worked examples each.
Downloadable templates for the integrity composite definition, the weekly delta attribution waterfall, the weekly read, the shared trust-and-safety audit, the quarterly creator-ecosystem audit brief, the regulatory-filing methodology footnote, and the ranking-launch readiness review checklist.
Worked examples drawn from consumer feed ecosystems at the head-of-quality level, anonymised.
The hand-built implementation playbook tuned to your feed, creator mix, and current cross-team ownership lines, delivered alongside course access.
30-day full refund window if the playbook does not match your role.

What you will have in hand by Day 1, Week 1, Month 1

Within 24 hours your account in the learning environment is provisioned and the tailored implementation playbook is delivered alongside it.

Weeks 1 to 2: modules 1 to 4 and the composite definition template applied to your current weekly read.

Weeks 3 to 4: modules 5 to 8 and the joint-ownership protocols installed with ranking and trust and safety.

Weeks 5 to 6: modules 9 to 12 and the quarterly audit, regulatory footnote, and readiness-review checklist installed.

Before and after

Before

You own a composite that moves weekly. The dip is real. The attribution gets rewritten in the room. The ranking team and trust and safety team read the dip differently. The weekly read takes a day to write and another half day to defend. The quarterly creator audit slips because the weekly is consuming the team.

After

You own a composite with a defended methodology, a waterfall that attributes a weekly delta in an hour, a shared read with the ranking lead that the head of product signs off on, a quarterly audit that runs on a cadence, and a regulator-facing methodology footnote that legal trusts.

What happens if you do not address this

The composite keeps dipping. Each dip is a separate fire drill. The ranking team starts shipping changes against their own internal metrics and the trust and safety team starts publishing their own number. The head of product asks why there are three reads of the same week. The feed ecosystem quality role gets restructured and the composite gets split into ranking-side and integrity-side dashboards, which is the worst of both worlds.

Who it is for

You lead feed ecosystem quality and recommendations at consumer scale. You report into a head of product or a head of engineering. You own a small team of data scientists and ML engineers who define integrity signals, audit ranking outputs, and sit in the room with trust and safety, legal, policy, and the ranking platform team. Your weekly read goes to the head of product and is referenced in policy filings. You are comfortable with offline evaluation, counterfactual estimation, slate-level metrics, and the politics of joint ownership with ranking. You are not looking for an introduction to recommender systems; you are looking for the handbook on how this role does its weekly read so it survives product review.

Who this is NOT for. This is not for an IC ranking engineer who wants to learn LightGCN or a two-tower model. It is not for a trust and safety policy analyst who reviews individual content decisions. It is not for someone new to consumer recommender systems. It is the handbook for the lead who already owns the composite and has to defend it.

How it arrives

Text-based course in the Art of Service learning environment, plus downloadable templates and worked examples for every module, plus the hand-built implementation playbook delivered alongside course access.

Time investment. Six weeks at four to six hours per week, including the time to apply the templates to your current weekly read.

Why $199 is the right number

There are excellent open-source writeups on individual recommender-system topics: two-tower architectures, slate-level evaluation, calibrated ranking. None of them are written for the head of feed ecosystem quality. This handbook is the role-specific one: the weekly read, the joint ownership with ranking and trust and safety, the regulator-facing brief, and the team you build to run it. The closest paid alternative is a consulting engagement at 50,000 USD and up.

FAQ

Is this generic recommender-systems theory?

No. It is the working handbook for the head of feed ecosystem quality. Templates and worked examples are at the level of the weekly read, not at the level of model architecture.

Does it require a specific stack?

No. The templates are stack-agnostic. They have been used by teams running internal ranking platforms, teams on managed ML platforms, and teams using a mix.

What does the implementation playbook contain?

It is hand-built for your role and your feed. It takes the integrity composite definition, the weekly delta waterfall, the joint-ownership protocols, and the quarterly audit, and tunes them to the signals you already publish, the teams you already share ownership with, and the policy context you already brief.

How long until the playbook arrives?

Within 24 hours of purchase, alongside course access.

What if the role is not the right fit?

Full refund within 30 days, no questions.

30-day money-back guarantee. If after a week of working through the materials this is not what you needed, reply to the receipt email and a full refund is processed. No questions, no forms.

Within 24 hours your account in the learning environment is provisioned and the tailored implementation playbook is delivered alongside it.