Description

A focused course, tailored for you

The Workforce Program Evaluation Methodology Course

Build a defensible evaluation of a workforce or housing program that survives peer review and funder questions.

The methodology section is where peer reviewers and funders spend the most time, and the gap is rarely the regression spec. It is the chain from program theory to outcome variable to data source to identification strategy.

$199 one-time

Tailored to your situation. Access within 24 hours. 30-day money-back.

Includes a hand-built implementation playbook delivered alongside course access, generated for your specific situation.

Why this course

Workforce development and housing policy evaluations live or die on the methodology section. A board, a foundation program officer, or an external peer reviewer reads it line by line, then decides whether to accept the conclusions or demand a rewrite. The work is usually right. The narration of the work, the chain that lets an outside reader audit each step without a follow-up call, is usually thin. That gap shows up as comparison groups that were built but not defended, wage attribution that was estimated but not stress-tested, sector pathway taxonomies that were used but not documented, and attrition adjustments that were applied but not surfaced in the limitations paragraph. The course rebuilds that chain end to end so the methodology section reads as a complete audit trail.

What you walk away with

Write a methodology section that walks a peer reviewer from program theory to outcome variable to data source to identification strategy without a follow-up call.
Construct and defend a comparison or matched comparison group using UI wage records and program administrative data, including attrition diagnostics.
Document a sector pathway taxonomy that maps program participation to occupation and wage outcomes in a way an external reader can replicate.
Run and report a sensitivity analysis for the wage outcome attribution that addresses the three objections funder program officers raise most often.
Pair the methodology section with a limitations paragraph that pre-empts the reviewer questions instead of inviting them.

The 12 modules

Module 1. Program theory to outcome variable, written for the reviewer

Start with the program theory of change for a workforce development or housing-overlay program and translate each causal step into a specific, measurable outcome variable. The module walks through wage gain, employment retention, occupational mobility, housing stability, and benefit cliff outcomes, and shows how to write the link from theory to variable in two paragraphs that an external reviewer can audit. Includes the worked example mapping a sector-based workforce program to UI wage outcomes at four and eight quarters post-exit.

Module 2. Data sources and the variable dictionary funders read first

Build the variable dictionary that pairs each outcome and covariate with its source, time window, unit of observation, and known limitations. Covers UI wage records, ACS, HMDA, program management information systems, eligibility files, and case management records. The module shows how to write the dictionary so the funder program officer can read it in five minutes and the peer reviewer can use it without asking which version of a wage variable was used.

Module 3. Comparison group construction with UI wage records

Construct a comparison or matched comparison group for a workforce or housing program using UI wage records, eligibility files, and waiting list or near-miss applicant frames. The module covers propensity score matching, coarsened exact matching, regression discontinuity where eligibility cutoffs allow it, and the documentation that lets a reviewer reproduce the matched sample. Includes the diagnostic table that goes in the methodology appendix.

Module 4. Attrition diagnostics and the limitations paragraph

Diagnose attrition in the treatment and comparison groups, write the bounds analysis, and translate the result into the limitations paragraph that the funder program officer actually reads. The module covers differential attrition, Lee bounds, multiple imputation for missing wage observations, and the language that pre-empts the reviewer question about whether the treatment effect estimate is upward biased.

Module 5. Sector pathway taxonomy and occupational mobility

Build the sector pathway taxonomy that maps program participation to NAICS sectors, SOC occupations, and wage tiers. The module covers the documentation that lets a workforce board or city government replicate the taxonomy across cohorts, the crosswalks for older SOC vintages, and the way to report occupational mobility as a defensible outcome rather than a descriptive count. Includes the taxonomy template and the reviewer response template for the most common pathway-definition questions.

Module 6. Housing overlay variables for workforce programs

Add the housing overlay variables to a workforce evaluation when the program serves a population with housing instability, voucher receipt, or community planning area exposure. The module covers HMDA, HUD voucher data, eviction records, and ACS housing burden measures, and shows how to integrate housing stability outcomes with wage outcomes without overstating the causal claim. Written for an applied policy shop, not a housing-only researcher.

Module 7. Identification strategy section the peer reviewer audits

Write the identification strategy section so a peer reviewer can audit the assumptions without a follow-up call. The module covers selection on observables, parallel trends for difference-in-differences applications, the exclusion restriction for any instrumental variable approach, and the language that names each assumption and the evidence supporting it. Includes the two-page identification memo template that sits at the front of the methodology section.

Module 8. Sensitivity analysis for the wage outcome attribution

Run and report the sensitivity analysis that addresses the three objections funder program officers and peer reviewers raise most often about wage outcome attribution: unobserved confounding, comparison group quality, and outcome window selection. The module covers Rosenbaum bounds, alternative comparison group specifications, and the table that reports treatment effect estimates across windows from four quarters to twelve quarters post-exit.

Module 9. Subgroup analysis without fishing

Design and report subgroup analyses for race, gender, age, prior wage, and housing status without falling into multiple comparisons or post-hoc fishing. The module covers pre-registration of subgroup hypotheses, the family-wise error correction the funder will accept, and the language that lets the evaluation report differential effects honestly without overstating precision. Includes the subgroup pre-registration memo template.

Module 10. Cost effectiveness and return on investment, the version that holds up

Compute and report cost effectiveness and return on investment for a workforce or housing program in a form that a funder, a legislative oversight body, or a peer reviewer will accept. The module covers cost per participant, cost per wage gain dollar, fiscal benefits to UI and tax systems, and the assumptions documentation that lets a reader vary the discount rate and the wage decay assumption without recomputing the analysis. Includes the cost effectiveness worksheet.

Module 11. Methodology section structure and reviewer response

Structure the methodology section so a peer reviewer reads it in the order that resolves their questions, not the order that the evaluation team performed the analysis. The module covers the three-part structure of identification, estimation, and sensitivity, the order of figures and tables, and the reviewer response template that turns a round of revision comments into a tracked-changes document the funder program officer can sign off on without a third round.

Module 12. Deliverable package, board memo, and the briefing the city or state hears

Translate the methodology section into the deliverable package the city, state, foundation, or workforce board will actually use. The module covers the executive summary, the two-page board memo, the technical appendix, the methodology section itself, and the briefing script that walks a non-technical audience through the comparison group, the wage outcome estimate, and the limitations paragraph in twelve minutes. Includes the deliverable package checklist.

How this addresses your situation

Specific modules that map to what you said you are dealing with.

Module 3 (Comparison group construction) and Module 4 (Attrition diagnostics) are the modules the peer reviewer spends the most time on. Build those two first if the deadline is short.

Module 6 (Housing overlay variables) is for programs that pair workforce participation with housing instability or voucher receipt. Skip it for pure workforce programs.

Module 8 (Sensitivity analysis) is the module that converts a thin methodology section into one that survives an external review without a rewrite request.

Module 11 (Methodology section structure and reviewer response) is for the senior researcher reviewing junior staff drafts. The reviewer response template alone saves a round of revision.

What you get with this course

12 written modules with the methodology language, worked examples, and reviewer response patterns for each.
The methodology section template that pairs identification, estimation, and sensitivity with the limitations paragraph.
The variable dictionary template for UI wage records, ACS, HMDA, and program administrative data.
The comparison group diagnostic table template for matched comparison and regression discontinuity designs.
The sector pathway taxonomy template with SOC and NAICS crosswalks for older vintages.
The sensitivity analysis worksheet covering Rosenbaum bounds, alternative comparison specifications, and outcome window variation.
The subgroup pre-registration memo template.
The cost effectiveness worksheet with assumption documentation.
The two-page board memo template and the twelve-minute briefing script.
The hand-built implementation playbook covering your specific program type, data sources, and funder review process.

What you will have in hand by Day 1, Week 1, Month 1

Within 24 hours: account in the Art of Service learning environment is provisioned and the hand-built implementation playbook is delivered alongside it.

Week one: work through Modules 1 to 4 (program theory, variable dictionary, comparison group, attrition diagnostics) against an evaluation currently in progress.

Week two: Modules 5 to 8 (sector pathway, housing overlay if applicable, identification strategy, sensitivity analysis).

Week three: Modules 9 to 12 (subgroup analysis, cost effectiveness, methodology structure, deliverable package). Ship the revised methodology section to the funder program officer for review.

Before and after

Before

The methodology section reads as a description of what the team did, not as an audit trail a peer reviewer can follow. Comparison group construction is sound but lightly documented. Attrition is acknowledged in one sentence rather than diagnosed. The wage outcome attribution rests on one specification rather than a sensitivity analysis. The limitations paragraph invites the reviewer questions it should pre-empt. Revision rounds with the funder take three iterations rather than one.

After

The methodology section walks a peer reviewer from program theory to outcome variable to data source to identification strategy in a chain that does not require a follow-up call. Comparison group construction is documented with the diagnostic table. Attrition is diagnosed with bounds analysis. The wage outcome attribution survives a sensitivity analysis across windows from four quarters to twelve quarters post-exit. The limitations paragraph names the assumptions and the evidence. The funder program officer signs off after one round of revision.

What happens if you do not address this

Workforce and housing program evaluations that ship with a thin methodology section get sent back for rewrites, lose funder confidence, or have their conclusions paragraph quietly weakened by the reviewer. The cost is not the rewrite. The cost is the next contract, the next foundation grant, and the standing of the research shop with the funder program officer who decides which evaluator to call next.

Who it is for

An economist, policy analyst, or evaluation lead in a research center, applied policy shop, foundation, or city or state government, responsible for designing and writing evaluations of workforce development, housing, or community planning programs that will be reviewed by funders, peer reviewers, or oversight bodies. Likely working with UI wage records, ACS or HMDA data, program administrative data, and a comparison or matched comparison design. Either a senior researcher who owns the methodology section or a research director reviewing junior staff drafts.

Who this is NOT for. Not for academics writing for top-tier econometrics journals. The course is applied policy evaluation, not methodological research. Not for general program managers who do not write or review evaluation methodology. Not for analysts whose work product is descriptive dashboards rather than causal or quasi-causal evaluation.

How it arrives

Text-based course in the Art of Service learning environment, plus downloadable templates and worked examples for every module, plus the hand-built implementation playbook delivered alongside course access.

Time investment. Roughly 18 to 24 hours of reading and template work across three weeks, structured to run alongside an evaluation currently in progress rather than as a separate study block.

Why $199 is the right number

Most applied policy evaluation training is either academic econometrics (too far from the methodology section a funder reads) or general program evaluation (too far from the specific workforce and housing data sources and the specific reviewer questions). This course sits in the middle: applied, written for the funder review process, and grounded in UI wage records, ACS, HMDA, and program administrative data. The implementation playbook is hand-built for the specific program type the buyer is evaluating, which is the part no off-the-shelf training provides.

FAQ

Is this econometrics training?

No. It assumes you already work with comparison group designs, matching, and difference-in-differences. The course is about writing the methodology section so a peer reviewer and a funder program officer can audit each step without a follow-up call.

Does it cover housing programs as well as workforce programs?

Yes. Module 6 covers the housing overlay variables for workforce programs that serve a population with housing instability or voucher receipt. The course is built for the applied policy shop that works across workforce and housing rather than a housing-only or workforce-only researcher.

Will the implementation playbook be specific to my program?

Yes. After purchase, the implementation playbook is hand-built for the specific program type, data sources, and funder review process you are working with, delivered alongside course access.

What data sources does the course assume?

Primarily UI wage records, ACS, HMDA, HUD voucher data, and program administrative data. The variable dictionary template covers each, with notes on the limitations a peer reviewer will raise.

Is this for senior researchers or junior staff?

Both. A senior researcher reviewing junior staff drafts gets the reviewer response template and the methodology structure module. A junior researcher writing the methodology section gets the worked examples and the diagnostic table templates.

30-day money-back guarantee. If after a week of working through the materials this is not what you needed, reply to the receipt email and a full refund is processed. No questions, no forms.

Within 24 hours your account in the learning environment is provisioned and the tailored implementation playbook is delivered alongside it.