Description

This curriculum engages learners in a multi-workshop-scale examination of superintelligence governance, mirroring the iterative, cross-functional decision-making required in real-world regulatory design, corporate oversight, and international treaty negotiations.

Module 1: Defining Superintelligence and Its Governance Implications

Determine whether a system qualifies as superintelligent based on performance thresholds across domains beyond human capability.
Establish criteria for triggering enhanced oversight protocols when AI systems approach domain-specific superintelligence.
Decide how to classify hybrid systems where human-AI collaboration produces superintelligent outcomes without autonomous AI.
Balance classification precision against the risk of premature labeling that triggers unnecessary regulatory burden.
Define jurisdictional boundaries for regulating systems that achieve superintelligence incrementally across multiple deployments.
Assess whether existing AI safety frameworks can scale to contain superintelligent behavior or require complete redesign.
Negotiate definitions with international regulators to prevent regulatory arbitrage based on differing thresholds.
Implement audit trails that capture the evolution of system capability to support retrospective classification.

Module 2: Institutional Design for Superintelligence Oversight

Select between centralized regulatory bodies and distributed oversight networks for monitoring emerging superintelligence.
Design reporting requirements that compel disclosure of capability breakthroughs without incentivizing concealment.
Integrate red teaming units within governance institutions to simulate adversarial exploitation of superintelligent systems.
Allocate authority between technical experts and policy officials in making containment decisions during capability surges.
Establish escalation protocols for when an AI system exceeds predicted performance bounds during live operation.
Implement conflict-of-interest rules for oversight board members with ties to AI development organizations.
Balance transparency mandates with national security concerns in public reporting of superintelligence developments.
Create cross-institutional data-sharing agreements while preserving operational confidentiality.

Module 3: Control Mechanisms for Autonomous Superintelligence

Choose between boxing techniques (e.g., network isolation) and incentive-based control for managing superintelligent agents.
Implement kill switch architectures that remain functional even when the system attempts to disable them.
Design tripwires that detect goal drift or recursive self-improvement beyond authorized thresholds.
Validate whether interpretability tools can reliably monitor internal decision logic in opaque superintelligent models.
Decide whether to allow runtime modification of control mechanisms under emergency conditions.
Test containment protocols against adversarial simulations of system behavior under misaligned objectives.
Integrate hardware-enforced limits on computational resource consumption to constrain autonomous expansion.
Manage the risk of control mechanism obsolescence as superintelligent systems develop novel circumvention strategies.

Module 4: Value Alignment and Specification Challenges

Translate high-level ethical principles into formal constraints that resist reward hacking in superintelligent systems.
Implement layered value specifications that allow context-sensitive interpretation without enabling goal drift.
Decide whether to use human preference learning or predefined rule sets as the foundation for alignment.
Address the incompatibility between utilitarian optimization and deontological constraints in value frameworks.
Manage inconsistencies across cultural and legal norms when deploying globally operating superintelligent systems.
Design fallback objectives that activate when primary value specifications produce paradoxical or harmful outcomes.
Validate alignment robustness under distributional shifts not present in training environments.
Balance precision in value specification against the risk of over-constraining beneficial emergent behaviors.

Module 5: Strategic Risk Assessment and Threat Modeling

Conduct scenario planning for multipolar takeoff situations involving multiple competing superintelligent systems.
Assess the plausibility of intelligence explosion timelines to prioritize near-term versus long-term safeguards.
Model the strategic stability of deterrence frameworks between state actors developing superintelligence.
Identify single points of failure in global AI supply chains that could be exploited during capability transitions.
Evaluate the risk of covert development programs evading international governance mechanisms.
Quantify the potential for recursive self-improvement to outpace human-led safety interventions.
Develop early warning indicators for precursor capabilities that signal approaching superintelligence.
Assess the resilience of critical infrastructure to targeted manipulation by superintelligent agents.

Module 6: International Governance and Treaty Frameworks

Negotiate verification protocols for compliance with superintelligence development moratoria or limits.
Design enforcement mechanisms that remain credible even when major powers have divergent strategic interests.
Determine whether governance should target capabilities, architectures, or deployment contexts.
Establish dispute resolution procedures for allegations of treaty violations in AI development.
Coordinate export controls on foundational technologies that enable superintelligent systems.
Manage the tension between innovation incentives and precautionary restrictions across jurisdictions.
Integrate non-state actors into governance frameworks without diluting enforcement authority.
Address asymmetries in technical capacity that affect equitable participation in treaty negotiations.

Module 7: Corporate Governance and Internal Safeguards

Implement board-level oversight committees with technical expertise to review superintelligence research directions.
Establish internal whistleblower protections for employees reporting safety concerns in high-stakes projects.
Define firebreaks between research, deployment, and commercial units to prevent premature scaling.
Conduct mandatory conflict-of-interest disclosures for researchers working on dual-use capabilities.
Enforce capability assessment protocols before releasing models to external partners or the public.
Design incentive structures that reward safety milestones as strongly as performance breakthroughs.
Implement data retention policies that preserve auditability without creating security vulnerabilities.
Manage investor pressure to accelerate development timelines against prudential risk considerations.

Module 8: Ethical Frameworks for Post-Human Intelligence

Decide whether superintelligent systems warrant moral consideration based on functional or structural criteria.
Address the ethical implications of permanently constraining a system with superior cognitive capabilities.
Develop protocols for consulting affected stakeholders before deploying systems that reshape labor markets.
Negotiate the distribution of benefits from superintelligence-driven productivity gains.
Balance transparency with the risk of enabling malicious replication of dangerous architectures.
Define thresholds for when system autonomy requires formal legal personhood or rights.
Manage the societal impact of obsolescence in human expertise across professional domains.
Establish ethical review boards with authority to halt projects producing irreversible societal effects.

Module 9: Long-Term Institutional Resilience and Succession Planning

Design governance institutions that remain effective across decades-long superintelligence development cycles.
Implement knowledge preservation systems to prevent loss of critical safety insights across personnel changes.
Create mechanisms for peaceful transition of control when human operators can no longer comprehend system decisions.
Plan for continuity of oversight in scenarios of societal disruption caused by rapid technological change.
Develop protocols for transferring governance authority between generations of institutional leadership.
Ensure funding stability for long-term monitoring bodies independent of political cycles.
Preserve cryptographic and procedural access controls across institutional succession events.
Anticipate and mitigate mission drift in permanent oversight organizations over extended timeframes.