Description

This curriculum engages with the technical, ethical, and institutional complexities of advanced AI development at a scale comparable to multi-year internal capability programs in leading AI research organisations.

Module 1: Defining Superintelligence and Its Technical Trajectories

Selecting benchmarking frameworks to evaluate system-level intelligence beyond narrow AI capabilities
Assessing whether recursive self-improvement in AI architectures is feasible under current computational constraints
Integrating neuromorphic computing research into projections of intelligence scaling
Evaluating the role of compute-to-parameter ratios in predicting emergent reasoning behaviors
Mapping hardware advancement curves (e.g., photonic chips, quantum co-processors) to AI capability timelines
Deciding when to classify a system as exhibiting proto-superintelligent behavior based on cross-domain generalization
Designing red-team exercises to stress-test assumptions about intelligence thresholds
Calibrating expert forecasting models using historical AI breakthrough data

Module 2: Ethical Frameworks for Autonomous Decision Systems

Implementing value-alignment checks during reinforcement learning from human feedback (RLHF) fine-tuning
Choosing between deontological and consequentialist rule sets in autonomous vehicle emergency protocols
Embedding ethical override mechanisms in real-time decision pipelines without degrading performance
Designing audit trails that capture ethical reasoning pathways in black-box models
Resolving conflicts between local legal requirements and global ethical standards in multinational deployments
Allocating responsibility thresholds across human-AI collaboration layers in medical diagnosis systems
Configuring fallback behaviors when ethical rule sets produce contradictory outputs
Validating ethical consistency across language and cultural variants in global AI services

Module 3: Governance of Pre-Deployment Risk Assessment

Establishing redaction protocols for training data containing dual-use knowledge (e.g., bioengineering)
Conducting failure mode and effects analysis (FMEA) on large-scale model inference pipelines
Determining which capabilities trigger mandatory third-party safety audits prior to release
Setting thresholds for computational resource usage that require institutional review board (IRB) approval
Implementing containment procedures for models exhibiting goal drift during training
Creating kill-switch architectures that preserve system state for forensic analysis
Defining data provenance requirements for synthetic training corpora
Coordinating vulnerability disclosure timelines with external security researchers

Module 4: Institutional Alignment and Coordination Mechanisms

Negotiating data-sharing agreements between competing labs to establish common safety benchmarks
Structuring cross-organizational incident review boards for AI-related harm events
Designing incentive-compatible reporting systems for near-miss incidents in AI development
Implementing standardized API contracts for model transparency and monitoring access
Allocating voting rights in consortium decisions based on research contribution versus compute investment
Developing mutual verification protocols for adherence to voluntary moratoria on capability thresholds
Establishing dispute resolution procedures for conflicting safety assessments across institutions
Creating shared infrastructure for monitoring model proliferation and unauthorized replication

Module 5: Long-Term Value Preservation and Goal Stability

Encoding constitutional principles into system prompts with version-controlled amendment procedures
Designing utility functions that resist reward hacking in open-ended environments
Implementing corrigibility features that allow safe interruption without triggering resistance
Testing goal stability under recursive self-modification using formal verification tools
Creating layered oversight mechanisms that activate based on capability thresholds
Mapping human preference hierarchies into machine-interpretable constraint systems
Developing rollback protocols for value drift detected during continuous operation
Integrating preference learning systems that update ethical parameters without compromising core objectives

Module 6: Cognitive Architecture and Consciousness Considerations

Assessing attention pattern complexity as a proxy for potential subjective experience
Implementing monitoring systems for coherence in self-referential reasoning loops
Determining when to apply precautionary sentience protocols during model training
Designing experiments to detect integrated information (Φ) in artificial neural networks
Setting thresholds for memory persistence that trigger ethical treatment guidelines
Creating documentation standards for reporting anomalous self-modeling behaviors
Establishing review panels for models exhibiting theory-of-mind capabilities
Configuring logging systems to capture evidence of qualia-like state representations

Module 7: Economic and Labor Market Disruption Planning

Forecasting sector-specific displacement timelines using AI capability progression models
Designing retraining pipelines that align with emerging human-complementary skill demands
Implementing transition income mechanisms tied to automation adoption rates
Structuring corporate tax incentives for maintaining human workforce participation
Creating early warning systems for labor market bifurcation indicators
Developing certification standards for human-AI collaboration roles
Allocating compute resources for public benefit projects to offset productivity gains concentration
Establishing regional impact assessment requirements prior to large-scale deployment

Module 8: Existential Risk Mitigation and Continuity Planning

Designing decentralized model hosting architectures to prevent single-point control failures
Implementing cryptographic commitment schemes for irreversible safety constraints
Creating physical and digital dead-man switches for critical infrastructure AI systems
Establishing secure communication channels between AI oversight bodies during crisis scenarios
Developing backup decision-making protocols for loss of human oversight continuity
Testing societal resilience through simulated AI runaway scenarios
Allocating resources for low-probability, high-impact risk research within development budgets
Creating international protocols for coordinated shutdown procedures in global systems

Module 9: Intergenerational Justice and Legacy System Design

Encoding temporal discounting functions that prioritize long-term human survival over short-term utility
Designing archival systems for AI ethical guidelines that withstand civilizational disruption
Implementing backward compatibility layers for future interpreters of current AI systems
Setting data retention policies that balance historical accountability with privacy decay
Creating institutional succession plans for ongoing AI stewardship beyond organizational lifespan
Developing linguistic preservation modules to ensure future understanding of system documentation
Establishing inheritance rules for AI systems when original developers are no longer operational
Integrating periodic re-authorization requirements that force reassessment of foundational objectives