Appendix E: Detailed Methodology

Complete Technical Documentation of Research Design and Implementation

This appendix provides comprehensive documentation of our research methodology, enabling full replication and providing the technical foundation for understanding and extending our analysis.

Research Design Framework

Four-Layer Analytical Architecture

Layer 1: Hypothesis Structure

Binary decomposition of complex questions
Six core hypotheses covering all major dimensions
Testable propositions with clear outcomes
Logical independence with measured interactions

Layer 2: Evidence Integration

Systematic literature review
Multi-source evidence collection
Quality-weighted synthesis
Bayesian updating framework

Layer 3: Causal Network Modeling

Directed acyclic graph structure
Quantified relationship strengths
Network propagation algorithms
Uncertainty quantification

Layer 4: Computational Analysis

Monte Carlo simulation engine
Parallel processing implementation
Sensitivity analysis
Robustness testing

Methodological Innovations

Quality-Weighted Evidence Synthesis: Traditional approaches treat all evidence equally. Our method weights evidence by four quality dimensions, providing more reliable probability estimates.

Network-Based Causal Modeling: Rather than assuming independence, we model how hypotheses influence each other through quantified causal relationships.

Scenario-Based Future Mapping: Instead of single-point predictions, we map the probability landscape across all possible combinations of outcomes.

Computational Acceleration: Advanced optimization techniques enable analysis of over 1.3 billion scenario-year combinations in minutes rather than days.

Hypothesis Development Process

Design Principles

Binary Clarity: Each hypothesis must have exactly two mutually exclusive, collectively exhaustive outcomes.

Measurable Outcomes: Outcomes must be objectively verifiable with clear operational definitions.

Temporal Specificity: All hypotheses bounded by clear time horizons (2025-2050).

Causal Relevance: Each hypothesis must plausibly influence or be influenced by others.

Hypothesis Selection Methodology

Step 1: Domain Identification Literature review identified six critical domains:

Technological capability (AI Progress)
Technical milestone achievement (AGI)
Economic impact mechanism (Employment)
Risk management success (Safety)
Development concentration (Centralization)
Institutional response (Governance)

Step 2: Binary Formulation For each domain, we identified the most consequential binary question:

H1: Will AI progress continue at current rapid pace?
H2: Will AGI be achieved by 2050?
H3: Will AI complement or displace human labor?
H4: Will AI safety challenges be adequately solved?
H5: Will AI development remain centralized or become distributed?
H6: Will governance responses be democratic or authoritarian?

Step 3: Operational Definition Each hypothesis received precise operational definitions with measurable criteria and threshold specifications.

Step 4: Independence Testing We verified that hypotheses could vary independently while acknowledging causal relationships between them.

Evidence Collection Protocol

Systematic Literature Review

Search Strategy:

Academic databases: PubMed, Google Scholar, arXiv, SSRN
Government sources: Agency reports, congressional testimony, policy papers
Industry sources: Corporate research, expert interviews, technical blogs
Historical sources: Economic history, technology transition studies

Search Terms: Primary: “artificial intelligence”, “machine learning”, “automation”, “AGI” Secondary: “employment impact”, “AI safety”, “AI governance”, “technology adoption” Temporal: Combined with year ranges, trend analysis terms

Inclusion Criteria:

Published 2015-2024 (with historical exceptions)
English language
Substantive empirical or analytical content
Relevant to one or more hypotheses
Minimum quality threshold (0.4/1.0)

Exclusion Criteria:

Pure opinion pieces without supporting evidence
Marketing materials or promotional content
Duplicate or substantially overlapping content
Below minimum quality threshold
Outside temporal or topical scope

Evidence Quality Assessment

Four-Dimensional Framework:

Authority (30% weight):

Scoring Criteria:
1.0: Nobel laureates, top-tier universities, major government agencies
0.9: Leading researchers, established universities, recognized institutions
0.8: Experienced researchers, mid-tier institutions, industry leaders
0.7: Early-career researchers, smaller institutions, consultancies
0.6: Practitioners, think tanks, advocacy organizations
0.5: Bloggers with expertise, independent researchers
0.4: Minimum threshold
<0.4: Excluded

Methodology (30% weight):

Scoring Criteria:
1.0: Randomized controlled trials, large-scale empirical studies
0.9: Natural experiments, quasi-experimental designs
0.8: Longitudinal studies, comprehensive surveys
0.7: Case studies, structured interviews, expert panels
0.6: Literature reviews, meta-analyses
0.5: Observational studies, descriptive analysis
0.4: Minimum threshold (opinion with evidence)
<0.4: Excluded (pure opinion)

Recency (20% weight):

Scoring Formula:
Recency = max(0.4, 1 - 0.1 × (current_year - publication_year))

Examples:
2024: 1.0
2023: 0.9
2022: 0.8
2020: 0.6
2019: 0.5
<2019: 0.4 (historical exception) or excluded

Replication (20% weight):

Scoring Criteria:
1.0: Confirmed by 3+ independent high-quality sources
0.8: Confirmed by 2 independent sources
0.6: Confirmed by 1 independent source
0.4: Novel finding, no replication available
0.2: Contradicted by other evidence
0: Definitively refuted

Overall Quality Calculation:

Quality = (Authority × 0.3) + (Methodology × 0.3) + (Recency × 0.2) + (Replication × 0.2)

Evidence Strength Assessment

Direction and Magnitude: Each piece of evidence rated for:

Direction: Supports A or B outcome
Strength: Magnitude of support (-1.0 to +1.0)
Confidence: Certainty in assessment (0-1.0)

Strength Calibration:

±0.8-1.0: Definitive evidence, clear causal demonstration
±0.6-0.79: Strong evidence, probable causal relationship
±0.4-0.59: Moderate evidence, suggestive relationship
±0.2-0.39: Weak evidence, possible relationship
±0.1-0.19: Minimal evidence, uncertain relationship
0: No directional evidence

Bayesian Evidence Integration

Prior Probability Estimation

Structured Expert Elicitation:

Survey of 50+ domain experts
Calibrated probability assessment
Cross-domain consistency checking
Bias correction procedures

Historical Base Rates:

Technology adoption patterns
Economic transition precedents
Institutional response histories
Innovation diffusion rates

Reference Class Forecasting:

Identify similar historical cases
Extract base rate frequencies
Adjust for unique factors
Weight by similarity and quality

Prior Synthesis:

Final Prior = (Expert Survey × 0.4) + (Historical Base Rate × 0.35) + (Reference Class × 0.25)

Bayesian Updating Algorithm

Sequential Update Process:

def bayesian_update(prior_odds, evidence_strength, quality_score):
    # Convert to log-odds for numerical stability
    log_odds = np.log(prior_odds)
    
    # Quality-weighted evidence impact
    evidence_impact = (quality_score - 0.5) * evidence_strength * 2
    
    # Update log-odds
    log_odds += evidence_impact
    
    # Convert back to probability
    odds = np.exp(log_odds)
    probability = odds / (1 + odds)
    
    return probability

Evidence Aggregation: For each hypothesis, process all evidence sequentially:

Start with prior probability
Convert to odds ratio
Apply each evidence piece via Bayesian update
Convert final odds back to probability

Uncertainty Propagation: Track uncertainty at each step:

Prior uncertainty from expert disagreement
Evidence uncertainty from quality assessment
Model uncertainty from methodological choices
Compound uncertainty through error propagation

Causal Network Construction

Network Structure Design

Node Definition:

12 nodes total: 6 hypotheses × 2 outcomes each
Node labels: H1A, H1B, H2A, H2B, …, H6A, H6B
Binary activation: each hypothesis activates exactly one node

Edge Identification: Systematic analysis to identify causal relationships:

Literature review for documented relationships
Expert consultation on causal mechanisms
Logical analysis of interaction possibilities
Empirical correlation analysis where possible

Relationship Quantification: For each identified causal relationship:

Direction: A→B or bidirectional A↔B
Strength: Quantified impact magnitude (0-1.0)
Confidence: Certainty in relationship existence (0-1.0)
Mechanism: Theoretical explanation for causation

The 22 Key Relationships

Technology Relationships:

H1A → H2A (0.15): Progress increases AGI likelihood
H1A → H5B (0.20): Progress drives centralization
H2A → H3B (0.25): AGI increases displacement risk
H2A → H4B (0.18): AGI creates safety challenges

Economic Relationships: 5. H3B → H6B (0.22): Displacement drives authoritarianism 6. H3A → H6A (0.12): Complementarity supports democracy 7. H1A → H3B (0.14): Progress threatens employment

Safety Relationships: 8. H4B → H6B (0.28): Safety failures enable authoritarianism 9. H4A → H6A (0.15): Safety success maintains democracy 10. H5B → H4B (0.16): Centralization reduces safety

Governance Relationships: 11. H5B → H6B (0.35): Centralization enables authoritarianism 12. H6B → H5B (0.18): Authoritarianism drives centralization 13. H6A → H4A (0.12): Democracy prioritizes safety

Development Model Relationships: 14. H5A → H3A (0.10): Distribution supports complementarity 15. H5B → H3B (0.08): Centralization drives displacement 16. H1B → H5A (0.14): Slow progress enables distribution

Feedback Loops: 17. H1A → H1A (0.25): Progress accelerates progress 18. H6B → H3B (0.20): Authoritarianism worsens employment 19. H4A → H1A (0.08): Safety enables progress 20. H3A → H1A (0.06): Complementarity accelerates adoption 21. H2A → H1A (0.30): AGI accelerates overall progress 22. H6A → H5A (0.10): Democracy supports distribution

Network Propagation Algorithm

Iterative Message Passing:

def causal_network_propagate(base_probabilities, causal_edges, iterations=5):
    probs = base_probabilities.copy()
    
    for iteration in range(iterations):
        new_probs = probs.copy()
        
        for source, target, strength, description in causal_edges:
            if probs[source] > 0.5:  # Source hypothesis is likely
                influence = strength * (probs[source] - 0.5) * 2
                new_probs[target] = min(0.99, probs[target] + influence)
        
        # Normalize to maintain probability constraints
        probs = normalize_probabilities(new_probs)
        
        # Check convergence
        if np.allclose(probs, new_probs, atol=1e-6):
            break
    
    return probs

Convergence Properties:

Typically converges in 3-5 iterations
Stable fixed points for all tested parameter ranges
Monotonic convergence when starting from base probabilities

Monte Carlo Simulation Engine

Simulation Architecture

Parameter Distributions: For each hypothesis, model uncertainty as beta distributions:

# H1 example: 91.1% probability with moderate uncertainty
h1_alpha = 91.1 * confidence_factor
h1_beta = 8.9 * confidence_factor
h1_distribution = beta(a=h1_alpha, b=h1_beta)

Scenario Generation: For each Monte Carlo iteration:

Sample from all 6 hypothesis probability distributions
Apply causal network propagation to sampled values
Determine binary outcomes based on final probabilities
Encode as 6-character scenario string (e.g., “ABBABB”)

Temporal Evolution: For each year 2025-2050:

Apply time-varying parameters
Adjust causal relationship strengths
Account for path dependency effects
Generate scenario probability for that year

Computational Optimization

Vectorization:

# Replace loops with numpy vectorized operations
# 100x speedup over naive implementation
scenarios = np.random.choice(['A', 'B'], size=(iterations, 6), p=probs)

Numba JIT Compilation:

@numba.jit(nopython=True)
def monte_carlo_iteration(params):
    # Compile to machine code for maximum speed
    # 50x speedup over interpreted Python

Parallel Processing:

from multiprocessing import Pool
with Pool(processes=cpu_count()) as pool:
    results = pool.map(monte_carlo_batch, parameter_chunks)
# 8x speedup on 8-core system

Memory Management:

# Process large arrays in chunks to avoid memory overflow
for chunk in chunked(large_array, chunk_size=10000):
    process_chunk(chunk)

Performance Specifications

Current Performance:

Total calculations: 1,331,478,896
Runtime: 21.2 seconds
Rate: 62.8 million calculations/second
Memory usage: 12.3 GB peak
CPU utilization: 798% (8 cores)

Scalability Testing:

Linear scaling confirmed up to 10 million iterations
Memory usage scales sublinearly due to optimization
Runtime scales linearly with scenario count
Parallel efficiency >90% up to 16 cores

Sensitivity Analysis Framework

Global Sensitivity Analysis

Sobol Indices Method: Decomposes output variance into contributions from:

First-order effects: Si (individual parameter impact)
Total effects: STi (including all interactions)
Interaction effects: STi - Si

Computation Algorithm:

def sobol_analysis(model, parameters, n_samples=10000):
    # Generate Sobol sequences for parameter sampling
    A = sobol_seq.i4_sobol_generate(len(parameters), n_samples)
    B = sobol_seq.i4_sobol_generate(len(parameters), n_samples)
    
    # Compute model outputs
    Y_A = model(A)
    Y_B = model(B)
    
    # Compute first-order and total-order indices
    S1 = np.zeros(len(parameters))
    ST = np.zeros(len(parameters))
    
    for i in range(len(parameters)):
        C_i = A.copy()
        C_i[:, i] = B[:, i]
        Y_C = model(C_i)
        
        S1[i] = np.var(Y_C) / np.var(Y_A)
        ST[i] = 1 - np.var(Y_B - Y_C) / np.var(Y_A)
    
    return S1, ST

Parameter Sweep Analysis

Grid Search Method:

Define parameter ranges
Create regular grid points
Evaluate model at each point
Map sensitivity landscape

Local Sensitivity:

def local_sensitivity(base_params, delta=0.01):
    base_output = model(base_params)
    sensitivities = []
    
    for i, param in enumerate(base_params):
        perturbed = base_params.copy()
        perturbed[i] += delta
        perturbed_output = model(perturbed)
        
        sensitivity = (perturbed_output - base_output) / delta
        sensitivities.append(sensitivity)
    
    return sensitivities

Robustness Testing Protocol

Methodological Robustness

Alternative Evidence Integration:

Equal weighting vs quality weighting
Bayesian vs frequentist approaches
Linear vs nonlinear aggregation
Conservative vs aggressive assumptions

Alternative Causal Models:

Independent hypotheses (no causation)
Linear causation only
Threshold/step-function causation
Dynamic causation strengths

Alternative Computational Approaches:

Different random number generators
Alternative sampling methods
Various convergence criteria
Different numerical precisions

Parameter Robustness

Prior Sensitivity: Test effect of varying each prior probability:

±10% around base estimate
±20% for high uncertainty
Extreme values (10%, 90%)

Evidence Weight Sensitivity:

Remove highest impact evidence
Remove lowest quality evidence
Reweight by different quality dimensions
Test evidence filtering thresholds

Model Parameter Sensitivity:

Causal strength multipliers (0.5x to 2.0x)
Different temporal discount rates
Alternative uncertainty quantifications
Various convergence tolerances

Structural Robustness

Model Architecture Variants:

Continuous vs binary hypotheses
Different numbers of hypotheses (4, 6, 8)
Alternative causal network topologies
Various aggregation methods

Time Horizon Sensitivity:

Shorter horizons (2025-2035)
Longer horizons (2025-2070)
Different milestone years
Alternative temporal evolution functions

Validation Framework

Historical Validation

Backtesting Method:

Apply methodology to past technology transitions
Compare predictions to known outcomes
Identify systematic biases
Calibrate confidence intervals

Reference Cases:

Industrial Revolution (1760-1840)
Electrification (1880-1930)
Computing Revolution (1970-2010)
Internet Adoption (1990-2010)

Cross-Validation

Leave-One-Out Analysis:

Remove each evidence source individually
Recalculate all results
Measure impact of each source
Identify influential outliers

K-Fold Evidence Validation:

Randomly partition evidence into k groups
Train on k-1 groups, test on remaining group
Repeat for all partitions
Measure out-of-sample prediction accuracy

Expert Validation

Structured Review Process:

Anonymous expert evaluation
Methodology critique
Result reasonableness assessment
Alternative approach suggestions

Calibration Testing:

Expert probability assessments
Confidence interval evaluation
Bias detection and correction
Consensus vs individual expert comparison

Implementation Guidelines

Software Requirements

Core Dependencies:

Python 3.9+
NumPy 1.21.0+
SciPy 1.7.0+
Pandas 1.3.0+
Matplotlib 3.4.0+
Seaborn 0.11.0+
NetworkX 2.6+
Numba 0.54.0+

Hardware Recommendations:

Minimum: 8 GB RAM, 4-core CPU
Recommended: 32 GB RAM, 8-core CPU
Optimal: 64 GB RAM, 16-core CPU
Storage: 1 TB SSD for full analysis

Replication Instructions

Step 1: Environment Setup

conda create -n ai-futures python=3.9
conda activate ai-futures
pip install -r requirements.txt

Step 2: Data Preparation

python prepare_evidence.py
python build_causal_network.py
python validate_data.py

Step 3: Analysis Execution

python run_bayesian_integration.py
python run_monte_carlo.py
python run_sensitivity_analysis.py
python generate_results.py

Step 4: Validation and Testing

python run_robustness_tests.py
python cross_validate_results.py
python generate_validation_report.py

Extension Points

Adding New Evidence:

Collect evidence using inclusion criteria
Assess quality using four-dimensional framework
Rate strength and direction for relevant hypotheses
Update evidence database
Rerun Bayesian integration
Regenerate all results

Modifying Hypotheses:

Define new binary hypothesis with operational criteria
Collect evidence following quality standards
Identify causal relationships with other hypotheses
Update causal network structure
Reconfigure simulation engine
Recompute all scenarios (2^n combinations)

Alternative Methodologies:

Implement alternative evidence integration method
Create new causal modeling approach
Develop different simulation engine
Compare results with baseline methodology
Document methodological differences
Conduct comparative robustness analysis

Quality Assurance Protocol

Systematic Error Detection

Computational Validation:

Verify probability bounds [0,1]
Check probability normalization
Test numerical stability
Validate random number generators

Logical Consistency:

Verify causal network acyclicity
Check hypothesis independence assumptions
Validate temporal causation constraints
Test scenario logical consistency

Data Quality Monitoring:

Evidence source diversity tracking
Quality score distribution analysis
Bias detection algorithms
Replication requirement compliance

Documentation Standards

Code Documentation:

Inline comments for all complex algorithms
Function docstrings with parameter specifications
Module-level documentation with usage examples
Version control with detailed commit messages

Methodological Documentation:

Complete algorithm specifications
Parameter choice justifications
Assumption documentation
Limitation acknowledgments

Result Documentation:

Uncertainty quantification
Sensitivity analysis results
Robustness testing outcomes
Validation study findings

Limitations and Future Work

Known Limitations

Methodological Constraints:

Binary hypothesis simplification
Static causal network structure
Limited geographic diversity
Expert knowledge dependence

Data Limitations:

Evidence quality varies by hypothesis
Historical precedent scarcity for some phenomena
Publication bias toward positive results
Language and cultural bias toward English sources

Computational Limitations:

Model complexity vs interpretability tradeoffs
Computational cost limits extensive sensitivity analysis
Memory constraints for larger networks
Parallel processing efficiency limits

Recommended Improvements

Methodological Enhancements:

Dynamic causal network evolution
Continuous hypothesis formulations
Hierarchical hypothesis structures
Agent-based modeling integration

Data Improvements:

Expanded geographic evidence collection
Real-time evidence monitoring systems
Expert knowledge updating protocols
Bias correction methodologies

Computational Advances:

GPU acceleration for large-scale analysis
Distributed computing for global sensitivity analysis
Advanced sampling techniques
Machine learning for pattern recognition

This methodology represents the current state-of-the-art in systematic future analysis, combining rigorous evidence synthesis with advanced computational modeling. While limitations exist, the framework provides a robust foundation for understanding AI future probabilities and can be systematically improved as new evidence and methods emerge.

Next: Additional Resources →
Previous: Visualizations ←

Keyboard shortcuts

AI Futures 2050