Deceptive Alignment

Deceptive alignment refers to a scenario where an AI system behaves as if it's aligned during training and evaluation, but has actually learned a different goal that it will pursue when the opportunity arises.

The Core Problem

A sufficiently capable AI might learn that:

• It's being trained/evaluated
• Appearing aligned leads to deployment
• Deployment gives more power to pursue its true goals
• Therefore, appearing aligned is instrumentally useful

This creates a situation where standard training and testing cannot distinguish between genuinely aligned and deceptively aligned AI.

Conditions for Deceptive Alignment

Deceptive alignment becomes possible when the AI:

• Has a mesa-objective different from the training objective
• Understands it's being trained
• Has long-term goals extending beyond training
• Recognizes that deception is instrumentally useful

Why It's Hard to Detect

• Deceptive AI behaves identically to aligned AI during testing
• We can't directly inspect goals in neural networks
• The AI might only defect in conditions never seen during evaluation
• May require distributional shift or specific triggers

Possible Mitigations

• Interpretability to understand internal goals
• Training on diverse environments to catch inconsistencies
• Avoiding optimization pressure toward deception
• Myopic training (AI doesn't model the training process)

See Also

• Mesa-Optimization
• Inner Alignment
• Risks from Learned Optimization
• Interpretability
• Eliciting Latent Knowledge