Sequence Models
Summary
Sequence models, particularly transformers and large language models, have revolutionized machine learning research and achieved remarkable success across various domains. These models, based primarily on attention mechanisms, have surpassed traditional recurrent and convolutional neural networks in tasks such as machine translation, demonstrating superior quality, improved parallelizability, and reduced training time. However, challenges remain in applying sequence models to purposeful adaptive behavior and interaction scenarios. Researchers have identified that these models may struggle with understanding cause and effect relationships, leading to incorrect inferences and auto-suggestive delusions. To address this limitation, recent work proposes treating actions as causal interventions and incorporating factual and counterfactual error signals during training, potentially enabling sequence models to better condition or intervene on data in supervised learning contexts.