What is the Cassandra Problem in AI?
LLMs such as ChatGPT have greatly impressed us with their capabilities. They are already beginning to fundamentally shape how we live, yet they are inhibited by a key problem: while they are technologically impressive, they essentially operate as expensive consensus mechanisms. The team behind Allora have named this phenomenon: the Cassandra Problem.
Cassandra was a Trojan princess and a priestess of Apollo, endowed with the gift of prophecy. According to Greek mythology, Apollo granted Cassandra the ability to foresee the future. However, when she spurned his romantic advances, Apollo cursed her so that no one would ever believe her predictions. This tragic twist rendered her prophecies futile, despite their accuracy.
Cassandra's most famous prophecy occurred during the Trojan War. She foresaw the downfall of Troy and specifically warned the Trojans about the Greek warriors hiding inside the wooden horse. Despite her desperate warnings, the Trojans ridiculed her, dismissed her as mad, and brought the horse into the city, leading to their ultimate destruction.
Cassandra weeps as Troy burns in the night…
What are the implications for AI?
Modern AI systems, including large language models (LLMs), often face a predicament similar to Cassandra's. Even though these systems can provide valuable insights, their outputs are frequently ignored or undervalued if they differ from what most people expect or believe. This happens because AI systems are usually trained to generate responses that align with common patterns found in large datasets, even if these responses are sometimes wrong or problematic.
The main issue is that these AI systems prioritise fitting in with the majority opinion over being accurate in specific situations. For example, imagine an AI trained on a massive amount of medical data to diagnose illnesses. If most of the data suggests that a common symptom leads to a common illness, the AI will usually diagnose the common illness, even if, in a specific case, the symptoms actually indicate a rare but serious disease.
Now, think about a network of AI models that combine their strengths to make decisions. They use a method called swarm intelligence to merge inferences from different models, typically relying on a weighted average based on each model's past performance or reputation to decide how much weight to give to each model's output. This means that if one model is highly accurate at diagnosing that rare disease, its output might still be ignored because it's overshadowed by the other models that favor the common diagnosis.
These specialised models are like Cassandra: they can provide excellent insights in certain situations, but their valuable contributions are often overlooked because the system favours the majority opinion. This leads to potentially missing critical, accurate insights that could make a big difference.
