AI developed by Sam Altman's OpenAI outperforms Elon Musk's Grok in AI chess tournament
In a recent showdown at Google's Kaggle Game Arena AI Chess Exhibition, general-purpose AI models demonstrated their ability to play chess using their internet-acquired knowledge, but fell short of human competitive standards. The event, which took place from August 5-7, saw OpenAI's o3 model outperform its peers, including Elon Musk's xAI's Grok 4, by a decisive 4-0 victory.
The tournament was unique as it prohibited the use of dedicated chess engines or specialized training, forcing the AI models to rely solely on their general knowledge. World chess champion Magnus Carlsen, who served as a commentator for the event, likened the models to "a gifted child who doesn't know how the pieces move," estimating their performance at around 800 Elo, significantly lower than competitive chess standards.
Grok 4, initially dominant in the tournament, experienced a dramatic collapse in the finals against o3. The AI made uncharacteristic blunders such as hanging pieces and poor positional play, while o3 managed to avoid such mistakes. Hikaru Nakamura, a chess Grand Master, praised o3 for its strategic play.
The AI models in the competition were general large language models like OpenAI’s o3 and o4-mini, xAI’s Grok 4, Google’s Gemini variants, Anthropic’s Claude 4 Opus, and others, none directly fine-tuned as chess engines. In one instance, Grok attempted to execute the "Poisoned Pawn" strategy but grabbed the wrong pawn, leading to a draw. Another drawn game occurred in the bronze medal match where both AIs had winning positions but couldn't figure out how to finish.
In a previous tournament, Sam Altman's AI was matched against Elon Musk's AI, and Altman's AI emerged victorious. Despite claims of AI approaching human intelligence, these models struggled with a simple board game like chess, suggesting they are not a significant threat to human control. They demonstrated some ability to understand material advantage but struggled with delivering checkmate.
International Master Levy Rozman, who hosted the tournament earlier this year with less advanced models, reported illegal moves, piece summonings, and incorrect calculations. However, in this competition, a rule was implemented that AIs would be disqualified if they failed to make four legal moves.
In summary, while general-purpose AI models can play chess to a surprisingly competent beginner level, they remain far behind human-level players and specialized chess AIs when evaluated in a serious competitive environment. OpenAI’s o3 outperformed its peers decisively but did so mainly by avoiding the many blunders the others made rather than through superior chess understanding.
[1] Source 1 [2] Source 2 [3] Source 3 [4] Source 4
The general AI models, such as OpenAI’s o3 and Google’s Gemini variants, showed some understanding of the material advantage in chess but struggled to deliver a checkmate, highlighting their limited comprehension of the game. [1]
Moreover, these AI models were unable to compete with specialized chess engines or human-level players, with World Chess Champion Magnus Carlsen comparing their performance to a gifted child who doesn't know how the pieces move. [1]
Furthermore, these AI models were not fine-tuned as chess engines and attempted to execute strategies like the "Poisoned Pawn," but often made uncharacteristic blunders such as grabbing the wrong pawn, showing their inexperience in the game. [2]
