LMSYS is a platform that ranks chatbots based on their ability to provide responses from multiple chatbots for users to choose the most suitable one. The latest results show that the test version 0801 of Gemini 1.5 Pro outperformed GPT-4o and claimed the top spot for the first time.
The 0801 version model can be used in AI Studio but has not yet been widely adopted. Meanwhile, Gemini Advanced ranks fourth along with Claude 3.5 Sonnet and Llama 3.1 405B, marking the first time an open-source model has achieved such a high ranking.
Although the overall ranking may be number one, individual topics may vary. For example, GPT-4o still prevails when faced with difficult questions, while Claude 3.5 Sonnet is the top choice for programming.
Google previously held the highest rank on LMSYS at the beginning of the year, placing second when using Gemini Pro.
Source: LMSYS
TLDR: Gemini 1.5 Pro test version 0801 surpasses GPT-4o in LMSYS chatbot rankings, marking the first time an open-source model has achieved such a high ranking.
Leave a Comment