Gemini Ultra Evaluation Unveils Parity with GPT-4, Yet Triumph Remains Elusive

After Google released the Gemini Ultra model for public use last night, comparisons with other models in the market, especially GPT-4, have begun. GPT-4, being the market leader, is now being challenged by Gemini Ultra.

Ethan Mollick, a professor at Wharton School of the University of Pennsylvania, had the opportunity to use Gemini Ultra before the general public for about a month. He recently posted his thoughts on his experience, stating that Gemini Ultra (also known as Gemini Advanced) is on the same level as GPT-4, making it the second leading model in the AI industry, alongside GPT-4.

However, Mollick mentioned that Gemini Ultra has not yet surpassed GPT-4 in a definitive way. Both models have their own strengths and weaknesses, but overall, they are on a similar level. GPT-4 excels in coding, while Gemini outperforms in explanation and communication by incorporating visuals and search results.

Mollick also pointed out that both models have their fair share of bugs and glitches. They tend to occur after interacting with the models for some time, suggesting that it might be a common issue for large-scale models like these.

In terms of chatbot personalities, Mollick discovered that Gemini is much friendlier and more helpful than ChatGPT, which is quite flat in its demeanor. Regarding security concerns, both models have guardrails in place, but their approaches differ. Gemini has a higher likelihood of responding to negative questions or scenarios.

TLDR: After the release of Gemini Ultra, comparisons with GPT-4 have begun. Ethan Mollick, a professor, stated that Gemini Ultra is on par with GPT-4. However, Gemini has not surpassed GPT-4 definitively. Both models have their strengths and weaknesses, with GPT-4 excelling in coding and Gemini in explanation and communication. Mollick also discovered bugs in both models after extended interaction. Gemini is more friendly and helpful as a chatbot, while ChatGPT is more flat in its responses. Both models have security measures, but Gemini is more likely to respond to negative situations.