Gemini

GPT-4o

when

There’s a new top dog in the world of generative artificial intelligence benchmarks and its name is Gemini 1.5 Pro.&nbsp;
The previous champ, OpenAI’s ChatGPT-4o, was finally surpassed on Aug. 1 when Google quietly launched an experimental release of its latest model.
Gemini’s latest update <a href="https://x.com/OfficialLoganK/status/1819041453978357899" target="_blank" rel="noopener noreferrer nofollow">arrived</a> without fanfare and is currently labelled as experimental. But it quickly gained the attention of the AI community across social media as reports began to trickle in that it was surpassing its rivals on benchmark scores.
<h2 id="artificial-intelligence-benchmarks">Artificial intelligence benchmarks</h2>
OpenAI’s ChatGPT has been the standard bearer for generative AI since the launch of GPT-3. Its latest model, GPT-4o, and its closest competitor, Anthropic’s Claude-3, have reigned supreme above most other models in most common benchmarks for the past year or so with little in the way of competition.
<figure><img src="https://s3.cointelegraph.com/uploads/2024-08/83086e66-a4ac-4502-91d5-cdcef1bd5962.jfif" alt="Google’s new Gemini AI model dominates benchmarks, beats GPT-4o and Claude-3">
 <figcaption style="text-align: center;">Source: <a href="https://x.com/lmsysorg/status/1819048821294547441" target="_blank" rel="noopener noreferrer nofollow">Large Model Systems Organization</a>.
 </figcaption>
</figure>
One of the most popular benchmarks is called the LMSYS Chatbot Arena. It tests models on a variety of tasks and assigns an overall competency score. GPT-4o received a score of 1,286 while Claude-3 earned a respectable 1,271.
A previous version of Gemini 1.5 Pro scored 1,261. But the experimental version (Gemini 1.5 Pro 0801) released on Aug 1 scored a whopping 1,300.
This indicates that it’s overall more capable than its competitors, but benchmarks aren’t necessarily an accurate representation of what an AI model can and can’t do.
<h2 id="community-excitement">Community excitement</h2>
Without deeper comparisons available, we’re entering an era where the AI chatbot market has matured enough to offer multiple options. It’s ultimately up to end-users to determine which AI model works best for them.
Anecdotally, there’s been a wave of excitement over the latest version of Gemini with users on social media <a href="https://venturebeat.com/ai/googles-gemini-1-5-pro-leaps-ahead-in-ai-race-challenging-gpt-4o/" target="_blank" rel="noopener noreferrer nofollow">calling</a> it “insanely good.” One Redditor went so far as to <a href="https://www.reddit.com/r/Bard/comments/1dernor/gemini_15_pro_is_insanely_good/" target="_blank" rel="noopener noreferrer nofollow">write</a> that it “blows 4o out of the water.”
It’s unclear at this time if the experimental version of Gemini 1.5 Pro will end up being the default going forward. While it remains generally available as of the time of this article’s publication, the fact that it’s in what's considered an early release or testing phase indicates that it’s possible the model could be rescinded or changed for safety or alignment reasons.
Related: <a href="https://cointelegraph.com/news/google-safer-transparent-ai-models" rel="noopener noreferrer nofollow">Google announces safety, transparency advancements in AI models</a>

Google’s new Gemini AI model dominates benchmarks, beats GPT-4o and Claude-3

Table of Contents

Cointelegraph

Other articles published on Aug 1, 2024

Understanding the Legal Challenges of Crypto Bankruptcies

ChainGPT's Breakthrough AI Features Boost Price: Will It Hold Amidst Upcoming Crypto Presales?

ZKX Protocol Closes, Triggering Over 50% Drop in ZKX Token Value

BlackRock Issues Urgent Warning Over Rising Crypto Scams Targeting ETF Investors

Massive Hack on Convergence: $210,000 Stolen in Minutes!

Altcoins With Huge Potential Bonk and Mpeppe (MPEPE)

Join our free newsletter for daily crypto updates!