Google has recently introduced the Gemini Ultra, and now they have released the upgraded version, Gemini 1.5 Pro. This middle-level (Pro) model is just as capable as the Gemini 1.0 Ultra, but with less processing power.
One of the standout features of Gemini 1.5 Pro is its support for input sizes up to 1 million tokens, the highest among all large-scale language models available today (Anthropic Claude 2.1 supports 200,000 tokens). This opens up more possibilities for utilizing the LLM model in a variety of tasks. Google compares it to a long audio clip of 11 hours, a 1-hour video, 700,000-word document, or 30,000 lines of code.
The architecture behind Gemini 1.5 is a new approach called Mixture-of-Experts (MoE), based on Google’s research from 2017. The idea is to create smaller specialized neural networks that work together, instead of building one large neural network like previous models. When it’s time to use the model, only the relevant networks are selected, resulting in significantly improved runtime efficiency.
Google demonstrated the capabilities of Gemini 1.5 by feeding it the entire transcript of the Apollo 11 astronauts’ conversations during their moon mission, a document spanning 402 pages. Gemini was able to understand the intricacies of the mission remarkably well.
Another example showcased Gemini’s abilities was analyzing the silent film Sherlock Jr. (1924), a 44-minute film, and extracting various events and details from it.
Currently, Gemini 1.5 is undergoing early testing with a standard token limit of 128,000. However, selected developers and organizational clients will have the opportunity to test the 1 million token version before anyone else.
TLDR: Google launched Gemini Ultra and followed it up with Gemini 1.5 Pro, an upgraded mid-level model. Gemini 1.5 Pro supports input sizes up to 1 million tokens, setting a new standard. It utilizes the Mixture-of-Experts architecture for improved efficiency. The model showcases impressive understanding capabilities, such as comprehending the Apollo 11 transcript and analyzing silent films. Early testing is underway, and selected users will have access to the 1 million token version.
Leave a Comment