Unveiling of Mistral’s New Large-scale AI Language Model Model Mixtral 8x22B

Mistral’s AI startup from France has introduced a new large language model (LLM) called Mixtral 8x22B, with a model size of 176 billion parameters, which is the next model after the previous Mixtral 8x7B model.

The operation of Mixtral 8x22B still utilizes the MoE (mixture of experts) approach, where each layer of each token selects 2 suitable models (referred to as experts) for processing, eliminating the need to run all parameters. Currently, Mistral has not disclosed all the details of the model, but it is estimated that the actual models used during runtime have approximately 38 billion parameters.

The entire file size is 262GB and can be downloaded from Mistral’s X or on Hugging Face, where files have been converted for accessibility.

Source: VentureBeat

TLDR: Mistral’s latest AI model, Mixtral 8x22B, boasts 176 billion parameters and uses a MoE approach for efficient processing. The total file size is 262GB and available for download.