Microsoft has released a small language model (SLM) called Phi-2, which has a whopping 2.7 billion parameters for customers to utilize on Azure AI Studio. Phi-2 is a product of the Machine Learning Foundations team under Microsoft Research. It is a Transformer model trained on 1.4 trillion tokens using 96 NVIDIA A100 GPUs, taking a total of 14 days to train. What sets Phi-2 apart is that it is a model developed solely by Microsoft, not relying on OpenAI.
The unveiling of Phi-2 took place at the Ignite event last month. Its main highlight is its size, as it is a 2.7 billion parameter model that performs comparably or even better than models larger than it by a factor of 25.
Microsoft tested Phi-2 against various popular benchmarks in the AI world, including 7 categories of Big Bench Hard (BBH) tasks such as common sense reasoning, language understanding, math, and coding. The results showed that Phi-2 was able to outperform larger models like Mistral-7B, Llama-2-13B, and even Llama-2-70B. Furthermore, Phi-2 even surpassed smaller models such as Google Gemini Nano 2, which has a size of 3.2 billion parameters.
In conclusion, Microsoft’s Phi-2 is a remarkable SLM with its immense number of parameters, outperforming both larger and smaller models in various AI benchmarks.
TLDR: Microsoft has introduced Phi-2, a small language model with 2.7 billion parameters. Developed internally by Microsoft, Phi-2 surpasses larger models and even outperforms smaller models, making it a significant breakthrough in the AI field.
Leave a Comment