It’s on Wednesday Microsoft Ignite conference, Microsoft declare two custom chips designed for accelerating AI tasks at home through your Azure cloud computing service: Microsoft Azure Maia 100 AI Accelerator and Microsoft Azure Cobalt 100 CPU.
Microsoft designed Maia specifically to run large language models such as GPT 3.5 Turbo and GPT-4, which support its Azure OpenAI services and Microsoft Copilot (formerly Bing Chat). Maia has 105 billion transistors that are manufactured on it 5 nm TSMC process. Meanwhile, the Cobalt is a 128-core ARM-based CPU designed to perform traditional computing tasks like the power of Microsoft Teams. Microsoft has no plans to sell either one, preferring them for internal use only.
As we’ve seen before, Microsoft wants to be the “Copilot company,” and it will need a lot of computing power to meet that goal. According to ReutersMicrosoft and other tech companies have struggled with the high cost of delivering AI services that can cost 10 times more than services like search engines.
Amid chip shortages that have driven up prices of high-end Nvidia AI GPUs, many companies are designing or considering designing their own AI accelerator chips, including AmazonOpenAI, IBM, and AMD. Microsoft also feels the need to make custom silicon to push its own services forward.
“Much like building a home lets you control every design choice and detail, Microsoft sees the addition of home chips as a way to ensure that everything is optimized for Microsoft’s cloud and AI operations,” the company wrote in your announcement blog. Then he adds poetry as a cookie cutter, “The chips will lean on custom server boards, placed between custom-made racks that fit easily inside existing Microsoft data centers. The application will work hand in hand with the software-design together to unlock new capabilities and opportunities.”
This isn’t Microsoft’s first foray into chip development. According to The VergeThe firm has long collaborated on silicon for its Xbox consoles and co-engineered chips for its line of surface tablets.
No technology company is an island, and Microsoft is no exception. The company plans to continue to rely on third-party chips both in the supply chain and possibly to satisfy its web of business partnerships. “Microsoft will also add the new Nvidia H200 Tensor Core GPU to its fleet next year to support large (sic) model inference without an increase in latency,” he wrote, referring to Nvidia’s announced AI-crunching GPU deficiency. . And it will continue to increase AMD MI300X– Accelerate virtual machines to Azure.
So, how well do the new chips do? Microsoft hasn’t released benchmarks yet, but the company seems happy with the chips’ performance-per-watt ratios, especially for Cobalt. “The architecture and implementation were designed with efficiency in mind,” Microsoft corporate VP of application product development Wes McCullough said in a statement. “We are making the most efficient use of transistors on silicon. Multiply the efficiency gains in servers across all our datacenters, it adds up to a great number of goodies.”