Microsoft's opportunity to become the leading AI platform for business has made it the world's most valuable public company, with Wall Street anticipating a coming productivity revolution.
Inside the software giant, the outlook is hopeful, but the reality is less glamorous and the path to success less clear.
That's according to interviews with several current top Microsoft executives, former executives, and several rank-and-file employees, who requested anonymity so they could share candid views on the future of the company's AI and its relationship with the $80 billion startup OpenAI. dollar.
There is real concern about whether Microsoft's new AI services will create enough value for enterprise customers who are paying more for this emerging technology. Internal AI projects have been halted or questioned, teams have been redeployed, while the OpenAI partnership is sucking up resources and generating discontent among some employees.
However, there is also a sense of optimism about Microsoft's ability to steadily improve its AI offerings and build on the trust it has built with thousands of companies that have used its software for decades. Generative AI may be at the forefront of startups, but Microsoft could win precisely because it is an old and somewhat boring company, not a fickle new startup.
“Customers trust Microsoft more than OpenAI because they have already bought into the Microsoft ecosystem,” one Microsoft AI researcher told BI. “It's more natural for organizations to use Microsoft rather than the offerings from OpenAI. All of these features aren't as groundbreaking as ChatGPT when they were first introduced, but they offer a lot of value for people in the productivity space.”
Microsoft opportunity
Consumer-facing chatbots are glamorous and fun, but the real money will be made if companies not only embrace great language models and generative AI, but find them useful for improving operations, sales, and other operations.
The first part is starting to happen, according to a recent Morgan Stanley survey of chief information officers, the executives who decide how billions of dollars are spent on technology each year. The bank found that AI and machine learning were top priorities for IT managers in the fourth quarter of 2023. More than two-thirds of these executives said AI had already impacted their IT budgets.
With such a huge opportunity on the horizon, CEO Satya Nadella has rallied practically the entire company around developing AI tools and services based on OpenAI's GPT models. This is the foundation of Microsoft's Copilot suite of offerings, which provides automated support for employees working on text documents, spreadsheets, cybersecurity issues, and a host of other tasks.
“Almost everyone I know is working on Copilot to some degree,” the Microsoft AI researcher told BI.
The pressure on nearly every team to prioritize generative AI in their applications has created some discontent among rank-and-file employees who say they've had to do more with less after major layoffs and salary freezes.
“One out of 10 times is a charm.”
Microsoft has also recently rolled out Copilot products internally, and employees told BI that they had mixed early results when it came to the reliability of the output. However, they expect this to improve over time.
Current and former executives said there was a race to add value to unreliable and somewhat “tricky” AI tools before the excitement faded and customers wondered whether they were really getting the return they expected from the much-hyped technology.
“About one in 10 times is a charm,” one Microsoft executive told BI. “The rest of the time the question is: Why even try?”
Microsoft spokesman Frank Shaw said the company is optimistic about its AI tools, and customers are seeing a positive impact even at this early stage. Customers get more final version of products than Microsoft employees, and the software giant is implementing feedback from all sources to improve the tools. “These things are quickly moving away from being partisan tricks,” he said.
“We're the ones shipping this,” Xu added, meaning Microsoft is the one bringing AI tools to market in a meaningful way.
Improve your AI value proposition
At the same time, there is growing pressure within OpenAI to become an actual company, rather than an AI research group. This led the startup to enter the world of Microsoft. In August, OpenAI launched its more powerful GPT Enterprise service for enterprise use.
“OpenAI’s gain is not Microsoft’s loss yet,” another Microsoft executive said. “It may happen in the future.”
Microsoft's answer was to create Copilots, which can be thought of as ChatGPT-like native experiences for existing enterprise customers. The company bundles these with its existing software, while offering customers access to many different large language models.
These generative AI tools and services are being sold as part of Microsoft's “digital transformation” offering, a long-term effort to persuade business customers to adopt the latest technologies, such as moving on-premises workloads to the cloud. Companies like Walmart and JPMorgan Chase have tapped Microsoft to build generative AI into their operations and products.
From FOMO to ROI
We are still in the FOMO phase of enterprise AI adoption. Companies are more interested in trying out this new technology, and less concerned about its cost. But Microsoft insiders know the conversation will soon turn to topics like return on investment.
There is a significant cost associated with this technology, according to one executive who spoke to BI, and the value is uncertain in the early stages of adoption of these tools.
“The ROI of the experiences they're building into their products is unclear, and it's a non-trivial cost,” this executive said, adding that there's an internal fear that enterprise customers will decide what they're getting isn't worth the money.
Value, not price
For now, it appears that companies offering this technology are not trying to compete strictly on price. Microsoft Copilots are mostly available as add-ons to existing products for $30 per user per month. Google charges roughly the same for similar AI tools.
“It is too early to assume that this will be a race to the bottom on prices,” another Microsoft executive said. “The problem Microsoft sees is that we just need to convince people that there's value in this. Companies aren't sure if this is too expensive. We're not looking at it like we need to make it cheaper, just add more value.”
The executive said customer feedback about Microsoft's AI tools has been “mixed” so far. “But it's still very early. The value proposition hasn't kicked in yet.”
“Gimmicky”, but it saves time
A C-level executive from a large Microsoft customer told BI that the company's Copilot tool for its Microsoft 365 suite of business applications is a “gimmicky” tool.
That person said the tools are limited in what they can do, and while they are useful, they are “not yet amazing” or indispensable.
However, upgrading Copilot to Microsoft 365 saves time by summarizing meetings and long email threads with a reasonable degree of reliability, the person said.
Dentsu's early co-pilot views
Ad agency Dentsu was in the Microsoft 365 Copilot early access program and plans to add up to 4,000 seats over the next month, according to Microsoft.
In an interview arranged by Microsoft, Shiva Vanavada, global chief product and technology officer at Dentsu, said employees reported saving 30 to 40 minutes per day using Microsoft 365 Copilot for tasks like summarizing team conversations and creating presentations and executive summaries.
Dentsu conducts monthly surveys to gauge sentiment, according to Vanavada, and 80% of employees had a very positive view of Copilot.
The remaining 20% have a mix of complaints, he said. People on the creative team don't like the Copilot functionality in PowerPoint because they want their presentations to look good and the AI versions don't meet that higher standard. Dentsu has training to help employees use Copilot to create more advanced deliverables and applications, Vanavada said.
Questions about internal GPU voltage
Microsoft is trying to make generative AI faster, better, and cheaper. But there are fundamental technological limitations that prevent the models from working accurately, said the people who spoke to BI.
These are also power-hungry workloads that rely heavily on Microsoft purchasing GPUs from Nvidia. Analysts at DA Davidson estimate that Microsoft spent $4.5 billion on Nvidia chips last year. A Microsoft executive told BI that this was within the company's actual spending. Shaw declined to comment on this.
Microsoft is making an internal effort to design its own AI chips to reduce reliance on Nvidia, but some employees see the effort as a waste of resources.
“Frankly, these efforts are a few years behind Nvidia, and all of these efforts require heavy lifting,” a Microsoft executive said. “You're playing in this rat race where the state of the art has evolved by the time you get there.”
Microsoft becomes OpenAI Technical Support
The group at the heart of Microsoft's enterprise AI dream is the “AI Platform” team run by Eric Boyd. This falls under Scott Guthrie's Cloud + AI organization.
The AI Platform team has gone through several iterations. What has historically been an organization with many internal AI plays has leaned heavily into the OpenAI partnership. This created some discontent and led to the departure of some executives who worked on Microsoft's local AI initiatives.
Insiders say Microsoft is focusing less on the internal services that previously made up Azure AI Services and more on the Azure OpenAI service.
Products like AI Cognitive Search, Azure AI Bot Service, and Kinect DK have practically disappeared, said one former executive who left as a result of the changes. These services exist in some form, but are either not part of the Azure AI org, have been renamed, or are bundled with other products, Shaw said.
“The previous Azure AI is basically just technical support for OpenAI,” a former Microsoft executive said. “Eric Boyd effectively keeps OpenAI a service. It's less of an innovation engine than it used to be. Now it's more IT for OpenAI. The beating heart of innovation is elsewhere.”
“It's not very innovative but it's a good business strategy,” this person added.
Azure OpenAI has hundreds of developers supporting Microsoft Azure cloud service customers that use OpenAI's GPT models. Some Microsoft employees work so closely with OpenAI that they have badges to enter OpenAI offices, and some OpenAI badge employees can enter Microsoft sites.
Microsoft isn't limited to OpenAI
The AI Platform team recently saw a major change in leadership after John Montgomery, the executive who previously ran the product, was replaced by former Instacart COO Asha Sharma.
The team offers what's called a “model as a service” in the industry, where companies like Microsoft provide access to AI models via APIs.
Along with its efforts with OpenAI, Microsoft is expanding the number of models it makes available to enterprise customers. The software giant already offers models of Cohere and Meta. It recently invested $16 million in French startup Mistral AI.
The Mistral models will be offered to Microsoft customers alongside about 1,600 other models including Cohere and Meta's Llama.
“Microsoft is a Big 10 company and will not limit its SaaS and cloud services to just OpenAI models,” a Microsoft executive said. “Making sure you have the widest variety of models in addition to that infrastructure is just good business.”
Mistral's investment may look like Microsoft is strategically moving away from OpenAI. This is useful for keeping antitrust regulators at bay, but this person said the size of the investment should clarify that perception.
“$16 million versus $13 billion,” this person said, referring to Microsoft’s investments in Mistral and OpenAI, respectively. “There is a desire for people to want drama but the math speaks for itself.”
Are you a Microsoft employee or someone else with a vision to share?
Contact Ashley Stewart via email (stewart@businessinsider.com), or send a secure message from a disabled device via Signal (+1-425-344-8242).
