Nadella: AI alignment will not 'just fall out' of scaling, and he no longer believes it will

Microsoft chairman and CEO Satya Nadella said he no longer believes AI alignment will come from making models more capable. He made the remarks on a new episode of the 20VC podcast, hosted by Harry Stebbings, according to The Next Web.

Asked what he had changed his mind about in the past year, Nadella said: "I used to always think that alignment will just fall out. I no longer believe that. So I think this alignment thing may be a harder, different problem."

Scaling worked, alignment did not follow

Nadella said he believed in scaling laws in 2018 and 2019. At that time, he said, Dario Amodei, Ilya Sutskever, Sam Altman, Greg Brockman and Elon Musk were "one happy family." Scaling, the idea that models improve with more data and computing power, has worked as he expected. Alignment, meaning AI acting in line with human values and goals, has not come with it, he said.

He said alignment mostly has to happen during training and may even need to start in pretraining, the first and largest stage of building a model. He also said the science of how models work internally remains elusive. "I don't think you can even RL your way to alignment," he said. Reinforcement learning (RL) is the method labs use to refine models after pretraining.

Zero trust around the model

Nadella wants companies to build security around models in the meantime. He called for zero trust, an approach that checks every action instead of trusting anything by default.

"If you really want to have agents that are more capable and yet less aligned, then you need to have a trust system that you can rely on," he said.

He said Microsoft's Agent 365 and Perception give every agent an identity and log its tool and model calls. The logs must be tamper-proof so an agent cannot change its own record. Companies then set policies and watch whether agents follow them. Microsoft added tools to monitor agents earlier this month, according to TNW. On 10 October, Nadella wrote on X that companies should treat AI models as insider risks.

Chain of thought and closed models

Nadella said nobody can rely on chain of thought, the step-by-step reasoning a model shows before it answers. "We know that the chain of thought itself can be unfaithful, and in fact it can be made worse when the model company says, 'Well, it's unfaithful and I'm going to hide it,'" he said. He did not name a company.

He wants trusted partners to receive the chain of thought in plain English so they can audit it and check it with other models. If makers of closed models do not see the need for more transparency, "we may have a problem," he said. He suggested that could push more customers toward open-weight models, whose weights anyone can download and run.

Open models and China

Stebbings pointed out that most open-weight models are Chinese. Nadella said AI without safety is bad for China, the US and the UK alike. "An unaligned open-weight Chinese model will probably attack even China," he said.

Microsoft will not release its own MAI models as open-weight, Nadella said. He described them as licensed weights, sold as a straightforward software business. He expects most AI use to run on cheap commodity tokens from many competing models, open and closed.

Microsoft reports quarterly earnings on Wednesday, GeekWire's Todd Bishop reported. They are its first results since it offered buyouts to about 60,000 staff.

What this means

Nadella's statement separates two things the industry often treats as linked: capability and alignment. By his account, scaling delivered capability as expected, while alignment remains an open problem that may need work from pretraining onward.

His practical conclusion is architectural. If alignment cannot be assumed, control has to sit outside the model: agent identity, tamper-proof logs of tool and model calls, and enforced policies. That is the design Nadella describes for Microsoft's Agent 365 and Perception.

His chain-of-thought comments set a condition for trust. In his view, reasoning traces that are unfaithful or hidden cannot support auditing, and he wants trusted partners to receive them in plain English. He framed closed-model transparency as the point at which customers might look to open-weight alternatives.

These are Nadella's views as stated on the podcast, not findings from a published study.