OpenAI has disbanded its AGI Readiness team, which advised the company on managing increasingly powerful AI systems and evaluating the world’s readiness to govern these technologies.
Reasons Behind the Departure
Miles Brundage, Senior Advisor for AGI Readiness, announced his departure via Substack. He attributed his decision to several factors, including hidden costs within the role, greater research efficiency outside of OpenAI, a desire to avoid bias, and his belief that he had fulfilled his goals at the company.
AGI, or artificial general intelligence, refers to a hypothetical system comparable to or exceeding human intelligence. Some experts believe such systems are on the horizon, while others doubt their feasibility. According to Brundage, neither OpenAI nor other advanced organizations—nor the world at large—are ready to handle AGI responsibly. He revealed plans to either establish a new nonprofit or join an existing one, focusing on AI research and policy advocacy. Brundage’s former subordinates from the AGI Readiness team will be reassigned to other departments within OpenAI.
Superalignment and Leadership Shifts
Last year, OpenAI launched a Superalignment department dedicated to research on managing AI systems that could potentially exceed human intelligence. This initiative aimed to prevent such systems from becoming uncontrollable. OpenAI committed 20% of its computing power over four years to support this project. However, the department ceased operations just a year later, in May, reminds NIX Solutions. Following its closure, OpenAI co-founders Ilya Sutskever and Jan Leike, who were leading the Superalignment efforts, announced their departure from the company.
Leike commented on the challenges of pursuing AGI, stating, “Building machines that are smarter than humans is an inherently dangerous endeavor. OpenAI takes on a huge responsibility on behalf of all humanity. But in recent years, safety culture and processes have taken a backseat to brilliant products.”
In September, further changes occurred when several other senior leaders—CTO Mira Murati, head of research Bob McGrew, and vice president of research Barret Zoph—also announced their departures from OpenAI on the same day.
We’ll keep you updated as OpenAI continues to evolve its approach to AI development and governance.