Events, programs, and opportunities from MAIA. Join mailing list
The Path to an AGI World
This week, we explore what the path to superintelligent AI systems could look like and what kind of world could lie on the other side. We begin with two foundational arguments: the orthogonality thesis and instrumental convergence. Together, they suggest that highly capable AIs need not share our goals, and that almost any goal gives it reasons to seek out resources and resist being shut down. Next, we examine the idea of an intelligence explosion, where AI R&D feedback loops at multiple layers (algorithms and data, chip quality, and chip manufacturing capacity) compound and drive progress extremely quickly. We then consider contrasting pictures of where this could lead. In “Machines of Loving Grace,” Dario Amodei sketches the upside: a world where AI compresses decades of progress in biology, medicine, and economic development, curing diseases and lifting billions out of poverty. We also discuss the possibility of gradual disempowerment where humans lose control not through sudden AI outmaneuvering or takeover but as human institutions gradually come to rely on AIs for economic labor, decision making, and even social needs. Finally, we consider how advanced AIs could lead to concentration of power in the hands of a few actors, amplifying inequality to an unprecedented degree.