The AI Two-Step
Doing the Socio-technical Dance
I sat down to write a short essay on the implications of AI for safety and reliability and I found no shortage of current research on potential failure modes (e.g., Weidinger et al., 2023; Gyevnar and Kasirzadeh, 2025).
But I struggled to find a fair perspective from which to judge the safety and reliability implications of AI. After all, AI is as likely to improve safety and reliability as it is to create new risks.
Then Todd M. LaPorte, from George Mason University, shared an essay by Henry Farrell and Cosma Rohilla Shalizi on "AI as a Social Technology". These authors argue that instead of jumping to the conclusion that a rogue AI will soon take over the world—a perspective they trace back to science fiction writing—it is more productive to start by thinking through how AI will interact with and reshape existing social institutions (they focus on bureaucracy).
Their essay led me to reframe my question about AI’s implications for safety and reliability. Instead of cataloging AI’s potential failure modes, I asked myself whether AI will be used to extend and enhance human capabilities or replace them?
People have been thinking a lot about this question, but primarily from the perspective of whether AI will create or destroy jobs. But it also has implications for safety, reliability and risk management.
In a well-known essay in the field of organization studies, Paul Adler and Bryan Borys (1996) distinguish between coercive and enabling bureaucracies. Coercive bureaucracies structure technology in ways that limit the reliance on employee inputs, while enabling bureaucracies seek to enhance the usability of technology.
To name just one implication of this distinction, the logic of repair is quite different. In a coercive bureaucracy, the inner workings of the technology are not easily accessible and repair becomes a task for specially-trained workers; in an enabling bureaucracy, technologies are more transparent and open to immediate intervention by employees.
The point is that if the social organizations that create and are created by AI are different, they are likely to produce different safety, reliability and risk profiles.
A “socio-technical” perspective on complex and emerging technologies is nothing new, but it seems oddly absent in discussions about AI. Perhaps the very framing of AI as an “autonomous” technology produces this neglect?
Taking a socio-technical perspective seriously requires doing what I call the “AI two-step.” If the first step is to focus on the failures of the technology itself, the second step is to ask how AI interacts with the social organization that surrounds it.
Carl Macrae (2022, 2024) is a master of the AI two-step and has developed what he calls the “SOTEC” framework for analyzing “autonomous and intelligent systems.” SOTEC unpacks the safety, reliability and risk implications of AI along five dimensions: structural, organizational, cultural, epistemic and technological.
Analyzing the fatal crash of an Uber self-driving vehicle, Macrae notes that the failure was not simply technological, but also included “weaknesses in supervisory systems, gaps in safety expertise and leadership, poor human–machine interfaces, and the absence of formalized safety management systems or regulatory requirements” (Macrae, 2022, 2006).
His socio-technical perspective leads him to stress the “the importance of expanding risk regulation and governance strategies to actively build ‘positive’ capabilities for resilience and safety as well as mitigate ‘negative’ sources of risk” (Macrae, 2024, 925).
Asking whether AI systems extend and enhance human capabilities or merely replace them with strictly autonomous machines is one way to inquire into how they will affect safety, reliability and risk management.
Join the dance floor and do the AI two-step!
Chris Ansell, Director of the Center for Catastrophic Risk Management
References
Adler, P. S., & Borys, B. (1996). Two types of bureaucracy: Enabling and coercive. Administrative Science Quarterly, 61-89.
Ghasemaghaei, M. (2026). When talk and walk diverge: how organizational-AI misalignment erodes integrity-based trust and reliance on AI. Journal of Management Information Systems, 43(1), 38-66.
Gyevnar, B., & Kasirzadeh, A. (2025). AI safety for everyone. Nature Machine Intelligence, 7(4), 531-542.
Macrae, C. (2024). Managing risk and resilience in autonomous and intelligent systems: Exploring safety in the development, deployment, and use of artificial intelligence in healthcare. Risk Analysis. https://doi.org/10.1111/risa.14273
Macrae, C. (2022). Learning from the failure of autonomous and intelli-gent systems: Accidents, safety and sociotechnical sources of risk. RiskAnalysis, 42(9), 1999–2025.
Weidinger, L., Rauh, M., Marchal, N., Manzini, A., Hendricks, L. A., Mateos-Garcia, J., ... & Isaac, W. (2023). Sociotechnical safety evaluation of generative ai systems. arXiv preprint arXiv:2310.11986.