Aydin Mohseni

I’m a mathematical philosopher, metascientist, and AI safety researcher. I’m an Assistant Professor of Philosophy at Carnegie Mellon University, and a core member of its Center for Conceptual Research on Safe AI and Institute for Complex Social Dynamics.

Research

I pursue three research programs. The first is on the foundations of AI safety. How should an artificial agent represent the world, value outcomes, and act? With Alex London, I extend decision theory so that an agent can refine its values as well as its beliefs, and prove a value-of-information theorem for learning about values. With Daniel Herrmann, Ben Levinstein, and Bruce Rushing, I show that causal reasoning reduces to subjective probability once interventions are treated as events in the world. I also study when we should delegate to AI systems whose aims may diverge from ours, and how to allocate compute when AI agents do safety research themselves. With Daniel, I co-direct ExProof AI.

The second is metascience: how scientific communities produce knowledge, and how to do it better. What a community learns depends on how individual inquiry is aggregated, connected, incentivized, and funded, and these properties often cannot be read off individual reasoning. A reform can improve every study and still worsen the published literature. I study grant funding, replication and reform, and how networks of scientists learn from one another.

The third is on method. I use Bayesian epistemology, decision theory, and evolutionary game theory, and I study when these tools mislead: when different models of evolution disagree about what evolves, and why, and when equilibrium concepts miss the outcomes that matter.

Teaching and mentoring

I teach decision theory, game theory, AI safety, logic, and philosophy of science at Carnegie Mellon; the full list is on the Teaching page. I mentor AI safety researchers through SPAR and advise master’s students.

Public writing

I write philosophy posts on LessWrong, among them Chance is in the Map, not the Territory and Evolution and the Low Road to Nash.

Previously

I did my PhD in Logic and Philosophy of Science at UC Irvine, with Cailin O’Connor and Simon Huttegger. My full history is on my CV.

Some years ago, I was a Peace Corps volunteer in Morocco, stationed near أزرو.

I have an Erdős number of 3, along the following path:

Aydin Mohseni → Brian Skyrms → Persi Diaconis → Paul Erdős

Contact

amohseni@cmu.edu. See also Working with me.