My dear friend Luuk is about to publish a small essay: AI, Human Stupidity, and Mathematical Progress. We talked about it a lot, and it’s been on my mind a lot.
Our search for understanding is one of the strongest motivators we have as a species. And I think it’s a moral endeavor. Not necessarily for spiritual reasons, but because of the stuff that falls out of it (calculus, electricity, the bomb, mRNA). It ends up having the biggest impact on all of us. For good and for bad.
Mathematics and theoretical physics have had a luxury here. The feedback loop between a theorem and its consequences was decades long. So you could love algebraic topology at 17, become an academic, spend a life on beautiful puzzles and never really feel responsible for the power you helped create. The distance did that for you.
But AI shrinks the distance.
The loop is now months to years. Nobody knows exactly, but very likely shorter than decades. And the people building it don’t seem to know if it’s good for us. Check Jacob Coxon’s resignation: alignment isn’t solved, the race continues anyway. (Small caveat: the account has nothing else on it, and Anthropic is heading for an IPO in a few weeks, so I can’t rule out this being a play. Who knows.) But whether that resignation is real or not doesn’t change a lot. You can see the rate of change by just being on the internet. What you can observe yourself already justifies thinking about this.
So if you’re a researcher: do you get to distance yourself? Say you won’t use it? Wait and see? I think that’s actually the selfish option. It’s coming anyway. You are one of the few people who understand the field well enough to judge what we should do with it. Either stop doing research, or take part in the public debate. Anything in between leaves the debate (and arguably mathematics itself) to the frontier labs.
Technology is moving from a supporting role to a very powerful accelerant. That makes governance a responsibility, not an option.
So my note to self: a shorter feedback loop to impact obliges you morally. Opting out of that responsibility seems to be a luxury of the past.