I email Andrew Gelman:
Looking at the current state of the social sciences, do you think the culture of research has genuinely changed since the replication crisis became widely discussed, or has it mostly generated new compliance rituals around pre-registration and open data while leaving the underlying incentive structure intact?
What do you think is the most consequential thing about American political behavior that the quantitative literature has established clearly, and what is the most consequential thing that journalists and political commentators continue to assert with confidence that the data do not support?
From your position as someone who thinks carefully about institutional incentives and measurement, what do you think the [Columbia University] administration consistently misread, and is there a statistical or social scientific way of understanding how institutions lose the ability to accurately perceive their own situation?
Andrew Gelman responds:
1. I’m loath to give an answer about the changes in the culture of research because I have not studied this systematically. My impression is that, yes, there’s more skepticism and less acceptance of noisy N=38 papers in psychology, etc., and less toleration for unfalsifiable evolutionary psychology and that sort of thing. On the other hand, perhaps this has just shifted from the science establishment to social media. Ten or fifteen years ago, there was a pipeline (partly abetted by Jeffrey Epstein) from researchers at top universities to publication in top journals to books, NPR, Ted, Gladwell, Freakonomics, etc., and lucrative speaking and consulting gigs. So you get people like Marc Hauser or Albert-Laszlo Barabasi or Brian Wansink or Dan Ariely doing the basic research (such as it is), academic middlemen such as Steven Levitt and Cass Sunstein as promulgators, and the universities, journals, and prestige news media as part of this system (as for example here).
Nowadays, though, social media runs on its own steam, and the models for academic junk science are researchers such as Andrew Huberman and Dr. Oz, who cut out the middleman and promote junk science directly, sell supplements, etc. And social media is full of fake news and AI slop. They don’t really need NPR, Ted, Gladwell, Freakonomics, etc., anymore; they can do it on their own. So, in short, yes, I do have the impression that science has reformed from the bad old days of 2010-2015 (about which, see this article with Simine Vazire), but maybe the public intellectuals don’t need academic science anymore; they can just make up whatever they want on their own.
2. My take on Columbia is similar to my take on many institutions, which is that they have an executive function but minimal legislative or judicial functions; I discussed this here and here. As a result, their decisions are made on consequentialist rather than proceduralist gounds, and over and over again the administration takes the seemingly reasonable decision to cover up misdeeds.
Gelman’s first answer is structurally interesting because he refuses the simple yes or no about whether research culture has reformed and instead maps the ecosystem change. The pipeline he describes, from noisy N=38 studies through academic middlemen to NPR and TED and Gladwell, has not been replaced by better science. It has been disintermediated. The junk science now goes direct to consumer via Huberman and supplements and social media, cutting out the credentialing layer entirely. That is a more pessimistic answer than it first appears. The reform he acknowledges inside academic science has coincided with the collapse of the institutional gatekeeping that made academic science matter to public culture. Science got more honest at the same moment it became less consequential to what people believe.
The Jeffrey Epstein reference is pointed. Gelman is naming something most academics avoid naming directly: that the pipeline from research to prestige media to speaking fees was itself a corruption system, and Epstein’s involvement with several figures in it was not incidental but symptomatic.
His second answer is the more original one. The executive-without-legislative-or-judicial-functions diagnosis explains something that gets described in other terms in my Columbia post. Columbia’s administration did not behave the way it did because of bad values or weak leadership in the ordinary sense. It behaved the way institutions without procedural constraints behave: consequentialist, cover-up-oriented, reactive to the most proximate pressure. The Shipman and Armstrong situations both fit that model exactly. When there is no judicial function to enforce norms and no legislative function to set them through deliberation, the executive defaults to managing outcomes rather than upholding principles.
I follow up with links to my Andrew Gelman profile and other Gelman posts as well as a final question: “Do you think the solution is to build the missing functions inside universities, and if so what would that look like, or is the structural problem too entrenched to address from within?”
Andrew responds:
Hi, thanks for sending. It’s kinda weird seeing myself discussed as a sociological object, but, fair enough, I’m a public figure. Your assessment is accurate that I’m not very good at strategic behavior so often I don’t even try. It’s similar to how I’m a bad negotiator so usually I’ll just try to make my goals clear and not try to optimize. Finally, regarding your question about what should universities do: I don’t know. I keep seeing Columbia do things that seem wrong to me (both morally and strategically wrong), but I also have the horrible feeling that if I were in charge of the institution I’d do no better.
This reply is worth sitting with carefully because it adds something no David Pinsof framework quite predicted.
The most striking thing is what Gelman confirms and what he does not contest. He does not push back on the sociological analysis. He does not say the framework mischaracterizes him. He says it is accurate. That is a significant data point. The person most positioned to reject the analysis as reductive accepts it as fair. This is either genuine self-knowledge or the defensive signal the prior analysis identified: I will not be the person who knew and said nothing, including about himself.
The self-description as bad at strategic behavior and bad at negotiation is the most interesting sentence. It confirms the autistic-adjacent diagnosis the arguing essay generated: someone who brings concrete practical rationality into domains where the intergroup dominance game is being played as the persuasion game. He is not saying this as a complaint. He is saying it as a description of how he operates. He makes his goals clear rather than optimizing around them. This is precisely the behavior Pinsof predicts from someone who keeps trying to have real arguments in pseudoargument domains and cannot fully stop doing it even when he understands the game.
The Columbia passage is the most revealing and the most human. He keeps seeing the institution do things that seem wrong to him both morally and strategically. But he has the horrible feeling that if he were in charge he would do no better. This is the Why Things Go to Shit essay stated personally. Things go to shit unless there is an incentive for them not to. Gelman has spent his career arguing that the problem is institutional incentive structures rather than bad people. His Columbia observation is the most direct available application of his own framework to his own institution: the wrongness is not produced by the specific people making the decisions. It is produced by the institutional incentive structure, which he would be equally subject to if he were in charge.
The word horrible is doing significant work. It is not a casual hedge. It is the precise emotional register of someone who has applied incentive determinism consistently enough to have lost the likability determinist consolation that better people would produce better outcomes. The horrible feeling is what incentive determinism feels like from the inside when you apply it honestly to yourself and your own institution. You cannot simply blame the administrators. You cannot simply vote them out and replace them with people who see more clearly. You would do no better. That is a genuinely uncomfortable place to arrive at and he is describing it accurately.
The prior analysis generated this prediction through multiple frameworks. The misunderstanding essay established that understanding the mechanism does not dissolve it. The Why Things Go to Shit essay established that things go to shit without countervailing incentives regardless of who is in charge. The incentives are everything essay established that behavior is determined by incentive structures and that knowing this does not exempt you from it. Gelman has now confirmed all three predictions simultaneously in three sentences about Columbia.
What the reply adds to the prior analysis is one thing the frameworks could not quite establish from outside: that the analysis is legible to its subject as accurate rather than as reductive, and that accuracy is experienced by the subject not as clarifying or liberating but as producing the horrible feeling that the problem goes all the way down. The oracle who knows he is the oracle does not escape the structural position. Gelman knows this about Columbia. He knows it about himself. The knowledge produces the horrible feeling rather than the relief that understanding is supposed to provide. That is the most honest available confirmation of everything the prior analysis built.
