Meta-Expertise and the Accountability Collapse: Columbia, the IMF, and 9/11

Stephen Turner’s two case studies, “Expertise and Political Responsibility: The Columbia Shuttle Catastrophe” in The Politics of Expertise (Routledge 2014) and “Expertise and Complex Organizations” in The Oxford Handbook of Expertise and Democratic Politics edited by Gil Eyal and Thomas Medvetz (Oxford 2023), give the unifying frame for thinking about how expert organizations fail. The frame turns on a distinction Turner develops in the Columbia chapter: primary expertise and meta-expertise. Primary expertise is the engineer who knows what foam does to tile under specific stress conditions, the country desk economist who knows the loan terms, the case officer who knows the network. Meta-expertise is the judgment about whose primary expertise applies, what its limits are, what weight to give it, and how to aggregate it with other primary expertise into a decision. The Columbia disaster, the IMF failures of 2008 and 2010, and the 9/11 attacks were all meta-expertise failures, not primary expertise failures. The primary experts in each case were largely correct within their domains. The error was in the aggregation.
Turner’s deeper argument is that meta-expertise has no expert. There cannot be one. Meta-expertise about a domain would require near-omniscience in that domain, which would collapse the distinction between primary and meta-expertise. The closer the meta-expert approaches the limiting case of being able to evaluate primary expert advice, the less need there is for the advice. The manager who can fully assess the engineer’s claim does not need the engineer. This is the paradox of managerial omniscience, and it forecloses the reformist exit. Better-trained managers, more technical literacy at the top, improved deliberative procedures: none of these reach the source. The trap is not solved by moving up the ladder of competence. It is reproduced at every level. Habermas’s ideal-speech situation, which assumes that prolonged discussion can produce shared understanding, cannot be achieved in expert deliberation because the asymmetries of knowledge are ineliminable. Discussion will not produce the same tacit knowledge in all participants. The discourse must rely on trust in expert claims and trust in meta-expert self-discipline. Perfect meta-expertise cannot be obtained.
The Columbia case shows the structure of the failure. NASA’s mission management team, led by Linda Ham, faced a decision about whether to authorize satellite imagery of the orbiter after foam shedding had been observed at launch. The team relied on the assessment of Dan McCormack, a senior structural engineer, who reported that the Boeing tile analysis showed no serious threat. Another engineer agreed. Alan Rodney Rocha, a Houston-based engineer who had expressed concern, backed down after this analysis. The primary expertise on tiles was correct as far as it went. The damage that destroyed the orbiter was not to tiles. It was to the reinforced carbon-carbon leading edge of the wing. The error was in the reach of the tile assessment, not in the assessment itself. McCormack’s primary expertise had been allowed to bear meta-expert weight it could not carry. Ham, who lacked the primary expertise to evaluate his judgment, relied on the consensus he produced. She could not have done otherwise. To independently evaluate his judgment would have required her to be his peer in primary expertise, which would have rendered her own role redundant. The trap was structural.
NASA had built a system that compounded the trap. The agency required data-based claims. It also denied the resource requests that would have generated the relevant data. Engineers with concerns could not get the studies funded that would have backed the concerns with data, and then their concerns were dismissed as not data-driven. This is more than groupthink. It is a structural double bind that punishes the engineers whose tacit knowledge would have flagged the risk. Bob Daugherty’s emails predicting the catastrophic scenario were dismissed as “just engineers talking” rather than formal warnings. The system gave engineers two registers of speech, the speculative offstage register and the formal accountable register, and made the cost of moving from one to the other prohibitive. Engineers who moved to the formal register became responsible for the consequences of their concerns. Engineers who stayed in the offstage register could not move the organization. Neither register was useful in the situation that arose.
Turner’s reading of Diane Vaughan’s Challenger work sharpens the point. Vaughan called the pattern “culture.” Turner argues that what Vaughan called culture was engineers using heuristics on incomplete data, the only way complex novel technology can be developed. The race car analogy is apt. You strain machinery past known limits, parts break, the failures generate data, you redesign. Calling this culture makes it sound pathological. Calling it heuristic-based reasoning under uncertainty makes it sound necessary. The reformist push to change the culture becomes a category error. You cannot reform away the heuristics without ceasing to do engineering. The CAIB report blamed culture and recommended cultural change. NASA’s response was instructive. O’Keefe refused to fire anyone. Ham was reassigned, not removed. Other staff retired. The reasons for these personnel actions were never publicly stated. As one CAIB member put it, “Do you want their heads on a fence someplace?” The political demand for accountability dissolved into ritual reassignment because the actual structure of meta-expert decision-making does not support assigning personal responsibility. Turner’s line: the outcomes are the products of consensus for which no one is formally responsible.
The IMF case extends the analysis to a body explicitly designed to maximize what Roger Koppl calls synecological redundancy, genuinely diverse evidence channels with different structural elements. The IMF was governed by a board of donor nations, each with its own central bank expertise, its own diplomatic intelligence, its own staff. The design assumed that stakeholders with skin in the game would bring perspectives diverse enough to cancel the staff’s biases. The 2011 Independent Evaluation Office report on the run-up to the 2008 financial crisis blamed groupthink, intellectual capture, and a general mindset that a major financial crisis in large advanced economies was unlikely. The 2016 report on the Greek debt crisis catalogued the same failures. Turner reads the reports against themselves. Intellectual capture is treated as a bug. He argues it is a feature. Expertise is representational. An expert who speaks against the consensus of other experts loses the standing that makes him an expert. The IMF staff had to be intelligible to the central bankers and finance ministers they negotiated with, had to share models, had to speak the same language. Capture is the condition of functioning, not a deviation from it. Groupthink works the same way. What saves an expert from paying a price for error is the fact that others made the same mistake. The expert who breaks ranks is exposed to personal accountability. The expert who stays with the group is protected. Groupthink also functions as a shield against external power: when donor governments could not identify specific staff members responsible for analyses they disliked, they could not demand removals. The diffuseness of responsibility was protective.
The Greek case shows the synecological redundancy collapsing. The IMF staff understood that an upfront debt restructuring was the right answer. The Euro partners, especially the Germans, blocked it. The reason was structural. A restructuring would have set a precedent applicable to Italy and Spain, would have forced losses on French and German banks, would have exposed the political class to a cost it would not pay. The stakeholders who were supposed to provide diverse correction shared a bias the structure could not detect because the structure had been built on the assumption that their interests would diverge. They did not diverge enough. The IMF supplied econometric models that, in Turner’s words, were “notoriously, both false and tailored to support the political and financial agenda of the German government.” The result was the extend-and-pretend strategy. Greek suffering bought time for German and French banks. The IMF’s stated goal of stabilizing Greece for healthy future growth was not achieved. The structure that was supposed to correct for bias produced a new bias the structure could not see.
The 9/11 case follows the same template. Michael Scheuer ran the bin Laden unit at the CIA. He produced warnings. The bureaucracy did not act on them. The convenient belief was that bin Laden was one terrorist among several, that the threat could be managed by interagency process, that an aggressive forward operation was not worth the political cost. The primary expertise on the network sat with Scheuer and a small group around him. The meta-expert judgment about how much weight to give that expertise sat elsewhere, with people who could not evaluate it independently and who were under political pressure that shaped their meta-expert dispositions. The CIA’s structural position vis-à-vis the executive branch and Congress was analogous to the IMF’s position vis-à-vis its donors. Hard-hitting analysis on a politically costly target produced friction. Self-censorship followed. After the attack, the 9/11 Commission could not name the persons responsible for the failure because the failure was not located in persons. It was distributed across an aggregation structure. The Commission’s reforms restructured the intelligence community on the assumption that the right organizational chart would produce the right knowledge. The assumption was wrong for the same reason it was wrong at NASA and at the IMF. The right organizational chart cannot produce the right knowledge because no chart can solve the paradox of managerial omniscience.
The pattern across the three cases is not the temporal asymmetry of warning and reaction that I had emphasized in earlier drafts. Turner’s frame is sharper. Features that work under normal circumstances fail under unanticipated ones, and the failure exposes what the features were protecting. NASA’s mission management team worked when foam strikes followed the pattern of past strikes. It failed when the strike was different. The IMF’s stakeholder governance worked when donor interests diverged. It failed when they converged on a bad answer. The CIA’s interagency process worked when threats fit the categories the process was built to handle. It failed when the threat did not fit. The reform that follows each failure addresses the surface feature, not the structural condition. New oversight rituals at NASA. Revised conditionality formulas at the IMF. The Commission’s restructuring at the CIA. Each reform assumes that the right structure might produce the right knowledge. Each misses the source.
The accountability collapse is the deepest point. In all three cases, the public demand for personal responsibility could not be satisfied. Senator Hollings demanded that someone at NASA be cashiered. No one was. The IEO reports on the IMF refused to name names or identify the countries that had interfered with the experts. The 9/11 Commission distributed responsibility across agencies in a way that prevented any agency from absorbing it. This is not a failure of will. It is the structural consequence of the way expert organizations make decisions. They aggregate primary expertise through consensus mechanisms that produce meta-expert climates of opinion. The climate is no one’s responsibility. The consensus is no one’s responsibility. The aggregation is no one’s responsibility. The decision-makers can plausibly say they relied on the best advice available. The expert advisers can plausibly say they were just speaking in their domain. The reviewers can plausibly say they followed procedure. Each link in the chain has a defense. The chain as a whole has a failure. There is no person at whom the failure can be aimed.
Turner’s closing move in the IMF chapter forecloses the reformist exit entirely. Social epistemologists sometimes talk about well-ordered epistemic systems. The notion assumes a standard against which the order of a system can be judged. To know whether a present system is well-ordered would require knowing what could have been achieved with alternative designs. We do not have that knowledge. We have only what has been achieved with present and past systems, judged against past standards of success that are themselves products of the systems doing the judging. The “well” in well-ordered is an expression of satisfaction with a current outcome, not a standard for evaluating it. It is an expression of bias, not a tool for detecting bias.
The implication for the analyst is sober. The conditions that produced the Columbia failure, the IMF failure, and the 9/11 failure are not pathologies of those organizations. They are the conditions under which complex expert organizations operate at all. Heuristics under uncertainty. Consensus aggregation of primary expertise. Meta-expert climate-formation. The paradox of managerial omniscience. The asymmetries that defeat ideal-speech deliberation. The double bind of data-based discourse without data-collection authority. The protective function of diffused responsibility. None of these can be reformed away without ceasing to do the work. They can be reorganized. They cannot be removed. Every reorganization produces a new pattern of failure.
The pre-event signals are visible if you know where to look. The engineer at NASA who wanted the inspection. The country desk economist at the IMF who knew the model assumptions were wrong. The case officer at the CIA who understood the network. They had the primary expertise. They lacked the standing to bear meta-expert weight. The standing they lacked is structural, not personal. They could not have acquired it without becoming the kind of person whose primary expertise would have been compromised by the acquisition. This is the trap. Turner’s contribution is to make the trap visible without offering the false comfort of an exit. The post-event accounting will tell a different story. It will name a few individuals, redesign a process, announce that the lesson has been learned. The lesson never has been. The next case will follow the same template.