Luke Ford

Having recounted Philip N. Cohen’s audits of the American Sociological Review and found them overstated, the obvious next question is whether he meets the standard he applies to other people.

He does.

The frame is his curriculum vitae. Screening it for peer-reviewed empirical articles with quantitative analysis published between 2015 and 2025 gives twenty papers. That window covers the years in which open science became half his public program.

The rule is the one frozen for the ASR recount. Count an article when publicly accessible code appears, without executing it, to cover the article’s quantitative results, and the data are either included or obtainable by outside researchers through a documented procedure that does not run through the authors. Restricted data do not disqualify; discretionary data do. Partial coverage does not count. No code was run.

The first thing the census turned up is a limit rather than a finding. For nineteen of the twenty articles I could not establish what the journal required at the date of publication.

Sociological Science is the exception, and it runs the safe direction: its mandatory package policy applies to manuscripts submitted after April 1, 2023, and Cohen’s article there appeared in January 2016. Everywhere else the trail runs into publisher framework documents rather than dated journal rules. Wiley announced its tiered framework in September 2017 and left the tier to each journal. Elsevier encourages rather than mandates. Springer Nature requires a statement and introduces no sharing mandate. Sage operates tiers and distinguishes journals that require public sharing from journals that encourage it, without saying which titles sat where in which years. Current submission guidelines cannot be read backward onto articles published a decade earlier.

So no article in this set is known to have appeared under a deposit mandate, and none is known not to have. That is worth stating plainly, because a reader will otherwise assume one or the other. Seven of the twenty carry a data-availability statement. A statement is not a deposit.

Thirteen of the twenty articles have a package. Sixty-five percent.

Under the same rule, the 2025 volume of ASR comes in at forty-four percent. Cohen’s own record beats the journal he has been auditing.

On sole-authored papers he is seven for seven. Every one. He is also one for one as first author of a multi-author paper. The gaps sit further down the author list: four of nine as second author, one of two as third, none of the one where he is fourth.

That distribution is what you would expect if deposit tracks whoever ran the analysis and assembled the files. It also means the seven-for-seven figure is the one that speaks to his own practice, and the rest speaks to his coauthors’.

The trajectory tracks his public program. Through 2017 he is one of five. From 2018 forward, the years in which he launched SocArXiv and began arguing in public about transparency, he is twelve of fifteen. His two 2015 papers, with Jeehye Kang in the Journal of Family Issues and with Lucia Lykke in Social Currents, have no packages and no data availability statements. His 2025 article on views of falling birth rates carries a statement pointing at the Pew source data and an OSF repository for the code.

The man changed his practice when he started telling other people to change theirs. That is the least he could have done and it is more than the record shows for most people who argue about this in public.

Three cautions. Twenty articles is a small denominator and the authorship cells run to one and two papers; nothing here supports a claim about how coauthorship affects deposit in general. The venue question is unresolved rather than answered, and the failure is instructive: across eleven titles and eleven years, I could not date a single journal-level sharing rule except one, which suggests the field documents its own requirements badly. And the rule I applied is his 2023 rule, which is stricter than his 2025 one; under the looser standard his rate would be higher.

The easy charge against a man who audits other people is that he would not survive his own audit. Cohen survives it. On the papers where he alone decided, he survives it completely, in a period when no journal in his set is known to have asked him to.

That makes the criticism in the other piece more interesting rather than less. The problem with his 2025 audit is that he changed the ruler between two measurements and read the difference as movement, which is a failure of the method rather than of the man, and which the man’s own thirty years of work would have caught in anybody else.

The ASR case ledger gives the coding rule this census applies.

What follows is the venue ledger behind the census: for each of the twenty articles, what could and could not be established about the journal’s requirement at the date of publication. All checks August 26, 2026.

Jeehye Kang and Philip N. Cohen published in the Journal of Family Issues in September 2015. No public deposit, code deposit or pre-publication check requirement was located for that date, and the article carries no data-availability statement. Sage’s later research-data framework distinguishes journals that require public sharing from journals that encourage it, and no evidence places this title in either group in 2015.

Lucia C. Lykke and Philip N. Cohen published in Social Currents in September 2015. No deposit, code, statement or check requirement was located, and the article carries no data-availability statement.

Cohen published alone in Sociological Science on January 25, 2016. Its mandatory replication-package policy applies to manuscripts submitted after April 1, 2023, so it cannot have governed this article. This is the one case in the set where the absence of a mandate rests on a dated document.

Jill E. Yavorsky, Philip N. Cohen and Yue Qian published in The Sociological Quarterly, online July 20, 2016. No public-deposit requirement was located. Wiley announced its tiered framework on September 14, 2017, fourteen months later, and left each journal to choose among encouragement, expectation and mandate.

Jeehye Kang and Philip N. Cohen published in Social Science and Medicine in April 2017. No public-deposit requirement was located. Elsevier’s November 30, 2016 initiative implemented data-citation standards to encourage sharing without mandating deposit, and its general policy describes sharing as encouraged unless a journal adopts a stronger rule.

Philip N. Cohen and Joanna R. Pepin published “Unequal Marriage Markets” in Socius in August 2018, with an OSF package. No public-deposit requirement was located. The contemporaneous Socius article “Institutionalizing Transparency” discusses stronger journal policies as reforms to be adopted rather than describing a rule the journal already had.

Cohen published “The Coming Divorce Decline” in Socius in August 2019. No public-deposit requirement was located; Sage’s guidance as catalogued in September 2019 supported data sharing rather than requiring it.

Brandon G. Wagner, Kate H. Choi and Philip N. Cohen published in Socius in December 2020 and supplied an OSF package. No public-deposit requirement was located.

Joanna R. Pepin and Philip N. Cohen published “Nation-Level Gender Inequality and Couples’ Income Arrangements” in the Journal of Family and Economic Issues in 2020. The title sat under Springer Nature’s general policy, which asks authors to supply documentation on request rather than to deposit publicly. When the journal adopted its own provisions could not be dated.

Cohen published alone in the European Journal of Environment and Public Health in June 2020 and linked data and code at OSF. No deposit, code or check requirement was located.

Jeehye Kang, Philip N. Cohen and Feinian Chen published in Child Indicators Research, online in 2020. No mandatory public-deposit rule was located. Springer Nature’s single statement policy arrived on March 22, 2023, more than two years later, and requires statements rather than deposit.

Cohen published “The Rise of One-Person Households” in Socius in December 2021, with data and code at OSF. No public-deposit requirement was located.

Cohen published alone in Social Determinants of Health in February 2022. No deposit, code, statement or check requirement was located. The OSF package was already public with the December 2021 preprint.

Cohen published “Rethinking Marriage Metabolism” in Population Research and Policy Review in 2023. A data-availability statement was required or being implemented under the March 2023 policy, which encourages repository deposit while permitting statements that describe restricted data. Public deposit was not required. Package.

Mónica L. Caudillo, Andrés Villarreal and Philip N. Cohen published in The ANNALS of the American Academy of Political and Social Science in 2023. No mandatory public-deposit rule was located for this title at this date, and the published record supplies no qualifying package.

Joanna R. Pepin and Philip N. Cohen published in Socius in 2024. A data-availability statement appears in the article and the replication materials are linked at GitHub. No deposit mandate was verified.

Hao-Chun Cheng and Philip N. Cohen published in Asian Population Studies in 2024. Taylor and Francis’s basic policy encourages a statement and data sharing without requiring deposit, and no evidence places this title in a mandatory tier. Package.

Ansgar Hudde and Philip N. Cohen published in Socius in 2024. The statement links code, output, GSS and IPUMS data, with materials at OSF. No mandatory public-deposit rule was located.

Hangqing Ruan, Melody Ge Gao, Yajie Xiong and Philip N. Cohen published in Socius in 2024. A statement rule may have applied; no mandatory deposit rule was located, and no qualifying replication package was located as of the check date.

Cohen published alone in Socius in 2025. The statement links the Pew source data and code at OSF. Sage’s policy prescribes what a statement must contain and distinguishes that from the stronger rule imposed by journals requiring repository deposit.

In November 2020 Philip N. Cohen posted an inspection of the American Sociological Review. He went through the quantitative research articles in four issues of Volume 85, looked for links to replication materials, found that four of fifteen provided them, and named the articles that did not.

In the same post he set out a theory of why researchers share. Authors share when they are personally compelled, when they are normatively compelled, or when they are formally compelled. The audit was an exercise in the first two. It made a public record of who had shared and who had not, on the premise that a scholar who sees his own paper listed under nothing will behave differently next time, and that a discipline that sees the list will develop an expectation.

He repeated the exercise in January 2024 and again in December 2025.

This note asks what those audits produced. It reports two tests of the personal-compulsion route, both returning zero events in every arm. It reports a recount of the 2025 audit under the 2023 standard. It reports the journal’s policy history across the period, which rules out formal compulsion. And it reports a detection problem, found by accident, that bears on every audit of this kind including this one.

None of these is a large finding. Together they narrow what a public audit can be doing.

The three audits do not use one rule. In January 2024 Cohen sorted twenty-five quantitative articles into three groups. Eight had what he called a full package, meaning data and code sufficient to replicate the paper, judged by inspection rather than execution. A further group supplied something less. The rest supplied nothing. His table and his prose disagree on the middle group, giving five in one place and six in the other, with the residual moving accordingly.

In December 2025 the middle group disappears. The positive category now holds articles providing packages of code and data, or information about getting the data. He counts materials hosted on personal websites and excludes materials described as available on request. Twelve of eighteen articles clear that bar.

He states the 2025 rule in the post. He then compares the resulting figure against the 2023 figure and reports that the journal is making progress.

I (meaning Claude and ChatGPT) recoded the eighteen 2025 articles under the 2023 standard and recoded his eight 2023 positives under the same rule, so the coder is constant as well as the criterion. The rule: count an article when publicly accessible code appears, without executing it, to cover the article’s quantitative results, and the data are either included or obtainable by outside researchers through a documented procedure administered independently of the authors. Restricted data do not disqualify an article; discretionary data do. A German scientific use file with a published application route counts. Swedish register data reachable through Statistics Sweden counts. Partial coverage of a multi-study article does not count. Materials not publicly accessible do not count.

All eight 2023 positives survive. Eight of the eighteen 2025 articles qualify.

That is 32.0 percent in 2023 and 44.4 percent in 2025, against a reported rise from 32.0 to 66.7. Counting the one article whose repository required an access request produces 50.0 percent as an upper bound. Roughly two thirds of the reported increase follows from the change in what was counted.

The full case ledger, with the rule stated first, a resolving link and a check date on every entry, and the grounds for each downgrade named, is available separately. It was sent to Cohen before publication. He declined to review it.

The 2020 audit created a natural comparison, and Cohen created it without meaning to. He inspected four of the six issues of Volume 85. The two he skipped contain quantitative research articles that were never listed, never named, and never assigned a sharing status in public. Assignment to the treated and untreated groups depends on which issues he happened to sample, which has no obvious relationship to any author’s propensity to deposit. Same journal, same volume, same editorial regime, same year, same author pool.

Volume 85 contains twenty-one quantitative research articles. Fifteen were audited. Eleven of those were baseline non-sharers named in public. Five comparable articles sat in the unaudited issues. One of those five had already deposited and is not at risk. One further case, in the December issue, appeared close enough to the audit date that its exposure is ambiguous, and results are reported both including and excluding it.

The prediction, if public naming works through personal compulsion, is that named authors should subsequently deposit at a higher rate than unnamed ones.

The result is zero in both arms. No article in either group has a located first public deposit dated after November 30, 2020.

The second test uses the 2023 audit, where Cohen distinguished severity. Some articles were listed as supplying something partial. Others were listed as supplying nothing. Both groups were named in the same post. If naming operates through embarrassment, the second group had more to be embarrassed about. The result is again zero in both arms across the seventeen cases at risk.

The finding is that the outcome does not occur. Post-publication first deposit by ASR authors, across two cohorts and thirty-three articles, is close to a nonexistent behavior. Whatever a public audit accomplishes, it does not accomplish it by prompting authors to go back and post what they did not post before.

The design has no statistical power worth reporting and none is claimed. Eleven against four or five cannot detect an effect of any plausible size. What the comparison establishes is descriptive: in a hand-checked census of two full volumes, the behavior the intervention would have to change occurred zero times.

While cataloguing the corrections Cohen made to his audits, a pattern appeared that bears on all of this. The 2023 audit has been corrected four times in ways that changed an article’s classification. In each case an author of the article wrote in. In each case the material existed when he looked and his method had not found it. Simone Zhang and David Johnson’s Harvard Dataverse deposit was published in January 2023, a year before the audit. Christof Brandtner’s materials had been available throughout. Ferry and colleagues’ OSF project predated the check.

A fourth case was found not by an author but by this recount. Cheng and colleagues were classified as supplying nothing. A Harvard Dataverse record tied to the article’s appendix had been public since August 11, 2022 and the article did not link to it.

Four detection failures in a volume of twenty-five articles is sixteen percent. Three surfaced because a particular author happened to complain. The fourth surfaced because someone re-ran the audit. There is no reason to think the four are all of them.

The implication runs at this note too. Audits that follow an article’s links measure what an article’s links reveal. My own searches used titles, DOIs and author names against six repositories, and Cheng shows the method sometimes recovers unlinked material. It gives no basis for estimating how often it misses. The eight of eighteen reported above is a floor and the true 2025 figure is probably higher, as with every figure in every audit of this kind. This note rests on a comparison between two rules applied by one coder to the same articles, so a shared false-negative rate does not disturb it. Anyone reading the levels rather than the difference should read them as lower bounds.

The correction record is small. Across the three audits there are six documented corrections. Four changed a classification, and all four were prompted by an author of the paper being reclassified. One was self-initiated and arithmetic, catching an article counted twice. One came from a third party and fixed a broken link. No classification change originated with anyone other than an affected author.

The obvious reading is that authors are the people who notice, which is true and sufficient to explain the pattern. Six cases cannot distinguish that from anything else. What the record does establish is that the audit’s error-correction ran through a channel available to people with standing to write to Cohen and an interest in the outcome. Cheng did not write in and the misclassification stood for two and a half years.

Cohen’s 2020 post argues that sociology operates on trust because almost nobody checks anybody’s work. His own audits were checked by the authors they audited and by no one else.

The remaining route in Cohen’s framework is formal compulsion, which would provide an institutional explanation for any rise between 2023 and 2025. It does not apply. The American Sociological Review did not adopt a mandatory deposit policy in the period. It does not require a data availability statement. It does not verify submitted materials. The American Sociological Association’s ethical standards ask members to make data available to qualified researchers after publication, which is a professional norm rather than a submission requirement. Sage operates a tiered research-data policy and leaves the tier to each journal; Acta Sociologica and Sociological Methods and Research adopted stronger tiers, and ASR did not.

The internal evidence agrees. Three of the eighteen quantitative articles in Volume 90 carry no data availability statement of any kind. That is what an unenforced regime looks like from the inside.

So across the window studied here, the journal imposed nothing, verified nothing, and the twelve-point rise that survives a constant rule occurred without a mandate. Two of the three routes Cohen named are ruled out or produced no measurable events. What remains is normative drift, or ordinary variation. Twelve points across denominators of eighteen and twenty-five is inside the range that sampling alone would produce.

Three questions that seem open are open in a narrower way than they look. Whether deposit requirements raise deposit rates: they do. Olivia Miske and colleagues, coding the historical policies of 62 social and behavioural science journals across 2009 to 2018, found data available for 82 of 511 papers where no policy applied, or 16.0 percent, against 43 of 68 where data and code were required, or 63.2 percent. Their two remaining tiers, data alone and the full requirement including a pre-publication check, rest on sixteen papers and five papers, so the figures of 87.5 and 100 percent establish nothing about what those regimes add.

Whether verification raises reproduction: the evidence is too thin to say. In the same study, weighted precise reproduction runs at 40.7 percent where no policy applied, across 87 papers, and 75.7 percent where data and code were required, across 38. The two tiers above rest on fourteen papers and four, and give 70.5 and 65.0 percent. Nothing about what a pre-publication check adds can be recovered from four papers. These are observational associations across journals rather than estimates of what any policy caused, and the study’s headline rates, 53.6 percent precise and 73.5 percent at least approximate, are conditional on obtaining usable data, which the investigators managed for 143 of the 600 sampled papers.

The same study gives the first direct measurement of reproduction in sociology journals. Across the six in the frame, ASR, the American Journal of Sociology, Demography, Social Forces, the European Sociological Review and the Journal of Marriage and Family, fifty-nine papers were sampled and sixteen attempted. Two reproduced precisely. Four more reproduced approximately. Ten did not reproduce. ASR contributed ten sampled papers, seven never attempted; of the three attempted, one reproduced precisely, one approximately and one not at all. Nothing rests on three papers, and the counts are worth reporting because they are what exists.

Whether anticipation of inspection produces better submissions: it does not. The Quarterly Journal of Political Science has reviewed submitted packages in house since 2005, on the stated theory that transparency would motivate authors to be cautious, and its review checked that the code ran, that results could be located, and that they matched the paper. Reviewing twenty-four papers, Nicholas Eubank reported that twenty needed modification and fourteen produced results differing from the manuscript when the authors’ own code was run. He drew the conclusion himself: that these problems occurred despite authors knowing their code would be checked demonstrates the necessity of checking it. These are pre-publication states, caught and corrected, so the published articles are not defective; what the record shows is that anticipation was not sufficient. The exercise cost about six hours and roughly $180 per paper.

The American Journal of Political Science, whose verification policy is the most studied in the social sciences, processed 149 manuscripts and completed verification for 127 by February 1, 2018. Eight reproduced every analysis on the first try. Sociology has one comparable case: for the Fragile Families Challenge special issue at Socius in 2019, two of fourteen manuscripts reproduced on the first attempt.

The counterweight comes from Management Science, where Fišar and colleagues assessed 419 packages published under a mandatory disclosure policy and classified 95.3 percent as fully or largely reproduced, conditional on reproducibility being verifiable at all; among the failures, 88 percent turned on restricted data access. That is an assessment of packages that had already cleared an editorial completeness check, run after publication by outside reviewers, and it is not a first-pass rate. Its median reviewer spent four hours per package, against six for the QJPS execution check and eight for the Odum workflow that bundled archival curation with verification. The three numbers measure different work and should not be read as one service at three prices.

Sociological Science shows where sociology sits. It requires a replication package as a condition of publication and states in its reproducibility policy that it will confirm a paper lists its package contents correctly and will not perform replication tests. No sociology journal executes what it collects. Cohen’s December 2025 post discusses Sociological Science and package requirements without mentioning Eubank or the QJPS record, and I found no citation to Eubank anywhere in Cohen’s published work. Sociologists have engaged the paper elsewhere: James Moody and coauthors cite it in the Annual Review of Sociology, as do Jeremy Freese and David Peterson, and Bianca Manago cites it in The American Sociologist.

No study has publicly named individual authors for deficient sharing and then measured whether those authors subsequently deposited. The adjacent literature measures a different thing. Daniel Krähmer, Laura Schächtele and Andreas Schneck emailed 1,028 authors of papers using European Social Survey data and asked for their analysis code; 385 supplied it, or 37.5 percent. Claudia Acciai, Jesper Schneider and Mathias Nielsen, testing whether authors respond differently to different kinds of requester, approached 1,634 authors whose papers said data were available on request, using fictitious prospective doctoral students; 226, or 14 percent, either shared or said they were willing to. Private request, private reply, nothing public at either end.

The closest existing design is a randomized trial offering BMJ Open authors an Open Data Badge and measuring verified public deposit. Two of 54 in control, two of 57 in treatment. No effect. That tests positive recognition rather than public criticism, which are different treatments, so a null on one does not settle the other. It is the only randomized evidence on author-level reputational incentives and public deposit, and it points the same way as the zeros reported here.

Public naming works on institutions. When a news organization singled out universities and hospitals for failing to report clinical trial results, reporting rates at the named institutions rose from an average of 35 percent to 76 percent, and Stanford went from about a third to 85 percent after hiring six full-time compliance staff. Institutions have compliance offices, legal exposure and reputational management. Individual scholars have none of these.

The two comparisons here are underpowered by construction and are offered as bounded case series rather than hypothesis tests. The 2023 severity comparison carries a selection problem: articles in the partial group had already deposited something, articles in the nothing group had not, so the groups differ in the disposition the outcome measures and the confound runs against the prediction. Negative searches are the weakest form of evidence and the Cheng case demonstrates their failure mode. Where a deposit’s first public date could not be established from repository metadata or an archived capture, the case is recorded as unknown rather than resolved.

Repository states change. The pages that carry the most weight in the recount were submitted to the Internet Archive on August 26, 2026 and are cited by timestamped capture. One of them, an OSF project the audit had recorded as requiring an access request in December 2025, still redirected an anonymous visitor to a sign-in page eight months later. A GitHub repository named in the same audit stood at the same three commits. Nothing here can establish what any other repository looked like on any earlier date. The corrections catalogue rests on what Cohen published: post text, appended notes, and comment threads. Corrections made by email and never noted would not appear.

A public audit of a journal’s replication practices, repeated three times over five years by a prominent scholar, produced no located instance of an author going back to deposit. The journal it audited imposed no requirement and ran no check across the period. The rise the third audit reported is roughly two thirds an artifact of a widened category and, on a constant rule, is not distinguishable from ordinary variation. The audit’s own errors were corrected only by the authors it named.

The audits may have shifted expectations in ways no design here could detect, and the discipline’s conversation about transparency is louder than it was. The route Cohen named first, the individual scholar moved to act by seeing his paper listed, is not where the action is. Six hours and a couple of hundred dollars per paper buys a check that a decade of counting has not.

The case ledger for the recount gives the coding rule, twenty-six entries, resolving links, check dates and archived captures.

Philip N. Cohen’s career can look unusually scattered if you read it by subject. He has written about women’s suffrage, racial inequality, housework, occupational segregation, cohabitation, marriage, divorce, same-sex parenting, genetics and race, fertility, COVID-19, peer review, preprints, open science, public sociology, and the politics of falling birth rates. His own bibliography now divides naturally into two large bodies of work, one on families and inequality and another on scholarly communication and science. But after reading the corpus chronologically, I think that division misses the strongest continuity.

Cohen has spent thirty years doing a sociology of the denominator.

By denominator I mean the rule that decides who has been counted and what has been grouped with what. Cohen’s characteristic move is to encounter an apparently stable social fact and ask what the rule was. Change the universe, the reference period, or the boundary of the category, and the fact may change. Two related habits grow out of that one, and I take them up later: a suspicion of explanations pitched at the level of the individual, and a suspicion of trends converted into directions of history.

Cohen’s first journal article, the 1996 “Nationalism and Suffrage: Gender Struggle in Nation-Building America,” suspects a category that looks more universal than it is. “Women’s suffrage” sounds like the political inclusion of women. Cohen instead reconstructs a movement in which leading white suffragists increasingly made their claim through white citizenship, national strength, racial hierarchy, and an alliance with white men. The category “women” concealed conflicting interests among women themselves. White suffragists could advance their own political status while helping exclude Black women and men from the polity. Cohen ends by insisting that feminist scholars treat those women as agents making purposeful choices.

Three years later, with Suzanne Bianchi, the same intellectual operation becomes statistical. “Marriage, Children, and Women’s Employment: What Do We Know?” begins from an apparently straightforward question: how much do married mothers work? But the Current Population Survey permits different reference periods and different universes. Cohen and Bianchi show that estimates of women’s full-time employment change substantially depending on which is used. Welfare policy was being justified partly by assumptions about how much mothers normally work. If the denominator changes the normal rate, it changes the policy comparison.

The following year, Cohen and Lynne Casper made the same kind of intervention in “How Does POSSLQ Measure Up?” The issue was cohabitation. A government measure that appeared to count unmarried partners was an operational construction whose performance could be tested against alternative ways of identifying couples.

That pattern runs through Cohen’s early inequality research. In “Individuals, Jobs, and Labor Markets: The Devaluation of Women’s Work,” with Matt Huffman, the wage attached to female-dominated work cannot be understood simply by comparing individuals. Jobs are situated within labor markets, and those contexts alter the relationship. In “Racial Wage Inequality: Job Segregation and Devaluation Across U.S. Labor Markets,” segregation and devaluation are analytically separated. In “The Gender Division of Labor: ‘Keeping House’ and Occupational Segregation in the United States,” the usual segregation statistic changes once women doing unpaid housework are restored to the division of labor.

The cumulative lesson is that inequalities attributed to persons are often properties of arrangements around persons. Households, jobs, establishments, occupations, labor markets, and policy regimes keep appearing between the individual and the outcome. Cohen’s 2007 work with Huffman on “Black Underrepresentation in Management across U.S. Labor Markets” asks why establishments operating in areas with larger Black populations are more likely to underrepresent Black workers in management.

Family research remains the largest substantive part of Cohen’s career, but much of it is an argument against treating “the family” as the natural unit at which family outcomes should be explained. Extended households change the employment possibilities of single mothers. National policy changes the relationship between individual bargaining resources and housework. Marriage markets differ by race. Economic shocks affect divorce and fertility. The family is nested inside structures that alter what family means.

Cohen’s second recurring suspicion is directed at trends that are turned too quickly into historical directions.

The titles tell part of the story. “Stalled Progress? Gender Segregation and Wage Inequality Among Managers, 1980–2000.” “Headed Toward Equality? Housework Change in Comparative Perspective.” “The ‘End of Men’ Is Not True.” “The Coming Divorce Decline.” “What’s the Story? Family Demography at the End of Progress.” “Rethinking Marriage Metabolism.”

Cohen encounters a real change and objects to the story attached to it. Women are advancing, therefore men are ending. Housework is becoming more equal, therefore equality is the destination. Marriage is declining, therefore marriage is disappearing. Fertility is falling, therefore population catastrophe is approaching. Cohen distrusts the conversion of movement into destiny.

When Hanna Rosin claimed that “young women” were outearning young men, Cohen unpacked “young women” and showed that the statistic described single, childless women under thirty who lived in metropolitan areas and worked full-time and year-round. Rosin said women had become a majority of managers. Cohen noticed that the underlying category was “managerial and professional,” which combined managers with female-dominated professions such as nursing and elementary education. The category created the headline.

Cohen begins that essay with a sentence that could serve as a manifesto for his career: statistics are not “mere technical details.” They are attempts to represent reality. That is why the boundaries matter.

The anti-teleological position becomes explicit in “What’s the Story? Family Demography at the End of Progress.” Cohen argues against imagining normal modernity as a track along which societies travel while pandemics, climate change, inequality, identity conflict, and political disorder interrupt the journey. Those things may be constitutive features of the world whose demography we are trying to explain. Even the second demographic transition begins to look suspicious if it is treated as another version of history moving in an incontrovertible direction.

This helps explain Cohen’s more recent interest in falling fertility. His 2025 article, “Negative Views of Falling Birth Rates in the United States Come Mostly from the Right Wing,” and his continuing work on population panic fit the older pattern. The subject has changed. The argumentative opponent has not. A demographic trend is being recruited into a large story about social decline. Cohen’s instinct is to separate the measured change from the narrative laid over it.

Return now to the first habit, because it is the one that carries him into a new field.

“Women.” “Managers.” “The Black middle class.” “Cohabitors.” “Married people.” “Young women.” “Science.” “Peer review.” Each can operate as a useful category. Each can also conceal enough heterogeneity to make a claim true only because unlike things have been pooled.

That is what makes the later open-science work a logical extension of his interests.

The 2022 collaborative paper “PReF: Describing Key Preprint Review Features” begins from a classification problem. “Preprint review” sounds like a recognizable practice, but the activities collected under that term differ in who initiates review, how reviewers are selected, whether identities are known, whether competing interests are checked, whether authors can respond, and what outcome the review produces. The solution is to decompose the label into standardized features so different systems can be compared on common dimensions.

That is Cohen’s old move in a new field.

In 1999, “women’s employment” had to be decomposed by universe and reference period before a trend could be interpreted. In 2013, “young women” had to be decomposed before Rosin’s claim could be evaluated. In PReF, “peer review” has to be decomposed before anyone can compare review systems or measure their evolution. The substantive distance between those projects is enormous. The methodological distance is small.

The same continuity appears at the level of causal explanation. Around 2018, Cohen’s bibliography begins to develop a second program alongside family demography: open scholarship, SocArXiv, peer review, scholarly infrastructure, and the production of knowledge. He never stops doing demography. His recent work includes marriage expectations, marriage events, fertility, mortality, and birth rates at the same time that he is publishing about scholarly communication.

The new program again moves the explanation away from individual behavior and toward context. In “The Scholarly Knowledge Ecosystem: Challenges and Opportunities for the Field of Information,” Micah Altman and Cohen define the scholarly ecosystem broadly enough to include stakeholders, markets, policies, organizations, norms, infrastructure, and educational systems. Scientific reliability is an outcome produced inside an institutional ecology.

The analogy with the earlier labor-market work fits. An employee’s outcome cannot be understood without the job, establishment, occupation, and labor market. A scientific paper cannot be understood without journals, repositories, incentives, review systems, publishers, universities, funders, and technical infrastructure. In both cases Cohen resists explanations that stop at the individual because the relevant machinery is operating one or more levels above.

There is a political continuity too.

Cohen’s 1996 suffrage paper is a critical account of white feminism, nationalism, and racial exclusion. His 2007 review essay “Confronting Economic Gender Inequality” is hardly value-neutral. What changes is that Cohen becomes increasingly explicit about what political commitments mean for epistemology.

The pivotal document is probably his 2015 “How Troubling Is Our Inheritance? A Review of Genetics and Race in the Social Sciences.” Cohen argues against Nicholas Wade’s attempt to connect population genetics to racial differences in social behavior. He says that he sets the evidentiary bar high for racial-genetic explanations because the consequences of a false positive are potentially severe. He calls this a political judgment and compares it to the asymmetric treatment of error in criminal justice and medical research.

That passage makes it difficult to describe Cohen’s later open-science work as an attempt to construct a value-free machine for deciding truth. He does not believe such a machine exists. Altman and Cohen’s scholarly-ecosystem work argues for information systems aligned with human values. The point of transparency is to make judgment inspectable.

That position is developed in Cohen’s 2025 book, Citizen Scholar: Public Engagement for Social Scientists. Its chapter sequence moves from description through open scholarship and peer review to social media, activism, and citizenship. The book’s argument is that scholarship and citizenship can be integrated, provided the scholar remains answerable to evidence.

Seen chronologically, there is therefore a critic-to-builder trajectory in Cohen’s career. By the early 2010s he is increasingly visible as an auditor of public and scholarly claims. Rosin’s statistics. Mark Regnerus’s same-sex parenting research. Nicholas Wade’s racial genetics. Alice Goffman’s methods. Claims about poverty, marriage, and family decline. The scholar stands outside the claim and asks whether it survives inspection.

Then Cohen becomes involved in building the infrastructure through which claims circulate. SocArXiv launches in 2016. He works on scholarly communication, preprints, review taxonomies, openness, editorial boards, and research infrastructure. The person who spent years inspecting products coming out of the scientific machine increasingly starts working on the machine.

His own scholarly-communication research states the methodological principle. In “Interventions in Scholarly Communication: Design Lessons from Public Health,” Altman, Cohen, and Jessica Polka argue that scholarly reform suffers from an absence of standard measurement, systematic data collection, and comparable evaluation. They mock the common intervention strategy as “Do it now, check sometime.” Their proposed remedy is to identify common measures and outcomes, build infrastructure for tracking them, incorporate assessment into interventions from the beginning, and create consistent longitudinal evidence.

PReF makes the same point in another form. If heterogeneous review practices are going to be compared across time, first define stable features that distinguish them. Only then measure the trend.

Public argument rewards compression. “ASR went from one-third to two-thirds” is intelligible in a way that a five-category taxonomy of data accessibility is not. A scholar trying to participate in public life must turn a complicated empirical world into categories, numbers, sentences, and stories. Yet Cohen’s scientific work repeatedly demonstrates that the compression is where trouble enters.

This tension sits near the center of Citizen Scholar. Cohen wants scholars to leave the seminar room and enter public argument without surrendering the standards that make scholarship worth hearing. But his own career suggests that the two activities place different pressures on description. The public intellectual needs a story. The empirical sociologist keeps discovering reasons the story is too clean.

His three informal audits of replication materials at the American Sociological Review are where that pressure becomes visible and countable, and I have taken them up separately.

That may be the most interesting trajectory in the entire corpus — a progression from studying how social arrangements produce observable outcomes to studying how knowledge arrangements produce observable facts, followed finally by an attempt to live inside both systems at once as a citizen scholar.

The objects keep getting larger. Women within families. Workers within jobs. Jobs within establishments. Establishments within labor markets. Families within policy regimes. Papers within journals. Scholars within an information ecosystem. But Cohen keeps asking variants of the same question: what structure surrounding the observation made it look the way it does?

And alongside that question is another: what disappeared when we named it?

“Women” can hide racial conflict. “Young women” can hide a highly selected demographic subgroup. “Managers” can become managers plus professionals. “Peer review” can hide radically different institutional processes.

Cohen’s sociology is strongest when he refuses to let the noun settle the empirical question.

That is why denominator skepticism seems a better description of his method. Before accepting a social fact, ask who counts. Ask what the comparison class is. Ask whether the category means the same thing in every observation. Ask whether a measured difference has been converted into a causal story, or a trend into a theory of history.

Those habits connect work that otherwise looks unrelated.

Cohen argues that statistics matter enough that their construction has to be exposed. Categories can mislead because reality exists outside them. A better denominator can produce a better description.

That faith in better description survives into the metascience work. The solution to unreliable scholarly communication is to expose provenance, make evidence inspectable, improve measurement, clarify categories, build institutions that permit criticism, and give outsiders enough information to decide whom to trust.

There is a revealing passage in Cohen’s 2023 chapter “How Do We Tell What’s True?” in which he refuses to put conventional peer review on a pedestal. Expert credentials, institutional reputation, disclosed political commitments, linked evidence, open criticism, and transparent sourcing all provide imperfect signals. Knowledge becomes more trustworthy when claims can be traced and interrogated.

That is also a description of Cohen at his best.

His enduring subject is the distance between a claim and the machinery that produced it.

For thirty years Cohen has been opening that machinery.

Andrew Gelman is the nearest single analogue. Substantive researcher, then methods critic, then a daily blog auditing other people’s published claims, then a participant in reforming what the field accepts.

The three acts match. Substantive researcher, then critic of method, then a public auditing operation, then a hand in what the field will accept. Cohen’s blog has run since 2008 and Gelman’s since 2004, and both use the same instrument: take a published claim, go to the numbers behind it, show the reader the arithmetic, name the paper and the author. Both write for a mixed audience of colleagues and journalists. Both get called a bully by people they have corrected.

Two differences. Gelman reruns. When he takes apart a paper he pulls the data where he can and does the analysis, and his commenters do it too, and the corrections that come out of that blog are corrections about what the numbers show. Cohen inspects. His audits ask whether a package exists and what it contains, and he says he does not run the code. That is the difference between an auditor and an inventory clerk, and it explains why his findings are about availability while Gelman’s are about results.

Gelman also audits his own side. His targets are chosen by whether a claim looks statistically implausible, and the record includes his own papers, his friends’ papers, and work whose conclusions he likes. That indifference is his central asset.

Kieran Healy holds Cohen’s epistemology. His work argues that the measure and the rendering are constitutive of the finding, and his 2017 essay against nuance argues that sociology overvalues theoretical elaboration and undervalues clear description. That last point is Cohen’s descriptive turn stated as a general position about the discipline, published before Cohen said it at Wisconsin, by a sociologist Cohen surely reads.

Healy built no audit. No annual counts, no named lists, no preprint server, no correction demands. He wrote a book about how to draw graphs and taught a generation of sociologists to think about rendering.

So the epistemology does not produce the enforcement. Two men hold roughly the same view of what a measure is and only one of them audits. Whatever explains Cohen’s project, it is not the belief that categories manufacture findings, because Healy holds that belief and does something else with it.

Jeremy Freese occupies the position in the discipline that Cohen might have taken. He is sociology’s designated metascience thinker, coauthor of the field’s review article on replication, a writer on preregistration and research transparency. He works through the literature rather than through counts and lists.

Freese and Peterson cite Nicholas Eubank. Cohen does not. The sociologist doing metascience as scholarship engaged the one journal that ran the experiment; the sociologist doing metascience as enforcement did not.

Otis Dudley Duncan (1921-2004) built the tools. The dissimilarity index he developed with Beverly Duncan became the standard measure of residential segregation, and he brought path analysis into sociology. Then he spent his later career arguing that sociology’s measurement was pre-scientific, that its quantities frequently did not measure what their names claimed, and that the discipline had adopted numerical machinery without asking what the numbers meant. His late book on social measurement is a sustained warning against exactly the pooling Cohen keeps undoing.

Duncan questioned the segregation index he had built and the causal modeling he had introduced. Cohen has done that with PReF, where he decomposed a category to build a measurement tool. He did not do it with the audit categories. Duncan is the standard, and the standard is set inside Cohen’s own lineage: this is a demographer working on segregation, which is Cohen’s own subject, who made auditing his own instruments the last act of his career.

Barry Wimpfheimer’s argument is that a category which arrived later, legal-analytic reading, has governed how earlier material gets read, and that the material looks different once you stop letting the later category set the terms. That is the same move on texts rather than on variables, which tells you the operation is a habit of asking what the naming did.

What the set adds up to is a way of locating Cohen.

The category operation is not his: Healy has it, Duncan had it, Wimpfheimer has it in another material. The three-act trajectory is not his: Gelman ran it a decade earlier in an adjacent field. The metascience is not his alone: Freese has been writing it in sociology throughout.

What is distinctive is the combination plus the enforcement. Cohen alone among these men publishes lists of named papers with a verdict beside each, repeatedly, over years. That is the residue after the comparisons subtract everything shared.

Philip N. Cohen has inspected the American Sociological Review three times to see whether its authors post the code and data behind their published results. The first inspection came in November 2020, the second in January 2024, the third in December 2025. Each time he read through the quantitative articles, looked for links to materials, and counted. He did not run any of the code. He says so.

The third post opens by reporting that the journal is making progress. Two-thirds of the quantitative articles in Volume 90 provided replication materials, he writes, against roughly a third in the previous audit and roughly a quarter in the first. Twelve of eighteen, up from eight of twenty-five.

The trouble is that the two numbers count different things.

In January 2024 Cohen sorted twenty-five quantitative articles into three columns. Eight had what he called a full package, by which he meant “data and code sufficient to replicate the papers.” Five or six others, depending on whether you read his prose or his table, supplied something less: partial data availability, a set of variable codes, fragments. The rest supplied nothing. The distinction between the first column and the second is the entire architecture of that audit.

In December 2025 the columns merge. His positive category now holds articles that “provide packages of code and data, or at least information about getting the data.” He adds that he counts materials parked on personal websites, which is not where replication files belong, and excludes materials described as available on request. Twelve of eighteen articles clear that bar.

He states the new rule in the post, in the sentence quoted above, for anyone who reads it. What he does next is compare the resulting figure against numbers produced by the older, narrower rule, and read the difference as movement in the journal.

I recoded the eighteen 2025 articles under the 2023 standard, one fixed rule, no code executed, matching his method. The rule: count an article when publicly accessible code appears to cover its quantitative results and the data are either included or obtainable by outside researchers through a documented procedure that does not run through the authors. Restricted data do not disqualify an article. A German scientific use file with a published application route counts. Swedish register data reachable through Statistics Sweden counts. Data that arrive only if the authors decide to send them do not.

Under that rule, eight of the eighteen 2025 articles qualify. Forty-four percent, against thirty-two percent in 2023.

So the journal improved, but the improvement he reports runs from thirty-two to sixty-seven percent, a gap of thirty-five points, and the improvement a constant rule can find runs from thirty-two to forty-four, a gap of twelve. Roughly two-thirds of the reported rise comes from the widening of the category.

Four articles produce the difference, and none of them is bad work. Fabiana Silva, Irene Bloemraad and Kim Voss deposited materials for “Frame Backfire” at the Open Science Framework, and Cohen’s own table records that the project required an access request when he checked it on December 9, timestamped to the minute. He counted it anyway, and elsewhere in the same post tells the journal that permission-gated projects should not be allowed. The barrier is still up. A reader who follows the article’s link today without an OSF account arrives at a sign-in page. Counting it regardless gives nine of eighteen, the outer bound of the recount. Mabel Abraham, Tristan Botelho and James Carter posted data and code covering the main results of two of the studies in “(Not) Getting What You Deserve”; the article contains more than two. Daniel Scott Smith and coauthors, writing on “How Values and Uncertainty Shape Scientific Advance in Peer Review,” posted a repository whose own documentation says it holds no data, excludes some code, and cannot reproduce the machine learning benchmarks without material the authors supply on request. Eight months after Cohen named it, the repository stands at the same three commits. David Brady, Aliza Luft and Ezra Zuckerman Sivan, reanalyzing another team’s work in “How Does Culture Matter for Attainment, and How Would We Know If It Did?”, posted a PDF of their code, which states that the underlying data came from the scholars whose work they were reanalyzing and cannot be passed along. It also carries author-specific file paths and merges against a dataset it does not supply, so an outside reader could not run it even holding the data.

The obvious defense is one Cohen has already made. In 2024 he described his own audit as a tiny study of poor quality. In 2025 he calls his methods “squishy.” A man who labels his instrument that way has told his readers what weight to put on it. The answer is the first line of the same post, where he writes that the journal is making progress and sets the three figures against each other to show it. The disclaimer and the directional claim sit in the same paragraph.

Cohen has spent thirty years teaching other researchers not to do this.

The operation runs through his corpus from the beginning. In 2000 he and Lynne Casper built a new historical measure of cohabitation because the standard one misclassified people and distorted the trend it was supposed to reveal. In 2002 he showed that part of the apparent decline in the marriage premium for men came from cohabitors sitting inside the never-married comparison group, and that pulling them out shrank the decline. He and Lynne Casper split multigenerational households into hosts and guests because knowing that someone lives in an extended household tells you nothing about their position in it. He and his coauthors argued that a Black middle class defined through married households misses the growing population of never-married professionals living alone. He recoded keeping house as an occupation and found that women leaving unpaid domestic work accounted for as much of the decline in occupational segregation as desegregation within paid work. With Matt Huffman he took the established association between local Black population share and racial wage inequality and split it into two candidate processes, segregation and devaluation, and reported that one survived the data and the other did not.

A category that pools things behaving differently will manufacture a finding. Three cases show that he knows this. The first is PReF, the preprint review framework he built with collaborators, a fourteen-author project from 2022. It begins from the observation that peer review names procedures too different to compare, and replaces the label with eight standardized descriptors so a reader can see what a given review consisted of. The point of the exercise is to make comparison possible by first fixing what is being compared.

The second is his 2013 essay “The End of Men Is Not True: What Is Not and What Might Be on the Road Toward Gender Equality” on Hanna Rosin’s The End of Men, which is combative from the first page. The claim that young women out-earn young men turns out, on inspection, to describe single childless full-time metropolitan workers, and Cohen walks through the restriction before reanalyzing the narrowed group and finding that men still come out ahead within education categories. He does the same to her claim about managers, which rests on a category folding female-dominated professions into a statement about management. Polemic did not cost him the method. He goes claim by claim, category by category, for pages.

The third is closest to home. In 2023, with Micah Altman and Jessica Polka, he published “Interventions in Scholarly Communication: Design Lessons from Public Health” in First Monday. The complaint there is that open science reforms get launched without stable measures or systematic evaluation. The prescription is common measures, assessment designed in from the start, and comparable outcomes tracked over time. That appeared two years before the 2025 audit, addressed open science reform specifically, and carried his name second.

So the ASR audits are a case of a technique he had, used under fire, and prescribed in print for this exact class of work, and then did not apply to a measurement he was building, in the one place where the measurement was going to be read as evidence of a trend.

Applying his 2023 standard to his 2025 cases moves sixty-seven percent to forty-four. The journal improved. It improved by about a third as much as reported, and what survives a constant rule is small enough that eighteen articles cannot separate it from ordinary year-to-year variation.

Cohen has spent years asking sociologists to correct the record when the record is wrong, and has corrected his own audit twice when authors wrote to tell him he had missed their packages. I sent him the coding rule and the twenty-six cases, with links and dates, before publishing this. He declined to read it. The ledger stands open, and if any of it is wrong, it will be fixed here. (V3)