The Hypothesis That Fits Too Well: Understanding HARKing and Preregistration

When the prediction turns out to have been written afterward

Have you ever read a research article whose hypothesis matched its results perfectly, down to the smallest detail? The researcher may indeed have been a sharp forecaster. But there is another possibility: the hypothesis was written only after the results came out.

Norbert Kerr called this practice HARKing, short for hypothesizing after the results are known. That is, a hypothesis born from the results is presented as if it had been there from the start (Kerr, 1998). Lishner (2021) breaks it into two elements: the hypothesis is adjusted after seeing the data, and the researcher then claims to have predicted that result. The first element is normal in science. It is the second that misleads readers.

Not all HARKing is equally serious

Kerr distinguished several versions. Some pick whichever hypothesis best fits the results, some add a rival hypothesis that is certain to lose so that it looks like a contest between theories, some quietly drop an initial hypothesis that failed, and some keep reporting the original hypothesis and then add a new one inspired by the data. According to Kerr, these forms are not equally serious.

Rubin (2017) sorts them into three labels. CHARKing, constructing a new hypothesis after the results and concealing its origin, is the riskiest. SHARKing, suppressing hypotheses that were not supported, is harmful when it concerns the main conclusions. RHARKing, retrieving a hypothesis from the literature after the results are known, is relatively safe as long as the source is cited.

Why researchers are tempted

Kerr saw the biggest driver in the publication process. In a survey of 156 behavioral scientists that he reported, 96% of respondents judged that a manuscript presenting a post hoc hypothesis as if it had been there from the start had a better chance of publication than an honest one. Some 52% had seen an editor or reviewer ask for the introduction to be rewritten to fit the results better. These figures describe respondents' perceptions and experiences in the late 1990s, not the number of articles that actually engaged in HARKing.

There is also an unintentional factor, hindsight bias: the feeling of having "known it all along" once the result is known. This bias is hard to remove, even when people have been warned.

What is lost when a hypothesis is disguised

Kerr listed twelve possible costs. A statistical coincidence can turn into theory and become hard to correct, because replications and null results are rarely published. A hypothesis written after the results are known is not truly tested by the data that gave rise to it, because it never had a chance to fail.

Information about what did not work disappears too, even though it could keep other researchers from going down the same dead end. Students, in turn, get a picture of science that is too smooth, so some feel untalented when their own research is full of failures.

Exploration is legitimate, as long as it is honest

The problem is not ideas that arise after seeing the data. Respondents to Kerr's survey actually supported adding new hypotheses, provided the original hypotheses are still reported. Marking a hypothesis as a later finding takes a single short sentence.

Baruch (2024) even argues that HARKing can be beneficial in the right situations, for example when a better theory emerges or an unexpected event occurs midway through data collection, as long as it is reported openly. He still rejects the practice of analyzing the data first and then writing a hypothesis to fit it.

Preregistration: locking in the order of events

Preregistration means uploading the hypotheses, study design, sample size, and analysis plan online before the data are collected or seen, complete with a timestamp that the public can check. Departures from the plan are reported. It is no cure-all: van den Akker and colleagues (2024) found no decrease in the proportion of positive results among preregistered studies. But readers can tell what was predicted from what was found.

At Anvesana, the design of the development study for Kompas Minat Indonesia, a vocational interest measure for adolescents, has been preregistered on the Open Science Framework (osf.io/nwv3d). Anyone will later be able to compare that plan with the final report.

What this means for practitioners

The instruments we use and the recommendations we follow rest on studies that may once have disguised a finding as a prediction. That does not mean every accurate prediction deserves suspicion. Start with one small habit: when reading the study behind an instrument, look for its preregistration link and check whether the report separates planned analyses from exploratory ones. If you are doing research yourself, write down your hypothesis and date it before opening the data.