A preface: why this almanac is publishing at all.
An introductory note on the open-notebook tradition the lab descends from, and on the decision to publish the work as it is being done rather than after it is safely concluded.
The lab notebook is older than the journal article. Michael Faraday filled close to sixteen thousand numbered diary entries at the Royal Institution, kept day by day from 1820 until shortly before his death in 1867; the diaries were transcribed and published, in seven volumes, only sixty years later. Newton's optical notebooks survive at the Cambridge University Library and have been excavated by historians for almost three centuries. James Watson and Francis Crick's lab books survive as the primary evidence of how the double helix was actually pieced together, including the false starts that the Nature paper never mentioned. The published article is the polished export. The notebook is the work.
In the second half of the twentieth century the journal article hardened into the dominant unit of scientific communication, and the notebook receded into archival territory. The result was efficient at communicating outcomes and increasingly poor at communicating process. By the early 2000s the gap had widened into what is now called the replication crisis: a 2015 collaboration led by the Center for Open Science attempted to replicate one hundred psychology studies and succeeded with roughly thirty-six percent of them, and similar exercises in cancer biology, economics and machine learning have produced comparable numbers.
In 2006 the chemist Jean-Claude Bradley, then at Drexel University, coined the term open-notebook science to describe the practice of publishing the laboratory record as it is being filled, in real time, including reagents that contaminated, hypotheses that died, and trials that produced nothing. Bradley's argument was uncomfortable but plain. A field whose visible record contains only the experiments that worked is a field that quietly compounds its own bias, and every reader downstream pays the cost. The proposed correction was structural rather than cultural: change what you publish, not how you write.
There is no insider information. Notebooks are made publicly available as the experiments are performed. Failures are reported as fully as successes.Bradley's working definition of open-notebook science, 2006
The practice has since fed several adjacent traditions. Preregistration: filing the analysis plan before the data is collected, so that the reader can tell genuine confirmation from post-hoc storytelling. The registered-report format, in which a journal accepts a study for publication on the basis of its protocol rather than its results. Public registries (the Open Science Framework, ClinicalTrials.gov, the AEA registry for economics) where time-stamped protocols can be sealed and later compared against what was actually published. Each of these is a partial implementation of the open-notebook idea, and each is most useful in fields where the temptation to selectively report is highest.
Runaric is run as an open notebook. Every preregistration is sealed before data is collected; every trial is logged, including the ones that find nothing; every retraction is filed alongside the original claim. The almanac is not a website that announces results. It is the notebook itself, kept where anyone can read along, attack the method, or write in with a question we could not have asked ourselves. The decision to publish at all, before there is much to publish, is a deliberate one: it makes the lab harder on itself than any reviewer could be, and it gives readers a record that is honest by construction.
There is an additional, smaller reason. A lab of one person, working from a small principality, on three axes that include subjects most institutions consider unserious, has nothing to gain from the journal-article posture and a great deal to lose. Publishing in the open is the only way the work becomes legible to people outside the room. It is also the only way the room stays honest about what is actually happening inside it.
Primary references.
The note above is an introduction to existing research, not a Runaric finding. The references below are the primary sources a reader can go check.
- Ref · 01
Bradley, J.-C. (26 September 2006). Open notebook science. Drexel COAS E-Learning Blog.
- Ref · 02
Open Science Collaboration (2015). Estimating the reproducibility of psychological science. Science 349, aac4716.
- Ref · 03
Faraday, M. (1932 to 1936). Faraday's Diary, 7 vols., ed. Thomas Martin. G. Bell & Sons.
- Ref · 04
Nosek, B. A., Ebersole, C. R., DeHaven, A. C. & Mellor, D. T. (2018). The preregistration revolution. Proceedings of the National Academy of Sciences 115, 2600 to 2606.