Notebooks as the Published Paper, How Ready are Scientific Communities?

Notebooks as the Published Paper How Ready are Scientific Communities? Kenton McHenry Associate Director of Software National Center for Supercomputing Applications University of Illinois Urbana-Champaign 7585f3da-7235-4dee-99e5-46d1635cabed.MP4 Resource Registry Workbench Sustainability Software as a Scholarly Object Notebooks Incentivize the community! •Drew Camron (UNIDATA) •Julien Chastang (UNIDATA) •Jeff Dozier (UCSB) •David Fulker (OPeNDAP) •Ryan Gooch (NREL) •Joseph Hardin (PNNL) •Keith Maull (NCAR) •Kenton McHenry (NCSA) •Chris Olson (UCSD) •Stephen Richard (U.S. Geoscience Information Network) •Bradley Spitzbart (Stony Brook University) •Lisa Tauxe (Scripps Institution of Oceanography) •Carol Willing (Project Jupyter) •Ilya Zaslavsky (UCSD) CfN’20-22 •https://github.com/earthcube2020 https://www.essoar.org •81 submissions –Largley Jupyter –2-3 reviewers per notebook –41 accepted –Published through Github/Jupyter Book/AGU ESSOAr system •Insights –Runnability –Science Tool focused vs Science focused submissions –Demographic changed, from researchers to students/early career https://earthcube2021.github.io/ec21_book Reviews Giglio, Donata; Carter, Shay, R.; Chastang, Julien, C; McHenry, Kenton; Schreiber, Lynne; Zaslavsky, Ilya (2022). Evaluation Rubric for Reviewing Standardized Notebooks. In EarthCube Organization Materials. UC San Diego Library Digital Collections. https://doi.org/10.6075/J0V40VCV Style Templates https://github.com/earthcube/NotebookTemplat es Notebooks as Scholarly Objects https://data.agu.org/notebooks-now/ •Tech-leaning roles and disciplines (e.g., RSEs, HPC, data science) •Reported the highest frequency of notebook use and creation •Viewed notebooks as valuable tools for reproducibility, workflow sharing, and education •However, they also voiced practical skepticism, calling notebooks “poor for building on top of” or “unreliable” compared with tested software packages •Common refrain: notebooks are great for demonstration, not software engineering Domain and Technical Culture Effects •Non-technical or traditional domains •Expressed less experience and more hesitation, emphasizing the need for context, reproducibility, and sustained access before treating notebooks as equivalent to papers •Many felt they could supplement but not replace traditional publications Domain and Technical Culture Effects •Among faculty (n = 14): •6 said “Yes,” 2 “No,” 6 “Maybe” to whether peer-reviewed notebooks should count as equivalent to papers •Comments centered on context, reusability, and reproducibility as determining factors •Several saw value if the notebook can stand alone scientifically or complement a paper, others felt it should remain supplementary •Among students •Most believed advisors/employers would not yet treat notebooks as equal to papers, suggesting institutional inertia remains a major barrier Faculty and Hiring Perceptions •Perceived added value of peer review: •~85 % saw moderate to great added value once a notebook is peer reviewed •Importance of peer review for reuse: •Most rated this as moderately important, not decisive •Motivations to publish notebooks: •41 → to allow others to reuse work/software •34 → to increase exposure •14 → to strengthen CVs •7 → promotion → Shows a focus on visibility and reuse, rather than formal credit systems Attitudes Toward Peer Review and Reuse •Long-term executability and sustainability •Dependencies, environment drift •Quality and reusability of notebook code •Poor structure, copy-paste culture •Reviewer burden •Difficulty of verifying code execution or handling large data. •Cultural acceptance •Lack of consensus among committees, funders, and journals. •Maturity gap •Some domains (e.g., education, Fortran-based modeling) remain ill-suited for notebooks. Common Barriers and Concerns Over 70 % of all respondents expressed willingness to support notebook adoption efforts, as authors, reviewers, organizers, or advisors •ECRs •Ready and eager •View notebooks as authentic research communication •Senior/Traditional researchers: •Cautious •recognize potential but question implementation •Tech fields: •Operationally ready but seek clearer review and sustainability standards •Non-technical fields: •Culturally and technically less ready Community Readiness and Next Steps •Generational and disciplinary divides are real •ECRs and tech domains are the most receptive. •Perceived legitimacy hinges on reproducibility, sustainability, and clear peerreview guidelines. •Institutional recognition (promotion, hiring) remains a major bottleneck. •Widespread willingness exists to participate in shaping the path forward suggesting a fertile moment for pilot notebook-publication programs and rubricbased peer review. In summary, the community is cautiously optimistic and partially ready. Younger, digitally fluent scientists are driving momentum, established structures must now evolve to legitimize and sustain notebook-based scholarship. Conclusion MONEY!!! Fun with AI: How does this transition different from the paper to digital transition? Fun with AI: How does this transition different from the paper to digital transition? •Similarities •Cultural, Infrastructure, Workflow disruption, Fragility, Review logisitcs, Lack of tooling, RISK!!! •The monetary incentive for going digital •Cheaper than printing and shipping paper! •Opportunities for licensing •Machine readable •The non-monetary incentives for going to notebooks •FAIR, yes, but not monetary •Compliance and funder alignment? •The monetary incentive for going to notebooks?? •Premium interactivity & data-as-a-service? •Platform licensing •Community and brand differentiation cc4abab5-1083-47de-916a-1aa82fe97330.MP4 Questions?