Stanford University
Implementing a Curriculum Vitae Addendum that Recognizes Open Scholarship Outputs
Stanford University’s leadership demonstrated their commitment to research integrity and transparency by framing the importance of faculty contributions to open scholarship, data, software, and code within their school’s evaluation criteria. Specifically, Stanford University School of Medicine’s associate dean of clinical and translational research and professor of Epidemiology and Population Health, Medicine, and Health Policy, Steven Goodman, and his team developed an optional Curriculum Vitae (CV) template. The template was designed by Mario Malički, associate director of Stanford Program on Research Rigor and Reproducibility (SPORR) and Zach Chandler, director of open scholarship strategy at the Center for Open and Reproducible Science at Stanford (CORES), to allow faculty to highlight their use of open practices designed to make their research reliable, robust, and transparent.
Open scholarship can improve research integrity: Stanford’s approach to faculty buy-in
At a recent HELIOS Open leadership meeting, Goodman (also the founder and director of SPORR) shared that by prioritizing research integrity (i.e. the integrity of the research, as opposed to the researcher), his implementation team found an effective way to incorporate faculty open scholarship contributions at the Stanford University School of Medicine.
Goodman described how higher education has grappled with various issues related to scientific integrity and research misconduct, including retractions linked to problems with data handling. This recently happened to a Nobel laureate on the Stanford faculty, who in response, instituted and made public a series of policy changes in his lab to improve transparency, use of virtual notebooks, internal replication, and review of data in lab meetings. Goodman indicated that these kinds of steps can help to solve research reliability problems with a systems approach.
The importance of administrator support and flexible approaches to institutional change
Goodman credits Linda Boxer, vice-dean of the Stanford School of Medicine, for endorsing the prototype and enabling their team’s work in creating the CV Addendum. Boxer’s support allowed the team to move forward with not only developing the template, but socializing it with faculty on campus in the fall of 2024.
Boxer insisted the template be optional – candidates would not be required to use it – but that it should also be highly visible. The Office of Academic Affairs website posted details on how to organize a CV and candidate statement, along with the Optional Guidelines - Detailed CV (Professoriate) with Rigor and Reproducibility Items and Optional - Sample CV (Professoriate) with Rigor and Reproducibility Items.
The school was intentional about making these items open and available for anyone to download. In the sample CV, candidates can include optional rigor and reproducibility (R&R) items with check marks and hyperlinks. Individuals can indicate if they have shared analysis plans, data or code availability, presentation materials, videos and publicly accessible articles – anything where someone can get a copy of the final manuscript, which includes PubMed Central. The team designed the template to be easy to use, with a light touch and simple format to give the opportunity for faculty to list any elements of open scholarship they would like to highlight.
Moving away from citation metrics in the Candidate’s Statement
The Stanford materials explicitly state that the School of Medicine does not use citation metrics or impact factors, asking candidates not to include them in their application. Goodman acknowledged that reviewers often still use them, but the team still felt it was important to explicitly include a statement about not using bibliometrics in internal guidelines.
The Candidate’s Statement form also includes new guidance on how to incorporate rigor and reproducibility practices as a companion to the optional CV format. It lists items that can be put into the personal statement, like use of blinding, masking, randomization, internal replication, reproducibility checks, metadata repositories, research projects with version control and time stamps, and data management standards within teams. The list includes:
Use and sharing of study protocols with analysis plans, code, data and meta-data.
Documented re-use of shared research products.
Use of data management practices and tools in the research group to assure version control and trackable data curation from point of acquisition to published research outputs (e.g. images, figures and tables).
Use of research methods or designs to maximize internal validity, e.g. detailed protocols, blinding, randomization, sample size calculations, independent replications and validation.
Use of validation for biological materials and reagents.
Use of independent code/data/figure/table reproducibility checks.
Registration and results reporting practices of all clinical research.
Use of a lab manual and orientation practices to teach R&R practices, tools, philosophy and expectations of all lab members.
How R&R practices are implemented in multi-team collaborations.
Development and/or dissemination of R&R tools and practices to others.
Recognitions for R&R practices received.
Goodman ensured this information is publicly accessible, and at the discretion of the candidate if they want them to be visible and valued.
Socializing open scholarship evaluation criteria with faculty
Goodman stated that Stanford is just starting to socialize and implement these open scholarship templates. The team has plans to make presentations across campus and publish articles to bring attention to the effort. The team also developed talking points for assistant deans and department chairs in explaining to faculty the motivation for these new elements:
“This optional rigor and reproducibility CV reflects the School of Medicine’s commitment to evaluating and rewarding its faculty, not just on its research products (i.e. mainly publications), whose ultimate validity can be hard to assess in the short term, but also on the reliability of its research methods.
For faculty who use extra time and resources to ensure the validity of published claims and to make their research transparent, these components provide a means for the institution to recognize and reward those efforts. Without such recognition, the dominant incentives are for more and faster publications.
As a backdrop, there is a list of the value of rigor and reproducibility practices to biomedical research. It supports evidence-based practice that leads to better patient care and improved health outcomes, minimizes bias, increases research efficiency, expediting progress by allowing other researchers to build on existing studies with confidence, use their methods, data or code, and promoting public trust.
Researchers who apply these principles of practice in their work demonstrate a willingness for their work to be scrutinized closely by the scientific community, community and confidence that will sustain that scrutiny.
Scientific integrity alignment with institutional priorities. It’s a way to provide evidence of impactful work because it can be more reliably built on and enhances leadership because it enables faculty to more effectively supervise.
Finally, it attracts funding.”
Goodman concluded by sharing that Stanford School of Medicine recently created a Dean's Task Force on Research Practice and Culture to evaluate research training and support, data management and oversight.
