Defining Open Source AI: Current Conversations within the Academic Community
Defining Open Source AI: Current Conversations within the Academic Community
The Open Source Initiative (OSI), a California public benefit corporation and not-for-profit community of technology experts, recently published the Open Source AI Definition – draft v. 0.0.9. The definition centers on the importance of AI systems that can be used, modified, and shared for any purpose, and studied and inspected transparently.
HELIOS Open consulted its Advisory Committee about the definition, hearing from committee and OSI board member, Sayeed Choudhury, at the September meeting. Choudhury shared information about the process OSI led to define open source AI (*see below for specific demographic and community participation information).
While the Mozilla Foundation recently endorsed OSI’s community-led definition, asserting it “is critical not just for redefining what “open source” means in the context of AI; it’s about shaping the future of the technology and its impact on society,” the definition’s release and nature of development is not without its critics: Meta, e.g., has developed its own open source AI definition, based on its Llama model. According to Choudhury, “it should be noted that Meta’s definition does not comply with the Open Source AI Definition. Fundamentally, the question relates to whether any single company or entity should be able to define something which will have such a profound impact on society.”
HELIOS Open, by design, does not endorse projects or initiatives, and individuals in the group were split on their support for both the initial definition and the process for developing it; however, one important consensus takeaway emerged:
We, as a community of higher education representatives, agree that one, single commercial entity should not define open source AI behind closed doors. In general, we should contribute to inclusive community-led efforts that center ethical AI values like openness/transparency and trust.
HELIOS Open Advisory Committee Recommendations and Considerations
Upon initial consideration of the definition, the group discussed recommendations, including improving the language around training data and its openness, which was developed with data privacy, copyright, and practicality considerations in mind. Stephen Jacobs, HELIOS Open Advisory Committee member and Director of Open@RIT, shared “if the definition doesn’t start by emphasizing the openness of training data out of the gate, I worry it will not get added in later. Over time negotiation tends to narrow, not broaden definitions.” OSI has shared that the definition and work is evolving, and conversation ensued about the differences between open source software and open source AI, including nuances with AI model code and training data and whether or not it is even possible to have full transparency into how an AI system is trained.
Some members believe that the OSI definition offers a standard to consider as the definition evolves. Future changes should be prompted by evaluating the definition against models, tracking changes in our understanding of the engineering aspects of these models, and (hopefully) assessing the merits of building and using publicly available data. Future conversations might also generate better understanding of issues related to reproducibility and replicability to better support our understanding of open source AI and scientific integrity.
HELIOS Open and Incentivizing Open Work
AI is an emerging priority area for higher education leaders. In discussions about the core components of an AI system, the HELIOS Open Advisory Committee reflected on the importance of AI systems that are also trustworthy, inclusive, and transparent. Tying directly to HELIOS Open’s goals to ensure campuses are incentivizing open scholarship practices in service of a more equitable and trustworthy research ecosystem, the group also validated the need to incentivize open work within academia like contributions to open source; individuals sharing research outputs openly; and individuals developing open curriculum or educational materials; among other activities.
HELIOS Open will be watching this space to better understand how open source AI might play a role in the academic setting.
Why Values and Ethics in AI Matter to Campus Leaders
As our research enterprise has become increasingly invested in AI, we emphasize the importance of AI systems that prioritize ethical AI considerations and values. Campus leaders are not only creating new positions for faculty, staff, and postdocs focused on AI, but also leveraging AI tools to maximize student outcomes and to promote research impact and efficiency. A definition can start a conversation and exploration, and one that is community-derived and values-aligned can help with future evaluation exercises.
Within academic institutions, libraries on campus are grappling with publisher AI restrictions that seek to limit institutions’ ability to use AI on publisher content. Many of the same companies are rapidly integrating AI into their own systems, often in ways that are not fully transparent and sometimes with limited notice. Particularly without transparency, vendors’ use of AI could pose new concerns, as we noted with regard to institutional data such as faculty activity reports in our analysis of the Barcelona Declaration. Campuses must be able to trust the systems that they rely on—regardless of whether they are commercial or non-commercial—and the implementation of AI in these systems should ensure that institutions’ interests are protected. Pursuing open source AI in research systems can be an important step in addressing these issues.
The federal government, which annually funds approximately $90 billion in academic research, is also invested in trustworthy AI systems. In May, the National Science Foundation's (NSF) pilot of the National AI Research Resource (NAIRR) announced its first round of awards to 35 projects and opened the application process for the second round of awards. The NAIRR pilot was established by President Biden's Executive Order on the Safe, Secure and Trustworthy Development and Use of AI and aims to democratize access to AI tools for all communities, specifically researchers at institutions of higher education, students, and small businesses.
HELIOS Open values community efforts that support openness and transparency for public benefit. Open source has long been an important component of digital infrastructure. Of all software, recent estimates suggest that 96% contain open source software. A Harvard Business School study noted that open source software has generated $8.8 trillion of value on the demand side, and reduced production costs on the supply side by a factor of 3.5. A community-based, consensus open source AI definition could yield even more benefits for universities and society at large, so we will eagerly watch the community process unfold and continue urging higher education leaders to incentivize open work on campus.
—-
*A note from OSI board member on OSI’s process for defining open source AI: The OSAID process serves as a reference model for diverse AI-related community engagement. OSI facilitated a dozen town halls and working group meetings; established a public forum for discussion and debate; provided presentations and opportunities for feedback at three dozen public events on five continents; 36 co-design working group/systems review volunteers representing 23 countries by birth or residence.
Of the working group members more than half (53%) are People of Color, more than one-third are Black, and slightly less (31%) are both Black and Indigenous. Women and femmes, including transgender women, account for 28% of the total and 63% of those individuals are Women of Color. Of this same volunteer group, nearly 30% of active co-designers are from academia, either as faculty or grad students.
Introducing Danny Anderson, President Emeritus, Trinity University, and Special Advisor to HELIOS Open
Throughout his academic career, Danny Anderson has recognized the value and promise of open scholarship. He joins HELIOS Open as Special Advisor and Executive Council Chair, leading a group of presidents and provosts invested in guiding HELIOS Open’s work to support incentives reform to explicitly reward open science and scholarship.
Biography: Anderson earned his bachelor of art’s degree in Spanish from Austin College in Sherman, Texas, and his master’s and doctorate in Spanish at the University of Kansas (KU). He started at the University of Texas at Austin (UT) as an assistant professor of Spanish and Portuguese. He went on to KU where he was a faculty member for 26 years, and dean of the College of Liberal Arts for five. Anderson served as president of Trinity University in San Antonio, Texas, from 2015-2022. Since retiring to Redlands, California, he has been working as an executive coach.
How did you become interested in open scholarship?
I spent most of my career as a faculty member at the University of Kansas (KU) in the field of Mexican literature. As a faculty member, I became aware of skyrocketing subscription costs and how they can erode library budgets. Our provost, David Shulenburger, was actively raising awareness and seeking solutions, and I got involved. Open access publishing was just starting. I joined early conversations about ensuring scholarly outputs disseminated through open access practices can be seen as of the same quality as traditional subscription and paywalled publications.
Since those initial faculty experiences, I have held various leadership roles that expanded my perspective on the importance of open access for faculty and research, for the needs of different disciplines, and for reclaiming faculty ownership of their research products. While I served as vice provost, I regularly attended faculty senate meetings. I recall a key meeting where faculty introduced and debated a new open access policy. During deliberation, strong emotions drove the conversation, including fear of change and misperceptions of open access publications. The motion was about to be defeated, but we began to discuss the processes and criteria scholars use to evaluate quality. Faculty representatives felt secure that they could trust the evaluation process they had created, and this changed the tide of the conversation. The faculty senate decided to take a chance and the policy was approved.
“The open access policy conversation reminded faculty that, as a community, we decide what is high quality; the outlet does not. ”
It was a galvanizing moment that pulled our faculty together to think about our research resources and values. We considered how we were standing up for the quality of research at our institution. Approving this open access policy gave the institution and its faculty a new way to focus on the assessment of research. KU was the first public research university to develop an open access policy, following similar policies at private schools – Harvard, MIT, and the Stanford School of Education.
Working collaboratively with champions was also valuable and deepened my interest in open scholarship. I worked with Lorraine Haricombe, then director of university libraries, at KU, and now the vice provost and director of UT libraries in Austin, Texas. I learned from her the different ways we can broaden the sense of what openness means to a library and the entire campus.
As president of Trinity University, I was pleased to continue participating in the broader effort to support and encourage sharing scholarship openly. SPARC invited me to speak at a conference about open access. My goal was to help librarians think about the concerns of presidents and provosts to help them refine their talking points as they sought greater participation from campus leaders in the open access movement. In 2019, I served on the National Academies Roundtable for Aligning Incentives for Open Scholarship. It’s one task to advocate for open access to support library budgets; it’s another challenge to figure out a process for incentivizing open scholarship behaviors among faculty and scholars, including sharing research outputs openly. My awareness deepened as I examined the trends in open scholarship, and pondered the need for significant change in faculty promotion and tenure policies. There are emerging research products that don’t fit the romantic concepts of individual originality. Promotion and tenure policies have to catch up in order to recognize and reward scholars who are changing their fields with new paradigms of practice for example, data sharing, sharing articles in open repositories, preprinting, etc. The change may seem daunting, but through shared governance processes, we wrote the rules. Through the same processes, we can change the rules to fit our current or changing circumstances.
It’s exciting to see how at some institutions the faculty senate, an influential dean, or an insightful department chair might be leading the way. At other institutions, the presidents and provosts have become champions.
Open scholarship is crucial for faculty careers, collaboration, and competition for grants. Forward-thinking institutions are adapting their tenure and promotion policies to encourage cutting-edge research and scholarship. Today, through HELIOS Open member institutions, I’m working with presidents, provosts, and senior research officers who are interested in taking action to support and develop incentives for open science and scholarship activities for faculty on their campuses.
What works in getting traction and support for open scholarship?
As an administrator, our function is to serve our faculty colleagues and help them be successful at their job. This means you have to think about their needs and motivations.
Open science practices, like making data and papers accessible and transparent, are among the ways that early career scholars can ensure the quality of their research in an era of concern about reproducibility. Sharing openly enables other individuals to reproduce an individual’s work, validating findings in new and rigorous ways. Because such validation is essential for the impact of their work, it helps them become better scientists. When data is shared more openly, it can speed up their work, and enable more interactions with the research.
Second, many early career researchers experience a dissonance between what they consider cutting edge research and what they hear on their campus about tenure and promotion expectations from senior colleagues.
“There is a tension between originality based in individualism and openness based in community.”
Our colleagues who work in open scholarship want individual recognition and reward for their contributions, which is why it’s important to think about how open science is written into promotion and tenure standards for the individual as well as for collaborative or team efforts. And the same faculty members also want scholarship impact in the world. They want their work to be real, and to be out there, accessible, and doing things that improve society. Open is a modality that creates a greater sense of reliability for the scientific endeavor.
Faculty members today often experience negative pressure given the public questioning of higher education’s value. Open science enables transparency into the scholarly activities on a campus, and can give a public view into its work. It is my hope that this kind of thorough transparency can demonstrate higher education’s value and combat the misperception that higher education creates a secret or elite group of insiders. Similarly, federal agencies fund a great deal of this research, and agencies are also embracing the ethos of open scholarship because it improves the quality of research and ensures public access to the research products the public funds.
Describe your work with HELIOS Open and how it has evolved.
HELIOS Open emerged as a formal organization in 2022, just as I was retiring. I was one of the National Academies Roundtable presidents who cosigned the call to action letter that led to the creation of HELIOS Open. I was contacted later to facilitate a NASA-funded meeting of presidents and provosts in January 2024. We gathered to discuss their role in modernizing tenure, promotion, review, and hiring to explicitly reward open science and scholarship activities. My role now is as a special advisor seeking to engage senior leaders and build our Executive Council of presidents and provosts guiding the work. This is a key moment, aligned with major policy changes, to think about how to effectively communicate, listen, and keep engagement strong.
What kind of feedback are you getting from administrators about advancing open practices?
I have seen presidents and provosts energized as they grasp the impact of open scholarship. They become engaged and excited partners with campus champions for open scholarship. For example, the provost at the University of Vermont, Patty Prelock, is energized about open scholarship because the faculty senate issued a resolution calling for the university to develop open science and open data policies and practices to facilitate their work. We look for those bright spots that are giving us models for how things can evolve. We want to share those stories, while respecting campus differences and local context. We want to encourage leaders to look at a model and adapt institutional change tactics to fit their institution.
We’ve also heard from and amplified department leaders who have had success. The University of Maryland Department of Psychology is an example of how to update promotion and tenure standards with open practices in mind. Michael Dougherty has shared the steps he took alongside his faculty to update hiring and advancement policies. He shares his story as a template that others are invited to use or adapt. The examples of individual champions having an impact on their campus community are fundamental for the change process underway at present.
Why do campus leaders need to take a position on open scholarship?
Campus leaders are most effective when they know how to be champions for ensuring that their faculty are effective – in the classroom, in the laboratory, in their research, and in their creative endeavors. Open science and open data are there to serve faculty, to help them be more effective and efficient.
Each president and provost has an opportunity to understand their institution’s unique story and help faculty stand out with the impact of their research, creative endeavors, and scholarship.
Presidents and provosts are effective in thinking about policy and taking action when there is energy emerging from the faculty to make a change. Campus leaders need to be ready to help guide energy and passion toward goals that are meaningful to ensure that open science achieves the goal of having better, faster, and transparent results.
What are you most excited about for the future?
I’m excited by the deep conversations currently underway to ensure open scholarship practices flourish and that faculty will be recognized and rewarded for their important contributions. This effort began originally in libraries and with individual researchers. Today, there is a robust conversation that includes grant officers from federal funding agencies and private philanthropies as organizations consider how to invest in research and scholarship that make a difference in society. Universities are engaged in this conversation at a time when we are often feeling like divisions can create barriers to working together.
Through my work with HELIOS Open and its 100+ institutional members, I see individuals across so many different organizations finding ways to form truly meaningful partnerships. That is the kind of collaboration that will help us strengthen the quality of scholarship and its impact on our world.
HELIOS Open Advisory Council Applauds Barcelona Declaration on Research Information
The Barcelona Declaration on Research Information, released April 16, 2024, calls for the “openness of information about the conduct and communication of research,” and asserts that this openness should be the norm. The signatories of the Barcelona Declaration on Open Research Information commit to taking a lead in transforming the way research information is used and produced. Signatories include funders and higher education institutions including the Gates Foundation and the Coimbra Group, which represents 40 European universities. [Click for a full list of signatories.]
The Declaration identifies a core contradiction in the research ecosystem:
“...a large share of all research information is locked inside proprietary infrastructures. It is managed by companies that are accountable primarily to their shareholders, not to the research community. As a research community, we have become strongly reliant on closed infrastructures. We have ended up assessing researchers and institutions based on non-transparent evidence. We are monitoring and incentivizing open science using closed data. We are also routinely making decisions based on information that is biased against less privileged languages, geographical regions, and research agendas. To advance responsible research assessment and open science and to promote unbiased high-quality decision making, there is an urgent need to make research information openly available through open scholarly infrastructures. Openness of research information must be the new norm.”
The HELIOS Open Member Advisory Committee applauds the Barcelona Declaration’s organizers and commitment to transparency and openness of research information. Open research information is a human and global endeavor, sometimes contained within national contexts and frameworks. Global open scholarship efforts can inform and enhance one another.
Below, we articulate ways in which the work of our 103 members, comprising major research-producing institutions across the United States, mirrors opportunities presented in the declaration, in order to help our members find their own points of alignment.
1. Make openness the default for the research information used and produced
We welcome the Barcelona Declaration’s core objective to ensure that openness is the norm for the research information we use and produce, for instance to assess researchers and institutions, to support strategic decision making, and to find relevant research outputs.
In our work to advance open scholarship incentives across over 100 HELIOS Open campuses, we advocate for academic institutions to explicitly account for openness within researcher and institution evaluations and infrastructure. In January 2024, we gathered at Florida International University at a NASA-funded workshop with over 50 presidents, provosts, and vice presidents for research and faculty to discuss how we can update our hiring, tenure, promotion, and review policies to explicitly reward open science and scholarship. We note the declaration’s push to ensure assessment and evaluation is built on open and transparent data and will support HELIOS Open leaders taking action to modernize research evaluation with this in mind.
*Open research information advances the goals underlying HELIOS Open’s efforts to modernize research assessment. Given the increasing use of technologies such as artificial intelligence (AI) across research activities, it is now much easier for vendors to convert institutional data (such as faculty activity reports) into research information that can then be monetized in ways that may not align with the best interests of faculty or institutions themselves. Making open research information central to evaluation can mitigate emerging types of institutional risk.
2. Work with services and systems that support and enable open research information
The Barcelona Declaration requires that for external and internal publishing systems and platforms 1) research information generated in publication processes (e.g., metadata of research articles and other outputs) be made openly available through open scholarly infrastructures 2) relevant research information can be exported and made open. This process should use standard protocols and identifiers where available.
Campuses across the United States are working hard to prepare for implementation of new federal public access policies that require all federally funded papers and associated data to be made immediately available upon publication. The guidance calls for agencies to share publication metadata, including funding information, and to require the use of persistent identifiers (PIDs).
*This federal policy can serve as a catalyst for institutions to develop policies regarding PIDs and open research information, regardless of whether scholarship is federally funded or not. HELIOS Open can support campuses in considering which infrastructure to adopt and what internal policies to create that advance the use of open research information.
3. Support the sustainability of infrastructures for open research information
The Barcelona Declaration advocates for good practices in community building and community governance, including providing fair and equitable contributions to the financial stability and development of research information infrastructures.
HELIOS Open’s Shared Open Infrastructure Working Group developed the “Scholarly Communication Infrastructure Guide: Buy, Build, or Partner” to support college and university leadership in making informed infrastructure decisions. The working group affirmed UNESCO’s vision that "Open science infrastructures should be organized and financed on a primarily not-for-profit and long-term vision, which enhance open science practices and guarantee permanent and unrestricted access to all, to the largest extent possible." In refining the key decision-making considerations, the group included governance as a key factor to support transparency and facilitate examination and prioritization of sustainable operations and community input.
*As research information infrastructure is extremely costly for campuses (in terms of time, money, and morale), campuses should carefully consider the alignment of their interests with those of research information system providers—and take proactive steps where necessary to ensure this alignment.
4. Support collective action to accelerate the transition to openness of research information
The Barcelona Declaration highlights the importance of sharing experiences and coordinating action to promote a system-wide transition from closed to open research information. Collective action is at the heart of HELIOS Open’s approach to supporting change among our 103 campus members.
*We will surface opportunities for our members to learn from and contribute to these emerging efforts to make open research information the default, including engagement with relevant organizations, such as the Coalition for Advancing Research Assessment (CoARA), the International Network of Research Management Societies (INORMS), and the Barcelona Declaration’s emerging joint roadmap for open research information (see https://barcelona-declaration.org/conference/).
The HELIOS Open Member Advisory Council urges campuses to ensure that research information serves institutional goals rather than constraining them.
The commitments articulated in the Barcelona Declaration provide a compelling blueprint for campus action. As each stage of the research lifecycle has moved online, vendors’ ability to collect and monetize information about institutions and faculty has expanded significantly. Campus engagement now can help avoid the use of institutional information by vendors for unintended and potentially adverse commercial purposes—harming campus culture through feelings of distrust and unproductive competitive pressure. Early proactive engagement by institutions can ensure that the collection and use of research information aligns with campus priorities and best interests, now and over the long term.
Further Reading:
Authors
HELIOS Open Advisory Council Members: Jim O'Donnell (Arizona State University), Sayeed Choudhury (Carnegie Mellon University), Dustin Fife (Colorado College), Tim McGeary (Duke University), Rita Teutonico (Florida International University), Camille Thomas (Florida State University), Caitlin Carter (HELIOS Open), Danny Anderson (HELIOS Open), Geeta Swamy (HELIOS Open and Duke University), Elisabeth Long (Johns Hopkins University), Chris Bourg (Massachusetts Institute of Technology), Kristi Holmes (Northwestern University), Athena Jackson (University of California, Los Angeles), Santi Thompson (University of Houston), Mairéad Martin (University of Illinois), Cynthia Logsdon (University of Louisville), Michael R Dougherty (University of Maryland), Lakeisha Harris (University of Maryland, Eastern Shore), Alicia Salaz (University of Oregon), Stephen Jacobs (Rochester Institute of Technology), Zach Chandler (Stanford University)
SPARC: Nick Shockey (SPARC)
HELIOS Open 2024 Programming and Structures
Vision: A community of U.S. higher education leaders engaged in collective action to advance open scholarship recognition, rewards, and resources to support open practices by default.
Values: We are working together to promote a more transparent, inclusive, equitable, and trustworthy research ecosystem.
High-Level Overview of 2024 Programming and Structures
HELIOS Open Community Calls every 6-8 weeks with the entire community.
These are opportunities to engage the entirety of the HELIOS Open network 1) to identify open scholarship focus areas that motivate and animate institutions, 2) to offer a medium for institutions with like interests to connect, and 3) to provide a path forward for identifying support structures to facilitate member collaborations that advance HELIOS Open goals.
Member Representative Advisory Committee* with shifting membership, annually. The goal is to have a community of HELIOS Open ambassadors that can advise and identify community-led opportunities for the HELIOS Open members to advance work, and then develop tangible next steps including milestones and deliverables for any proposed projects.
Working Groups and Affinity Groups will shift to mission-focused small group work that can form, storm, accomplish, and then return back into the larger overall community group. These will be community-led.
Quarterly dialogues between federal agencies and HELIOS Open leadership in the research office to establish direct communication channels with the federal agencies as they implement public access plans and as colleges and universities are thinking through compliance, infrastructure, and funding needs related to open scholarship and public access to research.
Senior Leadership Executive Council** This group will identify and advise on higher education’s open scholarship opportunities and assess and provide feedback on HELIOS Open work as it supports the National Academies Roundtable on Aligning Incentives for Open Scholarship and leadership goals.
Updated September 30, 2024, with lists of committee and council members:
*HELIOS Open Advisory Committee Members: Jim O'Donnell (Arizona State University), Sayeed Choudhury (Carnegie Mellon University), Dustin Fife (Colorado College), Tim McGeary (Duke University), Rita Teutonico (Florida International University), Camille Thomas (Florida State University), Caitlin Carter (HELIOS Open), Danny Anderson (HELIOS Open), Geeta Swamy (HELIOS Open and Duke University), Elisabeth Long (Johns Hopkins University), Chris Bourg (Massachusetts Institute of Technology), Kristi Holmes (Northwestern University), Athena Jackson (University of California, Los Angeles), Santi Thompson (University of Houston), Mairéad Martin (University of Illinois), Cynthia Logsdon (University of Louisville), Michael R Dougherty (University of Maryland), Lakeisha Harris (University of Maryland, Eastern Shore), Alicia Salaz (University of Oregon), Stephen Jacobs (Rochester Institute of Technology), Zach Chandler (Stanford University)
**HELIOS Open Executive Council Members: Bárbara González (Northern Illinois University), Patty Prelock (University of Vermont), Emily Chan (Colorado College), Susan Rundell-Singer (St. Olaf College), Greg Ball (University of Maryland), Fred Nafukho (University of Washington), Catherine Lucey (University of California San Francisco), Anthony Wutoh (Howard University)
Higher Education Leaders Convene to Explore Modernizing Hiring, Review, Promotion and Tenure to Explicitly Reward Open Scholarship
Fifty high-level college and university decision makers met on January 10 and 11 to discuss what they could do collectively to encourage open practices by changing incentive structures on campus. They collaborated at a workshop funded by NASA and hosted by Florida International University, in collaboration with HELIOS Open. The presidents, provosts, and vice provosts for faculty and research shared ideas about strategies for rewarding researchers’ open scholarship activities in structures and processes including tenure, promotion, review, and hiring. Many attendees agreed that updating tenure and promotion is one of the hardest institutional tasks, but that culture change is needed to achieve a more transparent, inclusive, and open incentives system.
Danny Anderson, President Emeritus, Trinity University, facilitated the meeting. After sharing remarks from HELIOS Open leadership, including Geeta Swamy, and recorded remarks from presidents Ron Daniels and Michael Crow, Anderson shared information about national and international efforts that advance open scholarship incentives, including the National Academies Roundtable on Aligning Incentives for Open Scholarship. Dr. Anderson also shared existing resources that could support leaders interested in taking action to enable change at their institutions. Recognizing how difficult it can be to enact new policies and navigate shared governance, he described useful models for complex change. Anderson encouraged campus leaders to be intentional, rather than reactive, and focus on the desired outcome when crafting their action plans.
Anderson asked attendees to form small groups to address the following questions related to changing processes and procedures to advance open scholarship incentives:
Where are you as academic leaders putting your attention– on problems that make you reactive or on outcomes you can achieve?
How are you relating to your campus? Are you taking on a reactive mindset or an outcomes mindset?
What actions are you taking based on your mindset?
Attendees received a handout with potential campus leadership actions suggested by the faculty and staff involved in HELIOS Open.
When the group reconvened , attendees reflected on faculty perceptions, realities, and needs related to career advancement, personal and professional values leaders should uphold in their policies and procedures, and how open scholarship can support faculty in career advancement. Participants explored their role in enabling change and brainstormed opportunities to replace archaic incentive structures. One attendee asked, “if something is open and freely accessible, can we start to ask how faculty are using it in their work to see where the work has grown and how it has led to breakthroughs?”
Next, a panel of federal agency representatives joined both virtually and in-person to share their intersecting goals for inter-agency coordination efforts on open science and public access. Marking the anniversary of the federal Year of Open Science, and what it might look like to enter an era of open science, it was clear that higher education leaders and federal agency representatives are eager to continue conversations with one another to ensure scholarship is transparent, accessible, efficient, and secure.
By the end of the day, the attendees suggested resources they might need to take action at their institutions, and brainstormed next steps to support faculty’s open scholarship activities, including:
Talking points to get faculty at all levels on board with change
Information about what types of (open and accessible) scholarly products, potentially those that are non-traditional, should and could be assessed
Better instructions (narrative documents) for external reviewers, and memorandums of understanding between departments and faculty describing how scholars would like their open scholarship activities to count in career advancement
Tips for creating a better onboarding process - one that includes better representation of open scholarship values and practices
More opportunities to connect with campus leaders to develop tangible action plans
The HELIOS Open network will continue to propel collective action in developing open scholarship incentives and to enable change.
Updated Name for HELIOS Open
The Higher Education Leadership Initiative for Open Scholarship is sharing news about a tweak to our “short” name. Moving forward, we will be going by the “HELIOS Open” moniker, rather than simply “HELIOS”.
We want to ensure our organization and work is easily disambiguated from other organizations with similar names. The best solution is to tweak our name to create a bit more distinction between organizations.
Going forward, we will be referring to our initiative in shorthand as HELIOS Open rather than HELIOS, and we ask our community to do the same.
Spotlight Series Recap: Generative AI and Open
The Higher Education Leadership Initiative for Open Scholarship (HELIOS Open) hosted a Spotlight Series focused on generative artificial intelligence (AI) and open scholarship.
The World Economic Forum defines Generative AI as “a category of artificial intelligence algorithms that generate new outputs based on the data on they have been trained. Unlike traditional AI systems that are designed to recognize patterns and make predictions, generative AI creates new content in the form of images, text, audio, and more.” In 2023, generative AI has seemingly made the leap from a topic vaguely on our collective radars to everything everywhere all at once. ChatGPT has emerged as a tool for the masses, with questions raised about how peer reviewers, editors, and professors will be able to discern between human and machine generated papers. Geoffrey Hinton, the so-called godfather of AI, quit Google with a stark warning about the ethical implications of AI. A raft of headlines like “Why We're Worried about Generative AI” appeared in prominent publications like Scientific American. Intrepid analysis revealed that the training datasets for a number of AI models are surprisingly small, pulling from heterogeneous sources.
Higher education is becoming increasingly aware that these concepts are of direct relevance to their missions and their research, including their open scholarship priorities. Three experts in this emerging space shared their insights.
First, Dr. Katie Shilton, Associate Professor in the College of Information Studies at the University of Maryland, College Park, and co-PI of the new NIST-NSF Institute for Trustworthy AI in Law & Society (TRAILS), explored ethical considerations of AI. For a long time, data has been a primary input of scholarship. Open scholarship has focused on making that data findable, accessible, interoperable, and reusable (FAIR) for other scholars. For generative AI, data is an input, but so are other research products like journal articles. The challenge is that Generative AI itself is not scholarship, but a replication of the language of scholarship. Data and papers are no longer only used to make predictions or new discoveries, but to feed large language models (LLM).
This establishes unprecedented ethical questions:
What are the right non-scientific generative uses of open scholarship?
What values or virtues do we want publicly-funded scholarship to support? How do we measure and track whether that is happening?
What roles do scholars play in managing the use of their scholarship?
Dr. Shilton posited that scholars may want to adopt open licenses that restrict their scholarship from some generative uses. There may be a need to develop licenses that require AI dashboards or other explanatory features, and FAIR data documentation practices are increasingly essential to support AI transparency.
Open scholarship is on the vanguard of these issues because open scholarship advocates are already thinking about the ethics of data sharing and can inform emerging norms. The principles this community develops, could guide content creators and generators as scholars and institutions navigate their relationships with generative AI tools and companies.
The NIST-NSF TRAILS Institute will support these efforts.
Dr. Susan Aaronson is also co-PI of the TRAILS program, Research Professor of international affairs and Director of the Digital Trade and Data Governance Hub at George Washington University. Dr. Aaronson posed key questions at the start of her talk:
How did the firms or organizations creating LLM get their data to train their programs?
Did they follow internationally accepted rules regarding copyright and personal data protection?
Is the data accurate, complete, and representative of the world, its people and their cultures?
Have the firms considered interoperability, transparency, data sovereignty, and values?
Are there considerations for policymakers and activists related to the expropriation of resources (data) and paying additional rents to big tech companies located in the West and China?
Dr. Aaronson explored the governance and regulation concerns for policymakers, scholars, and the public. The markets for AI are growing rapidly, but China and the U.S. hold 94% of all AI funding, with 73% of generative AI firms based in the U.S. According to Aaronson, these firms often use open-source methods, but often rely on trade secrets to protect their algorithms and to control and reuse the data they analyze. Policymakers, researchers, and international bodies are starting to explore these challenges and opportunities.
Dr. Aaronson concluded with an exploration of how US and China’s competitiveness could threaten open practices like data sharing. There is a belief that openness empowers China, and that is worrying to some U.S. policymakers and companies; however, openness is a norm of science and can yield comparative advantage. Science cannot progress without data sharing and cooperation.
Dr. Molly Kleinman, Managing Director of the Science, Technology, and Public Policy program at the University of Michigan, shared thoughts on AI's potential implications for scientific research. She began by highlighting her main takeaway: generative AI is trained on the past, and can only reproduce the past. Because generative AI can only “know” what it has been trained on, it cannot make anything truly novel, which has implications for its use in the conduct of and evaluation of scientific research. It will be increasingly important to be able to evaluate what is generated from AI. Scientists and scientists in training should be educated early on what AI can and cannot do.
In scholarly communication, there are questions about AI and trust in science, trust on campuses between faculty and students, how AI will be used in peer review, along with concerns about rightful authorship, research evaluation in tenure and promotion, and intellectual property. One thing that generative AI is good at is boiling down large quantities of information into brief summaries, but it is not so good at citing its sources. Dr. Kleinman noted that these tools will privilege highly cited articles that may not represent the field’s diversity or most novel findings.
Using and feeding generative AI also risks reinforcing Western, especially Anglo-American, dominance in science. The more common generative AI becomes as a tool in science, the more it will continue to reinforce the English language as the language of science. It is reasonable to assume that already-marginalized scientists may worry their scholarship will be less visible in a world with global, popularized use of AI tools. The aforementioned issues may challenge open science and scholarship because of valid fears of exclusion and extraction.
Colleges and universities have a role to play in ensuring equity and inclusion are considered when AI is incorporated in scholarly processes like researcher evaluation and peer review.
Similarly, scholarly publishers can monetize AI tools for researchers based on publishers’ proprietary content, and sell that information back to institutions, or hold these tools hostage unless they subscribe to their journals. Given the current issue of monopolies in scientific publishing and a small number of for-profit publishers’ control over a large swath of scholarly content, this has financial and ethical implications for higher education.
We may see pressure on academic institutions to adopt and use generative AI to stay up to date with the trends, so Dr. Kleinman concluded by reemphasizing the importance of training scientists to think critically about, and recognize the limitations, of generative AI.
HELIOS Open Members Co-Author Research Software Policy Recommendations to Federal Agencies
A cohort of HELIOS Open member representatives have joined with other open source experts to author a PLOS Biology perspective, "Policy recommendations to ensure that research software is openly accessible and reusable". The piece provides policymaking guidance to federal agencies on leveraging research software to maximize research equity, transparency, and reproducibility. It makes the affirmative case that to accurately be able to replicate and reproduce results and build on shared data, we must not only have access to the data themselves, but also understand exactly how they were used and analyzed. To this end, federal agencies in the midst of developing their responses to the White House Office of Science and Technology Policy (OSTP) memorandum on “Ensuring Free, Immediate, and Equitable Access to Federally Funded Research” can and should ensure that research software is elevated as a core component of the scientific endeavor.
The perspective outlines seven policy recommendations, encompassing how research software should be shared, curated, maintained, secured, and redistributed. HELIOS Open thanks all of the co-authors for their contributions to this piece: Lorena Barba (George Washington University), Philip Bourne (University of Virginia), Caitlin Carter (ORFG), Zach Chandler (Stanford University), Sayeed Choudhury (Carnegie Mellon University), Stephen Jacobs (Rochester Institute of Technology), Daniel S. Katz (University of Illinois Urbana Champaign), Stephanie Lieggi (University of California Santa Cruz), Erin McKiernan (ORFG), Beth Plale (Indiana University, Research Data Alliance - US), Greg Tananbaum (ORFG).
Spotlight Series Recap: Incentivizing Open in Reappointment, Promotion, Tenure, and Hiring
On March 22, 2023 the Higher Education Leadership Initiative for Open Scholarship (HELIOS Open) convened academic leaders to discuss incentivizing open scholarship practices in hiring, reappointment, promotion, and tenure (RPT).
Erin McKiernan, ORFG’s community manager, is also a professor at the National Autonomous University of Mexico in Mexico City. She moderated the session and started by sharing how close this topic is to her personally and professionally. Over the last few years she has worked with the ScholComm Lab, analyzing promotion and tenure guidelines at over a 100 institutions across the US. She also recently went through the promotion and tenure process herself. Despite the positive outcome, the process highlighted key challenges within current assessment frameworks and inconsistencies in what institutions value in their reward structures.
Read more about the ScholComm Lab’s RPT project on their blog.
McKiernan framed the day’s conversation: “when we are talking about incentives within promotion, tenure, and hiring, what we're really talking about is what universities value, what they recognize, and whether they are the same things.” In McKiernan’s research, she and her co-authors have discovered that what gets rewarded in these policies is not what universities always state they value. University mission statements often talk about the importance of community and public engagement for the betterment of society. Open scholarship practices like making our work openly available by sharing data, code, notebooks, and all kinds of outputs allow individuals to engage with the work, collaborate, and build on the work. There are many public aspects of what faculty do in their day-to-day work, including openly disseminating scholarly outputs, but tenure and promotion guidelines at many universities do not adequately reward public engagement and outreach that open scholarship practices enable.
Specifically, in analyzing tenure and promotion policies, the word “public” is mostly mentioned in the “service” category, which is a traditionally undervalued area, and one that often falls largely to women and minorities and other underrepresented groups at our institutions. When analyzing words like “impact” and “metrics,” traditional outputs like journal articles appeared as valued in a large percentage of documents. Open access came up in a very small percentage of documents, and often with negative connotations. There are clear opportunities for institutions to reform and improve these processes so that they incentivize these public aspects of faculty work.
McKiernan then introduced the panelists: Thad Potter, a 6th year PhD Candidate and the president of the National Association of Graduate- Professional Students; Sara Weston, assistant professor of psychology at the University of Oregon; and Alzada Tipton, Provost and Dean of the Faculty at Whitman College and co-lead for HELIOS Open’s Institutional and Departmental Policy Working Group. The following is a summary of the question and answer session.
How are you currently involved in RPT and hiring reform or advocacy?
Thad Potter: I'd love to see mentoring and student development included in our efforts to reform RPT. We recently partnered with the American Medical Student Association earlier this year, and wrote a piece called “Time to Reform Academic Publishing” in Issues in Science and Technology. In the article, we reflected on the recent White House Office of Science and Technology Policy (OSTP) memorandum “Ensuring Free, Immediate, and Equitable Access to Federally Funded Research,” and its relevance to early career researchers. Our current incentives within institutions do not align with goals of open scholarship and public access like equity. Change will be necessary to help support a more equitable system, and, for early-career researchers, these topics are increasingly important.
Sara Weston: My work is broadly at the intersection of individual differences in psychology and health. When I was hired I was hired under a job advertisement that specifically called for scholars who engaged in open science. When I arrived, I realized that even though I and a couple of colleagues had been hired with open science backgrounds, those values weren't reflected in our department’s tenure and promotion guidelines. I joined a two-person task force to revise our guidelines, and in doing so, met with many colleagues about what should be valued in tenure. We also explored how our values as a department could be better integrated into our policies, and developed a series of recommendations for changing our department guidelines. These recommendations have gone through a couple rounds of revisions, and now sit with our provost. Unfortunately, the momentum has slowed as we wait, and I’m happy to talk about why I think that's happened, and some of the things that we could do to push it through.
I would also like to second the importance of mentoring, which is an important part of public engagement as well. It prepares students to practice publicly engaged scholarship. We also need to make sure that when we’re doing this work, we're not leaving people behind or making it more difficult for people to engage in open and public scholarship.
Alzada Tipton: I'm an unusual person to be included on this panel because I'm not a scientist, but, when Whitman College was invited to become part of HELIOS Open, I was happy to step forward to co-lead the Institutional and Departmental Policy working group. It’s been a really interesting and terrific experience for me.
As the chief academic officer, I’ve witnessed a lot of faculty go through the RPT process. Before I arrived at Whitman, the institution had gone through a process to try to understand how to recognize diversity, equity, inclusion, and antiracism in RPT guidelines.
Whitman also received two Mellon grants for community engaged learning, with the goal to understand how to better recognize community engaged learning in RPT. The great thing about being a small Liberal Arts college in a small community is that there truly is this desire to share scholarship not only with the colleagues down the hallway, but also with the larger community.
The HELIOS Open working group I co-lead drafted a Joint Statement on Reforming Hiring, Reappointment, Tenure, and Promotion. The statement asks institutions to commit to engaging in a dialogue on developing and advancing hiring and RPT reform strategies that reflect the importance of open research and scholarship in shaping a positive research culture and achieving institutional missions. My president, who is a physicist, and the tenure and promotion committee signed on to the statement agreeing to revisit our RPT guidelines.
How can researchers at different stages (graduate students, professors at different stages, administration) push forward RPT and hiring reform?
Potter: I think there are inherent challenges for graduate students engaging in incentive structures or reform because we are in a strange spot on the student-to-employee spectrum. Graduate students’ roles change over the course of our time at the institution, and we are a very diverse group. I started my Ph. D. program by taking classes as I did as an undergraduate student, but, by the end of the program, my day to day life looks nothing like an undergraduate student’s, and the goal of a Ph. D. program is to elevate us to be peers to our instructors.
Defining incentives for students can be challenging because the career path out of graduate school is often varied. Many might go into the academies, which have a very different set of activities for students to undertake before they graduate than those who might go into industry, public policy, or other careers. Building an incentive structure around these different paths is difficult. I’m a big fan of developing an individual development plan, recognizing it's hard to mandate specific actions for the varied graduate student goals.
Our current system often incentivizes publishing in specific journals, but this is a challenge for many early career researchers who may not be able to pay high article processing charges (APC) some of the publishers levy to make scholarship open. Going from ‘pay to read’ to ‘pay to publish,’ doesn't seem ideal.
Weston: Starting at my institution with open science in the job description, I wish I had asked more about how the department is currently or planning to support open science. One thing I didn't recognize when I was interviewing as a postdoc is that there are opportunities to negotiate and bargain, and we should teach that to graduate students. You can ask for policy change.
When working on RPT materials at the University of Oregon, we were trying to address open science in a very broad way: through open access publishing, through recognizing forms of transparency, and through valuing community engagement and community-based participatory science. We ended up focusing solely on guidelines that increased transparency. I think we found ourselves fighting against the “bean counting” mentality that often happens in RPT: how many citations do you have? What's your h-index? How many of these journals did you publish in? Instead, we tried to advocate for painting a fuller picture of what a researcher’s science is like, what their science philosophy includes, how they approached research in general, and how they demonstrate their values through their work.
The changes meant more work for people applying for tenure, including more statements or annotated CVs. As a junior faculty member, I think it was important to say I'm willing to do more work in order to be assessed on more than my number of papers.
When we brought the recommendations to the department, we discovered this conversation is not solely about tenure; it’s tenure, promotion, and merit, which affects everyone in the department. Faculty members who are extremely prolific, including very senior researchers who have many collaborators and students, were concerned about undertaking more work to annotate every publication that goes on their CV. Many also questioned how external reviewers will handle any changes to the process because we don't have control over whether or not external reviewers will value bean counting or honor our new approaches. I found that point of contention very frustrating because we do have a choice as to whether we use letters from external reviewers. We could choose to say “this person didn't read our guidelines or didn't evaluate the candidate based on the things that we value.”
I think this is why HELIOS Open is really important. If many institutions sign on and signal they value transparency and openness, and then agree to teach our faculty how to review other people through that lens, it gives that credibility back to departments. It helps to move us along together and trust that reviewers from other institutions will value the same activities.
Tipton: Before I get into the question you asked, I want to underline something that Sara said about being able to ask for guidelines or policies upon hire. That's a really important point, and I want people to know that is exactly the right moment to do it; that's the time when a hiring person is looking for something that can bring the person to the institution, complete the search, and ensure it’s successful. At Whitman, we have memorandums of understanding with faculty who bring something different to the appointment.
From an administrator’s point of view and from my experience at small liberal arts colleges, it can already be quite difficult to jam our work into three traditional buckets: research, teaching, and service. It’s an asset that much of our work happens across the three areas. However, it becomes difficult to truly recognize this asset in the moment of tenure and promotion to associate professor, where the buckets often become much more distinct. We must develop a more generous way of recognizing work across the three areas.
While we are a bit more free of the bean counting we see at other institutions, we do value peer review. However, we operate in the current system with a volunteer community managed by closed access journals, and our tenure case reviewers are also volunteers. We must take a step back and consider that many of our RPT guidelines were written 20 to 30 years ago and our system relies heavily on volunteer labor under this less than ideal system. We need to think about how we work through the bottlenecks of closed access journals and, instead, work together with the many others focused on moving knowledge forward to support the scholarship of our colleagues without resting on closed structures just because it’s what we are used to.
How do we capitalize on our shared interest to effect change considering we have allies in the federal government, within student groups, and among higher education leaders?
Potter: It’s hard to directly engage with the faculty assessment process. Oftentimes, students aren't in the room when assessment and incentives are discussed, and we’re rarely part of that decision making process. I try to be involved in as much as I can, but it's hard to kind of do that on a local and national level. I try to make connections to junior faculty because they are the closest removed from my status as a student, and will one day be running the departments. It can be difficult to get tenured professors to do new things that they’re not already required or incentivized to do. However, it’s important to participate and take a step back and look at the broader picture to make better systems.
Weston: All professors have to engage in service, and while we generally try to protect junior faculty from doing too much at first, there are always opportunities to be engaged in something and to bring a concern at faculty meetings. Consider speaking up while honoring the bandwidth you have to take on service.
It’s also important to bring our colleagues who are a little bit less engaged into the conversation. Consider talking about open scholarship with your graduate mentor or other faculty members. Maybe if we had more people than just me asking about this, there would be quicker movement and more support for collective action.
Tipton: An interesting dynamic I’m seeing is that, for the more faculty oriented people in the virtual HELIOS Open working group meetings, some are pointing to the provost as an obstacle to RPT reform. As the provost, I’ve always said “well, it’s about the faculty.” I do believe that the faculty need to be the leaders in pushing for reform, while recognizing that many people dislike change or are extremely busy to the point where asking them to take on anything can feel quite burdensome. I can work with the faculty on the guidelines and recommendations they come up with and be supportive of their efforts.
I also want to reflect on Sara’s remarks and my thoughts about pre-tenured faculty who feel like they can't speak up about things, and to post-tenure faculty who feel like it’s no longer their problem to reform RPT. We need faculty to be willing to be leaders in this important issue. Sara’s voice, for example, is so important.
A way to push reform forward with colleagues may include talking about what open scholarship enables as part of others’ priorities for reforming RPT guidelines. For example, diversity, equity, inclusion, and antiracism is crucial to my faculty, and we can advance these efforts by opening up scholarship. Similarly, outside of the sciences, many faculty value public facing humanities work at Whitman. It has expanded our definition of “scholarship,” and provides us an opportunity to ask our faculty how we can give full credit for this work that they are so passionate about, and where they feel like they are making such a difference.
When we think about the college's mission, this is another entry into the reform conversation. Our mission statements often reflect the hope that we make a difference in the world, so why would we not reward activities that enable mission fulfillment. These are activities that would make a provost’s heart very happy.
McKiernan concluded the session by sharing the need to flip the narrative that reforming RPT in support of open scholarship’s goals is onerous, or an action that forces our faculty via unfunded mandates related to grant compliance. Instead, we want researchers to be credited for enabling our missions, for activities that go beyond bean counting, and for activities that are often relegated to the service bucket or other lower assessment categories.
HELIOS Open Collaborates on US Federal Government’s Year of Open Science
The Higher Education Leadership Initiative for Open Scholarship (HELIOS Open) is pleased to collaborate with NASA and other federal agencies in celebration of 2023 as the Federal Year of Open Science. Today, the White House launched this multi-agency initiative across the federal government to spark change and inspire open science engagement through events and activities that will advance adoption of open science. HELIOS Open will serve as a cross-sector collaborator, engaging across its 88 members to co-develop, promote, and advance a range of open science initiatives.
"We are excited to collaborate with federal agencies to make open science easier and more rewarding for both individuals and the organizations that support them," said Dr. Geeta Swamy, Duke University Associate Vice President for Research and Vice Dean for Scientific Integrity and Strategic Lead for HELIOS Open. "HELIOS Open member institutions are eager to develop actionable policies, resources, guidance, metrics, and infrastructure to advance open science scholarship. Cross-sector coordination through initiatives such as the Year of Open Science are critical to harmonizing and scaling these efforts."
"The Year of Open Science is a great step forward in promoting productive ways for researchers and scholars to communicate their work more openly and inclusively for the benefit of both science and society," said Dr. Michael Crow, President of Arizona State University and Co-Chair of the Higher Education Leadership Initiative for Open Scholarship (HELIOS Open). "HELIOS Open is excited to team with NASA to advance this important initiative."
Through its collaboration with NASA and other participating agencies on the 2023 Year of Open Science, HELIOS Open will serve as a critical conduit between higher education and the federal government on a range of open science activities, including:
Convening meetings on HELIOS Open campuses to discuss practical considerations like infrastructure and open scholarship good practices
Highlighting projects at HELIOS Open member institutions that advance the Year of Open Science agenda
Collaborating with federal agencies and allies from adjacent sectors (e.g., philanthropies, professional societies) to develop resources, tools, and incentives language to make the practice of open scholarship easier and more rewarding
Serving as a test bed for rapid, iterative, and transparent open scholarship experiments
Acting as a conduit to help optimize and implement agency open science plans across the 88 HELIOS Open member institutions
Awarding flash grants to stimulate and reward open activities at minority serving institutions.
Read more about the 2023 Year of Open Science in Nature and check out the new federal website to keep apprised of the latest agency developments.