For seven and a half million years, Deep Thought computed and calculated, and in the end announced that the answer was in fact Forty-two.
— The Hitchhiker’s Guide to the Galaxy, Douglas Adams
Traditional Problems
In the traditional model of research evaluation, “impact” is often reduced to a single number—the journal impact factor, which cannot possibly capture all dimensions of influence. The main problem researchers face is the narrow criteria of assessment – primarily citations in academic journals – which do not allow other forms of impact to gain recognition, such as policy adoption, public engagement, or technological applications.
Researchers rarely see the evaluation criteria that determine the funding of their projects, their promotion, or recognition. Success is measured by where you publish, not by who actually uses your research. Scientific knowledge often struggles to step down from the pedestal of “pure science” and enter real-world decision-making processes.
New tools and metrics are needed to measure impact.
What Is “Open Assessment and Metrics”
Open metrics are transparent, accessible, and diverse methods for measuring the impact of research. They capture not only the academic community but also influence on policy, business, and society. Open metrics are based on open data sources, use clear and transparent methodologies, and cover both quantitative and qualitative indicators.
Examples include:
- Alternative metrics that track mentions in social media, news coverage, and blog posts (e.g., Altmetrics).
- Open citation networks (e.g., OpenCitations).
- Policy citations in government documents (e.g., Overton).
- Statistics on the reuse of public datasets.
- Download counts and repository access metrics.
How Can “Open Assessment and Metrics” Solve Traditional Problems?
Open metrics broaden the focus of research evaluation: instead of only tracking journal citations, they assess whether researchers’ work leads to real change in society.
They rely on multidimensional indicators that include evidence of political, educational, or societal impact, rather than depending solely on journal impact factors. Open indicators allow for immediate visibility, tracking engagement in real time thanks to open data sources.
The open data on which metrics are based enables transparent, reproducible data collection and analysis. The connection to societal impact is made by tracking changes in policies, laws, and regulations that significantly affect communities. Open metrics reward authors who share datasets, software, and other knowledge by giving them credit and visibility.
Practical Steps for Using “Open Assessment and Metrics”?
Step 1: Publish open access with open data!
- The more accessible your work is, the more likely it will be discovered and adopted by diverse audiences.
Step 2: Use different open metrics platforms!
- Track your research via Altmetric.com, Dimensions, ImpactStory, and OpenCitations.
Step 3: Link your research with policy repositories!
- Publish your work on platforms connected to policy databases (e.g., Zenodo, OpenAIRE, and repositories linked to Overton).
Step 4: Anticipate pathways to impact in your research!
- Clearly describe who will use your research and how. Identify potential local or national authorities, NGOs, or community groups.
Step 5: Ensure clear communication with all stakeholders!
- Prepare short summaries, infographics, or plain-language briefs, and publish them alongside your academic articles.
Step 6: Monitor and adapt!
- Use feedback from open metrics platforms to manage indicators and adjust your dissemination strategy.
Good Practices and EU Examples
Horizon Europe Programme
- Requires applicants to describe both academic and societal impact, encouraging inclusion of open indicators and real-time tracking.
OpenAIRE
- Aggregates research graphs with open data from thousands of sources, making it easier to track impact beyond publications.
Altmetrics
- A pilot project of the European Commission testing the combination of traditional citation counts with policy mentions, downloads, and social engagement to create a richer impact profile.
EPRS (European Parliamentary Research Service)
- Engages directly with research results, citing open publications in policy analysis.
FOSTER+
- Supports open science training for European researchers by providing modules on measuring and communicating research impact using open, transparent indicators.
Conclusion
Open metrics reveal the scope of academic work and map the domino effect research has on society. By adopting open metrics, researchers not only broaden opportunities but also amplify impact.
Metaphorically speaking, as the Guide reminds us, it’s not just about the number – it’s about the question: who, exactly, are we influencing?