AboutCode blog! | AboutCode.org

This document provides a framework for evaluating and understanding the health of open source projects. Given the ubiquity of open source components in modern software supply chains, it is increasingly important to find viable ways of measuring such health.
There are several open source initiatives looking at this problem from different perspectives:
All of the three perspectives consider health as part of the journey. This analysis of the state of the art of project health focuses on the npm ecosystem, which has its own peculiarities. The npm ecosystem is a high-churn ecosystem where project sustainability and health are just as problematic as the technical aspects (vulnerabilities, technical debt, etc.), especially given that each project's governance structure must be able to handle the volume of pull requests and automated issue reports common in the JavaScript world.
Some papers focus on the understanding of dependencies in the npm space. Ahmed et al. discuss the existing technical lag within npm packages.1 The focus of this publication is not the current health of a specific package, but rather the time it takes for that package to be updated in the corresponding dependency tree: “We found that a large number of packages suffer from technical lag, where their outdated dependencies are several months behind the latest release.”
This includes ones that improve the internal processes at corporations when using open source as the Open Chain standards, those creating specific maturity models per vertical, such as the FINOS Foundation Open Source Maturity Model, and more generic ones like the Open Source Good Governance Initiative and the TODO Group.
Older, but well-established maturity models for processes are related to the Capability Maturity Model Integration (CMMI), which is used as a starting point for others, such as OpenBRR or QSoS.2 These are almost deprecated nowadays, but they were the starting point for maturity-related discussions in the corporate space.
A more general and academic approach to how organizations are approaching the topic of health can be seen in Linaker’s paper titled “Assessing open source software health in organizations’ intake processes: A qualitative study on the practitioners’ perspective”.3
Well-known examples are related to well-established open source foundations, such as the Eclipse Foundation, the Apache Software Foundation, or the Cloud Native Computing Foundation.
Their focus is to grow every new project to a common, shared base level of trust and maturity, making it easier for other members to adopt such technology. This typically includes diversity of organizations contributing to the project, adoption of the project processes and workflows, intellectual property requirements, licenses, and transparency in its operations.
Apache Software Foundation Maturity Model: The foundation has three levels of maturity for projects: starting as incubating, then moving to top-level projects, and finally, if abandoned or not used, marked with an archived status. They focus on code, licenses and copyright, releases, quality, community, consensus building, independence, and trademark and branding. The process is a self-assessment process, with tools to assist in the process.
Eclipse Foundation Development Process: This workflow includes a proposal reviewed by a formal review that includes the Project Management Committee, the Eclipse Management Organization, and input from the community. The main focus is having open, transparent, and demonstrable operations; community building; intellectual property processes; technical viability; and independence and vendor neutrality. In a mature phase, the Foundation expects the project to fully comply with the Eclipse Development Process.
Cloud Native Computing Foundation (CNCF) Project Lifecycle: The goal of this multi-stage evaluation is to ensure cloud native projects meet defined standards of maturity, security, and production readiness for adopters. It includes the criteria for each stage and the process for transitioning between levels. The four stages are:
CNCF provides an application process to grow from one step to another, which is publicly available and transparent. The project itself applies to any of the levels, and a group of experts reviews the application and the requirements. The adoption level of the technology is a strong requirement, including interviewing existing end-users.
The most innovative work in identifying open source health is happening in Community Health Analytics for Open Source Software (CHAOSS) and Open Source Security Foundation (OpenSSF), both "community-led" forums hosted under the umbrella of the Linux Foundation. CHAOSS is the only community focused on growing a body of knowledge around the health of open source. This initiative started in September 2017 and was founded as a new working group including universities, corporations, and open source foundations by the Linux Foundation. CHAOSS provides metrics, metrics models, and software to advance the understanding of open source health.
GrimoireLab and Augur (now forked by Red Hat and called CollectOSS) are the two most prominent open source tools to gather, analyze, and compose open source health information and trends.
GrimoireLab covers the most used pieces of infrastructure in open source, including source code management systems with Git or Mercurial; issue tracking systems like Bugzilla, Jira, GitHub, or Gitlab; review processes such as Gerrit, Gitlab, or GitHub; and communication channels like mailing lists or Slack channels. It takes care of the identities and affiliations of developers, storing and tracking the affiliation changes over time. The technology is flexible enough to add other semi-structured data sources through data analytics. A good example is the Linux Kernel, where code review takes place in mailing lists.
The new version of GrimoireLab, in development, is growing its scalability, now supporting 100,000 Git r