A Toolkit for Measuring the Impacts of Public Funding on Open Source Software Development

Cailean Osborne,Paul Sharratt,Dawn Foster,Mirko Boehm

2024-11-09

Abstract:Governments are increasingly employing funding for open source software (OSS) development as a policy lever to support the security of software supply chains, digital sovereignty, economic growth, and national competitiveness in science and innovation, among others. However, the impacts of public funding on OSS development remain poorly understood, with a lack of consensus on how to meaningfully measure them. This gap hampers assessments of the return on public investment and impedes the optimisation of public-interest funding strategies. We address this gap with a toolkit of methodological considerations that may inform such measurements, drawing on prior work on OSS valuations and community health metrics by the Community Health Analytics Open Source Software (CHAOSS) project as well as our first-hand learnings as practitioners tasked with evaluating funding programmes by the Next Generation Internet initiative and the Sovereign Tech Agency. We discuss salient considerations, including the importance of accounting for funding objectives, project life stage and social structure, and regional and organisational cost factors. Next, we present a taxonomy of potential social, economic, and technological impacts that can be both positive and negative, direct and indirect, internal (i.e. within a project) and external (i.e. among a project's ecosystem of dependents and users), and manifest over various time horizons. Furthermore, we discuss the merits and limitations of qualitative, quantitative, and mixed-methods approaches, as well as options for and hazards of estimating multiplier effects. With this toolkit, we contribute to the multi-stakeholder conversation about the value and impacts of funding on OSS developers and society at large.

Computers and Society,Software Engineering

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is the lack of understanding regarding the impact of public funds on open - source software (OSS) development. Specifically, although governments are increasingly emphasizing the support of goals such as software supply - chain security, digital sovereignty, economic growth, and national competitiveness in science and innovation through funding OSS development, there is still a lack of consensus on how to meaningfully measure the impact of these public funds. This gap in understanding hinders the evaluation of the return on public investment and impairs the ability to optimize public - interest funding strategies.

To address this issue, the author proposes a methodological toolkit for measuring the impact of public funds on OSS development. This toolkit draws on previous research on OSS valuation and community - health metrics, in particular the indicators developed by the CHAOSS project, as well as the author's practical experience in evaluating the Next Generation Internet initiative and the Sovereign Tech Agency funding plan.

### Main research contents include:

1. **Key considerations**:
- **Funding objectives**: The specific objectives of different funding projects may affect their effectiveness.
- **Project life cycle and social structure**: Consider the different stages of the project and its social structure (such as the contributor network).
- **Regional and organizational cost factors**: Cost differences in different regions and organizations will also affect the effectiveness of funding.

2. **Potential social, economic, and technological impacts**:
- These impacts may be positive or negative, direct or indirect, internal (i.e., within the project) or external (i.e., dependents and users in the project ecosystem), and will manifest within different time frames.

3. **Measurement methods**:
- Discusses the advantages and limitations of qualitative, quantitative, and mixed methods.
- Explores options and risks for estimating multiplier effects.

4. **Multi - stakeholder discussions**:
- Aims to promote multi - faceted discussions about the value and impact of public funds on OSS developers and society.

Through this toolkit, the author hopes to provide a framework for policymakers, researchers, and practitioners to more comprehensively understand and measure the impact of public funds on OSS development, thereby optimizing future funding strategies.

### Formula examples (if required):
- **Multiplier effect formula**:
\[
\text{Multiplier Effect}=\frac{\text{Total Impact}}{\text{Initial Investment}}
\]
where \(\text{Total Impact}\) is the total impact and \(\text{Initial Investment}\) is the initial investment.

### Summary:
This paper fills the current gap in understanding the impact of public funds on OSS development by proposing a systematic methodological toolkit, providing valuable guidance for future research and practice.