ArticleResearch integrity and peer review2026
Applicants as reviewers: a mixed methods evaluation of the risks, benefits, and potential of distributed peer review for grant funding allocations.
Article in Research integrity and peer review, 2026. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Not yet cited in PubMed.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
0 citing papers in PubMed.
No citing paper in PubMed yet.
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
9 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
backgroundPeer review-based funding evaluation systems are under increasing pressure. To help alleviate this, innovative approaches like Distributed Peer Review (DPR) - where applicants review proposals for the same funding call - are being explored. The Volkswagen Foundation trialled DPR alongside a conventional panel review process for an interdisciplinary call. Evaluation of this trial provides insights into the dynamics of (distributed) peer review.
methodsAll proposals (N = 140) underwent both DPR and the panel review process. In DPR, applicants (N = 323) reviewed four or five proposals, providing an overall score, scores for each funding criteria, and a comment. Proposals were ranked using a trimmed mean score. Expectation and feedback surveys were administered. The panel process involved shortlisting by VWS staff, review by panellists (N = 8), and a final panel discussion. Semi-structured interviews with applicants, panellists, and VWS staff (N = 20) explored perceptions and experiences of DPR, using thematic analysis. Quantitative analysis of review data and surveys utilised bootstrapping to assess decision stability and mixed effects models for consistency between reviewers. Qualitative and quantitative results were integrated in a convergent parallel design.
resultsDPR recommended ten proposals for funding, while the panel recommended eleven; three overlapped. Funding criteria strongly predicted overall score; 'Originality' was given most weight by both sets of reviewers. Stability of DPR funding decisions was modest; across 500 bootstrapped samples a mean of 5.49 out of ten previously funded proposals would still be funded. This may be explained by high inter-reviewer variability: more variance was attributable to between-reviewer differences (20.0%) than between-proposal differences (8.9%). Advantages of DPR included increased efficiency, faster decision times, detailed feedback to applicants, and democratisation of funding evaluation. Noted disadvantages were burden on applicants, absence of discussion, and perceived lack of reviewer expertise. Funders implementing DPR must consider the high initial effort, potential for 'gaming', and reduced direct influence.
conclusionsThis trial demonstrates the feasibility of DPR for an interdisciplinary funding programme. Implementing DPR requires funders to accept trade-offs and that DPR may not produce the same outcomes as existing systems. Building on established peer review principles, it may offer benefits, including shortening time to funding decision, and has the potential to contribute to the democratisation of funding evaluation.
Indexed as
Identifiers
What Socratic holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the Socratic graph.