Screening disagreements in a nursing systematic review occur when reviewers reach different decisions about whether a record or full-text report meets the review’s eligibility criteria. Resolve them by checking the protocol, examining the relevant evidence, discussing the reason for the difference and recording the final decision. This guide explains how to manage those steps and report them honestly.
What is a screening disagreement?
A screening disagreement arises when two people independently apply the same inclusion and exclusion criteria but make different decisions. One reviewer might include a report while another excludes it, or one might judge that there is not enough information to decide. The disagreement is a prompt to revisit the evidence and criteria, not proof that either person was careless.
Systematic reviews commonly screen titles and abstracts first, followed by potentially relevant full texts. The NIH Library screening guide describes both stages and explains that disagreements can be resolved through discussion or by involving another reviewer. Your protocol and university requirements determine the approach used in your review.
Why do reviewers disagree about eligibility?
Disagreements often reflect a difficult boundary in the criteria or incomplete information in a report. For example, an abstract may mention nurses but not clarify whether a nurse delivered the intervention. A full text may describe a mixed population without reporting whether results for the eligible group can be separated.
- Population: reviewers interpret a mixed sample differently.
- Intervention: the report does not clearly describe who delivered an activity.
- Setting: a programme spans both hospital and community care.
- Design: the authors use a label that does not match their methods.
- Outcomes: an eligible outcome appears in a table but not the abstract.
- Publication type: a conference abstract may be linked to a later full report.
Before changing a criterion, establish whether the problem is unclear wording, incomplete reporting or a genuine mismatch. A decision should be based on the review question and prespecified rules, not on whether the result appears favourable.
How do you resolve a disagreement step by step?
1. Compare the independent decisions
Identify the record, screening stage and each reviewer’s decision. Keep the original decisions visible if the software permits. The aim is to understand how the criterion was applied, not to replace the disagreement with an unexplained final label.
2. Return to the protocol
Read the precise inclusion and exclusion rule together. Check definitions, dates, eligible designs and examples agreed during calibration. If the review has a decision guide, consult that too. Do not rely solely on a general impression of whether the paper is interesting.
3. Locate the evidence in the report
Each reviewer should identify the passage supporting their decision. At the title-and-abstract stage, insufficient information may justify moving the record to full-text review under the team’s agreed rules. At full text, inspect methods, tables, appendices and any companion reports before deciding.
4. Discuss the reason for the difference
Explain which rule is disputed and why. For example, one reviewer may have interpreted “nurse-led” as requiring every session to be delivered by a nurse, while the other considered a multidisciplinary programme eligible. A documented definition can help the team apply the rule consistently to similar studies.
5. Involve a third reviewer when needed
If the two reviewers cannot agree, use the escalation process specified in the protocol. A third reviewer may facilitate discussion or make a decision if that role was agreed. Record the involvement and the reason for the final decision. Do not describe a third reviewer or independent second screening unless these genuinely occurred.
6. Record the final outcome
Document the screening stage, criterion, decision, rationale and who participated. For full-text exclusions, use a clear reason that matches the eligibility criteria. A structured log makes it easier to reconcile the selection record with the flow diagram and to explain decisions to a supervisor.
What does a worked screening disagreement look like?
Fictional example: A review is examining nurse-led education for adults managing a chronic condition. Its protocol includes programmes where a nurse delivers the core educational sessions. The team screens a report describing a programme delivered jointly by nurses and pharmacists.
Reviewer A includes the report because nurses were involved. Reviewer B marks it uncertain because the abstract does not specify who delivered the core sessions. The team advances the record to full-text review under its agreed title-and-abstract rules. In the full text, the intervention description states that pharmacists delivered the core sessions and nurses provided referral support.
The reviewers then compare those details with the protocol’s definition. If referral support alone does not meet the stated criterion, the team records an exclusion for ineligible intervention, with a note identifying the relevant methods section. If the full text remains unclear, they follow their protocol for seeking additional information and documenting uncertainty. This example illustrates a decision process; it is not a real study or result.
How should you document the decision?
A short log can include the record identifier, citation, screening stage, initial decisions, disputed criterion, supporting passage, resolution method and final outcome. Record dates and the reviewer’s role where required by the project. Use the same fields across reports so that the audit trail can be reviewed efficiently.
For instance, a full-text decision might read: “Excluded: core intervention delivered by a professional group outside the prespecified nurse-led criterion; decision agreed after discussion.” The wording must reflect what the report and your protocol actually say. Do not invent a quotation, decision date or reviewer to make the process appear more rigorous.
What happens if the eligibility criteria are unclear?
Pause and discuss the issue with the review team or supervisor. The first step is to distinguish a clarification from a substantive change. A clarification may explain how an existing rule applies to a borderline case. A change may alter which studies can qualify and affect decisions already made.
Document the wording, rationale and timing of any change. Follow your programme or review organisation’s amendment process. If the interpretation changes, assess whether previously screened records need to be revisited. Never quietly widen or narrow criteria simply because a particular paper has desirable findings.
How should disagreements appear in the methods and PRISMA report?
Describe what actually happened at each stage: how many people screened, whether they worked independently and how differences were resolved. The PRISMA 2020 resources support transparent reporting of the selection process. Follow your required reporting guideline and institution’s instructions.
The flow diagram summarises the movement of records and reports through selection; it is not a substitute for the decision log. Full-text exclusion counts should reflect the final recorded reasons and reconcile with the underlying records. For help checking those totals, read our guide to common PRISMA flow-diagram mistakes.
What if you are the only reviewer?
Report that fact accurately and follow the assessment brief. A supervisor may be able to check a sample or discuss difficult decisions if their role permits it, but this is not the same as independent duplicate screening of every record. State exactly what was checked and consider the limitation when interpreting the review.
The Cochrane Handbook discusses approaches to selecting studies for intervention reviews. Requirements vary across review types and institutions, so do not claim that a single-reviewer process meets a specific standard without checking that standard.
How can teams prevent recurring disagreements?
Before screening the full set, reviewers can pilot a sample against the protocol. Discuss unclear cases, refine the decision guide and record any agreed clarifications. During screening, schedule regular checks for recurring uncertainties and review whether the same rule is being applied consistently.
Software can organise decisions, but it cannot replace a clear protocol. Agree how to label uncertainty, how to link multiple reports of the same study and who will resolve a stalled case. If automation is used, report its role as required by the relevant guidance.
Frequently asked questions
Does a disagreement mean the paper must be excluded?
No. Reapply the eligibility criteria and inspect the available information. The final decision could be include, exclude or seek clarification, depending on the stage and the team’s rules.
Must every disagreement go to a third reviewer?
No. Many teams resolve differences by discussion and involve another reviewer only when needed. Follow the procedure in your protocol.
Can the eligibility criteria change during screening?
Sometimes an amendment is necessary, but changes should be justified, documented and approved where required. Consider the effect on earlier decisions.
Should I report a reviewer who only checked a sample?
Yes, if relevant to the review process. Specify that the person checked a sample rather than implying independent screening of all records.
Get support with your nursing systematic review
If you need feedback on your eligibility criteria, screening record or reporting, explore our nursing systematic review support. Share your protocol and anonymised decision log so the review can be assessed in context. You remain responsible for the genuine screening decisions and final submission.
Get My Nursing Assignment Quote
References
- Cochrane (n.d.) Chapter 4: Searching for and selecting studies. Available at: Cochrane Handbook, Chapter 4 (Accessed: 27 October 2026).
- National Institutes of Health Library (n.d.) Screening. Available at: NIH Library screening guidance (Accessed: 27 October 2026).
- PRISMA (n.d.) PRISMA 2020. Available at: PRISMA 2020 resources (Accessed: 27 October 2026).
