How to conduct a critical appraisal is one of the most important skills in nursing research because it helps you judge whether evidence is trustworthy, relevant and useful before you rely on it in a dissertation, literature review or evidence-based practice discussion. Critical appraisal is not simply describing a study and it is not completing a checklist mechanically. The aim is to examine the paper systematically, answer structured appraisal questions, justify each judgement with evidence from the article and explain how any strength or limitation affects confidence in the findings.
This guide shows you exactly how to conduct a critical appraisal using recognised tools such as CASP, JBI, MMAT, AMSTAR 2 and Cochrane RoB 2. The tools are used within one practical workflow. The purpose is not to compare them, but to show how you apply the appropriate framework to a study and then convert checklist answers into critical academic writing.
- Identify the study design before choosing an appraisal checklist.
- Read the paper once for meaning before completing the tool.
- Answer every question with evidence from the article, not assumptions.
- Record page numbers, quotations or methodological details that support each judgement.
- Explain the consequence of each limitation rather than merely saying it is a weakness.
- Do not convert CASP, JBI or MMAT into a simplistic percentage score unless your review protocol specifically requires a validated scoring approach.
What does it mean to conduct a critical appraisal?
Critical appraisal involves a careful and systematic examination of a research paper to judge its trustworthiness and its relevance to a particular clinical or academic question. CASP describes appraisal as a structured process that considers validity, results and relevance, while JBI frames its appraisal tools around the trustworthiness, relevance and results of published studies.
For nursing students, the important point is that appraisal must influence how evidence is used. A study with a strong design but substantial loss to follow-up may still contribute useful evidence, but its findings may deserve less confidence. A qualitative study may provide rich insight into patient experience while having limited transferability to another healthcare setting. The purpose is therefore to make a proportionate judgement rather than label every study as simply good or bad. International students applying evidence to British practice can extend this judgement with our guide to using UK evidence in a nursing dissertation.
How to conduct a critical appraisal step by step
Step 1: Identify the exact research question and study design
Start by locating the aim, research question or hypothesis. Then identify the study design. This determines which appraisal tool or checklist you should use. For example, a randomised controlled trial requires questions about randomisation, deviations from intervention and missing outcome data. A qualitative study requires questions about methodology, recruitment, reflexivity, analysis and the relationship between data and interpretation.
Do not choose a tool simply because it is familiar. CASP provides design-specific checklists for randomised trials, cohort studies, qualitative studies, case-control studies, systematic reviews and several other designs. JBI also provides design-specific tools across qualitative, quantitative, diagnostic, prevalence, review and textual evidence. Mixed-methods studies can be appraised using MMAT, while systematic reviews of healthcare interventions may be critically appraised using AMSTAR 2. RoB 2 is designed specifically for assessing risk of bias in randomised trials.
Step 2: Read the study once before opening the checklist
Read the full paper first. During this first reading, identify the clinical problem, sample, setting, intervention or phenomenon, outcome measures, main findings and authors’ conclusions. This prevents the appraisal from becoming a fragmented hunt for individual checklist answers.
On the second reading, complete the appraisal tool. Keep a separate evidence column where you record page numbers or short notes showing where the paper supports your answer. If the authors do not report something, write not reported rather than assuming it did not happen.
Step 3: Check whether the research design matches the question
This is one of the most important early appraisal decisions. Ask whether the chosen methodology can actually answer the research question. An RCT can test the effectiveness of an intervention. A cohort study can explore associations over time. A qualitative design is appropriate when the study aims to explore experiences, meanings, perceptions or processes.
If there is a mismatch, explain why it matters. For instance, a cross-sectional survey may identify an association between nurse burnout and medication errors, but it cannot establish which came first. Therefore, causal conclusions would be weaker.
How to conduct a critical appraisal using CASP questions
CASP checklists use structured questions to guide appraisal. The exact questions differ by design, but the reasoning process remains similar. The table below paraphrases the major questions you should work through rather than reproducing the official checklist wording.
| Appraisal question | What to look for in the paper | How to write the judgement |
|---|---|---|
| Is the research question clearly focused? | Population, intervention/exposure/phenomenon, outcome or purpose is identifiable. | Explain whether the aim is specific enough to guide the design and analysis. |
| Is the selected design appropriate? | Methodology matches the type of question being asked. | State why the design can or cannot answer the question convincingly. |
| Was recruitment appropriate? | Eligibility criteria, recruitment route, sampling strategy and participant flow. | Discuss selection bias or missing perspectives where relevant. |
| Were important sources of bias addressed? | Randomisation, confounding, blinding, measurement quality, researcher influence or attrition depending on design. | Name the likely bias and explain the direction or consequence. |
| Were the data collected appropriately? | Validated instruments, interview methods, follow-up procedures, consistency and timing. | Explain how data quality affects confidence in the findings. |
| Were the results analysed appropriately? | Statistical tests, thematic procedures, adjustment for confounders, handling of missing data. | Link analytical decisions to the strength of the conclusions. |
| What are the main findings? | Effect estimates, confidence intervals, themes, patterns and uncertainty. | Report the findings accurately before evaluating their meaning. |
| Are the findings relevant to nursing practice or your dissertation question? | Population, setting, care pathway, country, resources and clinical context. | Judge applicability without overstating generalisability. |
CASP advises users to record answers such as yes, no or cannot tell and then provide reasons. The important academic work occurs in the justification. A statement such as “sampling was appropriate” is incomplete unless you explain how recruitment supported or limited the study.
How to conduct a critical appraisal using JBI tools
JBI offers different appraisal tools for different study designs. Once you select the correct tool, move through every domain and document the evidence supporting your answer. JBI’s current tools cover designs including analytical cross-sectional studies, cohort studies, case-control studies, qualitative research, systematic reviews, diagnostic accuracy studies and quasi-experimental studies.
| JBI-style appraisal area | Questions to ask while reading | Why it matters |
|---|---|---|
| Participant selection | Were inclusion criteria clear? Did the sample represent the intended population or phenomenon? | Poor selection can produce systematic differences between participants and the population of interest. |
| Exposure or intervention measurement | Was exposure/intervention measured reliably and consistently? | Misclassification can distort associations or intervention effects. |
| Outcome measurement | Were outcomes measured using valid and reliable procedures? | Weak measurement reduces confidence in observed differences. |
| Confounding | Were important confounders identified and managed? | Uncontrolled confounding can create or hide associations. |
| Follow-up and missing data | Was follow-up sufficient? Were withdrawals explained? Was missing data handled appropriately? | Attrition can introduce bias if missingness differs between groups. |
| Analysis | Were statistical or qualitative analytical methods appropriate to the design and data? | Inappropriate analysis can produce misleading conclusions even when data collection was strong. |
When appraising qualitative research, JBI also asks users to consider methodological congruity: whether the philosophical perspective, methodology, research question, data collection, analysis and interpretation fit together. Our separate guide on how to appraise qualitative research examines this in more detail.
Using MMAT when conducting a mixed-methods critical appraisal
The Mixed Methods Appraisal Tool is useful when a paper or review contains qualitative and quantitative components. Start with the screening questions: is there a clear research question, and do the collected data allow the researchers to address it? You then appraise the relevant methodological component and the integration of methods.
| MMAT appraisal prompt | Evidence to record | Critical interpretation |
|---|---|---|
| Are the research questions clear? | Explicit aims or questions. | If unclear, explain how this makes methodological fit difficult to judge. |
| Do the data address the questions? | Alignment between questions, variables, interviews or observations. | Identify any questions that the collected data cannot answer. |
| Is each methodological strand adequately conducted? | Sampling, measurement, analysis and bias controls within each component. | Do not let one strong component conceal weaknesses in the other. |
| Is the integration of qualitative and quantitative evidence meaningful? | Joint displays, merged interpretation, explanation of disagreement or complementarity. | Explain whether integration adds insight beyond two parallel studies. |
| Are inconsistencies between components addressed? | Discussion of divergent qualitative and quantitative findings. | Unexplained contradictions may weaken the overall inference. |
The MMAT developers discourage reducing appraisal to a single numerical score because a total score can hide which methodological areas are problematic. For a dissertation, a domain-based judgement is usually more informative than writing that a study achieved “80% quality”.
Using RoB 2 to conduct critical appraisal of a randomised trial
For randomised trials, Cochrane RoB 2 focuses specifically on risk of bias. Rather than asking whether the whole study is generally “high quality”, assess bias across specific domains.
| RoB 2 domain | Questions to investigate | Potential consequence |
|---|---|---|
| Randomisation process | Was allocation random and concealed? Were baseline imbalances suspicious? | Problems can create systematic differences between intervention groups before treatment begins. |
| Deviations from intended interventions | Did participants receive the intended intervention? Could knowledge of allocation change behaviour or care? | Differences after assignment can distort the estimated intervention effect. |
| Missing outcome data | How many participants lacked outcome data, and why? | Missingness linked to outcome or treatment may bias results. |
| Measurement of outcomes | Were assessors blinded where possible? Were measurement methods comparable between groups? | Knowledge of treatment can influence subjective outcome assessment. |
| Selection of reported results | Were outcomes and analyses pre-specified? Were multiple analyses possible? | Selective reporting can exaggerate favourable findings. |
Your final judgement should explain which domain creates concern and how it affects the outcome you are using. This is stronger than simply stating that “the RCT had some bias”.
Using AMSTAR 2 when appraising a systematic review
If the article is a systematic review of healthcare interventions, AMSTAR 2 provides a structured way to examine review methods. Work through the review question, protocol, eligibility criteria, search strategy, study selection, data extraction, risk-of-bias assessment, synthesis methods, publication bias and conflicts of interest.
| AMSTAR 2 appraisal question | What you should verify |
|---|---|
| Was the review question defined before the review was conducted? | Population, intervention, comparator and outcomes are explicit and ideally supported by a protocol. |
| Was the literature search comprehensive? | Multiple appropriate databases, transparent search strategy, suitable dates and supplementary searching. |
| Was study selection and data extraction robust? | Independent or duplicate processes and clear disagreement procedures. |
| Was risk of bias assessed using an appropriate method? | Design-appropriate appraisal and explicit consideration of methodological limitations. |
| Did the synthesis account for study quality and heterogeneity? | Appropriate meta-analysis or narrative synthesis with justified methods. |
| Were publication bias and conflicts of interest considered? | Assessment where relevant and transparent funding/conflict declarations. |
Do not treat the presence of PRISMA reporting as proof that the review itself is methodologically strong. Reporting guidelines and critical appraisal tools have related but different purposes.
How to turn checklist answers into critical appraisal writing
This is where many nursing dissertations become descriptive. A completed checklist belongs in your working notes or appendix; the dissertation should communicate the meaning of the appraisal.
Use the sequence evidence → judgement → consequence → balanced conclusion.
| Stage | Example |
|---|---|
| Evidence | The study recruited 210 patients from a single specialist hospital and lost 22% of participants at follow-up. |
| Judgement | The attrition rate introduces concern because the characteristics of those lost were not fully reported. |
| Consequence | If participants with poorer outcomes were more likely to withdraw, the observed intervention effect may be overestimated. |
| Balanced conclusion | The study still provides relevant evidence, but its effect estimate should be interpreted cautiously alongside trials with more complete follow-up. |
For help developing this style, see our guide on how to write critically in a nursing dissertation. You can also organise appraisal findings alongside study characteristics using a nursing evidence table.
A practical critical appraisal worksheet
| Checklist question | Yes / No / Cannot tell | Evidence from article | Why it matters | Final judgement |
|---|---|---|---|---|
| Is the research question clear? | Page/section: | |||
| Is the design appropriate? | Page/section: | |||
| Is recruitment/sampling appropriate? | Page/section: | |||
| Are exposure/intervention/data collection methods sound? | Page/section: | |||
| Are outcomes/findings measured or derived appropriately? | Page/section: | |||
| Are major biases/confounders addressed? | Page/section: | |||
| Is the analysis appropriate? | Page/section: | |||
| Are the conclusions supported by the findings? | Page/section: | |||
| Are the findings relevant to your nursing question? | Population/setting comparison: |
Common mistakes when conducting a critical appraisal
Completing the checklist without explaining the answers
A row of yes/no answers does not demonstrate critical thinking. Every important judgement should have evidence and a consequence.
Calling everything a limitation
Appraisal should identify strengths as well as weaknesses. The aim is to judge confidence, not criticise every methodological decision.
Using the wrong checklist
A qualitative checklist cannot adequately appraise an RCT, and an RCT risk-of-bias tool cannot evaluate qualitative reflexivity. Always classify the design first.
Assuming “not reported” means “not done”
Write precisely. If the paper does not state whether allocation was concealed, the correct judgement may be that concealment is unclear rather than definitely absent.
Giving every flaw equal weight
Some issues are minor reporting gaps, while others directly threaten the validity of the findings. Explain which limitations are consequential.
Using a total score without justification
A numerical score can conceal serious problems in one domain. Domain-specific judgements are generally more useful for explaining confidence in evidence.
Frequently asked questions about how to conduct a critical appraisal
Should I use CASP or JBI for my nursing dissertation?
Use the tool required by your university or review protocol where one is specified. Otherwise, select a recognised tool appropriate to the study design and apply it consistently.
Do I need to include the completed checklist in my dissertation?
Many dissertations place completed appraisal checklists in an appendix and summarise the important judgements in the main text. Follow your assignment brief and supervisor guidance.
Can I use CASP for every paper?
CASP provides checklists for many common designs, but no single checklist is suitable for every form of evidence. The design must match the tool.
Can I exclude a study because it has one “No” answer?
Not automatically. Exclusion should follow your predefined review criteria. Methodological weaknesses may instead influence the weight you give a study during synthesis.
How do I appraise several studies without repeating myself?
Create an appraisal matrix. Record the same domains across studies, then synthesise recurring strengths and weaknesses. For example, you may note that three studies used valid outcome measures but all relied on single-centre samples.
Related Nursing Guides
- How to Conduct Thematic Analysis in a Systematic Review: Nursing Guide
- Effect Size for Nursing Research: SPSS and G*Power Guide
- G*Power for Nursing Research: Step-by-Step Guide
Conclusion
Knowing how to conduct a critical appraisal means moving systematically from the research question to the study design, sampling, data collection, analysis, risk of bias and relevance. CASP, JBI, MMAT, AMSTAR 2 and RoB 2 provide structured questions, but the checklist itself is only the beginning. Strong nursing appraisal records evidence for every judgement, identifies which methodological issues genuinely matter and explains how they change confidence in the findings. When you turn each checklist answer into an evidence-based judgement rather than a tick-box score, critical appraisal becomes a defensible part of literature review synthesis and evidence-based nursing practice.
References
- Critical Appraisal Skills Programme. (2026). Critical appraisal tools and resources. CASP. https://casp-uk.net/critical-appraisal-tools-and-resources/
- Critical Appraisal Skills Programme. (2026). CASP checklists. CASP. https://casp-uk.net/casp-tools-checklists/
- Hong, Q. N., Pluye, P., Fàbregues, S., Bartlett, G., Boardman, F., Cargo, M., Dagenais, P., Gagnon, M.-P., Griffiths, F., Nicolau, B., O’Cathain, A., Rousseau, M.-C., & Vedel, I. (2018). Mixed Methods Appraisal Tool (MMAT), version 2018: User guide. McGill University.
- JBI. (2026). Critical appraisal tools. https://jbi.global/critical-appraisal-tools
- Shea, B. J., Reeves, B. C., Wells, G., Thuku, M., Hamel, C., Moran, J., Moher, D., Tugwell, P., Welch, V., Kristjansson, E., & Henry, D. A. (2017). AMSTAR 2: A critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ, 358, j4008. https://doi.org/10.1136/bmj.j4008
- Sterne, J. A. C., Savović, J., Page, M. J., Elbers, R. G., Blencowe, N. S., Boutron, I., Cates, C. J., Cheng, H.-Y., Corbett, M. S., Eldridge, S. M., Hernán, M. A., Hopewell, S., Hróbjartsson, A., Junqueira, D. R., Jüni, P., Kirkham, J. J., Lasserson, T., Li, T., McAleenan, A., … Higgins, J. P. T. (2019). RoB 2: A revised tool for assessing risk of bias in randomised trials. BMJ, 366, l4898.
