Multiple comparisons in nursing research need attention when several statistical tests contribute to the same dissertation claim. Testing many outcomes, groups or time points can increase the opportunity for a false positive. A clear analysis plan helps you decide which comparisons belong together, what error rate matters and how to report the findings honestly.
For example, a nursing education project might compare three teaching methods across several questionnaire subscales. Reporting every result below 0.05 as a separate success can exaggerate the evidence. The answer is to define the research question and comparison family before selecting an adjustment.
This guide explains that decision through invented nursing examples. It focuses specifically on multiplicity, rather than repeating our statistical test selection guide or general SPSS analysis workflow.
What counts as multiple comparisons in nursing research?
Multiplicity occurs when an analysis provides several opportunities to reject statistical null hypotheses. These opportunities might involve separate patient outcomes, comparisons between several wards, questionnaire subscales or repeated follow-up assessments. Repeatedly reanalysing the same data with different inclusion rules creates another concern: the reported finding may depend on undisclosed analytical choices.
A family is a set of hypotheses considered together for an inferential purpose. It is not automatically every number in the dissertation. Equally, dividing one clinical claim into separate tables does not necessarily create independent families. The justification should follow the scientific question and the planned interpretation.
Imagine evaluating whether an education programme improves medication management. If knowledge, confidence and adherence are all used to claim that the programme works, they may need a shared multiplicity strategy. If one outcome was clearly designated primary before analysis, its planned test can have a different role from secondary outcomes. The protocol must make that distinction visible.
To plan multiple comparisons in nursing research, start with a family map. For each hypothesis, write the outcome, comparison, assessment time, primary or secondary status and intended conclusion. This makes it harder to exclude inconvenient tests after seeing their results. Discuss ambiguous cases with your supervisor before finalising the analysis.
Why can several tests produce misleading significance?
For a single valid test conducted at alpha 0.05, the Type I error rate is controlled at that level under the null hypothesis and the procedure’s assumptions. This does not mean that a significant result has a 5% probability of being false. Those are different probability statements.
For illustration, suppose 20 independent tests are performed and all their null hypotheses are true. The probability of at least one false rejection is 1 − (1 − 0.05)20, approximately 64.2%. This is a mathematical illustration under independence, not an estimate of the error rate in every nursing dissertation.
Clinical outcomes are often correlated, so the independence calculation may not describe your data. However, correlation does not remove the need for a plan. Avoid presenting the 64.2% figure as a universal risk or using it to justify an adjustment without considering the actual hypotheses.
There is also a practical cost to a stricter threshold: a genuine association may be harder to detect. Good planning balances error control, realistic sample size and the clinical importance of the outcomes. Our statistical power guide explains the wider design implications.
Which adjustment fits multiple comparisons in nursing research?
First decide what kind of mistake the analysis should control. Family-wise error rate concerns the probability of making one or more false rejections within the defined family. False discovery rate concerns the expected proportion of false rejections among all rejections, with the proportion treated as zero when nothing is rejected. These objectives should not be described as interchangeable.
| Method | Main objective | Potential use | Important caution |
|---|---|---|---|
| Bonferroni | Control family-wise error | A defined set of confirmatory comparisons | Can substantially reduce power as the family grows |
| Holm | Control family-wise error through ordered testing | Several confirmatory hypotheses requiring a step-down procedure | Use the complete family and stop the rejection sequence correctly |
| Benjamini–Hochberg | Control false discovery rate under applicable conditions | A justified exploratory discovery analysis | Does not provide the same protection as family-wise control |
| Tukey or Dunnett | Control error for a specified comparison structure | All pairwise means, or treatment groups versus a control, respectively | Check the model assumptions and the exact software procedure |
This table is a planning aid. It does not replace checking the design, dependence structure, model assumptions and university requirements. The method should be selected because it addresses the planned inference, not because one column produces more favourable results.
How does Bonferroni adjust multiple comparisons?
For m tests in a family, compare each raw p-value with alpha divided by m. Alternatively, multiply each raw p-value by m, cap the result at 1 and compare these adjusted values with the original alpha. Use one presentation consistently and explain which values appear in the table.
Suppose four planned outcomes form one family and alpha is 0.05. The per-test threshold is 0.0125. A raw p-value of 0.018 falls below 0.05 but does not meet that Bonferroni threshold. The corresponding adjusted value is 0.072. This does not establish that the intervention has no effect; it changes the strength of the statistical claim under the chosen rule.
Bonferroni does not require independence to control family-wise error when the individual tests are valid. Nevertheless, the individual analyses must still suit the data. Adjustment cannot repair biased sampling, dependent observations treated as independent or a wrongly specified outcome model.
How does Holm handle multiple comparisons?
Holm’s procedure orders the raw p-values from smallest to largest. For a family of m hypotheses, the first comparison uses alpha/m, the next alpha/(m − 1), and so on. Once a comparison fails its threshold, stop rejecting hypotheses in that sequence. This is a step-down approach to family-wise error control (Holm, 1979).
Keep a record of the original outcome labels when ordering the values. Otherwise, it is easy to assign an adjusted result to the wrong subscale. Software can produce adjusted p-values, but a small hand-worked example is useful for checking your understanding before reporting a larger analysis.
When should nursing research use Benjamini–Hochberg?
Benjamini and Hochberg (1995) introduced a procedure for controlling false discovery rate, establishing its control for independent test statistics. Its error target differs from the family-wise approach. Dependence conditions matter, so related questionnaire outcomes should not automatically be treated as satisfying every version of the method.
For an exploratory nursing survey, the aim might be to identify candidate associations for later confirmation. An FDR approach may be defensible if the research purpose and assumptions support it. It should not be introduced after a confirmatory analysis becomes non-significant, then presented as though it had always been planned.
An FDR target of 0.05 does not mean each selected association has a 5% probability of being false. Nor does it guarantee that exactly 5% of discoveries in this particular dataset are false. Keep the wording tied to the procedure’s long-run error criterion.
Worked example: four nursing education outcomes
The following figures are invented to demonstrate adjustment. They are not findings from a published study and should not be copied into an actual dissertation as research evidence. Suppose an approved project has four planned confirmatory outcomes within one family.
| Illustrative outcome | Raw p-value | Bonferroni-adjusted p-value | Holm-adjusted p-value |
|---|---|---|---|
| Knowledge score | 0.004 | 0.016 | 0.016 |
| Observed skill score | 0.012 | 0.048 | 0.036 |
| Confidence score | 0.030 | 0.120 | 0.060 |
| Documentation score | 0.200 | 0.800 | 0.200 |
At a family alpha of 0.05, both illustrated procedures reject the first two null hypotheses. The confidence result no longer meets the chosen rule, although its raw p-value is below 0.05. The fourth result does not meet either rule.
For Holm, the ordered thresholds are 0.0125, approximately 0.0167, 0.025 and 0.05. The first two raw p-values pass. The third does not, so the rejection sequence stops. Adjusted Holm p-values also enforce monotonicity across the ordered list; reporting software values is usually clearer than manually rebuilding a complex table.
The example does not show effect sizes, sample sizes or confidence intervals. A real results table needs the information necessary to assess magnitude and precision. A knowledge difference could be statistically detectable yet too small to matter educationally. Conversely, a clinically relevant effect could remain uncertain in a small sample.
How do you adjust multiple comparisons in nursing research using SPSS?
Choose the correct underlying model before choosing a post hoc option. IBM’s One-Way ANOVA documentation lists several comparison procedures, including Bonferroni, Tukey and Dunnett. Their availability in a menu does not make them equally suitable for every question (IBM, n.d.-a).
- Confirm the analysis unit. Identify whether rows represent independent nurses, patients, wards or repeated measurements.
- List the intended comparisons. Specify all pairs, comparisons with a control, planned contrasts or another justified set.
- Read the procedure documentation. Confirm what the selected option adjusts and which assumptions apply.
- Save the syntax. Retain the exact model and adjustment settings so the analysis can be reproduced.
- Inspect the output footnotes. Establish whether the displayed significance values and confidence limits are already adjusted.
- Match the software scope to the protocol. Check whether the programme corrected the same family that your methodology describes.
The last step is particularly important for several outcomes. IBM states that post hoc multiple-comparison tests in GLM Multivariate are performed for each dependent variable separately (IBM, n.d.-b). A correction within each outcome therefore should not automatically be described as controlling error across every outcome in the project.
Do not multiply an already adjusted p-value again unless you are implementing a clearly justified additional procedure. Double adjustment changes the analysis and can make the reported method incorrect. If the output is unclear, consult the syntax and documentation before writing the results.
SPSS features differ by procedure, edition and installed version. Avoid assuming that a method listed for one command appears in another dialogue. If a required adjustment is calculated outside the original procedure, document the software, full hypothesis family and calculation.
What about ANOVA, subgroups and repeated follow-ups?
A significant omnibus ANOVA does not identify which specific groups differ. The follow-up comparison strategy still needs justification. Conversely, a planned contrast and an unplanned search through all pairs serve different purposes. Describe the analysis you actually planned rather than applying a post hoc label to everything.
Repeated measurements introduce dependence as well as possible multiplicity. Running separate independent-samples tests at every follow-up can ignore the overall longitudinal question. Start with our repeated measures and mixed ANOVA guide when deciding how the design should be represented.
For subgroup analyses, avoid claiming that an intervention works differently between groups solely because one subgroup has p < 0.05 and another does not. A difference between two significance labels is not itself a test of interaction. Additional subgroup searches also require transparent reporting and an appropriate inferential plan.
Several sensitivity analyses may test the robustness of one finding rather than establish several separate successes. Still, selecting only the most favourable sensitivity result hides uncertainty. Explain why each analysis was undertaken and whether the conclusion changed across reasonable specifications.
How should you report multiple comparisons in nursing research?
The methodology should identify the hypothesis family, alpha level, adjustment method and whether analyses were confirmatory or exploratory. Include a reason for the choice. Avoid a sentence saying only that “SPSS corrected the results,” because it leaves the scope and procedure unclear.
Illustrative methods wording: “The four prespecified education outcomes were treated as one family of confirmatory hypotheses. Holm adjustment was applied to their p-values to control the family-wise error rate at 0.05. Raw and adjusted p-values are reported alongside the outcome estimates.”
Use that wording only when it accurately describes your approved and completed analysis. If the plan changed after the data were inspected, disclose the change. An honest amendment is more defensible than retroactively presenting a new strategy as prespecified.
Illustrative results wording: “The confidence outcome had a raw p-value of 0.030 and a Holm-adjusted p-value of 0.060. It did not meet the prespecified family-wise significance criterion. The estimated difference and its uncertainty were considered when interpreting the finding.”
Insert your actual estimate and interval in a real report. Label confidence intervals as ordinary or multiplicity-adjusted, as applicable. An ordinary 95% interval may not correspond to an adjusted hypothesis decision; readers need enough information to understand that distinction.
The discussion should address clinical meaning, design limitations and consistency with previous evidence. Do not describe a result that fails the adjustment rule as “almost proven.” Equally, do not replace an uncertain estimate with a categorical claim of no effect. Our SPSS discussion guide covers the broader interpretation process.
Common mistakes to remove before submission
- Counting only the significant tests when defining the family.
- Changing the adjustment method after comparing which gives the desired answer.
- Assuming software adjusted across all questionnaire outcomes.
- Using raw p-values in the text while the table reports adjusted values.
- Confusing Hochberg’s family-wise procedure with Benjamini–Hochberg FDR control.
- Treating an adjustment as a substitute for checking model assumptions.
- Reporting significance without outcome units, estimates or uncertainty.
When reviewing multiple comparisons in nursing research, trace one result from the analysis plan through syntax, output, results table and discussion. The family, method, value and conclusion should agree at every stage. Ask a reviewer to follow the same path without explanation from you.
Frequently asked questions
Does every nursing dissertation need Bonferroni correction?
No. The need for a multiplicity strategy depends on the hypotheses and intended claims. A single prespecified primary comparison differs from a large exploratory screen. Explain the decision rather than applying Bonferroni automatically.
Should I correct every p-value in the document together?
Not automatically. Define families through their scientific purpose and analysis plan. Unrelated descriptive material and separate inferential questions may have different roles. Do not split a shared claim into artificial families simply to avoid adjustment.
Can I report raw and adjusted p-values?
Yes, if they are clearly labelled and the interpretation follows the stated method. Include the complete relevant family, not just the results that remain significant. This gives readers a transparent account of the analysis.
Does Holm always produce a significant result?
No. It is a family-wise error procedure, not a way to guarantee significance. Its step-down structure can be less conservative than simple Bonferroni, but the result still depends on the data and the full comparison family.
Can exploratory findings still be useful?
Yes. They can generate questions for future research when labelled honestly. Their value depends on transparent methods, effect estimates and proportionate interpretation. Avoid presenting an exploratory association as a confirmed clinical conclusion.
Get nursing dissertation statistics support
If you are unsure which comparisons belong together, our nursing dissertation data analysis support can help review the analysis plan and explain legitimate output. Send the research question, rubric, approved protocol, anonymised variable list and relevant syntax or tables.
Support should help you understand and defend your own methods. Data must reflect the real study; a preferred p-value or significant result cannot be promised. Agree the review scope before sharing any information and follow your university’s requirements.
Get My Nursing Assignment Quote
Conclusion
Managing multiple comparisons in nursing research begins with a clear claim and a defined hypothesis family. Choose the error criterion before interpreting the results, check what the software actually adjusts and report estimates alongside uncertainty. A transparent non-significant result is useful evidence when its limitations are explained accurately.
References
- Benjamini, Y. and Hochberg, Y. (1995) ‘Controlling the false discovery rate: a practical and powerful approach to multiple testing’, Journal of the Royal Statistical Society: Series B (Methodological), 57(1), pp. 289–300. Original research paper.
- Holm, S. (1979) ‘A simple sequentially rejective multiple test procedure’, Scandinavian Journal of Statistics, 6(2), pp. 65–70. Original research paper.
- IBM (n.d.-a) One-Way ANOVA Post Hoc Tests. IBM SPSS Statistics 30 documentation. Procedure documentation (Accessed: 29 October 2026).
- IBM (n.d.-b) GLM Multivariate Analysis. IBM SPSS Statistics 30 documentation. Procedure documentation (Accessed: 29 October 2026).
Illustrative workspace photograph by Vandan Patel on Unsplash. The photograph does not show study data or a clinical research result.
