Systematic overview of systematic reviews of comprehensive school health services: methodology and select findings

Web Annex CSystematic overview of systematic reviews of comprehensive school health services: methodology and select findings

Acknowledgements

Drafting of the guideline

Mary Louisa Plummer, Child and Adolescent Health Consultant, United States of America, and David A. Ross, Department of Maternal, Newborn, Child and Adolescent Health and Ageing, WHO headquarters.

GRADE methodologist

Nandi Siegfried, Public Health Medicine Specialist, South Africa.

Steering Group (WHO staff unless otherwise noted)

Coordination

David Ross and Kid Kohl, Department of Maternal, Newborn, Child and Adolescent Health and Ageing.

Members

Jamela Al-Raiby, WHO Regional Office for the Eastern Mediterranean; Wole Ameyan, Department of Global HIV, Hepatitis and Sexually Transmitted Infections Programmes; Valentina Baltag, Department of Maternal, Newborn, Child and Adolescent Health and Ageing; Faten Ben-Abdelaziz, Department of Health Promotion; Paul Bloem, Department of Immunization, Vaccines and Biologicals; Sonja Caffe, WHO Regional Office for the Americas; Marie Clem Carlos, Department of Noncommunicable Diseases; Shelly Chadha, Department of Noncommunicable Diseases; Venkatraman Chandra-Mouli, Department of Sexual and Reproductive Health and Research; Katrin Engelhardt, Department of Nutrition and Food Safety; Kaia Engesveen, Department of Nutrition and Food Safety; Regina Guthold, Department of Maternal, Newborn, Child and Adolescent Health and Ageing; Joanna Herat, United Nations Educational, Scientific and Cultural Organization (UNESCO); Symplice Mbola Mbassi, WHO Regional Office for Africa; Rajesh Mehta, WHO Regional Office for South-East Asia; Denise Mupfasoni, Department of Control of Neglected Tropical Diseases; Martina Penazzato, Department of Global HIV, Hepatitis and Sexually Transmitted Infections Programmes; Marina Plesons, Department of Sexual and Reproductive Health and Research; Leanne Riley, Department of Noncommunicable Diseases; Chiara Servili, Department of Mental Health and Substance Use; Stéphanie Shendale, Department of Immunization, Vaccines and Biologicals; Marcus Stahlhofer, Department of Maternal, Newborn, Child and Adolescent Health and Ageing; Howard Sobel, WHO Regional Office for the Western Pacific; Martin Weber, WHO Regional Office for Europe; and Juana Willumsen, Department of Health Promotion.

Evidence review and synthesis

Systematic overview of systematic reviews of comprehensive school health services

Julia Levinson, Kid Kohl, Valentina Baltag and David Ross.

Systematic reviews of the effectiveness and acceptability of comprehensive school health services

Paul Montgomery, University of Birmingham, United Kingdom; Jacoby Patterson, Independent Senior Research Consultant, United Kingdom; and Anders M. Bach-Mortensen, University of Oxford, United Kingdom.

Review of global WHO health service interventions for 5–19-year-olds

Mary Plummer, Kid Kohl and David Ross.

Survey of expert opinion on school health services

Mary Plummer; Ace Chan, Stigma and Resilience Among Vulnerable Youth Centre (SARAVYC), School of Nursing, University of British Columbia, Vancouver, Canada; Kid Kohl; Ashley Taylor (SARAVYC); Elizabeth Saewyc (SARAVYC); and David Ross.

Brief exploratory review of school health services globally

Mary Plummer, Kid Kohl and Valentina Baltag.

Guideline Development Group

Rima Afifi, University of Iowa, United States of America; Habib Benzian, New York University, United States of America; Harriet Birungi, Population Council, Kenya; Rashida Ferrand, Biomedical Research and Training Institute, Zimbabwe; Jorge Gaete, Universidad de los Andes, Chile; Najat Gharbi, Ministry of Health, Morocco; Murthy Gudlavalleti Venkata Satyanarayana, Indian Institute of Public Health, India; Henrica J. M. Fransen, University of Tunis El Manar, Tunisia; Julia Levinson, Boston University, United States of America; Erin D. Maughan, National Association of School Nurses, United States of America; Ella Cecilia Naliponguit, Department of Education, Philippines; Atif Rahman, University of Liverpool, United Kingdom; Elizabeth Saewyc (Chair), University of British Columbia, Canada; Susan Sawyer, University of Melbourne, Australia; Hui-Jing Shi, Fudan University, China; and Sharlen Vigan, World Bank, Togo.

External Review Group

Bruce Dick, Adolescent Health Consultant, Switzerland; Chris Kjolhede, Bassett Health Care Network, United States of America; Regina Lee, Professor of Nursing, University of Newcastle, Australia; Maziko Matemvu, Her Liberty, Malawi; Antony Morgan, Glasgow Caledonian University, United Kingdom; Blanca Pianello Castillo, International Federation of Medical Students’ Associations, Spain.

Financial support

The United Kingdom Department for International Development, the Bill and Melinda Gates Foundation and the United States Agency for International Development.

Abbreviations

AMSTAR: A MeaSurement Tool to Assess systematic Reviews
NCS: non-randomized controlled study
PRISMA: Preferred Reporting Items for Systematic reviews and Meta-Analyses
RCT: randomized controlled trial
SHS: school health services
SOSR: systematic overview of systematic reviews

Glossary

A glossary of terms used throught the guidance and its web annexes is provided in the guidance document.

Web Annex C summarizes the methodology and select findings from the systematic overview of systematic reviews (SOSR) of comprehensive school health services (SHS) (1).

C.1. SOSR methodology

This overview was conducted using the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) (2). A protocol was developed a priori that outlined the overview objectives, aims, operational definitions, search strategy, inclusion/exclusion criteria and quality appraisal methods.

C.1.1. Search strategy

PubMed, Web of Science, ERIC, PsycINFO and the Cochrane Library were searched systematically. A detailed search strategy was iteratively developed in consultation with a librarian experienced in systematic reviews and an expert in SHS. The search strategy was developed for PubMed and then adapted for the other four databases. Searches were performed on 15 June 2018. Any existing overviews or systematic reviews of systematic reviews that emerged from the searches were not themselves included, but the systematic reviews within them were extracted and screened. Additionally, reference lists of included articles were scanned for any relevant systematic reviews.

C.1.2. Eligibility criteria

Systematic reviews were included in this overview if at least 50% of the studies within the systematic review fulfilled the following criteria: (a) participants were children (ages 5–9) or adolescents (ages 10–19) enrolled in schools; (b) interventions were within school-based or school-linked health services, involved a health provider and were of any duration or length of follow-up; (c) intervention effectiveness was compared to either no intervention, an alternative intervention, the same intervention in a different setting (not in schools), an active control or a waitlist control; (d) interventions aimed to improve some aspect of health; and (e) study designs were either randomized controlled trials (RCTs), non-randomized controlled studies (NCSs) or other non-randomized intervention studies. There were no date restrictions on publication of included systematic reviews.

In addition to these criteria for included studies, the systematic reviews themselves had to fulfil the following criteria: (a) include the words “systematic review” in the title or abstract; (b) outline inclusion criteria within the methods section; (c) be published in peer-reviewed journals and indexed before 15 June 2018; and (d) be published in the English language. In addition to systematic reviews that did not meet these inclusion criteria, systematic reviews were excluded if the review was superseded by a newer version.

C.1.3. Study selection

Citations identified from the systematic search were uploaded to Covidence systematic review software and duplicates were automatically deleted. Two reviewers screened all titles and abstracts using the inclusion/exclusion criteria and excluded all articles that were definitely ineligible. Articles that received conflicting votes (ineligible versus potentially or probably eligible) were discussed and consensus was reached. The same two reviewers screened the full text of all the potentially or probably eligible articles using a ranked list of the inclusion criteria. Reasons for exclusion were selected from the ranked list. If consensus was not possible during title/abstract or full-text screening, a third reviewer, who had the casting vote, would have been asked to independently screen the article. However, this was never required as consensus was always reached.

C.1.4. Data collection

One reviewer extracted summary data from each selected article using a customized standard form with independent data extraction performed for 15% of included systematic reviews by one of the other reviewers. There was 92% agreement between reviewers for all items within the standard form, with discrepancies only in level of detail. Data items included the research design of the systematic review and primary studies, sample description and setting, intervention characteristics, outcomes, meta-analysis results, quality appraisal and conclusions.

C.1.5. Synthesis of results

Due to the heterogeneity of the systematic reviews included in this overview, it was not possible to perform a meta-analysis. Outcome measures were collected from included studies.

C.1.6. Risk of bias

Risk of bias across systematic reviews was determined using Ballard and Montgomery’s four-item checklist for overviews of systematic reviews (3). These items include: (1) overlap (see below), (2) rating of confidence from the checklist for AMSTAR 2 (A MeaSurement Tool to Assess systematic Reviews 2) (4), (3) date of publication and (4) match between the scope of the included systematic reviews and the overview itself.

C.2. Select SOSR findings

Interventions with evidence for effectiveness addressed autism, depression, anxiety, obesity, dental caries, visual acuity, asthma and sleep (Table C.1). No review evaluated the effectiveness of a comprehensive SHS intervention addressing multiple health areas. Strongest evidence supports implementation of anxiety prevention programmes, indicated asthma education and vision screening with provision of free spectacles.

Table C.1

Findings from systematic overview of systematic reviews of comprehensive SHS.

C.3. SOSR conclusions

This SOSR presents multiple effective interventions that may be offered as a part of SHS delivered by a health provider. However, it is difficult to formulate an overarching answer about the effectiveness of SHS for improving the health of school-age children and adolescents due to the heterogeneity of systematic reviews found and the evident gaps in the systematic review literature. More than half of included systematic reviews analysed mental health and oral health interventions and no systematic reviews were found that assessed some other relevant health areas, such as vaccinations, communicable diseases and injuries. Further, no systematic reviews evaluated comprehensive SHS. In order for policy-makers and leaders in school health to make evidence-based recommendations on which services should be available in schools, who should deliver them and how they should be delivered, more systematic reviews must be done. These systematic reviews must assess routine, comprehensive SHS and the characteristics that make them effective, with special attention to content, quality, intensity, method of delivery and cost.

References¹

1.: Levinson J, Kohl K, Baltag V, Ross DA. Investigating the effectiveness of school health services delivered by a health provider: a systematic review of systematic reviews. PLoS One 2019;14(6):e0212603. doi:10.1371/journal.pone.0212603. [PMC free article: PMC6561551] [PubMed: 31188826] [CrossRef]

2.: Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6(7):e1000097. doi:10.1371/journal.pmed.1000097. [PMC free article: PMC2707599] [PubMed: 19621072] [CrossRef]

3.: Ballard M, Montgomery P. Risk of bias in overviews of reviews: a scoping review of methodological guidance and four-item checklist. Res Synth Methods 2017;8(1):92–108. doi:10.1002/jrsm.1229. [PubMed: 28074553] [CrossRef]

4.: Shea, BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of health care interventions, or both. BMJ 2017;358:j4008. doi:10.1136/bmj.j4008. [PMC free article: PMC5833365] [PubMed: 28935701] [CrossRef]

5.: Geryk LL, Roberts CA, Carpenter DM. A systematic review of school-based interventions that include inhaler technique education. Respir Med. 2017;132:21–30. doi:10.1016/j.rmed.2017.09.001. [PubMed: 29229099] [CrossRef]

6.: Walter H, Sadeque-Iqbal F, Ulysse R, Castillo D, Fitzpatrick A, Singleton J. Effectiveness of school-based family asthma educational programs in quality of life and asthma exacerbations in asthmatic children aged five to 18: a systematic review. JBI Database System Rev Implement Rep. 2016;14(11):113–138. doi:10.11124/JBISRIR-2016-003181. [PubMed: 27941517] [CrossRef]

7.: Hennegan J, Montgomery P. Do menstrual hygiene management interventions improve education and psychosocial outcomes for women and girls in low and middle income countries? A systematic review. PLoS One 2016;11(2):e0146985. doi:10.1371/journal.pone.0146985. [PMC free article: PMC4749306] [PubMed: 26862750] [CrossRef]

8.: Bastounis A, Callaghan P, Banerjee A, Michail M. The effectiveness of the Penn Resiliency Programme (PRP) and its adapted versions in reducing depression and anxiety and improving explanatory style: a systematic review and meta-analysis. J Adolesc. 2016;52:37–48. doi:10.1016/j.adolescence.2016.07.004. [PubMed: 27494740] [CrossRef]

9.: Brendel KE, Maynard BR, Albright DL. Effects of school-based interventions with US military-connected children: a systematic review. Res Soc Work Prac. 2014;24(6):649–58. doi:10.1177/1049731513517143. [CrossRef]

10.: Gold C, Wigram T, Elefant C. Music therapy for autistic spectrum disorder. Cochrane Database Syst Rev. 2006;19(2):CD004381. doi:10.1002/14651858.CD008958.pub2. [PubMed: 16625601] [CrossRef]

11.: Higgins E, O’Sullivan S. “What works”: systematic review of the “FRIENDS for Life” programme as a universal school-based intervention programme for the prevention of child and youth anxiety. Educ Psychol Pract. 2015;31(4):424–38. doi:10.1080/02667363.2015.1086977. [CrossRef]

12.: Kavanagh J, Oliver S, Caird J, Tucker H, Greaves A, Harden A et al. School-based cognitive-behavioural interventions: a systematic review of effects and inequalities. London: EPPI-Centre, University of London; 2009.

13.: Kavanagh J, Oliver S, Lorenc T, Caird J, Tucker H, Harden A et al. School-based cognitive-behavioural interventions: a systematic review of effects and inequalities. Health Sociol Rev. 2009;18(1):61–78. doi:10.5172/hesr.18.1.61. [CrossRef]

14.: McDonald A, Drey NS. Primary-school-based art therapy: a review of controlled studies. International Journal of Art Therapy 2018;23(1):33–44. doi:10.1080/17454832.2017.1338741. [CrossRef]

15.: Neil AL, Christensen H. Efficacy and effectiveness of school-based prevention and early intervention programs for anxiety. Clin Psychol Rev. 2009;29(3):208–15. doi:10.1016/j.cpr.2009.01.002. [PubMed: 19232805] [CrossRef]

16.: Sullivan AL, Simonson GR. A systematic review of school-based social-emotional interventions for refugee and war-traumatized youth. Rev Educ Res. 2016;86(2):503–30. doi:10.3102/0034654315609419. [CrossRef]

17.: Werner-Seidler A, Perry Y, Calear Al, Newby JM, Christensen H. School-based depression and anxiety prevention programs for young people: a systematic review and meta-analysis. Clin Psychol Rev. 2017;51:30–47. doi:10.1016/j.cpr.2016.10.005. [PubMed: 27821267] [CrossRef]

18.: Schroeder KJ, Travers J, Smaldone A. Are school nurses an overlooked resource in reducing childhood obesity? A systematic review and meta-analysis. J Sch Health 2016;86(5):309–21. doi:10.1111/josh.12386. [PMC free article: PMC4939612] [PubMed: 27040469] [CrossRef]

19.: Arora A, Khattri S, Mastura Ismail N, Kumbargere Nagraj S, Eachempati P. School dental screening programmes for oral health. Cochrane Database Syst Rev. 2017;12:CD012595. doi:10.1002/14651858.CD012595.pub3. [PMC free article: PMC6953367] [PubMed: 31425627] [CrossRef]

20.: Cooper AM, O’Malley LA, Elison SN, Armstrong R, Burnside G, Adair P et al. Primary school-based behavioural interventions for preventing caries. Cochrane Database Syst Rev. 2013;5:CD009378. doi:10.1002/14651858.CD009378.pub2. [PubMed: 23728691] [CrossRef]

21.: Marinho VCC, Worthington HV, Walsh T, Chong LY. Fluoride gels for preventing dental caries in children and adolescents. Cochrane Database Syst Rev. 2015;6:CD002280. doi:10.1002/14651858.CD002280.pub2. [PMC free article: PMC7138249] [PubMed: 26075879] [CrossRef]

22.: Stein C, Lopes Santos NM, Hilgert JB. Effectiveness of oral health education on oral hygiene and dental caries in schoolchildren: systematic review and meta-analysis. Community Dent Oral Epidemiol. 2018;46(1):30–7. doi:10.1111/cdoe.12325. [PubMed: 28815661] [CrossRef]

23.: Paul-Ebhohimhen VA, Poobalan A, van Teijlingen ER. A systematic review of school-based sexual health interventions to prevent STI/HIV in sub-Saharan Africa. BMC Public Health 2008;8, art. 4. doi:10.1186/1471-2458-8-4. [PMC free article: PMC2248569] [PubMed: 18179703] [CrossRef]

24.: Chung K-F, Chan M-S, Lan Y-Y, Lai CS-Y, Yeung W-F. School-based sleep education programs for short sleep duration in adolescents: a systematic review and meta-analysis. J Sch Health 2017;87(6):401–08. doi:10.1111/josh.12509. [PubMed: 28463450] [CrossRef]

25.: Evans JR, Morjaria P, Powell C. Vision screening for correctable visual acuity deficits in school-age children and adolescents. Cochrane Database Syst Rev. 2018;2(2):CD005023. doi:10.1002/14651858.CD005023.pub3. [PMC free article: PMC6491194] [PubMed: 29446439] [CrossRef]

Footnotes

1: All weblinks accessed 25 May 2021.

Publication Details

Copyright

Sales, rights and licensing. To purchase WHO publications, see http://apps.who.int/bookorders. To submit requests for commercial use and queries on rights and licensing, see http://www.who.int/about/licensing.

Third-party materials. If you wish to reuse material from this work that is attributed to a third party, such as tables, figures or images, it is your responsibility to determine whether permission is needed for that reuse and to obtain permission from the copyright holder. The risk of claims resulting from infringement of any third-party-owned component in the work rests solely with the user.

Some rights reserved. This work is available under the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 IGO licence (CC BY-NC-SA 3.0 IGO; https://creativecommons.org/licenses/by-nc-sa/3.0/igo).

Under the terms of this licence, you may copy, redistribute and adapt the work for non-commercial purposes, provided the work is appropriately cited, as indicated below. In any use of this work, there should be no suggestion that WHO endorses any specific organization, products or services. The use of the WHO logo is not permitted. If you adapt the work, then you must license your work under the same or equivalent Creative Commons licence. If you create a translation of this work, you should add the following disclaimer along with the suggested citation: “This translation was not created by the World Health Organization (WHO). WHO is not responsible for the content or accuracy of this translation. The original English edition shall be the binding and authentic edition”.

Any mediation relating to disputes arising under the licence shall be conducted in accordance with the mediation rules of the World Intellectual Property Organization (http://www.wipo.int/amc/en/mediation/rules/).

Publisher

World Health Organization, Geneva

NLM Citation

WHO guideline on school health services [Internet]. Geneva: World Health Organization; 2021. Web Annex C, Systematic overview of systematic reviews of comprehensive school health services: methodology and select findings.

First author, year and reference	Health area specified	Type(s) of interventions	Findings	Meta-analysis results
a. Findings from systematic reviews on asthma interventions
Geryk 2017 (5)	Asthma	Education	Improved inhaler technique	NA
Walter 2016 (6)	Asthma	Education	Improved daytime and night-time symptoms; physical activity intolerance; emergency hospital visits; and missed school or work days	NA
b. Findings from systematic reviews on menstrual management interventions
Hennegan 2016 (7)	Menstruation	Education, provision of sanitary products	Sanitary pad provision: moderate yet statistically insignificant effect on school attendance; overall trends toward improvements in menstruation knowledge, management practices, psychosocial outcomes and school attendance	School attendance: SMD = 0.49, 95% CI: −0.13, 1.11, p = 0.12
c. Findings from systematic reviews on mental health interventions
Bastounis 2016 (8)	Depression and anxiety	Education, prevention	Depression: non-significant, in favour of PRP programme; Anxiety: non-significant, in favour of control	Depression: MD = −0.23, 95% CI: −1.09, 0.62 Anxiety: SMD = 0.13, 95% CI: 0.00, 0.26
Brendel 2014 (9)	Well-being	Counselling	No statistically significant change	NA
Gold 2006 (10)	Autism	Therapy (music)	Small yet statistically significant effect sizes in favour of music therapy	Gestural communication: SMD = 0.50, 95% CI: 0.22, 0.79^* Verbal communication: SMD = 0.36, 95% CI: 0.15, 0.57^* Behavioural problems: SMD = −0.24, 95% CI:−0.45, −0.03^*
Higgins 2015 (11)	Anxiety	Prevention	Statistically significant improvement in self-reported anxiety	NA
Kavanagh 2009 (12,13)	Depression and anxiety	Counselling	Statistically significant reductions of depressive symptoms up to four weeks and three months follow-up	4 weeks: SMD = −0.16, 95% CI:−0.26, −0.05 Equivalent to reduction in 1.44 points on BDI^* 3 months: SMD = −0.21, 95% CI: −0.35, −0.07; equivalent to reduction in 1.9 points on BDI^*
McDonald 2018 (14)	Various	Therapy (art)	Improvements in outcomes on classroom behaviour, ODD and SAD	NA
Neil 2009 (15)	Anxiety	Prevention	Statistically significant reductions in anxiety symptoms at post-test and/or follow-up in 21 out of 27 primary trials	NA
Sullivan 2016 (16)	Trauma	Therapy	Improvements in trauma-related symptoms and impairment; negative effects for music therapy	NA
Werner-Seidler 2017 (17)	Depression and anxiety	Prevention, therapy	Small yet statistically significant effect sizes in favour of the intervention for both depression and anxiety	Depression: Hedges g = 0.23, 95% CI: 0.19, 0.28^* Anxiety: Hedges g = 0.20, 95% CI: 0.14, 0.25^*
d. Findings from systematic reviews on obesity interventions
Schroeder 2016 (18)	Obesity prevention and treatment	Education, counselling, prevention	Small but statistically significant reductions in all three BMI outcomes	BMI, attenuated due to high heterogeneity: SMD = −0.06, 95% CI: −0.17, −0.01^* BMIz score: SMD = −0.10, 95% CI: −0.15, −0.05^* BMI percentile: SMD = −0.41, 95% CI: −0.60, −0.21^*
e. Findings from systematic reviews on oral health interventions
Arora 2017 (19)	Oral health and dental care attendance	Screening, referrals	Insufficient evidence for conclusions on oral health outcomes or dental attendance	NA
Cooper 2013 (20)	Caries	Education, prevention	Insufficient evidence for conclusions on caries increment or plaque accumulation	NA
Marinho 2015 (21)	Caries	Prevention	Decrease in caries increment	PF = 0.28, 95% CI: 0.19, 0.36, p < 0.0001^*
Stein 2017 (22)	Caries and oral hygiene	Education	Decrease in mean plaque levels; improved oral hygiene; no change in gingivitis	Mean plaque levels: MD = −0.36, 95% CI: −0.59, −0.13, p = 0.004^* Oral hygiene: MD = −0.42, 95% CI: −0.69, −0.15, p = 0.002^* Gingivitis: MD = −0.07, 95% CI: −0.32, 0.19, p = 0.61
f. Findings from systematic reviews on sexual and reproductive health interventions
Paul-Ebhohimhen 2008 (23)	STIs and HIV	Education	Increased knowledge and attitudes; ineffective in changing risky behaviours	NA
g. Findings from systematic reviews on sleep interventions
Chung 2017 (24)	Sleep	Education	Statistically significant short-term benefits for all three outcomes	Weekday sleep time: SMD = 0.23, 95% CI = [0.17, 0.29], p = 0.0001^* Weekend sleep time: SMD = 0.46, 95% CI = [0.04, 0.86], p = 0.03^* Mood: SMD = 0.81, 95% CI: 0.17, 1.47, p = 0.01^*
h. Findings from systematic reviews on vision interventions
Evans 2018 (25)	Visual acuity	Education, screening, spectacles provision	Statistically significant increase in spectacles wear; no difference between provision of ready-made versus custom-made spectacles; no comparison of vision screening versus no vision screening	Free spectacles versus prescription: RR = 1.6, 95% CI = [1.34, 1.90], p < 0.00001^* Ready-made versus custom-made: RR = 0.98, 95% CI = [0.91, 1.05], p = 0.51