Work Description

Date: 13 December, 2021

Dataset Title: Data for: Global Service-Learning - A systematic review of principle and practice

Dataset Creators: Jason K Hawes, Rebecca Johnson, Lindsey Payne, Christian Ley, Caitlin A. Grady, Jennifer Domenech, Carly D. Evich, Andrew Kanach, Allison Koeppen, Kristen Roe, Audrey Caprio, Jessica Puente Castro, Paige LeMaster, Ernest R. III Blatchley

Dataset Contact: Jason K Hawes, jkhawes@umich.edu or jasonkhawes@gmail.com

Introduction: Global service-learning brings students, instructors, and communities together to support learning and community development across borders. In doing so, global service-learning practitioners act at the intersection of two fields: service-learning and international development. Critical scholarship in all three domains has highlighted the tensions inherent in defining and tracking “success” in community development. In response, service-learning and international development have turned considerable attention to documenting project characteristics, also known as best practices or success factors, which support equitable, sustainable community development. This database accompanies the article "Global Service-Learning - A systematic review of principle and practice," which presents a systematic synthesis of these fields’ best practices in the context of global service-learning. We propose 18 guiding principles for project design which aim to support practitioners in creating and maintaining justice-oriented, stakeholder-driven projects. This database contains the necessary reference material to trace the path of our analysis from abstract review to thematic synthesis. It also contains the final results of the thematic synthesis. To respect copyright restrictions, we have not made PDFs of all articles analyzed publicly accessible. Please contact the authors of this database or of the original article if you seek to access one of the articles we reference. For more information, see: Hawes, J. K. (2021). Global Service-Learning—A systematic review of principle and practice. International Journal of Research on Service-Learning and Community Engagement, 10(1).
(Introduction partially quoted from Hawes et al., 2021)

Methods: Systematic literature reviews employ strict inclusion and review criteria for the purposes of analyzing a body of literature with minimal bias and maximal transparency (Dixon-Woods, 2010). Early approaches to systematic review were criticized as oversimplifying complex bodies of literature, a challenge that prompted novel framings of systematic review which foregrounded qualitative complexity (Davies et al., 2013). This review adopts a variation on the thematic synthesis approach delineated by Thomas and Harden (2008), seeking to account for the context-driven nature of ID and SL assessments. This process takes place in four stages: abstract review, line-by-line article review, descriptive coding for themes, and analytical synthesis and framework development (Thomas & Harden, 2008).
**Abstract Review: Screening of documents** We used Thomson Reuters Web of Science to identify relevant best practices literature on ID and SL. The search method for ID was: TOPIC: ((“success* factor*” OR “best practic*” OR “framework”) AND TOPIC: (“internat* dev*”). The parallel search method for SL was: TOPIC: (“success* factor*” OR “best practic*” OR “framework”) AND TOPIC: (“serv* learn*”). We specified no date range and only included English-language, peer-reviewed articles. We extracted 488 ID articles and 228 SL articles on September 11, 2018. A number of articles have been published under these criteria since this date. While we have sought to reference relevant updates where useful, the long histories of service-learning and international development give us confidence that the synthesized principles remain relevant. Two individuals were assigned to each of the 716 article abstracts. Each pair used the Rayyan Abstract Review online platform to submit review results, blind of their counterpart’s suggestions (Ouzzani et al., 2016). Each coder suggested inclusion or exclusion, listed their certainty (categorical: low, medium, high), and provided an explanation in the case of exclusion. Preliminary rounds of coding and group meetings were used to refine inclusion and exclusion criteria, and coders agreed on inclusion or exclusion of 65% of abstracts. For the remaining 35% of abstracts, conflict resolution was conducted by committee consensus, with 3-4 coders collectively assessing and discussing each abstract. The final list of reviewed articles, as well as final inclusion and exclusion criteria, are available in the associated online repository.
**Article full-text review: Data extraction** Qualtrics survey software was used to compile full-text reviews.1 Each coder reviewed approximately 15 articles, extracting success factors, best practices, and frameworks of interest. The extracted excerpts were separated into three major categories, defined as follows. The first two categories were two different types of factors (a combination of success factors and best practices). We defined factors as project components or characteristics that were highlighted as driving success (success as defined by the authors of each article). Factors were identified as either primary or secondary. Primary factors included any factors that the article defined, analyzed, or synthesized while offering further construction, analysis, or validation. Secondary factors were those factors for which the article offered no additional construction, analysis, or validation beyond description and citation, in which case both the factor and the article being cited were extracted. Frameworks were defined as any theories or previously constructed frameworks explaining the characteristics of successful projects. Extracted frameworks were not included in the synthesis stage of analysis but were referenced for the final presentation of themes in this article. An assortment of meta-data was also collected for each article, including: original classification as ID or SL; type of article (e.g., review); and country of interest. We did not test interrater reliability. Instead, extensive training was conducted for each reviewer; we assessed the potential limitations of this method and accounted for this through the decision to only code for generic success factors instead of constructing an emergent codebook during extraction.
**Synthesis: Coding and framework development** Using these extracted factors, the four lead authors inductively coded for descriptive themes and refined these descriptive themes into a guiding principles framework. This was accomplished in a series of iterative coding stages (Figure 1). In stage one, three authors conducted by-hand pile-sorting of primary factors, eventually forming a series of overarching themes (e.g., Capacity Building, Stakeholders, Monitoring and Evaluation). In stages two and three, the same three authors further sorted each of these overarching themes into smaller sub-categories (e.g., Monitoring and Evaluation containing 1. Formative Evaluation Methods; 2. Summative Evaluation Methods; 3. Participatory Success Criteria Development) individually before collaboratively refining the emerging codebook. In stage four, the lead author conducted a similar pile-sorting exercise for the secondary factors, using these secondary factors and sources to validate the themes which emerged from our primary sample of literature. Finally, the four lead authors collaboratively refined the emergent codebook by binning related codes and adapting phrasing. Through this process, 18 guiding principles emerged, which were further binned into four thematic areas.

Davies, D., Jindal-Snape, D., Collier, C., Digby, R., Hay, P., & Howe, A. (2013). Creative learning environments in education—A systematic literature review. Thinking Skills and Creativity, 8, 80–91. https://doi.org/10/tmn
Dixon-Woods, M. (2010). Systematic reviews and qualitative methods. Qualitative Research: Theory, Method and Practice. 3rd Edn. London: Sage, 331–346.
Ouzzani, M., Hammady, H., Fedorowicz, Z., & Elmagarmid, A. (2016). Rayyan—A web and mobile app for systematic reviews. Systematic Reviews, 5(1), 210. https://doi.org/10.1186/s13643-016-0384-4
Thomas, J., & Harden, A. (2008). Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Medical Research Methodology, 8. Scopus. https://doi.org/10/c73jf5

(Methods quoted from Hawes et al., 2021)

File inventory:
Note: In these documents, "factors" refer to quotes extracted during full-text review. "Primary factors" refer to direct evidence from the article being read. "Secondary factors" refer to relevant information cited from a third-party source. Practically speaking, references to "codes" and to "principles" are the same. Principles are in reference to the outputs of the final thematic synthesis results, while codes are in references to the process of "coding" in qualitative text analysis. For the purposes of this database, since no preliminary coding schemes are presented here, "codes" and "principles" are funcitonally the same. Both "themes"/"thematic areas" and "categories" refer to the groups in which these "principles" or "codes" are binned. The phrase "themes" was used in final presentation of the article, but was not used during initial binning of the codes to avoid confusion with other uses of the word theme in qualitative text analysis. Again, for the purposes of this database, "themes" and "categories" are functionally the same.

Abstract review procedure and results - Presented in 1 PDF and 2 CSV files:
- Abstract_Review_Results-Complete_Results.csv: Displays the complete list of abstract reviewed, reviewers judgements, and final inclusion/exclusion ruling.
- Abstract_Review_Results-List_of_articles_for_full_text_review.csv: Simplified list of included articles in csv format
- Coding_Guide-Inclusion_Exclusion.pdf: Inclusion and exclusion criteria

Full text review procedure - Presented in 2 PDF files:
- Full-text_Review_Survey.pdf: Copy of the survey used by reviewers
- Master_List_of_Articles_included_in-Full_Text_Review.pdf: List of articles reviewed formatted as bibliography instead of csv.

Full text review results - Presented in 2 CSV files:
- GSL_Primary_Factors_with_Metadata.csv: Primary factors (see definition above) extracted during full-text review, with metadata for the article from which the factor was extracted.
- GSL_Secondary_Factors_with_Metadata.csv: Secondary factors (see definition above) extracted during full-text review, with metadata for the article from which the factor was extracted.

Thematic synthesis results - Presented in 4 CSV files and 1 PDF file:
- GSL_Principles_Codebook-Overview: Final codebook overview with definition and quantitative results. Includes sorting into thematic areas, final principles verbiage, and definition.
- GSL_Principles_Codebook-Primary_Factors_Sorted: List of all primary factors associated with each principle.
- GSL_Principles_Codebook-Secondary_Factors_Sorted: List of all secondary factors associated with each principle.
- GSL_Principles_Codebook-Articles_associated_with_each_code: For each principle, provides a list of articles from which factors for that principle were drawn. In other words, if any factor from an article ended up sorted into a principle, that article is listed in this document alongside the principle.
- Supplementary_Tables.pdf: Simple synopsis table of thematic areas and principles. Similar to GSL_Principles_Codebook-Overview.csv, but formatted as a docx/PDF.

Discipline: International Studies; Other: Service-learning, international development, higher education

Keywords: Service-learning, international development, global service-learning, best practices, equitable development, higher education, community engagement

Use and Access: This data set is made available under a Creative Commons Attribution-Noncommercial 4.0 International License (CC BY-NC 4.0).- http://creativecommons.org/licenses/by-nc/4.0/

For more information and to cite this work, see related manuscript:
Hawes, J. K. (2021). Global Service-Learning—A systematic review of principle and practice. International Journal of Research on Service-Learning and Community Engagement, 10(1).

Dataset citation:
Hawes, J., Johnson, R., Payne, L., Ley, C., Grady, C., Domenech, J., Evich, C., Kanach, A., Koeppen, A., Roe, K., Caprio, A., Puente Castro, J., LeMaster, P., Blatchley, E. Data for: Global Service-Learning - A systematic review of principle and practice [Data set], University of Michigan - Deep Blue Data. https://doi.org/10.7302/wazb-wk46