View Item 

Show simple item record

On the Reliability of Meta-Analytic Reviews

dc.contributor.authorYeaton, William H.en_US
dc.contributor.authorWortman, Paul M.en_US
dc.date.accessioned2010-04-13T19:51:44Z
dc.date.available2010-04-13T19:51:44Z
dc.date.issued1993en_US
dc.identifier.citationYeaton, William; Wortman, Paul (1993). "On the Reliability of Meta-Analytic Reviews." Evaluation Review 17(3): 292-309. <http://hdl.handle.net/2027.42/67840>en_US
dc.identifier.issn0193-841Xen_US
dc.identifier.urihttps://hdl.handle.net/2027.42/67840
dc.description.abstractThe article addresses the issue of intercoder reliability in meta-analyses. The current practice of reporting a single, mean intercoder agreement score in meta-analytic research leads to systematic bias and overestimates the true reliability. An alternative approach is recommended in which average intercoder agreement scores or other reliability statistics are calculated within clusters of coded variables. These clusters form a hierarchy in which the correctness of coding decisions at a given level of the hierarchy is contingent on decisions made at higher levels. Two separate studies of intercoder agreement in meta-analysis are presented to assess the validity of the model.en_US
dc.format.extent3108 bytes
dc.format.extent905148 bytes
dc.format.mimetypetext/plain
dc.format.mimetypeapplication/pdf
dc.publisherSage Publicationsen_US
dc.titleOn the Reliability of Meta-Analytic Reviewsen_US
dc.typeArticleen_US
dc.subject.hlbsecondlevelPublic Healthen_US
dc.subject.hlbtoplevelHealth Sciencesen_US
dc.description.peerreviewedPeer Revieweden_US
dc.contributor.affiliationumUniversity of Michiganen_US
dc.contributor.affiliationotherState University of New Yorken_US
dc.description.bitstreamurlhttp://deepblue.lib.umich.edu/bitstream/2027.42/67840/2/10.1177_0193841X9301700303.pdf
dc.identifier.doi10.1177/0193841X9301700303en_US
dc.identifier.sourceEvaluation Reviewen_US
dc.identifier.citedreferenceBangert-Drowns, R.L. 1986. Review of developments in meta-analytic method. Psychological Bulletin 99:388-99.en_US
dc.identifier.citedreferenceBullock, R.J., and D.J. Svyantek. 1985. Analyzing meta-analysis: Potential problems, an unsuccessful replication, and evaluation criteria. Journal of Applied Psychology 70:108-15.en_US
dc.identifier.citedreferenceChalmers, T.C., J. Berrier, H.S. Sacks, H. Levin, D. Reitman, and R. Nagalingham. 1987. Meta-analysis of clinical trials as a scientific discipline. II. Replicate variability and comparison of studies that agree and disagree. Statistics in Medicine 6:733-44.en_US
dc.identifier.citedreferenceCooper, H.M. 1982. Scientific guidelines for conducting integrative research reviews. Review of Educational Research 52:291-302.en_US
dc.identifier.citedreferenceDooley, D. 1990. Social research methods. 2d ed. Englewood Cliffs, NJ: Prentice-Hall.en_US
dc.identifier.citedreferenceDyken, M.L., and R. Pokras. 1984. The performance of endarterectomy for disease of the extracranial arteries of the head. Stroke 15:948-50.en_US
dc.identifier.citedreferenceEdwards, A.L. 1976. An introduction to linear regression and correlation. San Francisco: Freeman.en_US
dc.identifier.citedreferenceGanong, L.H. 1987. Integrative reviews of nursing research. Research in Nursing and Health 10:1-11.en_US
dc.identifier.citedreferenceGlass, G.V., B. Mc Gaw, and M.L. Smith. 1981. Meta-analysis in social research. Beverly Hills, CA: Sage.en_US
dc.identifier.citedreferenceHedges, L.V., and I. Olkin. 1985. Statistical methods for meta-analysis. New York: Academic Press.en_US
dc.identifier.citedreferenceHolt, R.R. 1982. Occupational stress. In Handbook of stress, edited by L. Goldberg and S. Breznitz. New York: Free Press.en_US
dc.identifier.citedreferenceHorwitz, R.I., and E.C. Yu. 1984. Assessing the reliability of epidemiologic data obtained from medical records. Journal of Chronic Disease 37:825-31.en_US
dc.identifier.citedreferenceHunter, J.E., and F.L. Schmidt. 1990. Methods of meta-analysis. Beverly Hills, CA: Sage.en_US
dc.identifier.citedreferenceHunter, J.E., F.L. Schmidt, and G.B. Jackson. 1982. Meta-analysis: Cumulating research findings across studies. Beverly Hills, CA: Sage.en_US
dc.identifier.citedreferenceJudd, C.M., E.R. Smith, and L.H. Kidder 1991. Research methods in social relations. 6th ed. Fort Worth, TX: Holt, Rinehart & Winston.en_US
dc.identifier.citedreferenceKaylor, J.A., D.W. King, and L.A. King. 1987. Psychological effects of military service in Vietnam: A meta-analysis. Psychological Bulletin 102:257-71.en_US
dc.identifier.citedreferenceLandman, J., and R. Dawes. 1982. Psychotherapy outcome: Smith and Glass' conclusions stand up under scrutiny. American Psychologist 37:504-16.en_US
dc.identifier.citedreferenceLangenbrunner, J. 1990. Quantitative synthesis methods: Scientific validity and utility for policy. A case study of carotid endarterectomy. Unpublished doctoral dissertation, University of Michigan.en_US
dc.identifier.citedreferenceLight, R. J. (ed.) 1983. Evaluation studies review annual, Vol. 8. Beverly Hills, CA: Sage.en_US
dc.identifier.citedreferenceMandler, G. 1967. Organization and memory. In The psychology of learning and motivation: Advances in research and theory, Vol. 1, edited by K. W. Spence and J. T. Spence, 327-72. New York: Academic Press.en_US
dc.identifier.citedreferenceMatt, G.E. 1989. Decision rules for selecting effect sizes in meta-analysis: A review and reanalysis of psychotherapy outcome studies. Psychological Bulletin 105:106-15.en_US
dc.identifier.citedreferenceOrwin, R.G., and D.S. Cordray. 1985. Effects of deficient reporting on meta-analysis: A conceptual framework and reanalysis. Psychological Bulletin 97:134-47.en_US
dc.identifier.citedreferenceSacks, H.S., J. Berrier, D. Reitman, V.A. Ancona-Berk, and T.C. Chalmers. 1987. Meta-analyses of randomized controlled trials. New England Journal of Medicine, 316:450-55.en_US
dc.identifier.citedreferenceSmith, M.L., and G.V. Glass. 1977. Meta-analysis of psychotherapy outcome studies. American Psychologist 32:752-60.en_US
dc.identifier.citedreferenceSmith, M.L., G.V. Glass, and T.I. Miller. 1980. Benefits ofpsychotherapy. Baltimore, MD: Johns Hopkins University Press.en_US
dc.identifier.citedreferenceStock, W.A., M.A. Okun, M.J. Haring, W. Miller, C. Kinney, and R.W. Ceurvorst. 1982. Rigor in data synthesis: A case study in reliability in meta-analysis. Educational Researcher 11:10-14, 20.en_US
dc.identifier.citedreferenceSusanto, Y.E., W.H. Yeaton, and P.M. Wortman 1990. Multilevel-multifacet meta-analysis: Occupational stress in health care workers. Unpublished manuscript.en_US
dc.identifier.citedreferenceVon Alven, W.H. 1964. Reliability engineering. Englewood Cliffs, NJ: Prentice Hall.en_US
dc.identifier.citedreferenceWarlow, C. 1984. Carotid endarteredomy: Does it work? Stroke 15:1068-76.en_US
dc.identifier.citedreferenceWortman, P.M. 1983 Evaluation research: A methodological perspective. Annual Review of Psychology 34:223-60.en_US
dc.identifier.citedreference—. 1987. Meta-analysis. New England Journal of Medicine 317:575-76.en_US
dc.identifier.citedreferenceWortman, P.M., and L.D. Greenberg. 1971. Coding, recoding, and decoding of hierarchical information in long-term memory. Journal of Verbal Learning and Verbal Behavior 10: 234-43.en_US
dc.identifier.citedreferenceYeaton, W.H., and P.M. Wortman. 1984. Evaluation issues in medical research systhesis. New Directions for Program Evaluation 24:43-56.en_US
dc.owningcollnameInterdisciplinary and Peer-Reviewed


Files in this item

Name:
10.1177_0193841X9301700303.pdf
Size:
883.9KB
Format:
PDF
View/Open

Show simple item record

Remediation of Harmful Language

The University of Michigan Library aims to describe library materials in a way that respects the people and communities who create, use, and are represented in our collections. Report harmful or offensive language in catalog records, finding aids, or elsewhere in our collections anonymously through our metadata feedback form. More information at Remediation of Harmful Language.

Accessibility

If you are unable to use this file in its current format, please select the Contact Us link and we can modify it to make it more accessible to you.