View Item 

Show simple item record

Categorizing metadata to help mobilize computable biomedical knowledge

dc.contributor.authorAlper, Brian S.
dc.contributor.authorFlynn, Allen
dc.contributor.authorBray, Bruce E.
dc.contributor.authorConte, Marisa L.
dc.contributor.authorEldredge, Christina
dc.contributor.authorGold, Sigfried
dc.contributor.authorGreenes, Robert A.
dc.contributor.authorHaug, Peter
dc.contributor.authorJacoby, Kim
dc.contributor.authorKoru, Gunes
dc.contributor.authorMcClay, James
dc.contributor.authorSainvil, Marc L.
dc.contributor.authorSottara, Davide
dc.contributor.authorTuttle, Mark
dc.contributor.authorVisweswaran, Shyam
dc.contributor.authorYurk, Robin Ann
dc.date.accessioned2022-02-07T20:25:27Z
dc.date.available2023-02-07 15:25:26en
dc.date.available2022-02-07T20:25:27Z
dc.date.issued2022-01
dc.identifier.citationAlper, Brian S.; Flynn, Allen; Bray, Bruce E.; Conte, Marisa L.; Eldredge, Christina; Gold, Sigfried; Greenes, Robert A.; Haug, Peter; Jacoby, Kim; Koru, Gunes; McClay, James; Sainvil, Marc L.; Sottara, Davide; Tuttle, Mark; Visweswaran, Shyam; Yurk, Robin Ann (2022). "Categorizing metadata to help mobilize computable biomedical knowledge." Learning Health Systems 6(1): n/a-n/a.
dc.identifier.issn2379-6146
dc.identifier.issn2379-6146
dc.identifier.urihttps://hdl.handle.net/2027.42/171602
dc.description.abstractIntroductionComputable biomedical knowledge artifacts (CBKs) are digital objects conveying biomedical knowledge in machine‐interpretable structures. As more CBKs are produced and their complexity increases, the value obtained from sharing CBKs grows. Mobilizing CBKs and sharing them widely can only be achieved if the CBKs are findable, accessible, interoperable, reusable, and trustable (FAIR+T). To help mobilize CBKs, we describe our efforts to outline metadata categories to make CBKs FAIR+T.MethodsWe examined the literature regarding metadata with the potential to make digital artifacts FAIR+T. We also examined metadata available online today for actual CBKs of 12 different types. With iterative refinement, we came to a consensus on key categories of metadata that, when taken together, can make CBKs FAIR+T. We use subject‐predicate‐object triples to more clearly differentiate metadata categories.ResultsWe defined 13 categories of CBK metadata most relevant to making CBKs FAIR+T. Eleven of these categories (type, domain, purpose, identification, location, CBK‐to‐CBK relationships, technical, authorization and rights management, provenance, evidential basis, and evidence from use metadata) are evident today where CBKs are stored online. Two additional categories (preservation and integrity metadata) were not evident in our examples. We provide a research agenda to guide further study and development of these and other metadata categories.ConclusionA wide variety of metadata elements in various categories is needed to make CBKs FAIR+T. More work is needed to develop a common framework for CBK metadata that can make CBKs FAIR+T for all stakeholders.
dc.publisherPatient‐Centered Clinical Decision Support Learning Network
dc.publisherWiley Periodicals, Inc.
dc.subject.otherFAIR principles
dc.subject.othermetadata
dc.subject.othertrust
dc.subject.othercomputable biomedical knowledge
dc.subject.otherdigital objects
dc.titleCategorizing metadata to help mobilize computable biomedical knowledge
dc.typeArticle
dc.rights.robotsIndexNoFollow
dc.subject.hlbsecondlevelBiomedical Health Sciences
dc.subject.hlbtoplevelHealth Sciences
dc.description.peerreviewedPeer Reviewed
dc.description.bitstreamurlhttp://deepblue.lib.umich.edu/bitstream/2027.42/171602/1/lrh210271.pdf
dc.description.bitstreamurlhttp://deepblue.lib.umich.edu/bitstream/2027.42/171602/2/lrh210271_am.pdf
dc.identifier.doi10.1002/lrh2.10271
dc.identifier.sourceLearning Health Systems
dc.identifier.citedreferenceKunze J, Baker T. The Dublin core metadata element set. RFC 5013; 2007 Aug.
dc.identifier.citedreferenceHarnisch L, Matthews I, Chard J, Karlsson M. Drug and disease model resources: a consortium to create standards and tools to enhance model‐based drug development. CPT Pharmacomet Syst Pharmacol. 2013; 2 ( 3 ): 1 ‐ 3. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3615532/.
dc.identifier.citedreferenceSandve GK, Nekrutenko A, Taylor J, Hovig E. Ten simple rules for reproducible computational research. PLoS Comput Biol. 2013; 9 ( 10 ). https://doi.org/10.1371/journal.pcbi.1003285.
dc.identifier.citedreferenceShao H, Sun D, Wu J, et al. paper2repo: GitHub repository recommendation for academic papers. Paper presented at: Proceedings of the Web Conference 2020; 2020:629‐639. https://dl.acm.org/doi/fullHtml/10.1145/3366423.3380145.
dc.identifier.citedreferenceBanks M. We need a GitHub for academic research. Slate Published Online April 20, 2017. https://slate.com/technology/2017/04/we-need-a-github-for-academic-research.html. Accessed May 27, 2020.
dc.identifier.citedreferenceGreenberg J. Metadata and digital information [ELIS classic]. In: McDonald JD, Levine‐Clark M, eds. Encyclopedia of Library and Information Science. 4th ed. Boca Raton, Florida, USA: CRC Press; 2017.
dc.identifier.citedreferenceRodriguez M. Research communication futures: a perspective on the FORCE11 scholarly communication institute. Ser Rev. 2018; 44 ( 4 ): 307 ‐ 312. https://www.tandfonline.com/doi/full/10.1080/00987913.2018.1555510.
dc.identifier.citedreferenceMusen MA, Bean CA, Cheung K‐H, et al. The center for expanded data annotation and retrieval. J Am Med Inform Assoc. 2015; 22 ( 6 ): 1148 ‐ 1152. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5009916/.
dc.identifier.citedreferenceGonçalves RS, O’Connor MJ, Martínez‐Romero M, et al. The CEDAR Workbench: An Ontology‐Assisted Environment for Authoring Metadata that Describe Scientific Experiments. Paper presented at: International Semantic Web Conference; 2017; Springer:103–110. https://link.springer.com/chapter/10.1007/978-3-319-68204-4_10.
dc.identifier.citedreferenceSchultes E, Strawn G, Mons B. Ready, set, GO FAIR: accelerating convergence to an internet of FAIR data and services. Paper presented at: DAMDID/RCDL; 2018:19–23. http://ceur-ws.org/Vol-2277/paper07.pdf.
dc.identifier.citedreferenceStarr J, Gastl A. isCitedBy: A metadata scheme for DataCite. Published online 2011. https://dlib.org/dlib/january11/starr/01starr.html.
dc.identifier.citedreferencePerez C. The RDA’s metadata standards directory: information gathering. Published Online 2013. https://rd-alliance.org/sites/default/files/CPerez-RDA-Metadata.pdf.
dc.identifier.citedreferenceZhang L, Powell JJ, Baker DP. Exponential growth and the shifting global center of gravity of science production, 1900–2011. Change Mag High Learn. 2015; 47 ( 4 ): 46 ‐ 49. https://www.tandfonline.com/doi/abs/10.1080/00091383.2015.1053777.
dc.identifier.citedreferenceDublin Core Metadata Initiative. Dublin core metadata element set, version 1.1.
dc.identifier.citedreferenceChong Q, Marwadi A, Supekar K, Lee Y. Ontology‐based metadata management in medical domains. J Res Pract Inform Technol. 2003; 35: 139.
dc.identifier.citedreferenceBuendía F, Gayoso‐Cabada J, Juanes‐Méndez JA, Sierra JL. Transforming unstructured clinical free‐text corpora into reconfigurable medical digital collections. Paper presented at: 2019 IEEE 32nd International Symposium on Computer‐Based Medical Systems (CBMS); 2019 Jun 5:IEEE:519‐522.
dc.identifier.citedreferenceDoerr M. The CIDOC conceptual reference module: an ontological approach to semantic interoperability of metadata. AI Mag. 2003; 24: 75.
dc.identifier.citedreferenceSicilia MA, Garcia E, Sanchez S, Rius A, Pages C. Specifying semantic conformance profiles in reusable learning object metadata. Paper presented at: Information Technology Based Proceedings of the Fifth International Conference onHigher Education and Training, 2004. ITHET 2004; May 31, 2004: IEEE:93‐97.
dc.identifier.citedreferenceMiksa T, Rauber A, Mina E. Identifying impact of software dependencies on replicability of biomedical workflows. J Biomed Informat. 2016; 64: 232 ‐ 254.
dc.identifier.citedreferenceDaniel R, Lagoze C, Payette SD. A metadata architecture for digital libraries. Paper presented at: Proceedings IEEE International Forum on Research and Technology Advances in Digital Libraries‐ADL’98; 1998 Apr 22: IEEE:276‐288.
dc.identifier.citedreferenceCaplan P. Understanding PREMIS. Washington, DC: Library of Congress. 2009.
dc.identifier.citedreferenceW3C. Integrity metadata. https://www.w3.org/TR/SRI/#integrity-metadata. Accessed May 3, 2020.
dc.identifier.citedreferenceLebo T, Sahoo S, McGuinness D, Belhajjame K, Cheney J, Corsar D, Garijo D, Soiland‐Reyes S, Zednik S, Zhao J. Prov‐o: The prov ontology. W3C recommendation; 2013;30.
dc.identifier.citedreferenceWroe C, Goble C, Greenwood M, et al. Automating experiments using semantic data in a bioinformatics grid. IEEE Intell Syst. 2004; 19: 48 ‐ 55.
dc.identifier.citedreferenceda Costa Pereira C, Dubois D, Prade H, Tettamanzi AG. Handling topical metadata regarding the validity and completeness of multiple‐source information: a possibilistic approach. Paper presented at: International Conference on Scalable Uncertainty Management; 2017 Oct 4; Cham: Springer:363‐376.
dc.identifier.citedreferenceBalshem H, Helfand M, Schünemann HJ, et al. GRADE guidelines: 3. Rating the quality of evidence. J Clin Epidemiol. 2011; 64: 401 ‐ 406.
dc.identifier.citedreferenceFriedman CP, Wyatt J. Evaluation Methods in Biomedical Informatics. Berlin/Heidelberg, Germany: Springer Science & Business Media; 2005.
dc.identifier.citedreferenceHalford GS, Wilson WH, Phillips S. Relational knowledge: the foundation of higher cognition. Trends Cogn Sci. 2010; 14 ( 11 ): 497 ‐ 505. https://doi.org/10.1016/j.tics.2010.08.005.
dc.identifier.citedreferenceFloridi L. Semantic information and the network theory of account. Synthese. 2012; 184 ( 3 ): 431 ‐ 454. https://link.springer.com/article/10.1007/s11229-010-9821-4.
dc.identifier.citedreferenceMüller M. Relational knowledge. Relational Knowledge Discovery. Cambridge: Cambridge University Press; 2012: 17 ‐ 37. https://doi.org/10.1017/CBO9781139047869.003.
dc.identifier.citedreferenceDi Iorio A, Caron B. PREMIS 3.0 ontology: improving semantic interoperability of preservation metadata. Paper presented at: Proceedings of the 13th International Conference on Digital Preservation; 2016:32–36.
dc.identifier.citedreferenceGuyatt GH, Oxman AD, Vist GE, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008; 336 ( 7650 ): 924. https://pubmed.ncbi.nlm.nih.gov/18436948/.
dc.identifier.citedreferenceHultcrantz M, Rind D, Akl EA, et al. The GRADE working group clarifies the construct of certainty of evidence. J Clin Epidemiol. 2017; 87: 4 ‐ 13. https://pubmed.ncbi.nlm.nih.gov/28529184/.
dc.identifier.citedreferenceSmith AM, Katz DS, Niemeyer KE. Software citation principles. Comput Sci. 2016; 2: e86.
dc.identifier.citedreferenceLamprecht AL, Garcia L, Kuzak M, et al. Towards FAIR principles for research software. Data Science. 2020; 3 ( 1 ): 37 ‐ 59.
dc.identifier.citedreferenceAlper BS, Richardson JE, Lehmann HP, Subbian V. It is time for computable evidence synthesis: the COVID‐19 knowledge accelerator initiative. J Am Med Informat Assoc. 2020; 27 ( 8 ): 1338 ‐ 1339.
dc.identifier.citedreferenceBowker GC, Star SL. Sorting Things Out: Classification and Its Consequences. Cambridge, Massachusetts, USA: MIT Press; 2000.
dc.identifier.citedreferenceFriedman CP, Flynn AJ. Computable knowledge: an imperative for learning health systems. Learn Health Syst. 2019; 3 ( 4 ): e10203. https://onlinelibrary.wiley.com/doi/full/10.1002/lrh2.10203.
dc.identifier.citedreferenceLehmann HP, Downs SM. Desiderata for sharable computable biomedical knowledge for learning health systems. Learn Health Syst. 2018; 2 ( 4 ): e10065. https://onlinelibrary.wiley.com/doi/full/10.1002/lrh2.10065.
dc.identifier.citedreferenceITU‐T Recommendation X.1255. Framework for discovery of identity management information. Approved on September 4, 2013. http://handle.itu.int/11.1002/1000/11951. Accessed December 4, 2020.
dc.identifier.citedreferenceWilkinson MD, Dumontier M, Aalbersberg IJ, et al. The FAIR guiding principles for scientific data management and stewardship. Sci Data. 2016; 3 ( 1 ). https://doi.org/10.1038/sdata.2016.18, https://www.nature.com/articles/sdata201618.
dc.identifier.citedreferenceMiddleton B, Platt JE, Richardson JE, Blumenfeld BH. Recommendations for Building and Maintaining Trust in Clinical Decision Support Knowledge Artifacts. Research Triangle Park, NC: Patient‐Centered Clinical Decision Support Learning Network. 2018. http://www.pccds-ln.org/sites/default/files/2018-09/TFWG%20White%20Paper_final.pdf.
dc.identifier.citedreferenceMiller E, Ogbuji O, Mueller V, MacDougall K. Bibliographic Framework as a Web of Data: Linked Data Model and Supporting Services. Library of Congress, Washington, DC, 2012. https://www.loc.gov/bibframe/pdf/marcld-report-11-21-2012.pdf
dc.identifier.citedreferenceHumphreys BL. De facto, De rigueur, and even useful: standards for the published literature and their relationship to medical informatics. Proc Annu Symp Comput Appl Med Care. 1990; 7: 2 ‐ 8. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2245563/.
dc.identifier.citedreferenceGold S, Batch A, McClure R, et al. Clinical concept value sets and interoperability in health data analytics. AMIA Annual Symposium Proceedings. Vol 2018, 480. Rockville, Maryland, USA: American Medical Informatics Association; 2018. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6371254/.
dc.identifier.citedreferenceHuff SM, Rocha RA, McDonald CJ, et al. Development of the logical observation identifier names and codes (LOINC) vocabulary. J Am Med Inform Assoc. 1998; 5 ( 3 ): 276 ‐ 292. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC61302/.
dc.identifier.citedreferenceSmith B, Ashburner M, Rosse C, et al. The OBO foundry: coordinated evolution of ontologies to support biomedical data integration. Nat Biotechnol. 2007; 25 ( 11 ): 1251 ‐ 1255. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2814061/.
dc.identifier.citedreferenceKirby JC, Speltz P, Rasmussen LV, et al. PheKB: a catalog and workflow for creating electronic phenotype algorithms for transportability. J Am Med Inform Assoc. 2016; 23 ( 6 ): 1046 ‐ 1052. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5070514/.
dc.identifier.citedreferenceBodenreider O, Nguyen D, Chiang P, et al. The NLM value set authority center. Stud Health Technol Inform. 2013; 192: 1224. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4300102/.
dc.identifier.citedreferencePeleg M, Boxwala AA, Tu S, et al. The InterMed approach to sharable computer‐interpretable guidelines: a review. J Am Med Inform Assoc. 2004; 11 ( 1 ): 1 ‐ 10. https://pubmed.ncbi.nlm.nih.gov/14527977/.
dc.identifier.citedreferenceAlper B, Mayer M, Shahin K, et al. Achieving evidence interoperability in the computer age: setting evidence on FHIR. BMJ Evid‐Based Med. 2019; 24 ( Suppl 1 ): A15.
dc.identifier.citedreferenceAvsec Ž, Kreuzhuber R, Israeli J, et al. The Kipoi repository accelerates community exchange and reuse of predictive models for genomics. Nat Biotechnol. 2019; 37 ( 6 ): 592 ‐ 600. https://pubmed.ncbi.nlm.nih.gov/31138913/.
dc.identifier.citedreferenceCooper G. Causal network discovery from biomedical and clinical data. Published Online 2018. http://hdl.handle.net/1853/59643. Accessed May 13, 2020.
dc.identifier.citedreferenceMüller R, Rogge‐Solti A. BPMN for healthcare processes. Paper presented at: Proceedings of the 3rd Central European Workshop on Services and their Composition (ZEUS 2011); 2011; Karlsruhe, Germany, vol 1.
dc.identifier.citedreferenceBelhajjame K, Corcho O, Garijo D, et al. Workflow‐centric research objects: a first class citizen in the scholarly discourse. Paper presented at: SePublica@ ESWC; 2012:1–12.
dc.identifier.citedreferenceHey T, Tansley S, Tolle K, et al. The Fourth Paradigm: Data‐intensive Scientific Discovery. Vol 1. Microsoft Research, Redmond, WA; 2009.
dc.identifier.citedreferenceWang W, Bleakley B, Ju C, et al. Aztec: A platform to render biomedical software findable, accessible, interoperable, and reusable. ArXiv Prepr ArXiv170606087. Published online 2017. https://arxiv.org/abs/1706.06087.
dc.identifier.citedreferenceWilliams MS, Buchanan AH, Davis FD, et al. Patient‐centered precision health in a learning health care system: Geisinger’s genomic medicine experience. Health Aff (Millwood). 2018; 37 ( 5 ): 757 ‐ 764. https://www.healthaffairs.org/doi/abs/10.1377/hlthaff.2017.1557.
dc.identifier.citedreferenceEtheredge LM. A rapid‐learning health system: what would a rapid‐learning health system look like, and how might we get there? Health Aff (Millwood). 2007; 26 ( Suppl 1 ): w107 ‐ w118. https://www.healthaffairs.org/doi/10.1377/hlthaff.26.2.w107.
dc.identifier.citedreferenceMatheny M, Israni ST, Ahmed M, Whicher D. Artificial intelligence in health care: The hope, the hype, the promise, the peril. Natl Acad Med Prepub. Published online, 2019; 2019. https://nam.edu/wp-content/uploads/2019/12/AI-in-Health-Care-PREPUB-FINAL.pdf.
dc.identifier.citedreferenceStead WW, Searle JR, Fessler HE, Smith JW, Shortliffe EH. Biomedical informatics: changing what physicians need to know and how they learn. Acad Med. 2011; 86 ( 4 ): 429 ‐ 434. https://pubmed.ncbi.nlm.nih.gov/20711055/.
dc.identifier.citedreferenceKahn R, Wilensky R. A framework for distributed digital object services. Corporation Natl Res Initiat Rest. Published online 1995. https://www.doi.org/topics/2006_05_02_Kahn_Framework.pdf.
dc.identifier.citedreferenceWittenburg P, Strawn G, Mons B, Boninho L, Schultes E. Digital Objects as Drivers towards Convergence in Data Infrastructures ( United Kingdom: Research Data Alliance; 2019 ). https://www.rd-alliance.org/sites/default/files/Digital_Objects_as_Drivers_towards_Convergence_in_Data.pdf
dc.identifier.citedreferenceBright TJ, Wong A, Dhurjati R, et al. Effect of clinical decision‐support systems: a systematic review. Ann Intern Med. 2012; 157 ( 1 ): 29 ‐ 43. https://pubmed.ncbi.nlm.nih.gov/22751758/.
dc.identifier.citedreferenceFlynn AJ, Shi W, Fischer R, Friedman CP. Digital knowledge objects and digital knowledge object clusters: unit holdings in a learning health system knowledge repository. 2016 49th Hawaii International Conference on System Sciences (HICSS). New York, NY, USA: IEEE; 2016: 3308 ‐ 3317. https://ieeexplore.ieee.org/document/7427597.
dc.identifier.citedreferenceMandl KD, Kohane IS, McFadden D, et al. Scalable collaborative infrastructure for a learning healthcare system (SCILHS): architecture. J Am Med Inform Assoc. 2014; 21 ( 4 ): 615 ‐ 620. https://pubmed.ncbi.nlm.nih.gov/24821734/.
dc.identifier.citedreferenceBoxwala AA, Rocha BH, Maviglia S, et al. A multi‐layered framework for disseminating knowledge for computer‐based decision support. J Am Med Inform Assoc. 2011; 18 ( Suppl 1 ): i132 ‐ i139. https://pubmed.ncbi.nlm.nih.gov/22052898/.
dc.identifier.citedreferenceFox J, Johns N, Rahmanzadeh A. Disseminating medical knowledge: the PROforma approach. Artif Intell Med. 1998; 14 ( 1–2 ): 157 ‐ 182. https://pubmed.ncbi.nlm.nih.gov/9779888/.
dc.identifier.citedreferenceCeusters W, Smith B. Aboutness: Towards foundations for the information artifact ontology. Published online 2015. http://ceur-ws.org/Vol-1515/regular10.pdf.
dc.identifier.citedreferenceFlynn AJ, Friedman CP, Boisvert P, Landis‐Lewis Z, Lagoze C. The knowledge object reference ontology (KORO): a formalism to support management and sharing of computable biomedical knowledge for learning health systems. Learn Health Syst. 2018; 2 ( 2 ): e10054. https://onlinelibrary.wiley.com/doi/full/10.1002/lrh2.10054.
dc.identifier.citedreferenceSimon HA, Newell A. Human problem solving: the state of the theory in 1970. Am Psychol. 1971; 26 ( 2 ): 145. https://doi.org/10.1037/h0030806.
dc.identifier.citedreferenceManifesto of the Mobilizing Computable Biomedical Knowledge Community Movement. Michigan, USA: University of Michigan; 2016. https://mobilizecbk.med.umich.edu/about/manifesto. Accessed May 27, 2020.
dc.identifier.citedreferenceLomotan EA, Meadows G, Michaels M, Michel JJ, Miller K. To share is human! Advancing evidence into practice through a National Repository of interoperable clinical decision support. Appl Clin Inform. 2020; 11 ( 01 ): 112 ‐ 121. https://pubmed.ncbi.nlm.nih.gov/32052388/.
dc.working.doiNOen
dc.owningcollnameInterdisciplinary and Peer-Reviewed


Files in this item

Name:
lrh210271.pdf
Size:
814.6KB
Format:
PDF
View/Open
Name:
lrh210271_am.pdf
Size:
477.5KB
Format:
PDF
View/Open

Show simple item record

Remediation of Harmful Language

The University of Michigan Library aims to describe library materials in a way that respects the people and communities who create, use, and are represented in our collections. Report harmful or offensive language in catalog records, finding aids, or elsewhere in our collections anonymously through our metadata feedback form. More information at Remediation of Harmful Language.

Accessibility

If you are unable to use this file in its current format, please select the Contact Us link and we can modify it to make it more accessible to you.