Work Description
Title: Analyzing Reviewer Responses to Systematic Review Search Methodology through Open Peer Review Open Access Deposited
Attribute | Value |
---|---|
Methodology |
|
Description |
|
Creator | |
Depositor |
|
Contact information | |
Discipline | |
Keyword | |
Resource type | |
Last modified |
|
Published |
|
Language | |
DOI |
|
License |
(2020). Analyzing Reviewer Responses to Systematic Review Search Methodology through Open Peer Review [Data set], University of Michigan - Deep Blue Data. https://doi.org/10.7302/acjm-cz18
Relationships
- This work is not a member of any user collections.
Files (Count: 47; Size: 1.33 MB)
Thumbnailthumbnail-column | Title | Original Upload | Last Modified | File Size | Access | Actions |
---|---|---|---|---|---|---|
TOWNSEND_OPR_Readme.txt | 2020-11-02 | 2020-11-12 | 5.5 KB | Open Access |
|
|
OPR_Journals_SearchStrategy.csv | 2020-11-02 | 2020-11-02 | 5.56 KB | Open Access |
|
|
OPR_Citations_Reviewed.csv | 2020-11-02 | 2020-11-02 | 169 KB | Open Access |
|
|
OPR_Qualtrics_Form.pdf | 2020-11-02 | 2020-11-02 | 65.9 KB | Open Access |
|
|
OPR_CommentProblem_Definitions.rtf | 2020-11-02 | 2020-11-02 | 9.23 KB | Open Access |
|
|
OPR_QUALTRICS_RAWDATA.csv | 2020-11-02 | 2020-11-02 | 407 KB | Open Access |
|
|
OPR_Qualtrics_Default_Report.csv | 2020-11-02 | 2020-11-02 | 321 KB | Open Access |
|
|
Q5_2Q8_3date_-_PMIDs.csv | 2020-11-12 | 2020-11-12 | 16.3 KB | Open Access |
|
|
Q5_2Q8_3date_-_Q5_2crosstab.csv | 2020-11-12 | 2020-11-12 | 22.7 KB | Open Access |
|
|
Q5_2Q8_3date_-_Q5_2Q8_3crosstab.csv | 2020-11-12 | 2020-11-12 | 1.93 KB | Open Access |
|
|
Q5_2Q8_3date_-_Q8_3crosstab.csv | 2020-11-12 | 2020-11-12 | 9 KB | Open Access |
|
|
Q5_3Q8_4translate_-_PMIDs.csv | 2020-11-12 | 2020-11-12 | 2.81 KB | Open Access |
|
|
Q5_3Q8_4translate_-_Q5_3crosstab.csv | 2020-11-12 | 2020-11-12 | 5.32 KB | Open Access |
|
|
Q5_3Q8_4translate_-_Q5_3Q8_4crosstab.csv | 2020-11-12 | 2020-11-12 | 2.11 KB | Open Access |
|
|
Q5_3Q8_4translate_-_Q8_4crosstab.csv | 2020-11-12 | 2020-11-12 | 1.21 KB | Open Access |
|
|
Q5_4Q8_5boolean_-_PMIDs.csv | 2020-11-12 | 2020-11-12 | 3.5 KB | Open Access |
|
|
Q5_4Q8_5boolean_-_Q5_4crosstab.csv | 2020-11-12 | 2020-11-12 | 21.9 KB | Open Access |
|
|
Q5_4Q8_5boolean_-_Q5_4Q8_5crosstab.csv | 2020-11-12 | 2020-11-12 | 2.07 KB | Open Access |
|
|
Q5_4Q8_5boolean_-_Q8_5crosstab.csv | 2020-11-12 | 2020-11-12 | 994 Bytes | Open Access |
|
|
Q5_5Q8_6subjheadings_-_Q5_5Crosstab.csv | 2020-11-12 | 2020-11-12 | 37.4 KB | Open Access |
|
|
Q5_5Q8_6subjheadings_-_Q5_5Q8_6c...s.csv | 2020-11-12 | 2020-11-12 | 1.71 KB | Open Access |
|
|
Q5_5Q8_6subjheadings_-_Q8_6crosstab.csv | 2020-11-12 | 2020-11-12 | 3.18 KB | Open Access |
|
|
Q5_6Q8_7text_-_PMID.csv | 2020-11-12 | 2020-11-12 | 11.2 KB | Open Access |
|
|
Q5_6Q8_7text_-_Q5_6crosstab.csv | 2020-11-12 | 2020-11-12 | 33.9 KB | Open Access |
|
|
Q5_6Q8_7text_-_Q5_6Q8_7crosstab.csv | 2020-11-12 | 2020-11-12 | 1.69 KB | Open Access |
|
|
Q5_6Q8_7text_-_Q8_7crosstab.csv | 2020-11-12 | 2020-11-12 | 6.83 KB | Open Access |
|
|
Q5_7Q8_8syntax_-_PMIDs.csv | 2020-11-12 | 2020-11-12 | 5.36 KB | Open Access |
|
|
Q5_7Q8_8syntax_-_Q5_7crosstab.csv | 2020-11-12 | 2020-11-12 | 24.2 KB | Open Access |
|
|
Q5_7Q8_8syntax_-_Q5_7Q8_8crosstab.csv | 2020-11-12 | 2020-11-12 | 1.85 KB | Open Access |
|
|
Q5_7Q8_8syntax_-_Q8_8crosstab.csv | 2020-11-12 | 2020-11-12 | 1.4 KB | Open Access |
|
|
Q5_8Q8_9limits_-_PMIDs.csv | 2020-11-12 | 2020-11-12 | 9.74 KB | Open Access |
|
|
Q5_8Q8_9limits_-_Q5_8crosstab.csv | 2020-11-12 | 2020-11-12 | 26 KB | Open Access |
|
|
Q5_8Q8_9limits_-_Q5_8Q8_9crosstab.csv | 2020-11-12 | 2020-11-12 | 2.25 KB | Open Access |
|
|
Q5_8Q8_9limits_-_Q8_9crosstab.csv | 2020-11-12 | 2020-11-12 | 4.22 KB | Open Access |
|
|
Q5_9Q8_10suppl_-_PMIDs.csv | 2020-11-12 | 2020-11-12 | 10.8 KB | Open Access |
|
|
Q5_9Q8_10suppl_-_Q5_9crosstab.csv | 2020-11-12 | 2020-11-12 | 25.3 KB | Open Access |
|
|
Q5_9Q8_10suppl_-_Q5_9Q8_10Crosstab.csv | 2020-11-12 | 2020-11-12 | 2.39 KB | Open Access |
|
|
Q5_9Q8_10suppl_-_Q8_10crosstab.csv | 2020-11-12 | 2020-11-12 | 7.53 KB | Open Access |
|
|
Q5_11Q8_1strategy_-_Crosstab.csv | 2020-11-12 | 2020-11-12 | 2.25 KB | Open Access |
|
|
Q5_11Q8_1strategy_-_PMIDs.csv | 2020-11-12 | 2020-11-12 | 7.89 KB | Open Access |
|
|
Q5_11Q8_1strategy_-_Q5_11Crosstab.csv | 2020-11-12 | 2020-11-12 | 27.3 KB | Open Access |
|
|
Q5_11Q8_1strategy_-_Q8_Crosstab.csv | 2020-11-12 | 2020-11-12 | 6.84 KB | Open Access |
|
|
Q5_5Q8_6subjheadings_-_PMIDs.csv | 2020-11-12 | 2020-11-12 | 8.25 KB | Open Access |
|
|
Q5_1Q8_2db_-_PMID.csv | 2020-11-30 | 2020-11-30 | 6.87 KB | Open Access |
|
|
Q5_1Q8_2db_-_Q5_1crosstab.csv | 2020-11-30 | 2020-11-30 | 12.6 KB | Open Access |
|
|
Q5_1Q8_2db_-_Q5_1Q8_2crosstab.csv | 2020-11-30 | 2020-11-30 | 1.15 KB | Open Access |
|
|
Q5_1Q8_2db_-_Q8_2crosstab.csv | 2020-11-30 | 2020-11-30 | 4.43 KB | Open Access |
|
This OPR_readme.txt file was generated on 2020-11-10 by WHITNEY A TOWNSEND
GENERAL INFORMATION
1. Title of Dataset: Analyzing Reviewer Responses to Systematic Review Search Methodology through Open Peer Review
2. Author Information
A. Primary Contact Information
Name: Whitney A Townsend
Institution: University of Michigan, Taubman Health Sciences Library
Address: 1135 Catherine, Ann Arbor MI 48109
Email: whitneyt@umich.edu
B. Alternate Contact Information
Name: Mark P MacEachern
Institution: University of Michigan, Taubman Health Sciences Library
Address: 1135 Catherine, Ann Arbor MI 48109
Email: markmac@umich.edu
B. Alternate Contact Information
Name: Jean Song
Institution: University of Michigan, Taubman Health Sciences Library
Address: 1135 Catherine, Ann Arbor MI 48109
Email: jeansong@umich.edu
3. Date of data collection: 2019-10-01 to 2020-10-20
4. Geographic location of data collection: Ann Arbor, MI USA
5. Information about funding sources that supported the collection of the data: None
SHARING/ACCESS INFORMATION
1. Licenses/restrictions placed on the data:
CC0 1.0 Universal (CC0 1.0) Public Domain Dedication - We dedicate this work to the public domain, waiving all our rights to the work worldwide under copyright law.
2. Links to publications that cite or use the data:
3. Links to other publicly accessible locations of the data:
4. Links/relationships to ancillary data sets:
5. Was data derived from another source?
Journals list was pulled from BMC Medicine & Public Health collection (54 titles in October 2019) https://www.biomedcentral.com/journals#Medicine-Public-Health
Citations were exported from the PubMed dataset https://pubmed.ncbi.nlm.nih.gov/
DATA & FILE OVERVIEW
1. File List:
OPR_Readme.txt - data descriptions and file descriptions (this file)
OPR_Journals_SearchStrategy.csv - The journal titles and search strategy used to generate the list of citations reviewed. Journals were identified from the BMC (BioMed Central) Medicine and Public Health collections
OPR_Citations_Reviewed.csv - The list of all citations reviewed (448 total), with reviewer initials (MM=Mark MacEachern; JS=Jean Song; WT=Whitney Townsend)
OPR_Qualtrics_Form.pdf - The review survey form used to screen for inclusion and to record the results of each citation and open peer review file review
OPR_CommentProblem_Definitions.rtf - Definitions and parameters used to determine the expectation for search comment of the final manuscript, and identify when a specified item should be marked as a "problem" on the OPR_Qualtrics_Form.pdf
OPR_Qualtrics_RawData.csv - Results of the citation review in Qualtrics; this data was exported directly from Qualtrics using the "Data: Export Data" function. After download, all fields prior to the form question were deleted (IP address, location, etc)
OPR_Qualtrics_Default_Report.csv - The default results report generated by Qualtrics using "Export Report" tool
SPSS Cross Tabulations Files - These files represent SPSS cross tabulation reports for each of the items classified as "potential problems". The cross tabulation identified which citations were flagged as problems by: peer reviewers only; peer reviewers and experts; and experts only. Each set of files includes the following for each potential problem:
Content of Files by Name Extension:
PMIDs.csv - includes comments and notes for citations that received peer review comments related to the search, indicating whether the comments were appropriate, whether authors responded to the comments, and whether the manuscript was changed to address the comments.
Q5_[x]Q8_[x]crosstab.csv - SPSS cross tabulation of expert and peer reviewer flagging for the specified potential problem
Q5_[x]crosstab.csv - all citations flagged by experts (MM, JS, WT) for the specified potential problem
Q8_[x]crosstab.csv - all citations flagged by peers (peer reviewers, editors) for the specified potential problem
Databases Searched
Q5_1Q8_2db - PMIDs.csv
Q5_1Q8_2db - Q5_2crosstab.csv
Q5_1Q8_2db - Q5_2Q8_3crosstab.csv
Q5_1Q8_2db - Q8_3crosstab.csv
Date Range
Q5_2Q8_3date - PMIDs.csv
Q5_2Q8_3date - Q5_2crosstab.csv
Q5_2Q8_3date - Q5_2Q8_3crosstab.csv
Q5_2Q8_3date - Q8_3crosstab.csv
Translation of the Research Question
Q5_3Q8_4translate - PMIDs.csv
Q5_3Q8_4translate - Q5_3crosstab.csv
Q5_3Q8_4translate - Q5_3Q8_4crosstab.csv
Q5_3Q8_4translate - Q8_4crosstab.csv
Boolean
Q5_4Q8_5boolean - PMIDs.csv
Q5_4Q8_5boolean - Q5_4crosstab.csv
Q5_4Q8_5boolean - Q5_4Q8_5crosstab.csv
Q5_4Q8_5boolean - Q8_5crosstab.csv
Subject Headings
Q5_5Q8_6subjheadings - PMIDs.csv
Q5_5Q8_6subjheadings - Q5_5Crosstab.csv
Q5_5Q8_6subjheadings - Q5_5Q8_6crosstabs.csv
Q5_5Q8_6subjheadings - Q8_6crosstab.csv
Text Words
Q5_6Q8_7text - PMID.csv
Q5_6Q8_7text - Q5_6crosstab.csv
Q5_6Q8_7text - Q5_6Q8_7crosstab.csv
Q5_6Q8_7text - Q8_7crosstab.csv
Spelling, Syntax, Line Numbers
Q5_7Q8_8syntax - PMIDs.csv
Q5_7Q8_8syntax - Q5_7crosstab.csv
Q5_7Q8_8syntax - Q5_7Q8_8crosstab.csv
Q5_7Q8_8syntax - Q8_8crosstab.csv
Limits
Q5_8Q8_9limits - PMIDs.csv
Q5_8Q8_9limits - Q5_8crosstab.csv
Q5_8Q8_9limits - Q5_8Q8_9crosstab.csv
Q5_8Q8_9limits - Q8_9crosstab.csv
Supplementary Searching
Q5_9Q8_10suppl - PMIDs.csv
Q5_9Q8_10suppl - Q5_9crosstab.csv
Q5_9Q8_10suppl - Q5_9Q8_10Crosstab.csv
Q5_9Q8_10suppl - Q8_10crosstab.csv
Presence of Reproducible Search Strategy
Q5_11Q8_1strategy - Crosstab.csv
Q5_11Q8_1strategy - PMIDs.csv
Q5_11Q8_1strategy - Q5_11Crosstab.csv
Q5_11Q8_1strategy - Q8_Crosstab.csv
2. Relationship between files, if important: See folder structure and descriptions above in item 1.
3. Additional related data collected that was not included in the current data package: None
4. Are there multiple versions of the dataset? No
METHODOLOGICAL INFORMATION
1. Description of methods used for collection/generation of data:
We conducted a search through BioMed Central's 54 medicine and public health journals that provide OPR documentation in order to identify systematic review papers published in 2017. For each article we determined if OPR data, reviewer and author comments, were accessible. If so, we assessed the search methodology and reporting quality of the search process with a grading rubric based on PRISMA and PRESS standards, and then mined peer reviewer comments for references to the search methodology.
2. Methods for processing the data:
Data was collected using Qualtrics, and cross tabulations performed for specific items in SPSS.
3. Instrument- or software-specific information needed to interpret the data:
4. Standards and calibration information, if appropriate:
5. Environmental/experimental conditions:
6. Describe any quality-assurance procedures performed on the data:
7. People involved with sample collection, processing, analysis and/or submission: Whitney A Townsend, Mark P MacEachern, Jean Song