Work Description

This OPR_readme.txt file was generated on 2020-11-10 by WHITNEY A TOWNSEND

GENERAL INFORMATION

1. Title of Dataset: Analyzing Reviewer Responses to Systematic Review Search Methodology through Open Peer Review

2. Author Information
A. Primary Contact Information
Name: Whitney A Townsend
Institution: University of Michigan, Taubman Health Sciences Library
Address: 1135 Catherine, Ann Arbor MI 48109
Email: whitneyt@umich.edu

B. Alternate Contact Information
Name: Mark P MacEachern
Institution: University of Michigan, Taubman Health Sciences Library
Address: 1135 Catherine, Ann Arbor MI 48109
Email: markmac@umich.edu

B. Alternate Contact Information
Name: Jean Song
Institution: University of Michigan, Taubman Health Sciences Library
Address: 1135 Catherine, Ann Arbor MI 48109
Email: jeansong@umich.edu

3. Date of data collection: 2019-10-01 to 2020-10-20

4. Geographic location of data collection: Ann Arbor, MI USA

5. Information about funding sources that supported the collection of the data: None

SHARING/ACCESS INFORMATION

1. Licenses/restrictions placed on the data:
CC0 1.0 Universal (CC0 1.0) Public Domain Dedication - We dedicate this work to the public domain, waiving all our rights to the work worldwide under copyright law.

2. Links to publications that cite or use the data:

3. Links to other publicly accessible locations of the data:

4. Links/relationships to ancillary data sets:

5. Was data derived from another source?
Journals list was pulled from BMC Medicine & Public Health collection (54 titles in October 2019) https://www.biomedcentral.com/journals#Medicine-Public-Health
Citations were exported from the PubMed dataset https://pubmed.ncbi.nlm.nih.gov/

DATA & FILE OVERVIEW

1. File List:

OPR_Readme.txt - data descriptions and file descriptions (this file)

OPR_Journals_SearchStrategy.csv - The journal titles and search strategy used to generate the list of citations reviewed. Journals were identified from the BMC (BioMed Central) Medicine and Public Health collections

OPR_Citations_Reviewed.csv - The list of all citations reviewed (448 total), with reviewer initials (MM=Mark MacEachern; JS=Jean Song; WT=Whitney Townsend)

OPR_Qualtrics_Form.pdf - The review survey form used to screen for inclusion and to record the results of each citation and open peer review file review

OPR_CommentProblem_Definitions.rtf - Definitions and parameters used to determine the expectation for search comment of the final manuscript, and identify when a specified item should be marked as a "problem" on the OPR_Qualtrics_Form.pdf

OPR_Qualtrics_RawData.csv - Results of the citation review in Qualtrics; this data was exported directly from Qualtrics using the "Data: Export Data" function. After download, all fields prior to the form question were deleted (IP address, location, etc)

OPR_Qualtrics_Default_Report.csv - The default results report generated by Qualtrics using "Export Report" tool

SPSS Cross Tabulations Files - These files represent SPSS cross tabulation reports for each of the items classified as "potential problems". The cross tabulation identified which citations were flagged as problems by: peer reviewers only; peer reviewers and experts; and experts only. Each set of files includes the following for each potential problem:

Content of Files by Name Extension:
PMIDs.csv - includes comments and notes for citations that received peer review comments related to the search, indicating whether the comments were appropriate, whether authors responded to the comments, and whether the manuscript was changed to address the comments.

Q5_[x]Q8_[x]crosstab.csv - SPSS cross tabulation of expert and peer reviewer flagging for the specified potential problem

Q5_[x]crosstab.csv - all citations flagged by experts (MM, JS, WT) for the specified potential problem

Q8_[x]crosstab.csv - all citations flagged by peers (peer reviewers, editors) for the specified potential problem

Databases Searched
Q5_1Q8_2db - PMIDs.csv
Q5_1Q8_2db - Q5_2crosstab.csv
Q5_1Q8_2db - Q5_2Q8_3crosstab.csv
Q5_1Q8_2db - Q8_3crosstab.csv

Date Range
Q5_2Q8_3date - PMIDs.csv
Q5_2Q8_3date - Q5_2crosstab.csv
Q5_2Q8_3date - Q5_2Q8_3crosstab.csv
Q5_2Q8_3date - Q8_3crosstab.csv

Translation of the Research Question
Q5_3Q8_4translate - PMIDs.csv
Q5_3Q8_4translate - Q5_3crosstab.csv
Q5_3Q8_4translate - Q5_3Q8_4crosstab.csv
Q5_3Q8_4translate - Q8_4crosstab.csv

Boolean
Q5_4Q8_5boolean - PMIDs.csv
Q5_4Q8_5boolean - Q5_4crosstab.csv
Q5_4Q8_5boolean - Q5_4Q8_5crosstab.csv
Q5_4Q8_5boolean - Q8_5crosstab.csv

Subject Headings
Q5_5Q8_6subjheadings - PMIDs.csv
Q5_5Q8_6subjheadings - Q5_5Crosstab.csv
Q5_5Q8_6subjheadings - Q5_5Q8_6crosstabs.csv
Q5_5Q8_6subjheadings - Q8_6crosstab.csv

Text Words
Q5_6Q8_7text - PMID.csv
Q5_6Q8_7text - Q5_6crosstab.csv
Q5_6Q8_7text - Q5_6Q8_7crosstab.csv
Q5_6Q8_7text - Q8_7crosstab.csv

Spelling, Syntax, Line Numbers
Q5_7Q8_8syntax - PMIDs.csv
Q5_7Q8_8syntax - Q5_7crosstab.csv
Q5_7Q8_8syntax - Q5_7Q8_8crosstab.csv
Q5_7Q8_8syntax - Q8_8crosstab.csv

Limits
Q5_8Q8_9limits - PMIDs.csv
Q5_8Q8_9limits - Q5_8crosstab.csv
Q5_8Q8_9limits - Q5_8Q8_9crosstab.csv
Q5_8Q8_9limits - Q8_9crosstab.csv

Supplementary Searching
Q5_9Q8_10suppl - PMIDs.csv
Q5_9Q8_10suppl - Q5_9crosstab.csv
Q5_9Q8_10suppl - Q5_9Q8_10Crosstab.csv
Q5_9Q8_10suppl - Q8_10crosstab.csv

Presence of Reproducible Search Strategy
Q5_11Q8_1strategy - Crosstab.csv
Q5_11Q8_1strategy - PMIDs.csv
Q5_11Q8_1strategy - Q5_11Crosstab.csv
Q5_11Q8_1strategy - Q8_Crosstab.csv

2. Relationship between files, if important: See folder structure and descriptions above in item 1.

3. Additional related data collected that was not included in the current data package: None

4. Are there multiple versions of the dataset? No

METHODOLOGICAL INFORMATION

1. Description of methods used for collection/generation of data:
We conducted a search through BioMed Central's 54 medicine and public health journals that provide OPR documentation in order to identify systematic review papers published in 2017. For each article we determined if OPR data, reviewer and author comments, were accessible. If so, we assessed the search methodology and reporting quality of the search process with a grading rubric based on PRISMA and PRESS standards, and then mined peer reviewer comments for references to the search methodology.

2. Methods for processing the data:
Data was collected using Qualtrics, and cross tabulations performed for specific items in SPSS.

3. Instrument- or software-specific information needed to interpret the data:

4. Standards and calibration information, if appropriate:

5. Environmental/experimental conditions:

6. Describe any quality-assurance procedures performed on the data:

7. People involved with sample collection, processing, analysis and/or submission: Whitney A Townsend, Mark P MacEachern, Jean Song