Data Sharing Workflow for Large Datasets (>1TB) with Globus
dc.contributor.author | Borda, Susan | |
dc.date.accessioned | 2018-02-21T17:08:42Z | |
dc.date.available | 2018-02-21T17:08:42Z | |
dc.date.issued | 2018-02-20 | |
dc.identifier.uri | https://hdl.handle.net/2027.42/142389 | |
dc.description | Poster | en_US |
dc.description.abstract | In the Deep Blue Data (DBD) repository, built on Fedora/Samvera, at the University of Michigan, we have datasets ranging in size from KBs to TBs. Like many institutions, we are struggling with an uptick in file sizes (total data stored in DBD will increase from 2.94TB in October to over 16TB by the end of December). Until recently we were unable to download files larger than 20GB from DBD, which significantly hampered our ability to serve the researchers in our university. However, as of early November, we have started testing the integration of Globus (https://www.globus.org/) with the Deep Blue Data interface as a means of downloading these larger files. Our poster will illustrate the workflow we have created to ingest these large datasets as well as the process of integrating Globus with our instance of Fedora/Samvera. We will be using an actual case study of the workflow from researcher network storage to final Globus download endpoint. The code for this Globus integration is available on GitHub. | en_US |
dc.language.iso | en_US | en_US |
dc.rights | CC0 1.0 Universal | * |
dc.rights.uri | http://creativecommons.org/publicdomain/zero/1.0/ | * |
dc.title | Data Sharing Workflow for Large Datasets (>1TB) with Globus | en_US |
dc.type | Other | en_US |
dc.subject.hlbsecondlevel | Information and Library Science | |
dc.subject.hlbtoplevel | Social Sciences | |
dc.description.peerreviewed | Peer Reviewed | en_US |
dc.contributor.affiliationumcampus | Ann Arbor | en_US |
dc.description.bitstreamurl | http://deepblue.lib.umich.edu/bitstream/2027.42/142389/1/Borda-IDCC2018-poster.pdf | en |
dc.description.bitstreamurl | http://deepblue.lib.umich.edu/bitstream/2027.42/142389/5/Borda-MIDAS2018-poster.pdf | en |
dc.description.filedescription | Description of Borda-IDCC2018-poster.pdf : Poster | |
dc.owningcollname | Library (University of Michigan Library) |
Files in this item
Remediation of Harmful Language
The University of Michigan Library aims to describe library materials in a way that respects the people and communities who create, use, and are represented in our collections. Report harmful or offensive language in catalog records, finding aids, or elsewhere in our collections anonymously through our metadata feedback form. More information at Remediation of Harmful Language.
Accessibility
If you are unable to use this file in its current format, please select the Contact Us link and we can modify it to make it more accessible to you.