CRESCYNT Toolbox: Data Repositories – Estate Planning for your Data

“Hypotheses come and go but data remain.”    – Ramon y Cajal

Taking care of our data for the long term is not just good practice, allowing us to share our data, defend our work, reassess conclusions, collaborate with colleagues, and examine broader scales of space and time – it’s also estate planning for our data, and a primary way of communicating with future scientists and managers.

egg-gold

Here are some great options for long-term data storage, highlighting repositories friendly to coral reef science.

First, there are some important repository networks useful for coral reef data – these can unify standards and offer collective search portals: we like DataONE (members here) and bioCaddie (members here).

KNB – the Knowledge Network for Biocomplexity offers open and private data uploads; ecological orientation. DataONE network.

NOAA CoRIS: Coral Reef Information System – often free to use and can accept coral reef related data beyond NOAA’s own data; contact them first.

BCO-DMO – Biological and Chemical Oceanography Data Management Office – if you have an NSF grant that requires data storage here, you’re fortunate. Good data management guidelines and metadata templates, excellent support staff. Now a DataONE member.

Dataverse – supported by Harvard endowments. There are multiple organizational dataverses – the Harvard Dataverse is free to use. bioCaddie member.

Zenodo – free to use, supported by the European Commission (this is a small slice of CERN’s enormous repository for the Large Hadron Collider). Assigns dois. We invite you to include the “Coral Reef” community when you upload. bioCaddie member.

NCBI – the National Center for Biotechnology Information is very broadly accepted for ‘omics data of all types. A bioCaddie member.

DataCite – not a repository, but if you upload a dataset at a repository that does not assign its own doi’s, you can get one at DataCite and include it when publishing your datasets.

We’ve not listed more costly repositories such as Dryad (focused on journal requirements) or repositories restricted to institutions. What about other storage options such as GitHub, Amazon Web Services, websites? Those have important uses, but are not curated repositories with long-term funding streams, so are not the best data legacy options.

eggs-stacked-imagesMost of these repositories allow either private (closed) or public (open) access, or later conversion to open access. Some have API’s for automated access within workflows. These are repositories we really like for storing and accessing coral reef work. Share your favorite long-term data repository – or experiences with any of the repositories listed here – in the comments.

CRESCYNT Toolbox: Data Repositories – Estate Planning for your Data

CRESCYNT Toolbox – Disaster Planning and Recovery

With computers, the question is not whether they will fail, but when.

tl;dr – It’s very practical to have cloud storage backup in addition to still-useful external hard drive backup routines. Here are some secure cloud alternatives.

itcrowd_giphyPersonal note. I’ve had hard drive failures due to lightning strike; simultaneous death of mirrored hard drives within a RAID; drenching from an upper floor emergency shower left flowing by a disgruntled chemistry student; and most recently, demise of my laptop by sudden immersion in salt water (don’t ask). By some intersection of luck and diligence, on each occasion recent backups were available for data recovery. In the most recent remake, it was a revelation how much work is now backed up via regular entry into the casual cloud.

This latest digital landing was mercifully soft (…cloudlike). Because of work portability, my recent sequential backup habit has been to a paid unshared Dropbox account; $10/mo is a bargain for peace of mind (beyond a certain size, restoration is not drag-and-drop). A surprising number of files these days are embedded in multiple team projects – much on Google Drive – so all of that was available, with revision history. Group conversations and files were on Slack and email. One auxiliary brain (iPhone)  was in a waterproof case with cloud backup, and another auxiliary brain (project/task tracking) was in a web app, KanbanFlow. Past years of long-term archives were already on external hard drives in two different cities. GitHub is an amazing place to develop, document, recover and share work in progress and products, but it is not a long-term curated data repository. For valuable datasets, the rule is to simplify formats, attach metadata, and update media periodically.

Thinking about your own locations for data storage and access? Check out this review of more secure alternatives to – and apps on top of – Dropbox. Some, like OwnCloud, can serve as both storage and linked access for platforms like Agave. A strength of some current analytical platforms is that they can access multiple data storage locations; for example, Open Science Framework can access Dropbox, Google Drive, GitHub, Box, figshare, and now Dataverse and Amazon Web Services as well.

A collaborator recently pointed out that the expense of any particular type of data storage is really the expense of its backup processes: frequency, automation, security, and combination of archiving media. Justifying the expense can come down to this question: What would it cost to replace these data? Some things are more priceless than others.

Disaster Planning and Recovery tools.  To go beyond data recovery in your planning, here’s an online guide for IT disaster recovery planning and cyberattacks. How much of a problem is this really? See Google’s real-time attack map (hit “play”). Better to plan than fear. You did update those default passwords on your devices, yes?

Feel free to share your own digital-disaster-recovery story in the comments.

CRESCYNT Toolbox – Disaster Planning and Recovery