Data Discovery Science Competition

 

Come Find the Data You Need!

Grow your skills and your data resources in the Data Discovery Science Competition at DataDiscoveryStudio.org

Data Discovery Studio is built to help scientists find data. It hosts records for over 1.6 million datasets and resources from over 40 repositories, and continues to grow. Individuals, classes, and sci-tech teams can compete in two categories:

(1) Build a shareable Collection of datasets from existing resources, and add links to other datasets and resources in other repositories. We show you how!

(2) Modify existing Jupyter Notebooks or create your own to explore a data type of interest. Guidance provided!

Find details at bit.ly/ddscompete or jump in and start exploring at DataDiscoveryStudio.org – enter by May 1, 2019.

Please share broadly! We built this NSF-funded EarthCube gateway for you, want it to be useful, and eagerly welcome your feedback.

_____

Note to coral reef scientists: the site already includes over 16,600 records of coral reef datasets and resources – enough to begin a select collection of your own to share with collaborators, and point to more.

More Details:

Both competition categories contribute to science. (1) Collections will gather and tag resources, including datasets, with a focus you choose for research or community use. (2) Jupyter Notebooks will help develop workflows for datasets with a common theme or structure. Both areas will help address real science questions, share results and resources with others, and will help improve the gateway for future users. The competition is suitable for individual, team, or class entry.

DDStudio’s strongest holdings in datasets and resources are in earth, ocean, atmospheric, hydrologic, geologic, planetary, ecological, and other geoscience domains, broadly defined. Try out a favorite search term at datadiscoverystudio.org – if DDStudio doesn’t have what you need, you can contribute a new link, or point us to repositories to add. DDStudio goes beyond search: some datasets can be explored right on site using example Jupyter Notebooks (e.g., sensor data) – or develop and contribute your own Notebooks.

You can’t win if you don’t enter – show up and represent your domain! bit.ly/ddscompete

New Opportunities – Spring 2018

We’ve been busy lately, including following up on the Data Rescue and Data Integration workshops with more materials to share (you can already access most of them here!), and wanted to share some new opportunities.

EarthCube All Hands Meeting

The EarthCube All Hands Meeting will be held June 6-8, 2018 in Washington, DC (our abstracts here). We are very excited to be able to bring a coral-reef-science guest or two to this meeting through reimbursement of travel and registration costs. Two people who work with coral reef data from a repository perspective will also attend with us as part of the CRESCYNT community. Please email us if you’re interested! (Registration is $250 before May 11)

CoralNET Software Update

CoralNET software for automated analysis of coral reef benthic imagery is getting a revision, and its developers would like your input. What feature improvements and new developments would you like to see? Add your feedback to this thread. More background here. (If you haven’t tried it yet, this is the time to do it – fresh users are a great source of important feedback.)

Nat’l Academies Workshop on Interventions to Increase Resilience of Coral Reefs

Attendance is free and open to the public, online or in person.

View the agenda and register.

 

>>>Go to the blog Masterpost or the CRESCYNT website or NSF EarthCube.<<<

CRESCYNT Data Science for Coral Reefs – Workshop Materials

Thanks to Jan Vincente and Ingrid Knapp for this pair of Data Rescue images.

Written while embedded in our CRESCYNT Data Science for Coral Reefs workshops. Amazingly, everyone who participated in workshop 1 – Data Science for Coral Reefs: Data Rescue – learned even more than they thought they would. We’ve had wonderful NCEAS trainers, spectacular participants with amazing datasets, and a lot of hard work over 4 days (March 7-10, 2018).

UPDATE: Here is the Data Rescue workshop agenda we used, with links to all of the training slides.

In the second intensive workshop – Data Science for Coral Reefs: Data Integration and Team Science – people will be introduced to R Studio and GitHub if they have not used them before, and then we will work on exploring techniques for integrating disparate datasets. We’ll start with a pair of datasets at a time, and efforts may involve extracting data from one dataset based on observations from another; upscaling, downscaling, resampling, or summarizing to make intervals and scales mesh – exactly the kind of process that coral reef researchers have said is a recurring challenge in asking bigger science questions.

UPDATE: Here is the Data Integration and Team Science workshop agenda we used, with links to all of those training slides and exercises.

Each workshop group is writing a paper to summarize and share lessons learned, so please stay tuned for those!

We experimented with an unusual process for these workshops: two days of training followed by two days of workathon. We’re liking it! Tell us what you think about these topics and training materials. What other workshop outputs would you like to see?

>>>Go to the blog Masterpost or the CRESCYNT website or NSF EarthCube.<<<

 

Chasing Coral is now on Netflix – A Powerful Film to See and Share

Coral Bleaching and its Aftermath – a scene from Chasing Coral. Credit: The Ocean Agency – XL Catlin Seaview Survey – Richard Vevers & Christophe Bailhache

Several of the coral reef scientists featured in the film Chasing Coral are CRESCYNT participants, including our PI, Dr Ruth D Gates, and we congratulate and thank them all for their eloquence, passion, deep experience, scientific integrity, and significant intellectual contributions to this powerful film. Chasing Coral‘s producers are making it available free for public screenings, and its focus now is educating audiences and moving people to action.

“It’s not too late for coral reefs…  indeed, for many other ecosystems that are facing challenges from climate change. It’s still possible to reduce the rate at which the climate is changing, and that’s within our power today.” – Dr Ove Hoegh-Guldberg

 

>>>Go to NSF EarthCube or the CRESCYNT website or the blog Masterpost.<<<

CRESCYNT at EarthCube All Hands Meeting 2017

EarthCube domain scientists, computer scientists, data scientists, and new members gathered in Seattle June 7-9, 2017 to communicate progress, connect over projects and science challenges, plan for future collaborative work, and welcome new participants.

Most of the presentations and posters from the meeting are available here. CRESCYNT program manager Ouida Meier delivered an invited talk on sci-tech matchmaking (video|slides, helped facilitate breakout sessions focused on clarifying requirements and resources for virtual workbenches (summary), and presented CRESCYNT coral reef use cases and workflow collaboration during a poster session. Discussion and collective brainstorming throughout the meeting was very dynamic and fruitful.

Download a larger pdf of the CRESCYNT poster – Earth Cube AHM 2017.

Read more EarthCube in the News.

 

>>>Go to NSF EarthCube or the CRESCYNT website or the blog Masterpost.<<<

Resources for Coral Reef Education – by Judy Lemus

We all recognize that communication and education about science concepts and the process of science is more important than ever.  Fortunately, coral reefs are charismatic ecosystems that inspire much curiosity, concern, and interest from many sectors of society.  While there is no shortage of stunning images and videos online, resources that combine these visuals with robust educational content can be more challenging to identify; they do exist and I’ve put together some of my favorites here. The list is not exhaustive, and we welcome your suggestions for great additions.

EDUCATIONAL WEBSITES. These resources provide educational information about coral reefs across multiple levels and concepts, often using multimedia.

Khaled bin Sultan Living Oceans Foundation Coral Reef Ecology Curriculum. The KSLOF has perhaps the most comprehensive website on coral reef ecology. The site is set up as a course with several units and resources with very nice graphics and high quality videos geared specifically for students and teachers. Lessons are aligned with the Next Generation Science Standards, Ocean Literacy Principles, and Common Core State Standards for K-12, but some of the material could easily be used in a college level course. A major downside to this site is that one must register to use it.

Smithsonian Ocean Portal. The Smithsonian’s website for coral and coral reefs is not as media-rich as the KSLOF, but does have a great deal of scientific information about corals.  Only a couple of lesson plans are offered, but the richness of the content lies in the embedded links to additional images and other stories. The science is backed up with oversight by Smithsonian coral reef biologist Nancy Knowlton.

MarineBio Coral Reefs. The MarineBio website is somewhat of a clearinghouse for other marine bio resources, but the educational content on coral reefs is good quality and quite extensive if you follow the links.  Like the Smithsonian site, there are links to both internal and external resources. The short videos featured throughout the site, generally from outside sources, are particularly engaging.

OTHER WEBSITES WITH EXTENSIVE INFORMATION ABOUT CORAL REEFS

National Ocean Service

NOAA Coral Reef Conservation Program

USGS Coral Reef Project

Coral Reef Alliance

Teach Ocean Science

ReefBase

Great Barrier Reef Foundation

Coral Triangle Initiative

Endangered Reefs, Threatened People

Coral Health Atlas

VIDEOS ABOUT CORALS AND CORAL REEFS. There are loads of videos of corals and coral reefs on the web; these excellent examples incorporate educational content.

Catlin Seaview

Chasing Coral (available through Netflix)

Climate Change: Coral Reefs on the Edge

Exploring the Coral Reef: Learn about Oceans for Kids

Corals Under Confocal

Coral bleaching caused by heating water (time-lapse)

Life Noggin – What Happens if All the Coral Dies? (animation)

Coral Bleaching Animation – HHMI BioInteractive Video (animation)

Coral Bleaching on the Great Barrier Reef (animation)

SCIENCE NEWS SITES. These science news websites regularly post stories on coral reefs.

ScienceDaily

LiveScience

---

Thanks to Dr. Judy Lemus for this cream-of-the-crop list. Judy is a Faculty Specialist in Science Education at the Hawaii Institute of Marine Biology; fortunately for us, she is also the Education Node Leader for CRESCYNT. You can download Judy’s list in pdf format.

>>>Go to NSF EarthCube or the CRESCYNT website or the blog Masterpost.<<<

CRESCYNT Toolbox – Workflows as Collaboration Space and Workbench Blueprint

Scientists need better ways to analyze and integrate their data and collaborate with other scientists; new computing technologies and tools can help with this. However, it’s difficult to overcome the challenge of disparate perspectives and the absence of a common vocabulary: this is true of multidisciplinary science teams, and true when scientists try to talk with computer scientists. Workflows, as a way to help design and implement a workbench, are needed both as a collaboration space and a blueprint for implementation.

Take a look at a recent presentation to the EarthCube science committee (video) or an earlier presentation offered at ASLO 2017 (slides and voice) to see a flexible and low-tech way to simultaneously (1) facilitate necessary sci-tech interactions for your own lab and (2) begin to sketch out a blueprint for work that needs to be done. Subsequent technical implementation is possible with new tools including Common Workflow Language (CWL) as a set of specifications, Dockers as modular and sharable containers for either fully developed tools or small pieces of code, and Nextflow as an efficient and highly scalable definitive software language to make the computational work happen. Look for a post in the near future by Mahdi Belcaid to describe the technical implementation of these workflows.

OPPORTUNITY! We will be hosting one or two in-person skills training workshops in the coming months, with your expenses covered by our NSF EarthCube CRESCYNT grant, focused particularly on training early career professionals, and will work through some challenging coral reef use cases and their cyberinfrastructure needs. We collected some great use cases at ICRS, but would like additional cases to consider, so we invite you to describe your own research challenges through this google form. Please contact us for more on this, or other issues. Thanks!

>>>Go to the blog Masterpost or the CRESCYNT website or NSF EarthCube.<<<

Cyber Tools and Resources for Research and Analysis – Workshop at ASLO

AGENDA with LINKS and REMOTE LOGIN information at bit.ly/cybertools-aslo

Join us for a great workshop!

• When: Sunday, Feb. 26, 2017, 9:00 am – 3:30 pm HST – PLUS optional visit to the LAVA visualization lab – back to convention center by 5:00 pm)
• Where: Hawaii Convention Center, Honolulu, HI, room 323 B
• How: email omeier@hawaii.edu to reserve your space (and your LUNCH).
• Cost to you: None, thanks to generous workshop guides & NSF EarthCube CRESCYNT funding
• Not registered yet? We still want you (but can’t promise lunch or transportation)

Can’t attend in person? Join the workshop online! (starts 11:00am PST, 2:00pm EST)

WORKSHOP TOPICS AND PRESENTERS:

• EarthCube: A Community-Driven Cyberinfrastructure for the Geosciences

Mohan Ramamurthy, UCAR [video]


• ECO-GEO Virtual Machine – an environmental genomics workbench

Elisha Wood-Charlson, UH Manoa [video]


• Multidisciplinary Challenges in Ocean Science Research

Jay Pearlman, J&F Ent, U Colo [video]


• SeaView: Connecting Ocean Data Repositories with Science and Visualization
  

Stephen Diggs, Scripps [video]


• Cyber Visualization Tools

Alberto Gonzalez, UH Manoa [video]


• Geoscience Papers of the Future (digital tools training)

Daniel Garijo, USC [video]


• More Cyber Tools for Research – the Toolbox

Ouida Meier, UH Manoa [video]


• EarthCube Integration and Test Environment

Emily Law, JPL [video]


• Workflow Assembly (hands-on conceptual assembly of data and tools)

Ouida Meier & participants


• Trip – visit the Laboratory for Advanced Visualization and Applications

Jason Leigh, UH Manoa

If you’re attending ASLO (Association for Sciences in Limnology and Oceanography) in Hawaii or will be in Honolulu on Feb. 26th and care about better ways to collaborate, solve data and workflow challenges, or take the next steps in the relentless digital revolution, join us in person!

We’re excited to be able to offer this workshop as a real-time webinar – please participate remotely if you can! We have an amazing lineup of presenters and workshop guides. Don’t miss out!

Funding gratefully acknowledged from NSF EarthCube CRESCYNT Coral Reef Science and Cyberinfrastructure Network, Ruth D. Gates, PI (crescynt.org)

Update: original post updated with slide links.

CRESCYNT Toolbox – What do we need on the workbench?

Science is a team sport. Collaborations allow us to ask more ambitious science questions, but also intensify the need to connect disparate datasets across scales of time and space. Solving data interoperability challenges requires technological solutions not yet in place, so we’re taking the initiative to review potential solutions.

We really want to include YOUR favorite tools and highest priority needs in this review, so please share those – we’ll report back.

A platform from EarthCube is some time and distance away, but we have a chance to start assembling tools already at hand and in use for coral reef research workflows and do some testing. The process also helps us ground the ideal in the practical.

What are  some criteria for a great infrastructure platform?

Ideally, solutions are: 1) modular, so when an improved tool is available it can be incorporated without restructuring the system; 2) free or low cost, so solutions are sustainable for most research labs; and 3) open source, allowing continued development from multiple disciplines and directions. However, we also want to start where people are, with the tools we’re already using – many of these are less than ideal but we make them work. That’s our starting place, and we want to hear about all of your tools.

It is tempting to set up a workbench for the challenge of analysis alone, but in a coral reef research lab we immediately crash into the realities of group data collection, field and lab work, physical specimens, and intersecting projects. All of these characteristics create additional layers of challenge. In the long run, infrastructure should help capture data and metadata generation at the source, and ease tracking, analysis, and replicability.

Good infrastructure solves more problems than it creates in compliance, skill demand, and management. An effective system helps graduate students and postdocs develop robust skills in managing data into the future, with guidelines that work for people, labs, and collaborators. Interoperability challenges must be solved for datasets that range from remote sensing to ecological surveys to bioinformatics work. Data cleaning, analysis, visualization, and mapping must be supported in flexible ways to clearly communicate research insights.

We have an opportunity to construct a preliminary array of tools and workflows on a cyberinfrastructure workbench for coral reef research, so if this is going to be a broadly useful start we need to know what you already like using and what more you wish you had. Dream big to start with, and along the way we’ll acknowledge the distinction between perfect and good-enough solutions.

Please complete this survey, or comment below. Thank you so much!

3D Mapping of Coral Reefs – How to Get Started – by John Burns

Rapid technological advancements are providing a suite of new tools that can help advance ecological and biological studies of coral reefs. I’ve studied coral health and disease for the last several years. One large gap in our research approach is the ability to connect changes in coral health to large-scale ecological processes. I knew that when corals died from disease it would alter the fundamental habitat of the system, which in turn would impact associated reef organisms. What I didn’t know was how to effectively document and quantify these changes. Sometimes we just need to alter our perspective to find the answers we are looking for. I starting reviewing methods used by terrestrial researchers to measure landscape changes associated with landslides and erosion. In doing so I came across structure-from-motion (SfM) photogrammetry, and it was immediately clear that this technique could improve our understanding of coral reef ecosystems. I spent the next few years developing methods to use this approach underwater, and have since used SfM to detect changes in reef structure associated with disturbances as well as improve our understanding of coral diseases.

 

The first question I am usually asked is, “How easy is it to use this technique and what does it cost?” The best answer I can provide is the logistic constraints depend on your research question. If you are interested in accuracy and controlling the parameters of the 3D reconstruction process, then you should use proprietary software like Agisoft PhotoScan and Pix4D. These programs give you full control, yet require more understanding of photogrammetry and substantial computing power. Autodesk ReCap can process images remotely, which reduces the need for a powerful computer, but also reduces your control over the 3D reconstruction process. At the most simple level, you can download the Autodesk 123D catch app on your phone and create 3D reconstructions in minutes! There are also multiple open-source software options, but they tend to be less powerful and lack a graphical user interface. My advice is to start small. Get started with some simple and free open source tools such as Visual SfM or Bundler. Collect a few sets of images and get some experience with the processing steps to determine if the model outputs are applicable for your research approach.

The second question I receive is, “What is the best way to collect the images?” Unfortunately, the answer is not to use the ‘auto’ setting on your camera and just take a bunch of pictures. Image quality will directly affect the resolution of your model, and is also important for stitching and spatial accuracy. Spend time to understand the principles of underwater photography. A medium aperture (f-stop of 8 to 11) will let in enough light in ambient conditions while not causing blur and distortion associated with depth of field. Since images are taken while moving through a scene, a high enough shutter speed is required that will eliminate blur and dark images. Since conditions can be highly variable, one must adapt to changes in light and underwater visibility while in the field. Cameras with auto-ISO can be helpful for dealing with changing light conditions while surveying. I also recommend DSLR or mirrorless cameras with high-quality fixed lenses, as they will minimize distortion and optimize overall resolution and clarity. For large areas I won’t use strobes because I take images from large distances off the reef, and this will typically create shadows in the images. I take images of the reef from both planar and oblique angles to capture as much of the reef scene as possible in order to eliminate ‘black holes’ in the resulting model. There is no ‘perfect approach,’ but you will need 70-80% overlap for accurate reconstruction. I swim in circular or lawn-mower patterns depending on the scene, and swear by the mantra that more is better (you can always throw out images later if there is too much overlap). It is worth investing time in experimenting with methods to develop a technique that works best for your study are and experimental design. SfM is a very flexible and dynamic tool, so don’t be afraid to create your own methods.

The third question is then, “How do you ground-truth the model for spatial accuracy?” This is a critical step that often gets overlooked. In order to achieve mm-scale accuracy the software must be able to rectify the model to known x,y,z coordinates. I use mailbox reflectors connected by PVC pipe to create ground control points (GCPs) with known distances. The red color and white outline of the reflectors is easily distinguished and identified by the software and saves a lot of time for optimizing the coordinates of the model. Creating functional GCPs is exceptionally important is spatial accuracy is required for your work. I also use several scale bars throughout my reef plots to check accuracy and scaling. This step of the process is critical for accurately measuring 3D habitat characteristics.

Maybe I’ve taken you too far into technical details at this point, but hopefully this helps for anyone looking to venture into the world of SfM. There is no perfect approach, and we must be adaptable as software continues to improve and new tools are constantly being created. We also need to continue to develop new methods for quantifying structure from 3D models. I export my models into geospatial software to extract structural information, but this step of the process can be improved with methods capable of annotating the true 3D surface of the models. As new software becomes available for annotating 3D surfaces we are entering an exciting phase with endless possibilities for collating and visualizing multiple forms of data. Being open-minded and creative with these techniques may provide new insight into how these environments function, and how we can protect them in the face of global stressors.

– Mahalo to John Burns for this in-depth guest posting. You can see more of his work, simultaneously beautiful and useful, at the Coral Health Atlas. Click on the image below for more of John’s remarkable 3D coral reef mapping work:

>>>Go to NSF EarthCube or the CRESCYNT website or the blog Masterpost.<<<