Last week, the Horizon 2020 PhenoMeNal project held a training workshop on e-infrastructures for metabolomics. It was a hands-on developers workshop looking at different cloud technologies and how they might be deployed and utilised for dealing with computational workflows in metabolomics, as well as for data management. The ISA team at Oxford is a partner on the project.
The focus of the workshop was on kick-starting the development of the analytics infrastructure for PhenoMeNal. It is envisaged that a Europe-wide cloud infrastructure will be deployed, which might be a mix of public and private clouds (private secure clouds for dealing with patient-identifiable data), with the compute elements taking the form of microservice containers. In this workshop, we learned about how we can create microservices containerised with Docker, and use them on the Google Cloud infrastructure orchestrated by the MANTL framework.
Hacking at the PhenoMeNal e-infrastructures workshop at SciLifeLab Uppsala, Sweden.
Now, you are probably wondering right now, “What does this have to do with ISA?”
During one of the hacking sessions in the workshop I worked with Ken Haug from the Metabolights (a popular metabolomics database that stores ISA tab files natively) team at EMBL-EBI, on working out a simple use case for our recent earlybird release of the Python ISA API. Here’s what we came up with.
First, we wrap up two file converters from the ISA API as Docker images (you can find these on Github):
- isatab2json – to convert ISA tab files to our ISA JSON format
- json2isatab – to convert JSON back to ISA tab.
Next, we create an iPython notebook using Jupyter that makes REST calls to our MANTL cluster running in Google Cloud. These REST calls simply ask MANTL to run each of the aforementioned Docker images as microservices. From our notebook, we can now:
- Convert an ISA tab to ISA JSON (which happens in a short-lived microservice in the cloud)
- Modify the ISA JSON (in a Jupyter notebook running in the cloud)
- Convert the modified ISA JSON back to ISA tab (again, in a microservice).
Within Google Cloud, a Jupyter notebook is deployed in its own microservice, where we can then call ISA API microservices that are created in their own containers on demand.
This effectively gives us the framework for a web-based ISA tab editor where ISA content can be modified by editing the JSON representation, something that the Metabolights team could use in the near future. Eventually, this may even lead to a web-based ISA creator.
You can check out the ISA microservices iPython notebook we created, but there’s quite a lot of overhead to set up the dependencies for the cloud infrastructure first. The intention here was to demonstrate how we can deploy ISA API services in a cloud, which is something that we plan to do with the PhenoMeNal project. However you don’t need to run the converters in the cloud and you can check out this standalone ISA API iPython notebook that you can run the same use case in a local Jupyter instance.
Please have a go yourself, and do give us any feedback via our ISA iPython notebooks project issue tracker. We hope to create more notebooks as demonstrators for how to use the new ISA API, so we would love for you to contribute any ideas and use cases.
