Contents

Release Notes¶

List of ISA-API changes and fixes:

isatools v0.13 package¶

  • Enhancements and Cleanup:

    • significant optimisation of isatab.dump() via reworking the study/assay graph implementation. This can improve up to 3 or 4 orders of magnitude the writing to ISA-tab for datasets with thousands of samples (PR #403). An additional tweak has been implemented to compute only once the study/assay process graph (8c2d09da6599dc89e3f6cfe8719c125c50fa3690)

    • All the major objects of the ISA models are now Commentable (PR #395)

    • isatools.create.connectors now supports both strings and Ontology Annotations for Subject Type and Sample Type (PR #405)

    • CI is now migrated from Travis CI to GitHub CI (issue #404)

    • Extension to documentation (new notebook documenting roundrip)

  • Breaking changes:

    • Changes to the isatools.create.connectors module to align it with the Datascriptor study JSON config. Now the Datascriptor study config has a “design” property holding all the study design information (PR #405)

  • Bug Fixes:

    • all names for protocols in studies generated by isatools.create.connectors from Datascriptor study configs are unique (issue #398)

    • identifiers for studies and investigations are coerced to strings (commit 7e9f468d633b54d356c136fdf1f0ea765d33279d)

isatools v0.12 package¶

  • Features:

    • Added the isatools.create.connectors module. This module contains the generate_study_design_from_config function to generate an isatools.create.StudyDesign from a JSON configuration document containing metadata about the study design, including sampling and assay plan. These JSON documents can be generated by the Datascriptor application.(issues #355, #356, #377)

    • Added support for observational variables to isatools.create.model.StudyArm (with the parameter source_characteristics) (PR #367)

    • StudyDesign.generate_isa_study()now generates one Assay per assay type (via StudyDesign._generate_samples()). (issue #369, #373)

    • The names of the assay nodes (extract, labelled, extract, protocol node, and data file) (issue #370, #373)

    • isatools.create.model.ProductNode now has an attribute extension (e.g. a file extension fo a DataFile node) (issue #378)

  • Enhancements and Cleanup:

    • Introduced support for (built-in, non-configurable) synonyms in protocol types. Protocol types have also been made case insensitive. (issue #382)

    • Refactored constants and error messages for the create mode, introducing the modules isatools.create.constants and isatools.create.errors (issue #342, #368)

    • Removed asterisk imports (*) from all the core modules (issue #368)

    • Removed leftover print() statements (issue #374)

    • overall test coverage increased from 67% to 75%. The increase is due both to an increase in the number of tests and the removal of some deprecated, experimental or spurious modules.

  • Breaking changes:

    • isatools.create.models has been renamed to isatools.create.model (issue #368)

    • IsaModelAttributeError has been replaced with AttributeError. We will favour standard Exceptions whenever applicable.

  • Bug Fixes:

    • Fixed typo in NAME_PROPERTY_ASSIGNMENT_ERROR (issue #328)

    • Silent expansion of Characteristic categories from str to OntologyAnnotation now supported. You can assign a string to a characteristic category and it will be automatically wrapped into an ontology annotation. (issue #332)

    • Fixed duplicated in OntologyAnnotation ids (issue #381)

    • Fixed raising of an error when correctly setting a comment value (issue #360)

    • Assigned correct header “Data Transformation Name” for protocol type “sequence analysis data transformation” (commit b8f2f55b4e046f1841ffba7d6fb190c87918375f)

  • New Documentation:

isatools v0.11 package¶

  • This release of isatools introduces a major overhaul of the pre-existing create mode for the generation of ISA documents from study design metadata. We have also introduced some (mostly) minor fixes to the core ISA API, enforcing a stricter validation on some objects.

  • Among the most noteworthy changes there are:

    • Support of complex longitudinal study designs with the introduction of Element, Treatment, StudyCell and StudyArm classes #318, #321

    • Increased flexibility of the sample and assay plans, with the introduction of the AssayGraph and the overhaul of the SampleAssayPlan class, now renamed SampleAndAssayPlan #329, #333, #296

    • Increased flexibility for Quality Control support through the QualityControlService utility class

    • Reworked serialization to JSON/deserialization from JSON of all the study design objects #313

    • Check added when submitting quantitative Parameter Values: only numerical values are allowed #347

  • For more information on the known issues with this release, please check the documentation at http://isatools.readthedocs.org

isatools v0.10 package¶

  • Various updates to the Create Mode for Metabolomics templating, found in isatools.create.models, and updates to address various general issues.

isatools v0.9 package¶

  • Various API packages have been refactored and moved. isatools.model.v1 is now in isatools.model. Packages isatools.validate, isatools.config, isatools.schemas have been removed. Some items from isatools.convert and isatools.io have been split into the isatools.io or isatools.net packages, where the latter has been created to distinguish parts of the ISA API that require network access

  • Issue #153 is still outstanding since version 0.4, as per below; new issue #235 where missing `Protocol REF`s are sometimes incorrectly inserted on loading ISA-Tab

  • New ISA creation functionality from planning objects in isatools.create.models currently only support a limited number of technology types (DNA microarray, DNA sequencing, mass spectrometry, and NMR)

  • SRA/ENA importer and Biocrates importer relies on XSLT2 processing only available with SAXON and requires .jar file to run

  • It has been decided to stop work on retrofitting support for Python 2.7 and focus on keeping up to date with the current and future Python versions

isatools v0.8 package¶

  • Issues #153 is still outstanding, as per below; new issue #208 (ISA-Tab validation issue)

  • SRA/ENA importer and Biocrates importer relies on XSLT2 processing only available with SAXON and requires .jar file to run

isatools v0.7 package¶

  • Issues #101, #153 are still outstanding, as per below

  • SRA/ENA importer and Biocrates importer relies on XSLT2 processing only available with SAXON and requires .jar file to run

isatools v0.6 package¶

  • Issues #146, #101, #153 are still outstanding, as per below

  • SRA/ENA importer and Biocrates importer relies on XSLT2 processing only available with SAXON and requires .jar file to run

  • We are aware that there may be some performance issues when loading and writing ISA-Tab documents with several thousand samples. This has been worked on and there is ongoing efforts to improve on the current performance of ISA-Tab I/O

isatools v0.5 package¶

  • All issues inherited from v0.4 (see below)

  • Currently only Python 3.4 and 3.5 is supported. Python 2.7 support is present in the py2 source branch on Github.

isatools v0.4 package¶

  • For certain of ISA-Tab table files, the ISA-Tab parser cannot disambiguate between process instances where a Name column is required to qualify a Protocol REF has been left blank. Utility functions have been written to detect these anomalies and to assist in correcting them, in the isatools.utils package. #146 (see detail after bullet points)

  • When converting to ISA JSON in using UUID or counter Identifier Types, some elements are not detected, such as Array_Design_REF #101

  • The ISA-Tab parser does not support reading Protein Assignment File, Peptide Assignment File, Post Translational Modification Assignment File columns, and therefore the isatab2* converters also do not support these #174

  • The SRA/ENA importer in sra2isatab relies on XSLT2 processing functionality only available with SAXON, so you must provide the JAR file yourself to use this

  • sra2isatab converter does not support SRA pools #153

  • The legacy functionality (marked in the documentation) relies on Java 1.6

To check for possible erroneous pooling events in an ISA-Tab archive, you can try something like:

>>> from isatools import utils
>>> utils.detect_isatab_process_pooling('tests/data/tab/MTBLS1/')
INFO: Converting ISA-Tab to ISA JSON...
INFO: Converting ISAtab to ISAjson for tests/data/tab/MTBLS1/
INFO: ... conversion finished.
Checking s_MTBLS1.txt
Checking a_mtbls1_metabolite_profiling_NMR_spectroscopy.txt
Possible process pooling detected on:  #process/Extraction1
Possible process pooling detected on:  #process/ADG_normalized_data.xlsx
[{'a_mtbls1_metabolite_profiling_NMR_spectroscopy.txt': ['#process/Extraction1', '#process/ADG_normalized_data.xlsx']}]
>>>

In this case, #process/Extraction1 is the pooling that we did not expect. This is a pooling on a single Extraction. From manual inspection of the ISA-Tab file a_mtbls1_metabolite_profiling_NMR_spectroscopy.txt we can then confirm that values are entirely missing from Extract Name, causing the parser to think the experimental graph converges on one process node. To rectify this, individual values should be put into this Name column. We can try fix erroneous pooling by filling out an empty Name column with a corresponding Protocol REF by doing the following:

>>> utils.insert_distinct_parameter(open('tests/data/tab/MTBLS1/a_mtbls1_metabolite_profiling_NMR_spectroscopy.txt', 'r+'), 'Extraction')
Are you sure you want to add a column of hash values in Extract Name? Y/(N)
>? Y

If successful, this will fill out the empty column with 8 character-long UUIDs (e.g. 4078cb03).

Please be aware that these utility functions detect_isatab_process_pooling() and insert_distinct_parameter() are there to help you manually fix your ISA-Tabs, not to automatically fix them for you. We wil address this issue in more depth in following releases.

isatools v0.3 package¶

  • required constraints on JSON schemas causes validation failure for @id objects, meaning some constraints using JSON schemas cannot be used for validation #108

  • Chained processes (i.e. a process followed by another process without any intermediate inputs and outputs, in ISAtab a Protocol REF columns followed by another Protocol REF columns without any materials in between) are not currently supported. It is not recommended to attempt to use such patterns with this version of the isatools package #111

  • When converting to ISA JSON in using UUID or counter Identifier Types, some elements are not detected, such as Array_Design_REF #101

  • The SRA/ENA importer in sra2isatab relies on XSLT2 processing functionality only available with SAXON, so you must provide the JAR file yourself to use this

  • The legacy functionality (marked in the documentation) relies on Java 1.6

isatools v0.2 package¶

  • required constraints on JSON schemas causes validation failure for @id objects, meaning some constraints using JSON schemas cannot be used for validation #108

  • When converting to ISA JSON in using UUID or counter Identifier Types, some elements are not detected, such as Array_Design_REF #101

  • Protocol REF columns must be present in order for the ISA-Tab to JSON conversion to pick up processes in the process sequences #111

  • Characteristics and Factor Values declared in assay tables in ISAtab are associated to Sample objects only. This means that when writing from Python objects, or converting from ISA JSON, to ISAtab these columns appear at the study table.

  • Chained processes (i.e. a process followed by another process without any intermediate inputs and outputs, in ISAtab a Protocol REF columns followed by another Protocol REF columns without any materials in between) are not currently supported. It is not recommended to attempt to use such patterns with this version of the isatools package #111

  • For experimental graph patterns to work, should follow relatively simple patterns. e.g. Straight Sample -> … -> Materials -> … -> Data paths (per assay), or simple splitting and pooling. See test package code for examples of what works.

  • No ISA JSON configurations have been included that correspond with the following default XML configurations: clinical_chemistry.xml and most are as yet untested.

isatools v0.1 package¶

  • Characteristics and Factor Values declared in assay tables in ISAtab are associated to Sample objects only. This means that when writing from Python objects, or converting from ISA JSON, to ISAtab these columns appear at the study table.

  • Chained processes (i.e. a process followed by another process without any intermediate inputs and outputs, in ISAtab a Protocol REF columns followed by another Protocol REF columns without any materials in between) are not currently supported. It is not recommended to attempt to use such patterns with this version of the isatools package #111

  • For experimental graph patterns to work, should follow relatively simple patterns. e.g. Straight Sample -> … -> Materials -> … -> Data paths (per assay), or simple splitting and pooling. See test package code for examples of what works.

For a full up-to-date list of issues, or to report an issue or ask a question, please see the issue tracker.

Downloading files stored in Github Hands-on Practicals