Skip to main content

Prototyping Tools

To provide an environment for testing the tools, we created a test Calvera instance (https://calvera-test.ornl.gov). You can deploy your tool on this instance and verify that it works as expected before deploying it to Calvera (https://calvera.ornl.gov).

Below are the instructions on how to do that.

info

https://calvera-test.ornl.gov is being used by the NDIP Team for development tests as well, so they might restart it from time to time without warning.

Add a new file to the prototype branch of the tools repository​

There is a special branch called prototype, where a commit automatically updates prototype tools on https://calvera-test.ornl.gov.

When developing new tools or updating existing ones, place the tool XML files in the GitLab repository called Galaxy Tools. Ensure your XML adheres to Galaxy's standards (for guidance on creating a Galaxy tool XML file, refer to this documentation). The XML file should be located in a subfolder of the tools/neutrons folder and follow naming conventions that reflect the tool's function.

note

All tools in a subfolder will be placed in a section with the subfolder name in the Galaxy tools panel. Tools in the root folder neutrons will be placed in the Uncategorized section.

  1. Clone and check out the prototype branch.
git clone --branch prototype https://code.ornl.gov/ndip/galaxy-tools
  1. Create an XML file for your tool in the tools/neutrons folder or it's subfolder.

<tool id="neutrons_asrp_spf_dev" name="Single Peak Fitting" profile="22.05" version="0.3.0">
    ...
</tool>
danger

Please make sure the tool id is unique. We recommend to add _dev suffix if the tool already exists on Galaxy and you are prototyping a new version of it.

Where the tool will run​

By default, the tool will run on a test VM in ORC cloud. If the tool needs a GPU, please add a resource requirement for a CUDA device:

<tool id="neutrons_asrp_spf_dev" name="Single Peak Fitting" profile="22.05" version="0.3.0">
...
    <requirements>
     <resource type="cuda_device_count_min">1</resource>
    </requirements>
...
note

If the tool needs to run somewhere else (or has some other specific requirements), it should be configured manually. Please talk to the NDIP team.

  1. Add tests for your tools (optionally, but recommended)

You can add tests to your tool file so that they will run automatically when changes are pushed to the prototype branch.

If any input is needed or output is expected, place this file in the test-data subfolder in a folder where the tool file is located.

As seen in the example test for the ASRP Single Peak Fitting tool below, the expected_spf_output.json file contains the results of running this tool with the given input file, NOM_Fe2O3_ramp_to_500K_at_temperature_95.05_K.gsa, and values. See Planemo documentation for more information.


<tests>
    <test>
        <param name="input_data" value="NOM_Fe2O3_ramp_to_500K_at_temperature_95.05_K.gsa"/>
        <param name="bank_id" value="2"/>
        <param name="peak_center" value="11900"/>
        <param name="xmin" value="11141"/>
        <param name="xmax" value="12660"/>
        <param name="background_type" value="Linear"/>
        <param name="units" value="TOF"/>
        <output name="output" file="expected_spf_output.json"/>
    </test>
</tests>
  1. Push your changes to the prototype branch. After a couple of minutes the tool should appear under the Prototype category in the corresponding section.

Modifying your tool​

  1. Push your changes to the prototype branch.

  2. Wait for the tools to be pushed to Calvera-test automatically; this may take a couple of minutes.

note

You need to reload the webpage in the browser to see updates to a tool. It is also a good idea to update the tool version with every push, e.g., 0.1.0, 0.1.1, etc., to ensure that the tool in Calvera-test is the correct version.

Displaying your prototype tool on the NOVA dashboard​

By default, prototype tools created through the NOVA application template will appear on https://nova-test.ornl.gov within the category you selected when running copier. If you wish to change the category, then you can re-run copier and change the answer to the tool category question.

Passing parameters to a tool in the NOVA dashboard​

Every tool on the NOVA dashboard can be autolaunched by opening a link in your browser. Each tool's autolaunch link can be copied with the button to the left of the tool name.

autolaunch.png

This link will have the following format: https://nova.ornl.gov/launch/{unique_tool_id}?parameter1=test&file_parameter2=/path/to/file/on/analysis/cluster

The parameters here correspond to the input parameters defined in your tool XML. Typically, these parameters are text values that are passed directly to the tool. However, if you need to pass a file to the tool, then you can append file_ to the parameter name from the tool XML. This will trigger the dashboard to ingest the file into Calvera and pass it to the tool.

Move the tool from prototype to production​

Please create an issue using this link: https://code.ornl.gov/ndip/galaxy-tools/-/issues/new, and indicate which tool(s) you want to move.

We'll remove the dev suffix from the tool id and reconfigure Calvera accordingly. If you want to make modifications to the tool later, you can create a new branch from the dev one, make your modifications, and create a merge request.