Skip to main content

Calvera Configuration (admin)

info

If you are not admin, just create an issue in a GitLab repository Galaxy Tools with a name of the tool file in the repository, we'll make sure it appears in Calvera.

Add a new file to the tools repo​

Tool XML files (see how to create an XML file here) are stored in a GitLab repository Galaxy Tools.

Tools are grouped into categories (corresponding to neutron scattering techniques), so name your file appropriately and put it in tools/neutrons/<category> folder. If the category does not exist, create a new one.

Modify tools config file​

We have to configure Calvera (or any other Galaxy instance) to start using the new tool. For Calvera we do this via Ansible playbook. Create a branch of Calvera Deployment repo, and make changes to the tool_conf.xml.j2 file:

  • select an appropriate section (or create a new one)

  • add a path to the tool file to this section. Depending on the type of the tool, the following options are available:

    - Add a prototype tool to Calvera-test

    A prototype tool is a special kind of the tool that users can push directly to Calvera-test. It is copied automatically to separate folder neutrons-dev, without any subfolders. Since this tool file is never copied to calvera, it does not harm to have it in the configuration file there. Eventually, when the tool is ready, it should be moved to its final location and this section will change (see below)

      <section id="neutrons_asrp" name="Automated Structural Refinement">
          <tool file="{{ galaxy_local_tools_dir }}/neutrons-dev/asrp_single_peak_fitting.xml"/>
      </section>

    - Add a tool to Calvera-test only

    If it is a normal tool (not prototype), but for some reason you only want to deploy it on Calvera-test, add a condition to it:

      <section id="neutrons_asrp" name="Automated Structural Refinement">
        {% if galaxy_test_tools %}
          <tool file="{{ galaxy_local_tools_dir }}/neutrons/powder_diffraction/asrp_single_peak_fitting.xml"/>
        {% endif %}
      </section>

    - Add a tool to both Calvera and Calvera-test

    This configuration will make the tool to be installed on both Calvera and Calvera-test.

      <section id="neutrons_asrp" name="Automated Structural Refinement">
          <tool file="{{ galaxy_local_tools_dir }}/neutrons/powder_diffraction/asrp_single_peak_fitting.xml"/>
      </section>

  • add a tool destination in job_conf.xml.j2 to run the tool on a specific computing resource (see the list of available destinations in the same file):

- id: "neutrons_asrp_spf"
  destination: "pulsar_cpu1_docker"

Adding a tool to the NOVA dashboard​

To add a tool to the NOVA dashboard, add it to the NOVA dashboard. In order to do so, you can add the tool ID (e.g. "neutrons_asrp") to https://code.ornl.gov/ndip/deployments/calvera/-/blob/dev/ansible/roles/ornl.nova-dashboard/templates/tools.json.j2. If you only want to show a tool on the test dashboard (nova-test.ornl.gov), then you should wrap the tool ID in an if block as shown in the example below.

{
  "sans": {
    "name": "Small-Angle Neutron Scattering",
    "description": "BIO-SANS, EQ-SANS, GP-SANS, USANS",
    "tools": [
      {%- if nova_test_tools -%}"neutrons_trame_sans",{%- endif -%}
      "interactive_tool_sasview"
    ]
  }
}

Moving a tool from prototype to production​

Assuming the tool was deployed from the prototype branch onto calvera-test, you first need to copy the tool XML file to the appropriate location in the dev branch. This will normally be into a directory like tools/neutrons/<category> were <category> is one of the categories of tools. In addition, if the tool had test data associated with it, you will also need to copy this from tools/neutrons/test-data on the prototype branch to tools/neutrons/<category>/test-data on the dev branch.

At the same time you copy the tool XML file, make sure to remove the -dev suffix from the tool name in the XML specification.

Decide if the tool is to be deployed on Calvera, Calvera-test, or both and follow the instructions above.

Here’s a cleaned-up version with grammar and clarity fixes:

Selecting Tool Destinations​

All NDIP tools run on one of the destinations configured in the job_conf.yml.j2 file.

Static Destinations​

Many tools currently run on a static, preconfigured destination. See the tools section of the job_conf.yml.j2 file.

Dynamic Destinations​

Alternatively, NDIP can dynamically select a destination based on tool requirements and available resources. To enable this, configure the tool to use dynamic_resource_orchestration in the job_conf file. Then, define available resources, allowed destinations per tool, and tool resource requirements in the resource_orchestration.yaml file. Use the existing file as a guide.

note

The order of allowed tool destinations in resource_orchestration.yaml is important as NDIP sequentially attempts to send tools to each destination, only moving to the next if the current one is unavailable.