Skip to main content

Pulsar

Pulsar is a service developed by the Galaxy team for running Galaxy jobs on remote servers. We maintain a fork with changes necessary for NDIP.

We provide an overview of the deployment below. If you need to add a new instance, or are interested in more details, then please review the instructions for adding a Pulsar instance.

Deployment​

We run Pulsar on multiple instance machines in various compute environments. It runs on VMs in ORC, a bare metal machine in the SNS analysis cluster, Frontier's Slate service, and NERSC's Spin platform. For the ORC VMs and our SNS machine, Pulsar is automatically deployed to each instance via a CI/CD job using Ansible. For Slate and Spin, we deploy Pulsar manually via Helm charts.

note

All deployed Pulsar instances can be accessed via ssh. You can find the username (typically cloud) and hostname/IP address for each Pulsar node on the infrastructure page. The SSH key for the Pulsar nodes is here.

Automated Deployment Details​

Deployment is managed through the pulsar Ansible role. This role includes:

Configuration​

Configuration values are defined in Ansible variable files. We use separate variable files for production and test instances of Pulsar.

Manual Deployment Details​

Both Slate and Spin deployments are performed manually via K8s Helm charts. For more information, please review the Helm charts for each service in our monorepo: