Pulsar
Pulsar is a service developed by the Galaxy team for running Galaxy jobs on remote servers. We maintain a fork with changes necessary for NDIP.
We provide an overview of the deployment below. If you need to add a new instance, or are interested in more details, then please review the instructions for adding a Pulsar instance.
Deployment
We run Pulsar on multiple instance machines in various compute environments. Below is the list of currently deployed instances.
For information about worker nodes infrastructure please refer to the infrastructure section
Pulsar Instances
| Name | Deployment |
|---|---|
| ORC small VM | Ansible via CI/CD job |
| ORC large VM | Ansible via CI/CD job |
| ORC test VM | Ansible via CI/CD job |
| SNS GPU machine | Ansible via CI/CD job |
| OLCF Frontier | Helm chart, manually |
| NERSC Perlmutter | Helm chart, manually |
All deployed Pulsar instances can be accessed via ssh. You can find the username (typically cloud) and hostname/IP
address for each Pulsar node on the infrastructure page. The SSH
key for the Pulsar nodes is here.
Automated Deployment Details
Deployment is managed through
the pulsar Ansible role.
This role includes:
Configuration templates for the Remote Data Broker and OIDC setup.
Tasks that run and monitor the status of the Pulsar server.
Configuration
Configuration values are defined in Ansible variable files. We use separate variable files for production and test instances of Pulsar.
Manual Deployment Details
Both Slate and Spin deployments are performed manually via K8s Helm charts. For more information, please review the Helm charts for each service in our monorepo: