Introduction
This section provides detailed information about services used within NDIP platform, incluing instruction for their deployment and configuration.
The NDIP consists of two primary components:
- Core Services – Internal services that are needed for NDIP platform to operate.
- Worked Node Services – These are the services that run on computational resources where users execute their jobs.
They are distributed across multiple environments:
- ORC cloud
- Neutron Science analysis cluster
- OLCF Frontier, running on Slate Kubernetes cluster
- NERSC Perlmutter, running on Spin Kubernetes cluster
note
For information about infrastructure supporting NDIP please refer to the infrastructure section
Currently in use:
| Name | Function | Type | Details |
|---|---|---|---|
| Galaxy | Main component of the NDIP platfrom | core | link |
| PostgreSQL | Internal Galaxy database | core | link |
| Pulsar | Accepts jobs from Galaxy and runs them on compute resources | compute | link |
| RabbitMQ | Message broker communication between Galaxy and Pulsar | core | link |
| Rucio | Manages NDIP data | core | link |
| Remote Data Broker | Facilitates data movement between worker nodes and Galaxy | core/compute | link |
| Apache Kafka | Stores data ingress requests coming from ingress proxy | core | link |
| Ingress Proxy | Listens for ingress requests and sends them to Kafka | core | link |
| Prometheus Stack | Monitoring and alerting | core | link |
| Node Exporter | Exposes hardware/system metrics for Prometheus | core/compute | link |
In the following sections, we provide details about these services.