Open Research Cloud VMs
Open Research Cloud (ORC) operates on the RedHat OpenStack Platform. If you have access to the NDIP project, you can view its resources on the dashboard.
To request access, submit a ticket here. Use
Security Level: Moderate and Project Name: NDIP.
Do not use the dashboard for deployments. It is only meant for validation and troubleshooting.
Infrastructure Management
Deployments and updates are managed via Terraform and Ansible using a monorepo.
With this repository, we manage:
- Security groups
- Shared and local storage
- Network ports (IPs)
- Virtual machines
Additionally, shared storage is mounted using an Ansible playbook.
Configuration
Current configuration
resources*.yaml files describes all cloud resources currently deployed. To add or remove a VM or modify configurations, these are the primary files you will need to update. We currently have the main YAML file and two separate files to configure Calveras and Pulsars.
Available parameters
For a complete list of possible parameters, please refer to the example file.
Default values
Additionally, there is a file with default settings that are applied across all instances.
Deployment
To deploy (apply) the current configuration, we use GitLab CI/CD jobs. The deployment process involves several stages:
- Build (optional): This stage builds the Docker container with all necessary dependencies installed.
- Init: Initializes Terraform, with the Terraform state stored in GitLab.
- Validate: Validates the Terraform configuration.
- Plan: Generates or updates the Terraform plan, outlining the intended changes in the job output.
- Apply: Applies the Terraform plan, resulting in updates to the cloud resources.
The first three stages are accessible for every GitLab branch, allowing for modifications, validation, and previewing the potential impact.
Once everything looks good, you can create a Merge Request (MR), and when it is merged, the apply stage will become available in the main branch.
Legacy VM deployments
We still have a couple of resources deployed from legacy repositories. These are:
- Shared CEPH storage we use for storing NDIP data
- Kubernetes cluster we use to deploy monitoring services.
Please see documentation (when available) or code in those repositories for more information.