Relations Reference - Concourse CI Machine Charm

Overview

Relations connect Concourse CI units to external services (PostgreSQL, Prometheus) and coordinate between Concourse components (web servers and workers). The Concourse CI Machine Charm defines three relation types: requires (consumes data), provides (exposes data), and peers (coordinates between units).

Note: Relations are established using juju integrate <app1>:<endpoint1> <app2>:<endpoint2> (Juju 3.x) or juju relate (Juju 2.x).

Relation Summary

Relation Name	Type	Interface	Status	Applied To
`postgresql`	requires	`postgresql_client`	Required	Web server units
`monitoring`	provides	`prometheus_scrape`	Optional	All units
`tsa`	provides	`concourse-ci_tsa`	Optional	Web server units
`flight`	requires	`concourse-ci_tsa`	Optional	Worker units
`vault-kv`	requires	`vault-kv`	Optional	Web server units
`peers`	peer	`concourse-ci_peers`	Automatic	All units (same app)

Required Relations

postgresql (requires)

Interface postgresql_client

Limit 1 connection

Applied To Web server units

Status Required

Description: Provides PostgreSQL database backend for the Concourse web server. Stores pipelines, build metadata, resource versions, and authentication data. Uses Juju secrets API for secure credential management.

Supported PostgreSQL Versions:

PostgreSQL 16/stable (recommended)
Any charm providing postgresql_client interface with Juju secrets support

Data Exchanged:

From PostgreSQL: Database name, host, port, username, password (via Juju secret), connection string
To PostgreSQL: Database name request, schema initialization

Usage Example:

# Deploy PostgreSQL 16
juju deploy postgresql --channel 16/stable --base ubuntu@24.04

# Deploy Concourse CI
juju deploy concourse-ci-machine concourse-ci --config mode=auto

# Integrate
juju integrate concourse-ci:postgresql postgresql:database

Warning: Web server units will enter Blocked status without this relation. Worker-only units (mode=worker) do not require this relation.

Related Configuration: None (credentials managed automatically via relation)

Troubleshooting:

Blocked status "Database not connected": Ensure postgresql relation is established and PostgreSQL is ready
Connection errors: Check PostgreSQL logs (juju debug-log --include postgresql) and network connectivity
Schema errors: Concourse auto-initializes database schema on first connection. Check Concourse logs for migration failures.

Optional Relations

monitoring (provides)

Interface prometheus_scrape

Limit Unlimited

Applied To All units

Status Optional

Description: Exposes Prometheus metrics endpoints for monitoring Concourse CI performance, resource usage, and job statistics. Enables integration with Prometheus and Grafana for observability.

Metrics Endpoints:

Port 9391: Core Concourse metrics (web server and worker metrics)
Port 9358: Per-job status metrics (when enable-metrics=true and concourse-exporter is installed)

Data Exchanged:

To Prometheus: Scrape targets (unit IP, port 9391/9358), job name, labels

Usage Example:

# Enable metrics in Concourse
juju config concourse-ci enable-metrics=true

# Deploy Prometheus
juju deploy prometheus-machine prometheus

# Integrate monitoring relation
juju integrate concourse-ci:monitoring prometheus:target

# Access Prometheus UI and query Concourse metrics
# Example PromQL: concourse_jobs_running{team="main"}

Available Metrics (Port 9391):

concourse_builds_*: Build counts, durations, statuses
concourse_workers_*: Worker states, container counts
concourse_db_*: Database connection pool metrics
concourse_steps_*: Pipeline step execution metrics
concourse_http_*: Web server HTTP request metrics

Available Metrics (Port 9358, requires concourse-exporter):

concourse_job_status: Per-job status (succeeded, failed, errored)
concourse_job_last_build_duration: Last build duration per job

Related Configuration: enable-metrics (boolean, default: false)

Note: Metrics are always exposed on port 9391 regardless of enable-metrics. The configuration option controls concourse-exporter installation (port 9358).

tsa (provides)

Interface concourse-ci_tsa

Limit Unlimited

Applied To Web server units

Status Optional

Description: Provides TSA (Transportation Security Administration) endpoint for worker registration. TSA is the SSH-based gateway that workers connect to for authenticating and registering with the Concourse web server. Used when deploying separate web and worker applications (mode=web + mode=worker).

Data Exchanged:

To Workers: TSA host key, TSA public key, web server IP/hostname, TSA port (default: 2222)

Usage Example:

# Deploy web server
juju deploy concourse-ci-machine web --config mode=web

# Deploy workers
juju deploy concourse-ci-machine worker -n 2 --config mode=worker

# Integrate web TSA endpoint with workers
juju integrate web:tsa worker:flight

Note: This relation is not needed in mode=auto deployments, where TSA keys are distributed via peer relations automatically.

Related Configuration: None (TSA port and keys managed automatically)

flight (requires)

Interface concourse-ci_tsa

Limit 1 connection

Applied To Worker units

Status Optional

Description: Consumes TSA endpoint from web server to retrieve TSA keys and connection details. Worker units use this relation to authenticate and register with the web server's TSA gateway.

Data Exchanged:

From Web Server: TSA host key, TSA public key, web server IP/hostname, TSA port
To Web Server: Worker public key (for authorized_keys)

Usage Example:

# Workers require TSA endpoint from web server
juju integrate worker:flight web:tsa

Warning: Worker units in mode=worker will enter Blocked status without this relation. Workers cannot function without connecting to a web server's TSA.

Related Configuration: None (worker keys and TSA connection managed automatically)

vault-kv Relation

vault-kv (requires)

Interface vault-kv

Limit 1

Applied To Web server units

Status Optional

Description: Connects the Concourse web server to HashiCorp Vault for pipeline credential management. When established, Concourse automatically resolves ((secret-name)) variable references in pipelines by looking them up in the Vault KV store. Credentials are provisioned via AppRole authentication.

Compatible Providers:

vault charm (2.0/stable — ubuntu@24.04)

Data Exchanged:

vault_url: Vault server HTTPS URL (provided by Vault)
ca_certificate: Vault TLS CA certificate PEM (provided by Vault)
mount: KV mount path, e.g. charm-web-concourse (provided by Vault)
credentials: Juju secret ID containing role-id and role-secret-id (provided by Vault)
mount_suffix: Suffix for the KV mount name, set to concourse (provided by requirer)
egress_subnet / nonce: Unit identity data used by Vault to issue AppRole credentials (provided by requirer)

Effect on Concourse Config: When the relation is ready, the following environment variables are written to config.env on the web unit:

CONCOURSE_VAULT_URL
CONCOURSE_VAULT_AUTH_BACKEND=approle
CONCOURSE_VAULT_AUTH_PARAM=role_id:...,secret_id:...
CONCOURSE_VAULT_CA_CERT (written to /etc/concourse/vault-ca.pem)
CONCOURSE_VAULT_PATH_PREFIX=/charm-web-concourse

Integration Example:

# Deploy and integrate
juju deploy vault --channel 2.0/stable
juju integrate vault:vault-kv web:vault-kv

# Initialize Vault (save unseal key and root token)
juju ssh vault/leader -- vault operator init -key-shares=1 -key-threshold=1 -format=json

# Unseal
juju ssh vault/leader -- vault operator unseal <unseal-key>

# Authorize the charm
juju add-secret vault-root-token token="<root-token>"
juju grant-secret vault-root-token vault
juju run vault/leader authorize-charm secret-id=<secret-id> --wait 2m

# Verify
juju ssh web/0 -- sudo grep VAULT /var/lib/concourse/config.env

Note: Vault-kv credentials override the manual vault-url charm config option when both are set. See the Vault How-To Guide for the complete setup procedure.

Troubleshooting:

Message	Cause	Solution
vault blocked "Please initialize Vault"	Vault not initialized	Run `vault operator init` and `vault operator unseal`
VAULT vars missing from config.env	Charm not authorized	Run `juju run vault/leader authorize-charm`
Pipelines fail with "credential not found"	Secret not in correct KV path	Write secrets to `charm-web-concourse/<team>/<pipeline>/<name>`

Peer Relation

peers (peer)

Interface concourse-ci_peers

Scope Same application

Applied To All units (automatic)

Status Automatic

Description: Automatically coordinates data between units in the same application. Used in mode=auto deployments to distribute TSA keys, worker keys, and admin password across all units. No manual configuration required—Juju creates this relation automatically.

Data Exchanged:

TSA host key: Generated by leader, shared with all units
TSA public key: Generated by leader, shared with all units
Worker keys: Generated by workers, shared with leader for TSA authorization
Admin password: Generated by leader, shared with all units
Session signing key: Generated by leader, shared with all units

Usage: Automatic—no manual action required.

Note: Peer relation enables zero-configuration key distribution in mode=auto deployments. Workers automatically receive TSA keys from the leader and register without manual SSH key copying.

Related Configuration: None (peer coordination is automatic)

Relation Applicability Matrix

Relation	mode=auto (leader)	mode=auto (follower)	mode=web	mode=worker
`postgresql`	✅ Required	❌ N/A	✅ Required	❌ N/A
`monitoring`	✅ Optional	✅ Optional	✅ Optional	✅ Optional
`tsa` (provides)	✅ Available	❌ N/A	✅ Available	❌ N/A
`flight` (requires)	❌ N/A	✅ Auto-connected	❌ N/A	✅ Required
`vault-kv`	✅ Optional	❌ N/A	✅ Optional	❌ N/A
`peers`	✅ Automatic	✅ Automatic	✅ Automatic	✅ Automatic

Relation Examples

auto Mode (Fully Automated)

# Deploy with auto mode (peer relation handles key distribution)
juju deploy concourse-ci-machine concourse-ci -n 3 --config mode=auto
juju integrate concourse-ci:postgresql postgresql:database
juju integrate concourse-ci:monitoring prometheus:target

# All TSA keys distributed automatically via peers relation ✨

web + worker Mode (Separate Applications)

# Deploy web and workers as separate apps
juju deploy concourse-ci-machine web --config mode=web
juju deploy concourse-ci-machine worker -n 2 --config mode=worker

# Required relations
juju integrate web:postgresql postgresql:database
juju integrate web:tsa worker:flight  # Key exchange between apps

# Optional monitoring
juju integrate web:monitoring prometheus:target
juju integrate worker:monitoring prometheus:target

GPU Workers with Monitoring

# Deploy web server
juju deploy concourse-ci-machine web --config mode=web

# Deploy GPU workers
juju deploy concourse-ci-machine gpu-worker \
  --config mode=worker \
  --config compute-runtime=cuda \
  --config tag="cuda,gpu"

# Deploy CPU workers
juju deploy concourse-ci-machine cpu-worker -n 2 \
  --config mode=worker \
  --config tag="cpu,high-mem"

# Integrate all applications
juju integrate web:postgresql postgresql:database
juju integrate web:tsa gpu-worker:flight
juju integrate web:tsa cpu-worker:flight
juju integrate web:monitoring prometheus:target
juju integrate gpu-worker:monitoring prometheus:target
juju integrate cpu-worker:monitoring prometheus:target

Web with Vault Credential Management

# Deploy web, workers, and Vault
juju deploy concourse-ci-machine web --config mode=web
juju deploy concourse-ci-machine worker -n 2 --config mode=worker
juju deploy vault --channel 2.0/stable

# Wire up relations
juju integrate web:postgresql postgresql:database
juju integrate web:tsa worker:flight
juju integrate vault:vault-kv web:vault-kv

# Initialize and unseal Vault, then authorize the charm
# (see howto/use-vault.html for full procedure)

Troubleshooting Relations

Blocked Status

Message	Cause	Solution
"Database not connected"	Web server missing `postgresql` relation	`juju integrate <app>:postgresql postgresql:database`
"TSA endpoint not available"	Worker missing `flight` relation	`juju integrate <worker>:flight <web>:tsa`
"Waiting for peer relation"	Peer relation not established (rare)	Wait for Juju to establish peer relation (automatic)

Viewing Relations

# View all relations for an application
juju status --relations

# View specific relation status
juju show-unit concourse-ci/0 | grep -A 5 "relation-info"

# Debug relation data (for charm developers)
juju run concourse-ci/0 -- relation-get -r <relation-id> - <remote-unit>

Removing Relations

# Remove a relation
juju remove-relation concourse-ci:monitoring prometheus:target

# Re-establish if needed
juju integrate concourse-ci:monitoring prometheus:target

Warning: Removing the postgresql relation from a web server will cause the web service to stop and enter Blocked status. Pipelines and build data remain in the database and will be restored when the relation is re-established.