Test Matrix Reference

Supported versions, platforms, and deployment scenarios

Overview

This reference documents tested and supported configurations for the Concourse CI Machine Charm. The charm undergoes continuous integration testing against various platforms, versions, and deployment scenarios.

Note: Configurations marked as Tested are verified in CI. Configurations marked as Supported are expected to work but may not be explicitly tested.

Ubuntu Base Versions

Ubuntu Version Status Juju Base Notes
Ubuntu 24.04 LTS (Noble Numbat) Tested ubuntu@24.04 Recommended. Primary development and testing platform.
Ubuntu 22.04 LTS (Jammy Jellyfish) Supported ubuntu@22.04 Should work but not explicitly tested. containerd and PostgreSQL 16 available.
Ubuntu 20.04 LTS (Focal Fossa) Unsupported ubuntu@20.04 End of Standard Support: April 2025. PostgreSQL 16 not available in default repositories.

Concourse CI Versions

Version Status Release Date Notes
8.0.1 (latest) Tested January 2025 Current stable release. Default when version="".
7.14.2 Tested December 2024 Previous stable release. Fully supported.
7.x series Supported 2024 All 7.x versions should work with this charm.
6.x series Experimental 2023 May work but not recommended. PostgreSQL 16 compatibility unknown.

PostgreSQL Versions

Version Channel Status Notes
PostgreSQL 16 16/stable Tested Recommended. Fully supports Juju secrets API.
PostgreSQL 15 15/stable Experimental May work if charm supports postgresql_client interface with secrets.
PostgreSQL 14 14/stable Unsupported Older interface. Not recommended.
Note: The charm requires PostgreSQL charms that support the postgresql_client interface with Juju secrets API. PostgreSQL 16/stable is the only officially tested version.

Juju Versions

Juju Version Status Notes
Juju 3.6+ Tested Recommended. Full secrets API support.
Juju 3.1 - 3.5 Supported Should work. Secrets API available from 3.1+.
Juju 2.x Unsupported No secrets API. PostgreSQL 16 charm requires Juju 3.x.

Deployment Platforms

Platform Provider Status Notes
LXD Containers localhost Tested Primary testing platform. Full feature support including GPU passthrough.
Bare Metal manual Supported Full support. GPU support works natively without passthrough configuration.
Virtual Machines maas, openstack, etc. Supported Full support. GPU passthrough depends on hypervisor capabilities.
AWS EC2 aws Supported Machine charm compatible. GPU instances (P3, P4, G4) supported.
Azure VMs azure Supported Machine charm compatible. GPU instances (NC-series) supported.
Kubernetes k8s, microk8s Unsupported Machine charm not compatible. Use concourse-web and concourse-worker K8s charms instead.

Deployment Modes

Mode Status Test Coverage Notes
mode=auto Tested CI regression tests Leader becomes web, followers become workers. Fully automated key distribution.
mode=web + mode=worker Tested CI regression tests Separate applications. TSA key exchange via relations.

GPU Support Matrix

NVIDIA CUDA

GPU Model Status Driver Version Notes
NVIDIA RTX 3070 Tested 580.95+ Primary testing GPU. Full support for CUDA workloads.
NVIDIA RTX 4000 series Supported 545.0+ Ada Lovelace architecture. Requires recent drivers.
NVIDIA RTX 3000 series Supported 470.0+ Ampere architecture. Widely used for ML workloads.
NVIDIA RTX 2000 series Supported 418.0+ Turing architecture. Full CUDA support.
NVIDIA Tesla (T4, V100, A100) Supported 450.0+ Data center GPUs. Optimized for compute workloads.
NVIDIA GTX 1000 series Supported 390.0+ Pascal architecture. Older but still functional.

AMD ROCm

GPU Model Status ROCm Version Notes
AMD Radeon RX 7900 XT/XTX Tested 7.2.0+ RDNA 3 architecture (gfx1100). Native support, no workarounds needed.
AMD Radeon RX 6000 series Supported 5.0.0+ RDNA 2 architecture (gfx1030). Native support.
AMD Radeon Pro (W6000, W7000) Supported 5.0.0+ Professional cards. Full ROCm support.
AMD Instinct (MI200, MI100) Supported 5.0.0+ Data center GPUs. Primary ROCm target.
AMD Ryzen 7000 series (iGPU) Experimental 7.2.0+ Integrated GPU (gfx1103). Requires HSA_OVERRIDE_GFX_VERSION=11.0.0 workaround. Lower performance due to shared memory.
AMD Ryzen 5000 series (iGPU) Experimental 5.0.0+ Integrated GPU (gfx90c). Requires HSA_OVERRIDE_GFX_VERSION=9.0.0 workaround.
Note: Integrated AMD GPUs (APUs) require the HSA_OVERRIDE_GFX_VERSION environment variable workaround. See ROCm Verification Reference for details.

Feature Test Matrix

Feature Status Test Coverage Notes
Basic Deployment Tested CI regression Auto mode, web+worker mode, single unit
PostgreSQL Integration Tested CI regression Database connection, schema initialization, secrets
Worker Scaling Tested CI regression Add/remove workers, automatic registration
Shared Storage (LXC) Tested CI regression LXC disk device mounting, binary sharing
GPU Workers (NVIDIA) Tested CI regression nvidia-container-toolkit, GPU passthrough, PyTorch verification
GPU Workers (AMD) Tested CI regression amd-container-toolkit, ROCm runtime, PyTorch verification
General Folder Mounting Tested CI regression Read-only and writable folder mounting from /srv
Worker Tags Tested CI regression Manual tags, GPU-generated tags (cuda, rocm)
Version Upgrades Tested CI regression Upgrade/downgrade Concourse versions, automatic worker upgrades
Prometheus Metrics Supported Manual testing Metrics endpoint exposed, Prometheus relation works
TLS/HTTPS Experimental Manual testing Requires manual certificate configuration. No TLS relation yet.
Vault Integration Supported Configuration tested Configuration options available, end-to-end integration not CI-tested

CI Test Commands

The charm includes a comprehensive test script that validates all major features. CI runs the full regression suite on every commit.

# Run full regression test suite
./scripts/deploy-test.sh

# Skip cleanup to inspect deployment
./scripts/deploy-test.sh --skip-cleanup

# Start from specific test step
./scripts/deploy-test.sh --goto=gpu --skip-cleanup

Test Steps (in CI)

  1. Basic Deployment: Deploy PostgreSQL + Concourse (auto mode, 3 units)
  2. Verify Active Status: Wait for all units to reach "Active" status
  3. Shared Storage: Deploy with shared-storage=lxc, verify binary sharing
  4. General Folder Mounting: Mount folders to /srv, verify read-only and writable mounting
  5. Worker Tags: Deploy worker with custom tags, verify in fly CLI
  6. GPU Workers (NVIDIA): Deploy with compute-runtime=cuda, verify GPU detection
  7. GPU Workers (AMD): Deploy with compute-runtime=rocm, verify GPU detection
  8. Version Upgrade: Upgrade from 7.14.2 to 8.0.1, verify workers auto-upgrade
  9. Cleanup: Remove all applications and storage

Known Issues and Limitations

Reporting Issues

If you encounter issues with a specific configuration:

  1. Check if your configuration is marked as Tested or Supported
  2. Review the Troubleshooting Guide
  3. Open an issue on GitHub Issues with:
    • Ubuntu version (lsb_release -a)
    • Juju version (juju version)
    • Charm version (juju status --format=yaml | grep charm-rev)
    • Deployment platform (LXD, bare metal, cloud provider)
    • Relevant logs (juju debug-log --include concourse-ci)