Setting Up Shared Storage

Learn how to deploy Concourse CI with shared storage to save disk space and speed up deployments

📚 What You'll Learn: By completing this tutorial, you'll deploy Concourse CI with shared storage that reduces disk usage by 62% and speeds up upgrades by 63% compared to traditional deployments.

Why Shared Storage?

When you deploy multiple Concourse CI units without shared storage, each unit downloads its own copy of the binaries (~1GB each). With 3 units, that's 3GB of duplicated data!

Shared storage solves this by:

Having the leader download binaries once to a shared volume
Workers detect and reuse those binaries automatically
Reducing total disk usage from N×1GB to ~1.15GB
Speeding up deployments and upgrades significantly

┌─────────────────────────────────────────────┐ │ Shared Storage Volume │ │ /var/lib/concourse/ │ ├─────────────────────────────────────────────┤ │ ├── bin/ ← Leader writes once │ │ │ ├── concourse │ │ │ └── gdn │ │ ├── keys/ ← Shared TSA keys │ │ └── worker/ ← Per-unit state │ │ ├── concourse-0/ │ │ ├── concourse-1/ │ │ └── concourse-2/ │ └─────────────────────────────────────────────┘ ▲ ▲ ▲ │ │ │ Leader Unit Worker Unit 1 Worker Unit 2

Before You Start

This tutorial assumes you have:

Completed the first deployment tutorial (or understand Juju basics)
Juju 3.1+ installed with a controller
LXD cloud (we'll use local disk shared storage for simplicity)
~20 minutes of time

💡 About this tutorial: We'll use LXD's built-in disk device passthrough to create shared storage. This is the supported shared storage mode for the charm.

Step 1: Create a Model

Let's start fresh with a new model for this deployment.

1 Create the shared-storage model

juju add-model concourse-shared

Step 2: Deploy PostgreSQL

The database is still required for Concourse CI web server.

2 Deploy PostgreSQL

juju deploy postgresql --channel 16/stable --base ubuntu@24.04

Step 3: Deploy with Shared Storage

Here's where it gets interesting! We'll deploy with the shared-storage=lxc configuration.

3 Deploy Concourse CI with LXC shared storage mode

juju deploy concourse-ci-machine concourse -n 3 \
  --config mode=auto \
  --config shared-storage=lxc \
  --channel edge \
  --base ubuntu@24.04

What's happening?

--config shared-storage=lxc: Tells the charm to expect shared storage
Units will wait for you to configure the LXC shared folder
The leader will download binaries once everyone can see the shared storage

Step 4: Connect Database

4 Create the database relation

juju integrate concourse:postgresql postgresql:database

Step 5: Check Status (Units Will Be Waiting)

5 Watch deployment status

juju status --watch 5s

You should see units showing: "Waiting for shared storage mount"

This is expected! Press Ctrl+C to stop watching.

💡 What's happening: The units know they're supposed to use shared storage, but the LXC disk device hasn't been added yet. Let's fix that!

Step 6: Configure LXC Shared Storage

Now we'll set up the actual shared storage using the provided script.

6 Create shared storage directory on host

mkdir -p /tmp/concourse-shared

7 Run the setup script

cd /home/sylee/projects/concourse-ci-machine
./scripts/setup-shared-storage.sh concourse /tmp/concourse-shared

This script will:

Find all LXC containers for the "concourse" application
Add a disk device mounting /tmp/concourse-shared to /var/lib/concourse
Enable UID/GID mapping so permissions work correctly

✅ Script Output: You should see messages like "Added device 'concourse-storage' to container juju-xxxxx-0"

Step 7: Watch the Magic Happen

Within 10 seconds (thanks to test mode), units will detect the shared storage and complete setup automatically!

8 Watch deployment complete

juju status --watch 5s

You'll see the progression:

concourse/0 (leader): Downloading binaries to shared storage
concourse/1-2 (workers): Detecting binaries, skipping download
All units: Active and ready!

Step 8: Verify Shared Storage is Working

Let's confirm the binaries are actually shared.

9 Check that units share the same filesystem

# Check unit 0
juju ssh concourse/0 "df /var/lib/concourse | tail -1"

# Check unit 1  
juju ssh concourse/1 "df /var/lib/concourse | tail -1"

The device name should be the same for both units! This proves they're using shared storage.

10 Verify binaries exist only once

juju ssh concourse/0 "ls -lh /var/lib/concourse/bin/"

You should see:

concourse  (~900MB)
gdn        (~20MB)

These binaries are shared across all 3 units - not duplicated!

What You've Accomplished

Congratulations! You've deployed Concourse CI with shared storage. Here's what you gained:

✅ 62% disk space savings (3 units: ~1.15GB instead of 3GB)
✅ Faster deployments (workers don't download binaries)
✅ Faster upgrades (coordinated, single download)
✅ Same functionality (everything works exactly the same)

🎉 Success! Your Concourse CI deployment is now using shared storage efficiently!

Testing the Efficiency

Let's add another worker to see how fast it is with shared storage.

11 Add a new worker unit

juju add-unit concourse

Wait for the new unit to start, then run the setup script again:

./scripts/setup-shared-storage.sh concourse /tmp/concourse-shared

Watch how quickly the new unit becomes active! It doesn't download anything - it just uses the existing shared binaries.

Next Steps

Now that you understand shared storage, explore these advanced topics:

GPU Workers: Try the GPU workers tutorial
Dataset Mounting: Learn how to inject datasets into GPU tasks
Understanding the Architecture: Read how shared storage works internally

Troubleshooting

Units stuck on "Waiting for shared storage mount"

Solution: Make sure you ran the setup-shared-storage.sh script. Check it completed without errors.

Permission denied errors

Solution: The script uses shift=true for UID mapping. If you still see permission errors, check the LXC device configuration:

lxc config device show juju-xxxxx-0

Binaries not appearing

Solution: Check the leader unit is active and has completed download:

juju debug-log --include concourse/0 --replay | grep -i download

🎓 Tutorial Complete! You now know how to deploy Concourse CI with efficient shared storage. This is a production-ready pattern for multi-worker deployments!