gitcaddy-runner/GUIDE.md

GitCaddy Runner - User Guide

A CI/CD runner for Gitea Actions that executes workflows defined in .gitea/workflows/ directories. This runner connects to your Gitea instance and picks up jobs to execute them in Docker containers or directly on the host machine.

Table of Contents

• Installation
• Getting Started
• Registration
• Running the Daemon
• Configuration
• Labels and Execution Environments
• Docker Deployment
• Kubernetes Deployment
• Common Workflows
• Artifact Upload Helper
• Disk Space Management
• Cache Server
• Troubleshooting
• Best Practices

Installation

From Binary

Download the latest release for your platform:

# Linux AMD64
wget https://git.marketally.com/gitcaddy/gitcaddy-runner/releases/download/v1.0.0/gitcaddy-runner-1.0.0-linux-amd64

# macOS ARM64
wget https://git.marketally.com/gitcaddy/gitcaddy-runner/releases/download/v1.0.0/gitcaddy-runner-1.0.0-darwin-12-arm64

# Windows AMD64
wget https://git.marketally.com/gitcaddy/gitcaddy-runner/releases/download/v1.0.0/gitcaddy-runner-1.0.0-windows-amd64.exe

Make the binary executable:

chmod +x gitcaddy-runner-1.0.0-linux-amd64
sudo mv gitcaddy-runner-1.0.0-linux-amd64 /usr/local/bin/gitcaddy-runner

From Source

Requirements:

• Go 1.23 or later
• Docker (if using Docker-based workflows)

git clone https://git.marketally.com/gitcaddy/gitcaddy-runner.git
cd gitcaddy-runner
make build
sudo mv gitcaddy-runner /usr/local/bin/

Verify Installation

gitcaddy-runner --version

Getting Started

Prerequisites

1. Gitea Instance: You need a running Gitea instance (v1.21 or later recommended)
2. Docker (optional but recommended): For running workflows in containers
3. Registration Token: Obtain from your Gitea instance:
   • Go to your repository → Settings → Actions → Runners
   • Click "Create new runner" and copy the registration token

Quick Start

1. Generate a configuration file:

gitcaddy-runner generate-config > config.yaml

2. Register the runner:

gitcaddy-runner register \
  --instance https://gitea.example.com \
  --token YOUR_REGISTRATION_TOKEN \
  --name my-runner \
  --labels "ubuntu-latest:docker://docker.gitea.com/runner-images:ubuntu-latest"

3. Start the runner daemon:

gitcaddy-runner daemon

Registration

The runner must be registered with your Gitea instance before it can execute jobs.

Interactive Registration

Run the registration command without flags for an interactive setup:

gitcaddy-runner register

You'll be prompted for:

• Gitea instance URL: https://gitea.example.com/
• Registration token: Obtained from Gitea UI
• Runner name: Defaults to hostname if left empty
• Labels: Comma-separated list of labels (or press Enter for defaults)

Non-Interactive Registration

For automation or scripts:

gitcaddy-runner register \
  --no-interactive \
  --instance https://gitea.example.com \
  --token YOUR_TOKEN \
  --name production-runner \
  --labels "ubuntu-latest:docker://docker.gitea.com/runner-images:ubuntu-latest,ubuntu-22.04:docker://docker.gitea.com/runner-images:ubuntu-22.04"

Ephemeral Runners

For single-use runners (useful for auto-scaling):

gitcaddy-runner register \
  --ephemeral \
  --instance https://gitea.example.com \
  --token YOUR_TOKEN \
  --name ephemeral-runner

Ephemeral runners automatically unregister after completing one job.

Registration File

Registration information is stored in .runner (by default). This file contains:

• Runner UUID and token
• Runner name
• Configured labels
• Gitea instance address

Important: Keep this file secure as it contains authentication credentials.

Running the Daemon

Basic Usage

Start the runner daemon to begin accepting jobs:

gitcaddy-runner daemon

With a custom configuration file:

gitcaddy-runner daemon --config /path/to/config.yaml

Run Once Mode

Execute a single job and then exit:

gitcaddy-runner daemon --once

This is useful for:

• Testing runner setup
• Ephemeral runners in auto-scaling environments
• Scheduled job execution

Background Execution

Run as a background service using systemd:

Create /etc/systemd/system/gitcaddy-runner.service:

[Unit]
Description=GitCaddy Runner
After=network.target docker.service
Requires=docker.service

[Service]
Type=simple
User=gitea-runner
WorkingDirectory=/home/gitea-runner
ExecStart=/usr/local/bin/gitcaddy-runner daemon --config /home/gitea-runner/config.yaml
Restart=always
RestartSec=10

[Install]
WantedBy=multi-user.target

Enable and start:

sudo systemctl daemon-reload
sudo systemctl enable gitcaddy-runner
sudo systemctl start gitcaddy-runner
sudo systemctl status gitcaddy-runner

Configuration

Generate Default Configuration

gitcaddy-runner generate-config > config.yaml

Configuration File Structure

log:
  level: info  # trace, debug, info, warn, error, fatal

runner:
  file: .runner                # Registration file path
  capacity: 1                  # Number of concurrent jobs
  timeout: 3h                  # Job timeout
  shutdown_timeout: 0s         # Graceful shutdown timeout
  insecure: false              # Skip TLS verification
  fetch_timeout: 5s            # Job fetch timeout
  fetch_interval: 2s           # Job fetch interval
  github_mirror: ''            # Mirror for GitHub actions
  envs:                        # Extra environment variables
    NODE_ENV: production
  env_file: .env               # Load environment from file
  labels:                      # Runner labels
    - "ubuntu-latest:docker://docker.gitea.com/runner-images:ubuntu-latest"

cache:
  enabled: true
  dir: ""                      # Cache directory (default: $HOME/.cache/actcache)
  host: ""                     # Cache server host (auto-detected)
  port: 0                      # Cache server port (0 = random)
  external_server: ""          # External cache server URL

container:
  network: ""                  # Docker network (empty = auto-create)
  privileged: false            # Run containers in privileged mode
  options: ""                  # Additional docker run options
  workdir_parent: workspace    # Parent directory for job workspaces
  valid_volumes:               # Allowed volume mounts (glob patterns)
    - '**'
  docker_host: ""              # Docker daemon socket (auto-detected)
  force_pull: true             # Always pull images
  force_rebuild: false         # Rebuild images even if present
  require_docker: false        # Require Docker daemon
  docker_timeout: 0s           # Docker daemon startup timeout

host:
  workdir_parent: ""           # Host workspace directory (default: $HOME/.cache/act)

Environment Variables

Legacy environment variables (deprecated but still supported):

• GITEA_DEBUG: Enable debug logging
• GITEA_TRACE: Enable trace logging
• GITEA_RUNNER_CAPACITY: Set runner capacity
• GITEA_RUNNER_FILE: Set registration file path
• GITEA_RUNNER_ENVIRON: Set environment variables (comma-separated key:value)
• GITEA_RUNNER_ENV_FILE: Load environment from file

Note: Configuration file settings take precedence over environment variables.

Labels and Execution Environments

Labels determine which jobs a runner can execute and how they run.

Label Format

label-name:schema:argument

• label-name: Identifier used in workflow runs-on
• schema: Execution environment (docker or host)
• argument: Additional configuration (e.g., Docker image)

Docker Labels

Run jobs in Docker containers:

labels:
  - "ubuntu-latest:docker://docker.gitea.com/runner-images:ubuntu-latest"
  - "ubuntu-22.04:docker://docker.gitea.com/runner-images:ubuntu-22.04"
  - "node:docker://node:18"
  - "python:docker://python:3.11"

Workflow usage:

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - run: echo "Running in Ubuntu container"

Host Labels

Run jobs directly on the host machine:

labels:
  - "self-hosted:host"
  - "linux:host"
  - "macos:host"

Workflow usage:

jobs:
  build:
    runs-on: self-hosted
    steps:
      - uses: actions/checkout@v3
      - run: echo "Running on host"

Custom Images

Use custom Docker images:

labels:
  - "custom-env:docker://registry.example.com/my-build-env:latest"

Multiple Labels

A runner can have multiple labels to support different job types:

labels:
  - "ubuntu-latest:docker://docker.gitea.com/runner-images:ubuntu-latest"
  - "ubuntu-22.04:docker://docker.gitea.com/runner-images:ubuntu-22.04"
  - "self-hosted:host"
  - "gpu:host"

Docker Deployment

Basic Docker Run

docker run -d \
  --name gitcaddy-runner \
  -e GITEA_INSTANCE_URL=https://gitea.example.com \
  -e GITEA_RUNNER_REGISTRATION_TOKEN=YOUR_TOKEN \
  -e GITEA_RUNNER_NAME=docker-runner \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -v /data/runner:/data \
  marketally/gitcaddy-runner:latest

Docker Compose

Create docker-compose.yml:

version: '3'

services:
  runner:
    image: marketally/gitcaddy-runner:latest
    environment:
      - GITEA_INSTANCE_URL=https://gitea.example.com
      - GITEA_RUNNER_REGISTRATION_TOKEN=YOUR_TOKEN
      - GITEA_RUNNER_NAME=compose-runner
      - GITEA_RUNNER_LABELS=ubuntu-latest:docker://docker.gitea.com/runner-images:ubuntu-latest
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - ./runner-data:/data
    restart: unless-stopped

Start:

docker-compose up -d

Docker-in-Docker (DinD)

For isolated Docker environments:

version: '3'

services:
  runner:
    image: marketally/gitcaddy-runner:latest-dind
    privileged: true
    environment:
      - GITEA_INSTANCE_URL=https://gitea.example.com
      - GITEA_RUNNER_REGISTRATION_TOKEN=YOUR_TOKEN
      - DOCKER_TLS_CERTDIR=/certs
    volumes:
      - runner-data:/data
      - docker-certs:/certs
    restart: unless-stopped

volumes:
  runner-data:
  docker-certs:

Rootless Docker

For enhanced security:

version: '3'

services:
  runner:
    image: marketally/gitcaddy-runner:latest-dind-rootless
    environment:
      - GITEA_INSTANCE_URL=https://gitea.example.com
      - GITEA_RUNNER_REGISTRATION_TOKEN=YOUR_TOKEN
      - DOCKER_HOST=unix:///run/user/1000/docker.sock
    volumes:
      - runner-data:/data
    restart: unless-stopped
    security_opt:
      - seccomp:unconfined

volumes:
  runner-data:

Kubernetes Deployment

Using Docker-in-Docker

See examples/kubernetes/dind-docker.yaml:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: gitcaddy-runner-vol
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi
---
apiVersion: v1
kind: Secret
metadata:
  name: runner-secret
type: Opaque
data:
  token: <base64-encoded-registration-token>
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: gitcaddy-runner
spec:
  replicas: 1
  selector:
    matchLabels:
      app: gitcaddy-runner
  template:
    metadata:
      labels:
        app: gitcaddy-runner
    spec:
      volumes:
      - name: docker-certs
        emptyDir: {}
      - name: runner-data
        persistentVolumeClaim:
          claimName: gitcaddy-runner-vol
      containers:
      - name: runner
        image: git.marketally.com/gitcaddy/gitcaddy-runner:latest
        command: ["sh", "-c", "while ! nc -z localhost 2376 </dev/null; do echo 'waiting for docker daemon...'; sleep 5; done; /sbin/tini -- run.sh"]
        env:
        - name: DOCKER_HOST
          value: tcp://localhost:2376
        - name: DOCKER_CERT_PATH
          value: /certs/client
        - name: DOCKER_TLS_VERIFY
          value: "1"
        - name: GITEA_INSTANCE_URL
          value: http://gitcaddy-http.gitcaddy.svc.cluster.local:3000
        - name: GITEA_RUNNER_REGISTRATION_TOKEN
          valueFrom:
            secretKeyRef:
              name: runner-secret
              key: token
        volumeMounts:
        - name: docker-certs
          mountPath: /certs
        - name: runner-data
          mountPath: /data
      - name: daemon
        image: docker:23.0.6-dind
        env:
        - name: DOCKER_TLS_CERTDIR
          value: /certs
        securityContext:
          privileged: true
        volumeMounts:
        - name: docker-certs
          mountPath: /certs

Apply:

kubectl apply -f dind-docker.yaml

Using Rootless Docker

See examples/kubernetes/rootless-docker.yaml for a more secure rootless setup.

Common Workflows

Basic CI Workflow

Create .gitea/workflows/ci.yaml:

name: CI

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      
      - name: Set up Go
        uses: actions/setup-go@v4
        with:
          go-version: '1.21'
      
      - name: Run tests
        run: go test ./...
      
      - name: Build
        run: go build -v ./...

Multi-Platform Build

name: Build

on: [push]

jobs:
  build-linux:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - run: make build-linux
  
  build-macos:
    runs-on: macos
    steps:
      - uses: actions/checkout@v3
      - run: make build-macos
  
  build-windows:
    runs-on: windows
    steps:
      - uses: actions/checkout@v3
      - run: make build-windows

Using Caching

name: Build with Cache

on: [push]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      
      - name: Cache dependencies
        uses: actions/cache@v3
        with:
          path: |
            ~/.cache/go-build
            ~/go/pkg/mod
          key: ${{ runner.os }}-go-${{ hashFiles('**/go.sum') }}
          restore-keys: |
            ${{ runner.os }}-go-
      
      - name: Build
        run: go build ./...

Using Artifacts

Important: GitCaddy Runner does not support actions/upload-artifact@v4 or actions/download-artifact@v4. Use v3 instead:

name: Build and Upload

on: [push]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      
      - name: Build
        run: make build
      
      - name: Upload artifact
        uses: actions/upload-artifact@v3  # Note: v3, not v4
        with:
          name: my-artifact
          path: dist/
  
  deploy:
    needs: build
    runs-on: ubuntu-latest
    steps:
      - name: Download artifact
        uses: actions/download-artifact@v3  # Note: v3, not v4
        with:
          name: my-artifact
          path: dist/
      
      - name: Deploy
        run: ./deploy.sh

Using Services

name: Integration Tests

on: [push]

jobs:
  test:
    runs-on: ubuntu-latest
    services:
      postgres:
        image: postgres:15
        env:
          POSTGRES_PASSWORD: postgres
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5
    steps:
      - uses: actions/checkout@v3
      
      - name: Run integration tests
        env:
          DATABASE_URL: postgres://postgres:postgres@postgres:5432/test
        run: go test -tags=integration ./...

Artifact Upload Helper

For large files or unreliable networks, use the built-in upload helper with retry logic:

gitcaddy-upload \
  -url https://gitea.example.com/api/v1/repos/owner/repo/releases/1/assets \
  -token YOUR_TOKEN \
  -file ./large-artifact.zip \
  -retries 10

Options:

• -url: Upload endpoint URL
• -token: Authentication token
• -file: File to upload
• -retries: Maximum retry attempts (default: 5)

The upload helper automatically:

• Retries on network failures
• Pre-warms connections
• Uses chunked uploads for large files
• Provides detailed progress logging

Disk Space Management

The runner includes automatic disk space monitoring and cleanup capabilities.

Automatic Cleanup

The runner automatically monitors disk usage and triggers cleanup when:

• Disk usage exceeds 85% (warning threshold)
• Disk usage exceeds 95% (critical threshold)

Cleanup targets:

• Old cache directories (>24 hours)
• Old work directories (>48 hours)
• Old artifact staging files (>72 hours)
• System temp files (>24 hours)
• Build tool caches (>7 days): Go, npm, pip, Maven, Gradle, NuGet, Cargo, etc.

Manual Cleanup

Trigger cleanup manually:

gitcaddy-runner cleanup --config config.yaml

Output example:

Cleanup completed: freed 15728640000 bytes, deleted 1234 files in 5.2s

Monitoring Disk Space

The runner reports disk space in its capabilities:

{
  "disk": {
    "path": "/workspace",
    "total_bytes": 107374182400,
    "free_bytes": 32212254720,
    "used_bytes": 75161927680,
    "used_percent": 70.0
  }
}

Best Practices

1. Monitor disk usage: Check runner logs for disk space warnings
2. Configure cleanup: Adjust cache retention in config.yaml
3. Use ephemeral runners: For auto-scaling environments
4. Separate volumes: Use dedicated volumes for runner data

Cache Server

The runner includes a built-in cache server for actions/cache.

Automatic Cache Server

Enabled by default in the configuration:

cache:
  enabled: true
  dir: ""      # Default: $HOME/.cache/actcache
  host: ""     # Auto-detected
  port: 0      # Random available port

The cache server URL is automatically provided to jobs via ACTIONS_CACHE_URL.

External Cache Server

Use an external cache server:

cache:
  enabled: true
  external_server: "http://cache.example.com:8080/"

Standalone Cache Server

Run the cache server independently:

gitcaddy-runner cache-server \
  --dir /path/to/cache \
  --host 0.0.0.0 \
  --port 8080

Options:

• --dir: Cache storage directory
• --host: Listen address
• --port: Listen port

Troubleshooting

Runner Not Picking Up Jobs

Check registration:

cat .runner

Verify the runner appears in Gitea UI (Settings → Actions → Runners).

Check labels: Ensure workflow runs-on matches a runner label.

Check logs:

gitcaddy-runner daemon --config config.yaml

Look for connection errors or authentication issues.

Docker Connection Issues

Error: Cannot ping the docker daemon

Solutions:

1. Verify Docker is running:

docker ps

2. Check Docker socket path:

ls -la /var/run/docker.sock

3. Add user to docker group:

sudo usermod -aG docker $USER
newgrp docker

4. Specify Docker host in config:

container:
  docker_host: "unix:///var/run/docker.sock"

Timeout Issues

Error: Job timeout after 3 hours

Solution: Increase timeout in config:

runner:
  timeout: 6h

Disk Space Issues

Error: No space left on device

Solutions:

1. Run manual cleanup:

gitcaddy-runner cleanup

2. Increase disk space or use dedicated volume

3. Configure more aggressive cleanup:

cache:
  dir: /path/to/larger/disk

SSL/TLS Certificate Issues

Error: x509: certificate signed by unknown authority

Solution (not recommended for production):

runner:
  insecure: true

Better solution: Install proper CA certificates:

sudo cp your-ca.crt /usr/local/share/ca-certificates/
sudo update-ca-certificates

Actions Not Found

Error: unable to resolve action

Solution: Configure GitHub mirror if needed:

runner:
  github_mirror: 'https://github.mirror.example.com'

Artifact Upload Failures

Error: actions/upload-artifact@v4 not supported

Solution: Use v3 instead:

- uses: actions/upload-artifact@v3  # Not v4
  with:
    name: my-artifact
    path: dist/

For large files with unreliable networks, use the upload helper:

gitcaddy-upload -url $URL -token $TOKEN -file artifact.zip

Best Practices

Security

1. Keep registration file secure: The .runner file contains authentication credentials
2. Use secrets: Never hardcode sensitive data in workflows
3. Limit volume mounts: Restrict valid_volumes in configuration
4. Use rootless Docker: When possible, use rootless containers
5. Regular updates: Keep runner and Docker images up to date

Performance

1. Use caching: Cache dependencies to speed up builds
2. Parallel jobs: Increase capacity for concurrent job execution
3. Local images: Use force_pull: false for frequently used images
4. Dedicated volumes: Use fast storage for workspaces and cache

Reliability

1. Monitor disk space: Watch for low disk space warnings
2. Set appropriate timeouts: Balance between job completion and resource usage
3. Use health checks: Monitor runner process and Docker daemon
4. Graceful shutdown: Configure shutdown_timeout for clean termination
5. Log rotation: Implement log rotation to prevent disk fill

Scalability

1. Ephemeral runners: Use --ephemeral for auto-scaling
2. Multiple runners: Deploy multiple runners for load distribution
3. Label-based routing: Use labels to route jobs to appropriate runners
4. Resource limits: Set Docker resource limits in container.options

Maintenance

1. Regular cleanup: Run periodic manual cleanup or rely on automatic cleanup
2. Update images: Keep runner images and workflow images current
3. Monitor logs: Check logs for warnings and errors
4. Test configuration: Validate config changes in staging first
5. Backup registration: Keep backup of .runner file

Workflow Best Practices

1. Use specific versions: Pin action versions (e.g., @v3, not @latest)
2. Minimize steps: Combine related commands
3. Use artifacts wisely: Only upload necessary files
4. Clean up: Remove temporary files in workflows
5. Test locally: Use gitcaddy-runner exec to test workflows locally

Example Production Setup

# config.yaml
log:
  level: info

runner:
  capacity: 4
  timeout: 2h
  shutdown_timeout: 5m
  fetch_interval: 2s
  labels:
    - "ubuntu-latest:docker://docker.gitea.com/runner-images:ubuntu-latest"
    - "ubuntu-22.04:docker://docker.gitea.com/runner-images:ubuntu-22.04"

cache:
  enabled: true
  dir: /mnt/fast-storage/cache

container:
  network: gitea-actions
  force_pull: false
  workdir_parent: /mnt/fast-storage/workspace
  valid_volumes:
    - /mnt/artifacts/**
  docker_host: unix:///var/run/docker.sock

host:
  workdir_parent: /mnt/fast-storage/host-workspace

This configuration provides:

• 4 concurrent jobs
• 2-hour job timeout with 5-minute graceful shutdown
• Fast storage for cache and workspaces
• Restricted volume mounts for security
• Optimized image pulling