# GitCaddy Runner - User Guide

A CI/CD runner for Gitea Actions that executes workflows defined in `.gitea/workflows/` directories. This runner connects to your Gitea instance and picks up jobs to execute them in Docker containers or directly on the host machine.

## Table of Contents

- [Installation](#installation)
- [Getting Started](#getting-started)
- [Registration](#registration)
- [Running the Daemon](#running-the-daemon)
- [Configuration](#configuration)
- [Labels and Execution Environments](#labels-and-execution-environments)
- [Docker Deployment](#docker-deployment)
- [Kubernetes Deployment](#kubernetes-deployment)
- [Common Workflows](#common-workflows)
- [Artifact Upload Helper](#artifact-upload-helper)
- [Disk Space Management](#disk-space-management)
- [Cache Server](#cache-server)
- [Troubleshooting](#troubleshooting)
- [Best Practices](#best-practices)

## Installation

### From Binary

Download the latest release for your platform:

```bash
# Linux AMD64
wget https://git.marketally.com/gitcaddy/gitcaddy-runner/releases/download/v1.0.0/gitcaddy-runner-1.0.0-linux-amd64

# macOS ARM64
wget https://git.marketally.com/gitcaddy/gitcaddy-runner/releases/download/v1.0.0/gitcaddy-runner-1.0.0-darwin-12-arm64

# Windows AMD64
wget https://git.marketally.com/gitcaddy/gitcaddy-runner/releases/download/v1.0.0/gitcaddy-runner-1.0.0-windows-amd64.exe
```

Make the binary executable:

```bash
chmod +x gitcaddy-runner-1.0.0-linux-amd64
sudo mv gitcaddy-runner-1.0.0-linux-amd64 /usr/local/bin/gitcaddy-runner
```

### From Source

Requirements:
- Go 1.23 or later
- Docker (if using Docker-based workflows)

```bash
git clone https://git.marketally.com/gitcaddy/gitcaddy-runner.git
cd gitcaddy-runner
make build
sudo mv gitcaddy-runner /usr/local/bin/
```

### Verify Installation

```bash
gitcaddy-runner --version
```

## Getting Started

### Prerequisites

1. **Gitea Instance**: You need a running Gitea instance (v1.21 or later recommended)
2. **Docker** (optional but recommended): For running workflows in containers
3. **Registration Token**: Obtain from your Gitea instance:
   - Go to your repository → Settings → Actions → Runners
   - Click "Create new runner" and copy the registration token

### Quick Start

1. **Generate a configuration file**:

```bash
gitcaddy-runner generate-config > config.yaml
```

2. **Register the runner**:

```bash
gitcaddy-runner register \
  --instance https://gitea.example.com \
  --token YOUR_REGISTRATION_TOKEN \
  --name my-runner \
  --labels "ubuntu-latest:docker://docker.gitea.com/runner-images:ubuntu-latest"
```

3. **Start the runner daemon**:

```bash
gitcaddy-runner daemon
```

## Registration

The runner must be registered with your Gitea instance before it can execute jobs.

### Interactive Registration

Run the registration command without flags for an interactive setup:

```bash
gitcaddy-runner register
```

You'll be prompted for:
- **Gitea instance URL**: `https://gitea.example.com/`
- **Registration token**: Obtained from Gitea UI
- **Runner name**: Defaults to hostname if left empty
- **Labels**: Comma-separated list of labels (or press Enter for defaults)

### Non-Interactive Registration

For automation or scripts:

```bash
gitcaddy-runner register \
  --no-interactive \
  --instance https://gitea.example.com \
  --token YOUR_TOKEN \
  --name production-runner \
  --labels "ubuntu-latest:docker://docker.gitea.com/runner-images:ubuntu-latest,ubuntu-22.04:docker://docker.gitea.com/runner-images:ubuntu-22.04"
```

### Ephemeral Runners

For single-use runners (useful for auto-scaling):

```bash
gitcaddy-runner register \
  --ephemeral \
  --instance https://gitea.example.com \
  --token YOUR_TOKEN \
  --name ephemeral-runner
```

Ephemeral runners automatically unregister after completing one job.

### Registration File

Registration information is stored in `.runner` (by default). This file contains:
- Runner UUID and token
- Runner name
- Configured labels
- Gitea instance address

**Important**: Keep this file secure as it contains authentication credentials.

## Running the Daemon

### Basic Usage

Start the runner daemon to begin accepting jobs:

```bash
gitcaddy-runner daemon
```

With a custom configuration file:

```bash
gitcaddy-runner daemon --config /path/to/config.yaml
```

### Run Once Mode

Execute a single job and then exit:

```bash
gitcaddy-runner daemon --once
```

This is useful for:
- Testing runner setup
- Ephemeral runners in auto-scaling environments
- Scheduled job execution

### Background Execution

Run as a background service using systemd:

Create `/etc/systemd/system/gitcaddy-runner.service`:

```ini
[Unit]
Description=GitCaddy Runner
After=network.target docker.service
Requires=docker.service

[Service]
Type=simple
User=gitea-runner
WorkingDirectory=/home/gitea-runner
ExecStart=/usr/local/bin/gitcaddy-runner daemon --config /home/gitea-runner/config.yaml
Restart=always
RestartSec=10

[Install]
WantedBy=multi-user.target
```

Enable and start:

```bash
sudo systemctl daemon-reload
sudo systemctl enable gitcaddy-runner
sudo systemctl start gitcaddy-runner
sudo systemctl status gitcaddy-runner
```

## Configuration

### Generate Default Configuration

```bash
gitcaddy-runner generate-config > config.yaml
```

### Configuration File Structure

```yaml
log:
  level: info  # trace, debug, info, warn, error, fatal

runner:
  file: .runner                # Registration file path
  capacity: 1                  # Number of concurrent jobs
  timeout: 3h                  # Job timeout
  shutdown_timeout: 0s         # Graceful shutdown timeout
  insecure: false              # Skip TLS verification
  fetch_timeout: 5s            # Job fetch timeout
  fetch_interval: 2s           # Job fetch interval
  github_mirror: ''            # Mirror for GitHub actions
  envs:                        # Extra environment variables
    NODE_ENV: production
  env_file: .env               # Load environment from file
  labels:                      # Runner labels
    - "ubuntu-latest:docker://docker.gitea.com/runner-images:ubuntu-latest"

cache:
  enabled: true
  dir: ""                      # Cache directory (default: $HOME/.cache/actcache)
  host: ""                     # Cache server host (auto-detected)
  port: 0                      # Cache server port (0 = random)
  external_server: ""          # External cache server URL

container:
  network: ""                  # Docker network (empty = auto-create)
  privileged: false            # Run containers in privileged mode
  options: ""                  # Additional docker run options
  workdir_parent: workspace    # Parent directory for job workspaces
  valid_volumes:               # Allowed volume mounts (glob patterns)
    - '**'
  docker_host: ""              # Docker daemon socket (auto-detected)
  force_pull: true             # Always pull images
  force_rebuild: false         # Rebuild images even if present
  require_docker: false        # Require Docker daemon
  docker_timeout: 0s           # Docker daemon startup timeout

host:
  workdir_parent: ""           # Host workspace directory (default: $HOME/.cache/act)
```

### Environment Variables

Legacy environment variables (deprecated but still supported):

- `GITEA_DEBUG`: Enable debug logging
- `GITEA_TRACE`: Enable trace logging
- `GITEA_RUNNER_CAPACITY`: Set runner capacity
- `GITEA_RUNNER_FILE`: Set registration file path
- `GITEA_RUNNER_ENVIRON`: Set environment variables (comma-separated `key:value`)
- `GITEA_RUNNER_ENV_FILE`: Load environment from file

**Note**: Configuration file settings take precedence over environment variables.

## Labels and Execution Environments

Labels determine which jobs a runner can execute and how they run.

### Label Format

```
label-name:schema:argument
```

- **label-name**: Identifier used in workflow `runs-on`
- **schema**: Execution environment (`docker` or `host`)
- **argument**: Additional configuration (e.g., Docker image)

### Docker Labels

Run jobs in Docker containers:

```yaml
labels:
  - "ubuntu-latest:docker://docker.gitea.com/runner-images:ubuntu-latest"
  - "ubuntu-22.04:docker://docker.gitea.com/runner-images:ubuntu-22.04"
  - "node:docker://node:18"
  - "python:docker://python:3.11"
```

Workflow usage:

```yaml
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - run: echo "Running in Ubuntu container"
```

### Host Labels

Run jobs directly on the host machine:

```yaml
labels:
  - "self-hosted:host"
  - "linux:host"
  - "macos:host"
```

Workflow usage:

```yaml
jobs:
  build:
    runs-on: self-hosted
    steps:
      - uses: actions/checkout@v3
      - run: echo "Running on host"
```

### Custom Images

Use custom Docker images:

```yaml
labels:
  - "custom-env:docker://registry.example.com/my-build-env:latest"
```

### Multiple Labels

A runner can have multiple labels to support different job types:

```yaml
labels:
  - "ubuntu-latest:docker://docker.gitea.com/runner-images:ubuntu-latest"
  - "ubuntu-22.04:docker://docker.gitea.com/runner-images:ubuntu-22.04"
  - "self-hosted:host"
  - "gpu:host"
```

## Docker Deployment

### Basic Docker Run

```bash
docker run -d \
  --name gitcaddy-runner \
  -e GITEA_INSTANCE_URL=https://gitea.example.com \
  -e GITEA_RUNNER_REGISTRATION_TOKEN=YOUR_TOKEN \
  -e GITEA_RUNNER_NAME=docker-runner \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -v /data/runner:/data \
  marketally/gitcaddy-runner:latest
```

### Docker Compose

Create `docker-compose.yml`:

```yaml
version: '3'

services:
  runner:
    image: marketally/gitcaddy-runner:latest
    environment:
      - GITEA_INSTANCE_URL=https://gitea.example.com
      - GITEA_RUNNER_REGISTRATION_TOKEN=YOUR_TOKEN
      - GITEA_RUNNER_NAME=compose-runner
      - GITEA_RUNNER_LABELS=ubuntu-latest:docker://docker.gitea.com/runner-images:ubuntu-latest
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - ./runner-data:/data
    restart: unless-stopped
```

Start:

```bash
docker-compose up -d
```

### Docker-in-Docker (DinD)

For isolated Docker environments:

```yaml
version: '3'

services:
  runner:
    image: marketally/gitcaddy-runner:latest-dind
    privileged: true
    environment:
      - GITEA_INSTANCE_URL=https://gitea.example.com
      - GITEA_RUNNER_REGISTRATION_TOKEN=YOUR_TOKEN
      - DOCKER_TLS_CERTDIR=/certs
    volumes:
      - runner-data:/data
      - docker-certs:/certs
    restart: unless-stopped

volumes:
  runner-data:
  docker-certs:
```

### Rootless Docker

For enhanced security:

```yaml
version: '3'

services:
  runner:
    image: marketally/gitcaddy-runner:latest-dind-rootless
    environment:
      - GITEA_INSTANCE_URL=https://gitea.example.com
      - GITEA_RUNNER_REGISTRATION_TOKEN=YOUR_TOKEN
      - DOCKER_HOST=unix:///run/user/1000/docker.sock
    volumes:
      - runner-data:/data
    restart: unless-stopped
    security_opt:
      - seccomp:unconfined

volumes:
  runner-data:
```

## Kubernetes Deployment

### Using Docker-in-Docker

See `examples/kubernetes/dind-docker.yaml`:

```yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: gitcaddy-runner-vol
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi
---
apiVersion: v1
kind: Secret
metadata:
  name: runner-secret
type: Opaque
data:
  token: <base64-encoded-registration-token>
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: gitcaddy-runner
spec:
  replicas: 1
  selector:
    matchLabels:
      app: gitcaddy-runner
  template:
    metadata:
      labels:
        app: gitcaddy-runner
    spec:
      volumes:
      - name: docker-certs
        emptyDir: {}
      - name: runner-data
        persistentVolumeClaim:
          claimName: gitcaddy-runner-vol
      containers:
      - name: runner
        image: git.marketally.com/gitcaddy/gitcaddy-runner:latest
        command: ["sh", "-c", "while ! nc -z localhost 2376 </dev/null; do echo 'waiting for docker daemon...'; sleep 5; done; /sbin/tini -- run.sh"]
        env:
        - name: DOCKER_HOST
          value: tcp://localhost:2376
        - name: DOCKER_CERT_PATH
          value: /certs/client
        - name: DOCKER_TLS_VERIFY
          value: "1"
        - name: GITEA_INSTANCE_URL
          value: http://gitcaddy-http.gitcaddy.svc.cluster.local:3000
        - name: GITEA_RUNNER_REGISTRATION_TOKEN
          valueFrom:
            secretKeyRef:
              name: runner-secret
              key: token
        volumeMounts:
        - name: docker-certs
          mountPath: /certs
        - name: runner-data
          mountPath: /data
      - name: daemon
        image: docker:23.0.6-dind
        env:
        - name: DOCKER_TLS_CERTDIR
          value: /certs
        securityContext:
          privileged: true
        volumeMounts:
        - name: docker-certs
          mountPath: /certs
```

Apply:

```bash
kubectl apply -f dind-docker.yaml
```

### Using Rootless Docker

See `examples/kubernetes/rootless-docker.yaml` for a more secure rootless setup.

## Common Workflows

### Basic CI Workflow

Create `.gitea/workflows/ci.yaml`:

```yaml
name: CI

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      
      - name: Set up Go
        uses: actions/setup-go@v4
        with:
          go-version: '1.21'
      
      - name: Run tests
        run: go test ./...
      
      - name: Build
        run: go build -v ./...
```

### Multi-Platform Build

```yaml
name: Build

on: [push]

jobs:
  build-linux:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - run: make build-linux
  
  build-macos:
    runs-on: macos
    steps:
      - uses: actions/checkout@v3
      - run: make build-macos
  
  build-windows:
    runs-on: windows
    steps:
      - uses: actions/checkout@v3
      - run: make build-windows
```

### Using Caching

```yaml
name: Build with Cache

on: [push]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      
      - name: Cache dependencies
        uses: actions/cache@v3
        with:
          path: |
            ~/.cache/go-build
            ~/go/pkg/mod
          key: ${{ runner.os }}-go-${{ hashFiles('**/go.sum') }}
          restore-keys: |
            ${{ runner.os }}-go-
      
      - name: Build
        run: go build ./...
```

### Using Artifacts

**Important**: GitCaddy Runner does not support `actions/upload-artifact@v4` or `actions/download-artifact@v4`. Use v3 instead:

```yaml
name: Build and Upload

on: [push]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      
      - name: Build
        run: make build
      
      - name: Upload artifact
        uses: actions/upload-artifact@v3  # Note: v3, not v4
        with:
          name: my-artifact
          path: dist/
  
  deploy:
    needs: build
    runs-on: ubuntu-latest
    steps:
      - name: Download artifact
        uses: actions/download-artifact@v3  # Note: v3, not v4
        with:
          name: my-artifact
          path: dist/
      
      - name: Deploy
        run: ./deploy.sh
```

### Using Services

```yaml
name: Integration Tests

on: [push]

jobs:
  test:
    runs-on: ubuntu-latest
    services:
      postgres:
        image: postgres:15
        env:
          POSTGRES_PASSWORD: postgres
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5
    steps:
      - uses: actions/checkout@v3
      
      - name: Run integration tests
        env:
          DATABASE_URL: postgres://postgres:postgres@postgres:5432/test
        run: go test -tags=integration ./...
```

## Artifact Upload Helper

For large files or unreliable networks, use the built-in upload helper with retry logic:

```bash
gitcaddy-upload \
  -url https://gitea.example.com/api/v1/repos/owner/repo/releases/1/assets \
  -token YOUR_TOKEN \
  -file ./large-artifact.zip \
  -retries 10
```

Options:
- `-url`: Upload endpoint URL
- `-token`: Authentication token
- `-file`: File to upload
- `-retries`: Maximum retry attempts (default: 5)

The upload helper automatically:
- Retries on network failures
- Pre-warms connections
- Uses chunked uploads for large files
- Provides detailed progress logging

## Disk Space Management

The runner includes automatic disk space monitoring and cleanup capabilities.

### Automatic Cleanup

The runner automatically monitors disk usage and triggers cleanup when:
- Disk usage exceeds 85% (warning threshold)
- Disk usage exceeds 95% (critical threshold)

Cleanup targets:
- Old cache directories (>24 hours)
- Old work directories (>48 hours)
- Old artifact staging files (>72 hours)
- System temp files (>24 hours)
- Build tool caches (>7 days): Go, npm, pip, Maven, Gradle, NuGet, Cargo, etc.

### Manual Cleanup

Trigger cleanup manually:

```bash
gitcaddy-runner cleanup --config config.yaml
```

Output example:

```
Cleanup completed: freed 15728640000 bytes, deleted 1234 files in 5.2s
```

### Monitoring Disk Space

The runner reports disk space in its capabilities:

```json
{
  "disk": {
    "path": "/workspace",
    "total_bytes": 107374182400,
    "free_bytes": 32212254720,
    "used_bytes": 75161927680,
    "used_percent": 70.0
  }
}
```

### Best Practices

1. **Monitor disk usage**: Check runner logs for disk space warnings
2. **Configure cleanup**: Adjust cache retention in config.yaml
3. **Use ephemeral runners**: For auto-scaling environments
4. **Separate volumes**: Use dedicated volumes for runner data

## Cache Server

The runner includes a built-in cache server for `actions/cache`.

### Automatic Cache Server

Enabled by default in the configuration:

```yaml
cache:
  enabled: true
  dir: ""      # Default: $HOME/.cache/actcache
  host: ""     # Auto-detected
  port: 0      # Random available port
```

The cache server URL is automatically provided to jobs via `ACTIONS_CACHE_URL`.

### External Cache Server

Use an external cache server:

```yaml
cache:
  enabled: true
  external_server: "http://cache.example.com:8080/"
```

### Standalone Cache Server

Run the cache server independently:

```bash
gitcaddy-runner cache-server \
  --dir /path/to/cache \
  --host 0.0.0.0 \
  --port 8080
```

Options:
- `--dir`: Cache storage directory
- `--host`: Listen address
- `--port`: Listen port

## Troubleshooting

### Runner Not Picking Up Jobs

**Check registration**:

```bash
cat .runner
```

Verify the runner appears in Gitea UI (Settings → Actions → Runners).

**Check labels**: Ensure workflow `runs-on` matches a runner label.

**Check logs**:

```bash
gitcaddy-runner daemon --config config.yaml
```

Look for connection errors or authentication issues.

### Docker Connection Issues

**Error**: `Cannot ping the docker daemon`

**Solutions**:

1. Verify Docker is running:

```bash
docker ps
```

2. Check Docker socket path:

```bash
ls -la /var/run/docker.sock
```

3. Add user to docker group:

```bash
sudo usermod -aG docker $USER
newgrp docker
```

4. Specify Docker host in config:

```yaml
container:
  docker_host: "unix:///var/run/docker.sock"
```

### Timeout Issues

**Error**: Job timeout after 3 hours

**Solution**: Increase timeout in config:

```yaml
runner:
  timeout: 6h
```

### Disk Space Issues

**Error**: No space left on device

**Solutions**:

1. Run manual cleanup:

```bash
gitcaddy-runner cleanup
```

2. Increase disk space or use dedicated volume

3. Configure more aggressive cleanup:

```yaml
cache:
  dir: /path/to/larger/disk
```

### SSL/TLS Certificate Issues

**Error**: `x509: certificate signed by unknown authority`

**Solution** (not recommended for production):

```yaml
runner:
  insecure: true
```

**Better solution**: Install proper CA certificates:

```bash
sudo cp your-ca.crt /usr/local/share/ca-certificates/
sudo update-ca-certificates
```

### Actions Not Found

**Error**: `unable to resolve action`

**Solution**: Configure GitHub mirror if needed:

```yaml
runner:
  github_mirror: 'https://github.mirror.example.com'
```

### Artifact Upload Failures

**Error**: `actions/upload-artifact@v4 not supported`

**Solution**: Use v3 instead:

```yaml
- uses: actions/upload-artifact@v3  # Not v4
  with:
    name: my-artifact
    path: dist/
```

For large files with unreliable networks, use the upload helper:

```bash
gitcaddy-upload -url $URL -token $TOKEN -file artifact.zip
```

## Best Practices

### Security

1. **Keep registration file secure**: The `.runner` file contains authentication credentials
2. **Use secrets**: Never hardcode sensitive data in workflows
3. **Limit volume mounts**: Restrict `valid_volumes` in configuration
4. **Use rootless Docker**: When possible, use rootless containers
5. **Regular updates**: Keep runner and Docker images up to date

### Performance

1. **Use caching**: Cache dependencies to speed up builds
2. **Parallel jobs**: Increase `capacity` for concurrent job execution
3. **Local images**: Use `force_pull: false` for frequently used images
4. **Dedicated volumes**: Use fast storage for workspaces and cache

### Reliability

1. **Monitor disk space**: Watch for low disk space warnings
2. **Set appropriate timeouts**: Balance between job completion and resource usage
3. **Use health checks**: Monitor runner process and Docker daemon
4. **Graceful shutdown**: Configure `shutdown_timeout` for clean termination
5. **Log rotation**: Implement log rotation to prevent disk fill

### Scalability

1. **Ephemeral runners**: Use `--ephemeral` for auto-scaling
2. **Multiple runners**: Deploy multiple runners for load distribution
3. **Label-based routing**: Use labels to route jobs to appropriate runners
4. **Resource limits**: Set Docker resource limits in `container.options`

### Maintenance

1. **Regular cleanup**: Run periodic manual cleanup or rely on automatic cleanup
2. **Update images**: Keep runner images and workflow images current
3. **Monitor logs**: Check logs for warnings and errors
4. **Test configuration**: Validate config changes in staging first
5. **Backup registration**: Keep backup of `.runner` file

### Workflow Best Practices

1. **Use specific versions**: Pin action versions (e.g., `@v3`, not `@latest`)
2. **Minimize steps**: Combine related commands
3. **Use artifacts wisely**: Only upload necessary files
4. **Clean up**: Remove temporary files in workflows
5. **Test locally**: Use `gitcaddy-runner exec` to test workflows locally

### Example Production Setup

```yaml
# config.yaml
log:
  level: info

runner:
  capacity: 4
  timeout: 2h
  shutdown_timeout: 5m
  fetch_interval: 2s
  labels:
    - "ubuntu-latest:docker://docker.gitea.com/runner-images:ubuntu-latest"
    - "ubuntu-22.04:docker://docker.gitea.com/runner-images:ubuntu-22.04"

cache:
  enabled: true
  dir: /mnt/fast-storage/cache

container:
  network: gitea-actions
  force_pull: false
  workdir_parent: /mnt/fast-storage/workspace
  valid_volumes:
    - /mnt/artifacts/**
  docker_host: unix:///var/run/docker.sock

host:
  workdir_parent: /mnt/fast-storage/host-workspace
```

This configuration provides:
- 4 concurrent jobs
- 2-hour job timeout with 5-minute graceful shutdown
- Fast storage for cache and workspaces
- Restricted volume mounts for security
- Optimized image pulling