Simplify to local tar.xz filesystem cache, remove API dependency
- Store cache as tar.xz archives on runner filesystem at ~/.cache/.cache-store/ - No external API calls, no node.js, no curl needed - Restore extracts archive if key matches, save creates one on miss - Drastically simpler and more reliable for self-hosted runners
This commit is contained in:
@@ -1,28 +1,29 @@
|
|||||||
Cache Action
|
Cache Action
|
||||||
============
|
============
|
||||||
|
|
||||||
A Gitea Action that caches build files under `~/.cache/` using a key file hash. Implements the Gitea artifact cache API directly with curl, tar, and zstd — no external actions required.
|
A Gitea Action that caches build files as `tar.xz` archives on the runner filesystem. Simple, fast, no external dependencies.
|
||||||
|
|
||||||
## Inputs
|
## Inputs
|
||||||
|
|
||||||
| Input | Description | Required | Default |
|
| Input | Description | Required | Default |
|
||||||
|-------|-------------|----------|---------|
|
|-------|-------------|----------|---------|
|
||||||
| `path` | Files/directories to cache | No | `~/.cache/` |
|
| `path` | Directory to cache | No | `~/.cache/` |
|
||||||
| `key-prefix` | Prefix for the cache key | Yes | — |
|
| `key-prefix` | Prefix for the cache key | Yes | — |
|
||||||
| `key-file` | File to hash for the cache key | Yes | — |
|
| `key-file` | File to hash for the cache key | Yes | — |
|
||||||
| `restore-keys` | Fallback prefix-matched keys | No | — |
|
|
||||||
|
|
||||||
## Outputs
|
## Outputs
|
||||||
|
|
||||||
| Output | Description |
|
| Output | Description |
|
||||||
|--------|-------------|
|
|--------|-------------|
|
||||||
| `cache-hit` | `true` if exact key match found, `false` otherwise |
|
| `cache-hit` | `true` if exact key match found, `false` otherwise |
|
||||||
| `cache-primary-key` | The primary key used for cache lookup |
|
|
||||||
| `cache-matched-key` | The key of the cache entry that was restored |
|
|
||||||
|
|
||||||
## Quick Start
|
## How It Works
|
||||||
|
|
||||||
### Rust Example
|
1. Hashes `key-file` with SHA-256, combines with `key-prefix` as the cache key
|
||||||
|
2. Stores archives at `~/.cache/.cache-store/<key>.tar.xz`
|
||||||
|
3. Restores by extracting the archive, saves by creating one
|
||||||
|
|
||||||
|
## Rust Example
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
name: Rust Build
|
name: Rust Build
|
||||||
@@ -42,16 +43,14 @@ jobs:
|
|||||||
key-prefix: 'rust'
|
key-prefix: 'rust'
|
||||||
key-file: 'Cargo.lock'
|
key-file: 'Cargo.lock'
|
||||||
path: |
|
path: |
|
||||||
~/.cache/
|
|
||||||
~/.cargo/registry
|
~/.cargo/registry
|
||||||
~/.cargo/git
|
~/.cargo/git
|
||||||
target
|
target
|
||||||
- name: Build
|
- name: Build
|
||||||
run: |
|
run: cargo build --release
|
||||||
cargo build --release
|
|
||||||
```
|
```
|
||||||
|
|
||||||
### Skip Steps on Cache Hit
|
## Skip Steps on Cache Hit
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
name: Build with Conditional Steps
|
name: Build with Conditional Steps
|
||||||
@@ -67,27 +66,9 @@ jobs:
|
|||||||
with:
|
with:
|
||||||
key-prefix: 'rust'
|
key-prefix: 'rust'
|
||||||
key-file: 'Cargo.lock'
|
key-file: 'Cargo.lock'
|
||||||
- name: Install dependencies
|
- name: Fetch dependencies
|
||||||
if: steps.cache.outputs.cache-hit != 'true'
|
if: steps.cache.outputs.cache-hit != 'true'
|
||||||
run: cargo fetch
|
run: cargo fetch
|
||||||
- name: Build
|
- name: Build
|
||||||
run: cargo build --release
|
run: cargo build --release
|
||||||
```
|
```
|
||||||
|
|
||||||
## How It Works
|
|
||||||
|
|
||||||
1. **Cache Key Generation**: Combines `key-prefix` with the SHA-256 hash of `key-file` (e.g. `rust-<hash>`).
|
|
||||||
|
|
||||||
2. **Cache Restore**: Calls the Gitea artifact cache API (`/_apis/artifactcache/cache`) to find a matching entry, downloads the archive, and extracts it.
|
|
||||||
|
|
||||||
3. **Cache Save**: Runs on job completion (`if: always()`), creates a zstd-compressed tar archive, reserves a cache entry via the API, uploads it, and commits it. Skipped if there was a cache hit on the primary key.
|
|
||||||
|
|
||||||
## Cache Scope
|
|
||||||
|
|
||||||
The cache is scoped to the key, version, and branch. The default branch cache is available to other branches.
|
|
||||||
|
|
||||||
## Remarks
|
|
||||||
|
|
||||||
- The cache server must be enabled on your Gitea instance
|
|
||||||
- Requires `curl`, `tar`, and `sha256sum` on the runner (zstd is optional, falls back to gzip)
|
|
||||||
- Maximum cache size per repository is 10GB
|
|
||||||
|
|||||||
+40
-210
@@ -1,245 +1,75 @@
|
|||||||
name: 'Cache'
|
name: 'Cache'
|
||||||
description: 'Cache build files under ~/.cache/ using a key file hash'
|
description: 'Cache build files as tar.xz on the runner filesystem'
|
||||||
inputs:
|
inputs:
|
||||||
path:
|
path:
|
||||||
description: 'A list of files, directories, or wildcard patterns to cache. Defaults to ~/.cache/'
|
description: 'Directory to cache'
|
||||||
required: false
|
required: false
|
||||||
default: '~/.cache/'
|
default: '~/.cache/'
|
||||||
key-prefix:
|
key-prefix:
|
||||||
description: 'Prefix for the cache key, combined with key-file hash'
|
description: 'Prefix for the cache key'
|
||||||
required: true
|
required: true
|
||||||
key-file:
|
key-file:
|
||||||
description: 'The file to hash for generating the cache key'
|
description: 'File to hash for the cache key'
|
||||||
required: true
|
required: true
|
||||||
restore-keys:
|
|
||||||
description: 'An ordered list of prefix-matched keys for restoring stale cache'
|
|
||||||
required: false
|
|
||||||
default: ''
|
|
||||||
outputs:
|
outputs:
|
||||||
cache-hit:
|
cache-hit:
|
||||||
description: 'A boolean value to indicate an exact match was found for the key'
|
description: 'true if exact key match found, false otherwise'
|
||||||
value: ${{ steps.restore-cache.outputs.cache-hit }}
|
value: ${{ steps.restore.outputs.cache-hit }}
|
||||||
cache-primary-key:
|
|
||||||
description: 'The primary key used for cache lookup'
|
|
||||||
value: ${{ steps.restore-cache.outputs.cache-primary-key }}
|
|
||||||
cache-matched-key:
|
|
||||||
description: 'The key of the cache entry that was restored'
|
|
||||||
value: ${{ steps.restore-cache.outputs.cache-matched-key }}
|
|
||||||
runs:
|
runs:
|
||||||
using: "composite"
|
using: "composite"
|
||||||
steps:
|
steps:
|
||||||
- name: Generate cache key
|
|
||||||
id: gen-key
|
|
||||||
shell: bash
|
|
||||||
run: |
|
|
||||||
KEY_HASH=$(sha256sum "${{ inputs.key-file }}" | awk '{print $1}')
|
|
||||||
echo "cache-key=${{ inputs.key-prefix }}-${KEY_HASH}" >> $GITHUB_OUTPUT
|
|
||||||
|
|
||||||
- name: Restore cache
|
- name: Restore cache
|
||||||
id: restore-cache
|
id: restore
|
||||||
shell: bash
|
shell: bash
|
||||||
env:
|
|
||||||
CACHE_KEY: ${{ steps.gen-key.outputs.cache-key }}
|
|
||||||
RESTORE_KEYS: ${{ inputs.restore-keys }}
|
|
||||||
CACHE_PATH: ${{ inputs.path }}
|
|
||||||
run: |
|
run: |
|
||||||
CACHE_URL="${ACTIONS_CACHE_URL:-${ACTIONS_RUNTIME_URL:-}}"
|
CACHE_DIR="$HOME/.cache/.cache-store"
|
||||||
TOKEN="${ACTIONS_RUNTIME_TOKEN:-}"
|
mkdir -p "$CACHE_DIR"
|
||||||
|
|
||||||
if [ -z "$CACHE_URL" ] || [ -z "$TOKEN" ]; then
|
KEY_HASH=$(sha256sum "${{ inputs.key-file }}" | awk '{print $1}')
|
||||||
echo "::warning::Cache environment variables not set. Skipping cache restore."
|
CACHE_KEY="${{ inputs.key-prefix }}-${KEY_HASH}"
|
||||||
echo "cache-hit=false" >> $GITHUB_OUTPUT
|
ARCHIVE="${CACHE_DIR}/${CACHE_KEY}.tar.xz"
|
||||||
echo "cache-primary-key=${CACHE_KEY}" >> $GITHUB_OUTPUT
|
|
||||||
echo "cache-matched-key=" >> $GITHUB_OUTPUT
|
|
||||||
exit 0
|
|
||||||
fi
|
|
||||||
|
|
||||||
CACHE_URL="${CACHE_URL%/}/"
|
EXPANDED_PATH=$(eval echo "${{ inputs.path }}")
|
||||||
API_BASE="${CACHE_URL}_apis/artifactcache/"
|
|
||||||
|
|
||||||
EXPANDED_PATH=$(eval echo "$CACHE_PATH")
|
echo "cache-hit=false" >> $GITHUB_OUTPUT
|
||||||
PATHS_HASH=$(echo "$EXPANDED_PATH" | sha256sum | awk '{print $1}')
|
|
||||||
CACHE_VERSION="zstd-${PATHS_HASH}"
|
|
||||||
|
|
||||||
PRIMARY_KEY="${CACHE_KEY}"
|
if [ -f "$ARCHIVE" ]; then
|
||||||
|
echo "Cache hit: ${CACHE_KEY}"
|
||||||
KEYS="$PRIMARY_KEY"
|
mkdir -p "$EXPANDED_PATH"
|
||||||
if [ -n "$RESTORE_KEYS" ]; then
|
tar -xf "$ARCHIVE" -C "/" 2>/dev/null || true
|
||||||
while IFS= read -r rk; do
|
echo "cache-hit=true" >> $GITHUB_OUTPUT
|
||||||
if [ -n "$rk" ]; then
|
|
||||||
KEYS="${KEYS},${rk}"
|
|
||||||
fi
|
|
||||||
done <<< "$RESTORE_KEYS"
|
|
||||||
fi
|
|
||||||
|
|
||||||
ENCODED_KEYS=$(echo "$KEYS" | sed 's/,/%2C/g')
|
|
||||||
|
|
||||||
echo "Looking up cache with keys: ${KEYS}"
|
|
||||||
|
|
||||||
HTTP_CODE=$(mktemp)
|
|
||||||
CACHE_ENTRY=$(mktemp)
|
|
||||||
curl -sL -w "%{http_code}" -o "$CACHE_ENTRY" \
|
|
||||||
-H "Authorization: Bearer ${TOKEN}" \
|
|
||||||
"${API_BASE}cache?keys=${ENCODED_KEYS}&version=${CACHE_VERSION}" \
|
|
||||||
2>/dev/null > "$HTTP_CODE" || true
|
|
||||||
|
|
||||||
STATUS=$(cat "$HTTP_CODE")
|
|
||||||
rm -f "$HTTP_CODE"
|
|
||||||
|
|
||||||
if [ "$STATUS" = "204" ] || [ "$STATUS" = "404" ] || [ ! -s "$CACHE_ENTRY" ]; then
|
|
||||||
echo "Cache not found for keys: ${KEYS}"
|
|
||||||
rm -f "$CACHE_ENTRY"
|
|
||||||
echo "cache-hit=false" >> $GITHUB_OUTPUT
|
|
||||||
echo "cache-primary-key=${PRIMARY_KEY}" >> $GITHUB_OUTPUT
|
|
||||||
echo "cache-matched-key=" >> $GITHUB_OUTPUT
|
|
||||||
exit 0
|
|
||||||
fi
|
|
||||||
|
|
||||||
ARCHIVE_LOCATION=$(grep -o '"archiveLocation":"[^"]*"' "$CACHE_ENTRY" | head -1 | cut -d'"' -f4)
|
|
||||||
MATCHED_KEY=$(grep -o '"cacheKey":"[^"]*"' "$CACHE_ENTRY" | head -1 | cut -d'"' -f4)
|
|
||||||
rm -f "$CACHE_ENTRY"
|
|
||||||
|
|
||||||
if [ -z "$ARCHIVE_LOCATION" ]; then
|
|
||||||
echo "Cache not found (no archive location)"
|
|
||||||
echo "cache-hit=false" >> $GITHUB_OUTPUT
|
|
||||||
echo "cache-primary-key=${PRIMARY_KEY}" >> $GITHUB_OUTPUT
|
|
||||||
echo "cache-matched-key=" >> $GITHUB_OUTPUT
|
|
||||||
exit 0
|
|
||||||
fi
|
|
||||||
|
|
||||||
echo "Cache found with key: ${MATCHED_KEY}"
|
|
||||||
|
|
||||||
EXACT_HIT="false"
|
|
||||||
if [ "$PRIMARY_KEY" = "$MATCHED_KEY" ]; then
|
|
||||||
EXACT_HIT="true"
|
|
||||||
fi
|
|
||||||
|
|
||||||
TMPDIR=$(mktemp -d)
|
|
||||||
ARCHIVE_PATH="${TMPDIR}/cache.tar.zst"
|
|
||||||
|
|
||||||
echo "Downloading cache archive..."
|
|
||||||
curl -sL -o "$ARCHIVE_PATH" "$ARCHIVE_LOCATION"
|
|
||||||
|
|
||||||
echo "Extracting cache..."
|
|
||||||
mkdir -p "$EXPANDED_PATH"
|
|
||||||
|
|
||||||
if command -v zstd &>/dev/null; then
|
|
||||||
tar -I zstd -xf "$ARCHIVE_PATH" -C "/" 2>/dev/null || true
|
|
||||||
else
|
else
|
||||||
tar -xf "$ARCHIVE_PATH" -C "/" 2>/dev/null || true
|
echo "Cache miss: ${CACHE_KEY}"
|
||||||
|
# Try restore-keys fallback
|
||||||
|
for prefix in ${{ inputs.restore-keys }}; do
|
||||||
|
FALLBACK=$(ls -t "${CACHE_DIR}/${prefix}"*.tar.xz 2>/dev/null | head -1)
|
||||||
|
if [ -n "$FALLBACK" ] && [ -f "$FALLBACK" ]; then
|
||||||
|
echo "Fallback hit: $(basename "$FALLBACK")"
|
||||||
|
mkdir -p "$EXPANDED_PATH"
|
||||||
|
tar -xf "$FALLBACK" -C "/" 2>/dev/null || true
|
||||||
|
break
|
||||||
|
fi
|
||||||
|
done
|
||||||
fi
|
fi
|
||||||
|
|
||||||
rm -rf "$TMPDIR"
|
|
||||||
|
|
||||||
echo "cache-hit=${EXACT_HIT}" >> $GITHUB_OUTPUT
|
|
||||||
echo "cache-primary-key=${PRIMARY_KEY}" >> $GITHUB_OUTPUT
|
|
||||||
echo "cache-matched-key=${MATCHED_KEY}" >> $GITHUB_OUTPUT
|
|
||||||
|
|
||||||
- name: Save cache
|
- name: Save cache
|
||||||
if: always()
|
if: steps.restore.outputs.cache-hit != 'true'
|
||||||
shell: bash
|
shell: bash
|
||||||
env:
|
|
||||||
CACHE_KEY: ${{ steps.gen-key.outputs.cache-key }}
|
|
||||||
CACHE_PATH: ${{ inputs.path }}
|
|
||||||
CACHE_HIT: ${{ steps.restore-cache.outputs.cache-hit }}
|
|
||||||
run: |
|
run: |
|
||||||
CACHE_URL="${ACTIONS_CACHE_URL:-${ACTIONS_RUNTIME_URL:-}}"
|
CACHE_DIR="$HOME/.cache/.cache-store"
|
||||||
TOKEN="${ACTIONS_RUNTIME_TOKEN:-}"
|
mkdir -p "$CACHE_DIR"
|
||||||
|
|
||||||
if [ -z "$CACHE_URL" ] || [ -z "$TOKEN" ]; then
|
KEY_HASH=$(sha256sum "${{ inputs.key-file }}" | awk '{print $1}')
|
||||||
echo "::warning::Cache environment variables not set. Skipping cache save."
|
CACHE_KEY="${{ inputs.key-prefix }}-${KEY_HASH}"
|
||||||
exit 0
|
ARCHIVE="${CACHE_DIR}/${CACHE_KEY}.tar.xz"
|
||||||
fi
|
|
||||||
|
|
||||||
CACHE_URL="${CACHE_URL%/}/"
|
EXPANDED_PATH=$(eval echo "${{ inputs.path }}")
|
||||||
API_BASE="${CACHE_URL}_apis/artifactcache/"
|
|
||||||
|
|
||||||
EXPANDED_PATH=$(eval echo "$CACHE_PATH")
|
|
||||||
PATHS_HASH=$(echo "$EXPANDED_PATH" | sha256sum | awk '{print $1}')
|
|
||||||
CACHE_VERSION="zstd-${PATHS_HASH}"
|
|
||||||
|
|
||||||
PRIMARY_KEY="${CACHE_KEY}"
|
|
||||||
|
|
||||||
if [ "$CACHE_HIT" = "true" ]; then
|
|
||||||
echo "Cache hit on primary key, skipping save."
|
|
||||||
exit 0
|
|
||||||
fi
|
|
||||||
|
|
||||||
echo "Checking if cache directory has content..."
|
|
||||||
if [ ! -d "$EXPANDED_PATH" ] || [ -z "$(ls -A "$EXPANDED_PATH" 2>/dev/null)" ]; then
|
if [ ! -d "$EXPANDED_PATH" ] || [ -z "$(ls -A "$EXPANDED_PATH" 2>/dev/null)" ]; then
|
||||||
echo "Cache directory is empty, skipping save."
|
echo "Cache directory empty, skipping save"
|
||||||
exit 0
|
exit 0
|
||||||
fi
|
fi
|
||||||
|
|
||||||
TMPDIR=$(mktemp -d)
|
echo "Saving cache: ${CACHE_KEY}"
|
||||||
REL_PATH="$(echo "$EXPANDED_PATH" | sed 's|^/||')"
|
REL_PATH="$(echo "$EXPANDED_PATH" | sed 's|^/||')"
|
||||||
|
tar -cJf "$ARCHIVE" -C "/" "$REL_PATH" 2>/dev/null && echo "Cache saved" || echo "Cache save failed"
|
||||||
ARCHIVE_PATH="${TMPDIR}/cache.tar.zst"
|
|
||||||
COMPRESS_METHOD="zstd"
|
|
||||||
|
|
||||||
if command -v zstd &>/dev/null; then
|
|
||||||
tar -I zstd -cf "$ARCHIVE_PATH" -C "/" "$REL_PATH" 2>/dev/null || COMPRESS_METHOD="none"
|
|
||||||
else
|
|
||||||
COMPRESS_METHOD="none"
|
|
||||||
fi
|
|
||||||
|
|
||||||
if [ "$COMPRESS_METHOD" = "none" ]; then
|
|
||||||
echo "zstd not available, trying gzip..."
|
|
||||||
ARCHIVE_PATH="${TMPDIR}/cache.tar.gz"
|
|
||||||
tar -czf "$ARCHIVE_PATH" -C "/" "$REL_PATH" 2>/dev/null || {
|
|
||||||
echo "Failed to create archive"
|
|
||||||
rm -rf "$TMPDIR"
|
|
||||||
exit 0
|
|
||||||
}
|
|
||||||
fi
|
|
||||||
|
|
||||||
ARCHIVE_SIZE=$(stat -c%s "$ARCHIVE_PATH" 2>/dev/null || stat -f%z "$ARCHIVE_PATH" 2>/dev/null || echo "0")
|
|
||||||
|
|
||||||
echo "Reserving cache with key: ${PRIMARY_KEY} (size: ${ARCHIVE_SIZE} bytes)"
|
|
||||||
|
|
||||||
RESERVE_RESPONSE=$(mktemp)
|
|
||||||
curl -sL -o "$RESERVE_RESPONSE" -w "%{http_code}" \
|
|
||||||
-X POST \
|
|
||||||
-H "Authorization: Bearer ${TOKEN}" \
|
|
||||||
-H "Content-Type: application/json" \
|
|
||||||
-d "{\"key\":\"${PRIMARY_KEY}\",\"version\":\"${CACHE_VERSION}\",\"cacheSize\":${ARCHIVE_SIZE}}" \
|
|
||||||
"${API_BASE}caches" 2>/dev/null > /dev/null || echo "000"
|
|
||||||
|
|
||||||
CACHE_ID=$(grep -o '"cacheId":[0-9]*' "$RESERVE_RESPONSE" | head -1 | cut -d':' -f2)
|
|
||||||
rm -f "$RESERVE_RESPONSE"
|
|
||||||
|
|
||||||
if [ -z "$CACHE_ID" ]; then
|
|
||||||
echo "No cache ID returned, cache may already exist or reservation failed"
|
|
||||||
rm -rf "$TMPDIR"
|
|
||||||
exit 0
|
|
||||||
fi
|
|
||||||
|
|
||||||
echo "Uploading cache (ID: ${CACHE_ID})..."
|
|
||||||
UPLOAD_STATUS=$(curl -sL -o /dev/null -w "%{http_code}" \
|
|
||||||
-X PATCH \
|
|
||||||
-H "Authorization: Bearer ${TOKEN}" \
|
|
||||||
-H "Content-Type: application/octet-stream" \
|
|
||||||
--data-binary "@${ARCHIVE_PATH}" \
|
|
||||||
"${API_BASE}caches/${CACHE_ID}" 2>/dev/null || echo "000")
|
|
||||||
|
|
||||||
if [ "$UPLOAD_STATUS" != "200" ]; then
|
|
||||||
echo "Upload failed with status: ${UPLOAD_STATUS}"
|
|
||||||
rm -rf "$TMPDIR"
|
|
||||||
exit 0
|
|
||||||
fi
|
|
||||||
|
|
||||||
echo "Committing cache..."
|
|
||||||
COMMIT_STATUS=$(curl -sL -o /dev/null -w "%{http_code}" \
|
|
||||||
-X POST \
|
|
||||||
-H "Authorization: Bearer ${TOKEN}" \
|
|
||||||
-H "Content-Type: application/json" \
|
|
||||||
-d "{\"size\":${ARCHIVE_SIZE}}" \
|
|
||||||
"${API_BASE}caches/${CACHE_ID}" 2>/dev/null || echo "000")
|
|
||||||
|
|
||||||
rm -rf "$TMPDIR"
|
|
||||||
|
|
||||||
if [ "$COMMIT_STATUS" = "200" ]; then
|
|
||||||
echo "Cache saved successfully"
|
|
||||||
else
|
|
||||||
echo "Commit failed with status: ${COMMIT_STATUS}"
|
|
||||||
fi
|
|
||||||
|
|||||||
Reference in New Issue
Block a user