Touch archives on hit and prune the cache store after restore
Record last use by touching the archive so the retention policy can expire by access time, and run prune.sh after every restore (even on miss) to keep the shared store trimmed: keep the 3 newest archives per key-prefix family, expire anything older than 7 days, hard-cap the store at 2 GB (oldest first, trumping the keep set), never touch files younger than 1h, and drop stale run workspaces under ~/.cache/act older than 2 days.
This commit is contained in:
@@ -22,6 +22,26 @@ A Gitea Action that caches build files as `tar.xz` archives on the runner filesy
|
||||
1. Hashes `key-file` with SHA-256, combines with `key-prefix` as `<prefix>-<hash>`
|
||||
2. Stores archives at `~/.cache/.cache-store/<key>.tar.xz`
|
||||
3. Restores by extracting the archive to `/`, saves by creating one
|
||||
4. Touches the archive on every hit, so the retention policy can expire by
|
||||
last use
|
||||
|
||||
## Cache Retention
|
||||
|
||||
Archives are content-addressed, so old keys are never needed again — the store
|
||||
is pruned automatically on every restore/save (best-effort, safe under
|
||||
concurrent runners):
|
||||
|
||||
- The newest `CACHE_KEEP_N` (default `3`) archives per key-prefix family are
|
||||
always kept, so reverting e.g. `Cargo.lock` still hits an older key
|
||||
- Anything older than `CACHE_TTL_DAYS` (default `7`) outside the keep set is
|
||||
deleted
|
||||
- A hard size cap `CACHE_CAP_GB` (default `2`) deletes oldest-first when
|
||||
exceeded
|
||||
- Files younger than 1 hour are never touched (in-flight save protection)
|
||||
- Stale run workspaces under `~/.cache/act/*/hostexecutor` older than
|
||||
`CACHE_WORKSPACE_TTL_DAYS` (default `2`) are removed
|
||||
|
||||
Override the defaults per-job via `env:` (e.g. `env: { CACHE_KEEP_N: 5 }`).
|
||||
|
||||
## Usage
|
||||
|
||||
|
||||
+11
-2
@@ -40,9 +40,18 @@ runs:
|
||||
if [ -f "$ARCHIVE" ]; then
|
||||
echo "Cache hit: ${CACHE_KEY}"
|
||||
mkdir -p "$EXPANDED_PATH"
|
||||
tar -xf "$ARCHIVE" -C "/" 2>/dev/null || true
|
||||
echo "cache-hit=true" >> $GITHUB_OUTPUT
|
||||
if tar -xf "$ARCHIVE" -C "/" 2>/dev/null; then
|
||||
# record last use so the retention policy can expire by access time
|
||||
touch "$ARCHIVE"
|
||||
echo "cache-hit=true" >> $GITHUB_OUTPUT
|
||||
else
|
||||
echo "Cache restore failed"
|
||||
echo "cache-hit=false" >> $GITHUB_OUTPUT
|
||||
fi
|
||||
else
|
||||
echo "Cache miss: ${CACHE_KEY}"
|
||||
echo "cache-hit=false" >> $GITHUB_OUTPUT
|
||||
fi
|
||||
|
||||
# Trim the shared cache store and stale workspaces (best-effort)
|
||||
bash "${{ github.action_path }}/prune.sh" || true
|
||||
|
||||
@@ -0,0 +1,90 @@
|
||||
#!/usr/bin/env bash
|
||||
# Prune the shared runner build-cache store (~/.cache/.cache-store) and stale
|
||||
# run workspaces (~/.cache/act). Best-effort: safe to run from any job on any
|
||||
# runner, since all runners on a host share the same store.
|
||||
#
|
||||
# Policy (per family, where a family is the key-prefix, e.g. `leptos-target`):
|
||||
# - the newest CACHE_KEEP_N archives per family are always kept (rollback
|
||||
# safety: reverting Cargo.lock must still hit an older key)
|
||||
# - any archive older than CACHE_TTL_DAYS outside the keep set is deleted
|
||||
# - a hard CACHE_CAP_GB size cap deletes oldest-first when exceeded
|
||||
# - files younger than 1 hour are never touched (in-flight save protection;
|
||||
# saves are atomic, see the save action: write .tmp then mv)
|
||||
#
|
||||
# Overrides via env: CACHE_KEEP_N (default 3), CACHE_TTL_DAYS (default 7),
|
||||
# CACHE_CAP_GB (default 2), CACHE_WORKSPACE_TTL_DAYS
|
||||
# (default 2)
|
||||
|
||||
set -u
|
||||
|
||||
CACHE_DIR="$HOME/.cache/.cache-store"
|
||||
KEEP_N="${CACHE_KEEP_N:-3}"
|
||||
TTL_SECS=$(( ${CACHE_TTL_DAYS:-7} * 86400 ))
|
||||
CAP_BYTES=$(( ${CACHE_CAP_GB:-2} * 1024 * 1024 * 1024 ))
|
||||
SAFETY_SECS=3600
|
||||
WORKSPACE_TTL_DAYS="${CACHE_WORKSPACE_TTL_DAYS:-2}"
|
||||
|
||||
[ -d "$CACHE_DIR" ] || exit 0
|
||||
NOW=$(date +%s)
|
||||
|
||||
# stale tmp files left by interrupted saves
|
||||
find "$CACHE_DIR" -maxdepth 1 -name '*.tar.xz.tmp' -mmin +60 -delete 2>/dev/null
|
||||
|
||||
# newest KEEP_N archives per family are protected from deletion
|
||||
declare -A protected=()
|
||||
declare -A fam_count=()
|
||||
while IFS=$'\t' read -r mt name; do
|
||||
fam=$(sed -E 's/-[0-9a-f]{64}\.tar\.xz$//' <<< "$name")
|
||||
if [ -n "$fam" ] && [ "${fam_count[$fam]:-0}" -lt "$KEEP_N" ]; then
|
||||
protected[$name]=1
|
||||
fam_count[$fam]=$(( ${fam_count[$fam]:-0} + 1 ))
|
||||
fi
|
||||
done < <(find "$CACHE_DIR" -maxdepth 1 -name '*.tar.xz' -printf '%T@\t%f\n' | sort -rn)
|
||||
|
||||
# deletion candidates: not protected, older than 1h, oldest first
|
||||
declare -a candidates=()
|
||||
while IFS=$'\t' read -r mt name; do
|
||||
[ "${protected[$name]:-0}" = "1" ] && continue
|
||||
mtime=$(printf '%.0f' "$mt")
|
||||
[ $(( NOW - mtime )) -lt "$SAFETY_SECS" ] && continue
|
||||
candidates+=("$mtime|$name")
|
||||
done < <(find "$CACHE_DIR" -maxdepth 1 -name '*.tar.xz' -printf '%T@\t%f\n' | sort -n)
|
||||
|
||||
# TTL pass: drop archives past their retention window
|
||||
deleted=0
|
||||
for c in "${candidates[@]:-}"; do
|
||||
[ -n "$c" ] || continue
|
||||
mtime="${c%%|*}"; name="${c#*|}"
|
||||
if [ $(( NOW - mtime )) -gt "$TTL_SECS" ]; then
|
||||
rm -f "$CACHE_DIR/$name" && deleted=$((deleted + 1))
|
||||
fi
|
||||
done
|
||||
[ "$deleted" -gt 0 ] && echo "prune: deleted $deleted archive(s) older than ${CACHE_TTL_DAYS:-7} day(s)"
|
||||
|
||||
# size-cap pass: keep the store under CAP_BYTES, oldest first. The cap is a
|
||||
# hard backstop and overrides keep-N protection (only the 1h safety floor is
|
||||
# absolute), so the size bound always holds even for small families.
|
||||
total=$(du -sb "$CACHE_DIR" 2>/dev/null | awk '{print $1}')
|
||||
if [ "${total:-0}" -gt "$CAP_BYTES" ]; then
|
||||
while IFS=$'\t' read -r mt name; do
|
||||
[ "${total:-0}" -le "$CAP_BYTES" ] && break
|
||||
mtime=$(printf '%.0f' "$mt")
|
||||
[ $(( NOW - mtime )) -lt "$SAFETY_SECS" ] && continue
|
||||
if [ -f "$CACHE_DIR/$name" ]; then
|
||||
size=$(stat -c %s "$CACHE_DIR/$name")
|
||||
rm -f "$CACHE_DIR/$name" && total=$(( total - size ))
|
||||
fi
|
||||
done < <(find "$CACHE_DIR" -maxdepth 1 -name '*.tar.xz' -printf '%T@\t%f\n' | sort -n)
|
||||
echo "prune: store size now $(du -sh "$CACHE_DIR" 2>/dev/null | cut -f1) (cap ${CACHE_CAP_GB:-2}G)"
|
||||
fi
|
||||
|
||||
# stale run workspaces: every run gets a fresh ~/.cache/act/<id>/hostexecutor
|
||||
# dir that is never reused, so old ones are dead weight (they can hold entire
|
||||
# target/ dirs); nothing younger than the floor is touched
|
||||
ws_deleted=0
|
||||
while IFS= read -r ws; do
|
||||
[ -n "$ws" ] || continue
|
||||
rm -rf "$ws" && ws_deleted=$((ws_deleted + 1))
|
||||
done < <(find "$HOME/.cache/act" -maxdepth 2 -type d -name hostexecutor -mtime +"$WORKSPACE_TTL_DAYS" 2>/dev/null)
|
||||
find "$HOME/.cache/act" -maxdepth 1 -type d -empty -delete 2>/dev/null || true
|
||||
[ "$ws_deleted" -gt 0 ] && echo "prune: removed $ws_deleted stale workspace(s) older than ${WORKSPACE_TTL_DAYS} day(s)"
|
||||
Reference in New Issue
Block a user