Test Audit by @garrytan · Skillet

Test Audit

Add skill

Find low-value or duplicate tests and the test-only code they keep alive. (gstack)

Permissions

Files

SKILL.md

test-audit

Find low-value or duplicate tests and the test-only code they keep alive. (gstack)

preamble-tier: 2
version: 1.0.0
triggers: audit the test suite, find low-value tests, prune useless tests
allowed-tools: Bash, Read, Write, Edit, Glob, Grep, AskUserQuestion

When to invoke this skill

Report-only unless you approve a batch. Use for /test-audit.

Preamble (run first)

_SS=
[ -x  ] || _SS=
 --skill  --model  --parent-pid  \
  ||

2 files · 46 KB39 KB

Version history

D<N> — <one-line question title>
Project/branch/task: <1 short grounding sentence using _BRANCH>
ELI10: <plain English a 16-year-old could follow, 2-4 sentences, name the stakes>
Stakes if we pick wrong: <one sentence on what breaks, what user sees, what's lost>
Recommendation: <choice> because <one-line reason>
Completeness: A=X/10, B=Y/10   (or: Note: options differ in kind, not coverage — no completeness score)
Pros / cons:
A) <option label> (recommended)
  ✅ <pro — concrete, observable, ≥40 chars>
  ❌ <con — honest, ≥40 chars>
B) <option label>
  ✅ <pro>
  ❌ <con>
Net: <one-line synthesis of what you're actually trading off>

eval "$(~/.claude/skills/gstack/bin/gstack-slug 2>/dev/null)"
_BRANCH=$(git branch --show-current 2>/dev/null | tr -cd 'a-zA-Z0-9._/-') || :; _BRANCH=${_BRANCH:-unknown}
_PROJ="${GSTACK_HOME:-$HOME/.gstack}/projects/${SLUG:-unknown}"
if [ -d "$_PROJ" ]; then
  echo "--- RECENT ARTIFACTS ---"
  find "$_PROJ/ceo-plans" "$_PROJ/checkpoints" -type f -name "*.md" 2>/dev/null | xargs -r ls -t 2>/dev/null | head -3
  [ -f "$_PROJ/${BRANCH:-unknown}-reviews.jsonl" ] && echo "REVIEWS: $(wc -l < "$_PROJ/${BRANCH:-unknown}-reviews.jsonl" | tr -d ' ') entries"
  [ -f "$_PROJ/timeline.jsonl" ] && tail -5 "$_PROJ/timeline.jsonl"
  if [ -f "$_PROJ/timeline.jsonl" ]; then
    _LAST=$(grep "\"branch\":\"${_BRANCH}\"" "$_PROJ/timeline.jsonl" 2>/dev/null | grep '"event":"completed"' | tail -1)
    [ -n "$_LAST" ] && echo "LAST_SESSION: $_LAST"
    _RECENT_SKILLS=$(grep "\"branch\":\"${_BRANCH}\"" "$_PROJ/timeline.jsonl" 2>/dev/null | grep '"event":"completed"' | tail -3 | grep -o '"skill":"[^"]*"' | sed 's/"skill":"//;s/"//' | tr '\n' ',')
    [ -n "$_RECENT_SKILLS" ] && echo "RECENT_PATTERN: $_RECENT_SKILLS"
  fi
  _LATEST_CP=$(find "$_PROJ/checkpoints" -name "*.md" -type f 2>/dev/null | xargs -r ls -t 2>/dev/null | head -1)
  [ -n "$_LATEST_CP" ] && echo "LATEST_CHECKPOINT: $_LATEST_CP"
  if [ -f "$_PROJ/decisions.active.json" ]; then
    echo "--- ACTIVE DECISIONS (recent, scope-relevant) ---"
    ~/.claude/skills/gstack/bin/gstack-decision-search --recent 5 2>/dev/null
    echo "--- END DECISIONS ---"
  fi
  echo "--- END ARTIFACTS ---"
fi

~/.claude/skills/gstack/bin/gstack-question-log '{"skill":"test-audit","question_id":"<id>","question_summary":"<short>","category":"<approval|clarification|routing|cherry-pick|feedback-loop>","door_type":"<one-way|two-way>","options_count":N,"user_choice":"<key>","recommended":"<key>","session_id":"SESSION_ID"}' 2>/dev/null || true

~/.claude/skills/gstack/bin/gstack-question-preference --write '{"question_id":"<id>","preference":"<pref>","source":"inline-user","free_text":"<optional original words>"}'

~/.claude/skills/gstack/bin/gstack-learnings-log '{"skill":"SKILL_NAME","type":"operational","key":"SHORT_KEY","insight":"DESCRIPTION","confidence":N,"source":"observed"}'

~/.claude/skills/gstack/bin/gstack-skill-end --skill "test-audit" --outcome OUTCOME \
  --session-id "SESSION_ID" --tel-start "TEL_START" --used-browse USED_BROWSE \
  --error-message "ERROR_MESSAGE" --failed-step "FAILED_STEP" 2>/dev/null || true

setopt +o nomatch 2>/dev/null || true  # zsh compat
eval "$(~/.claude/skills/gstack/bin/gstack-slug 2>/dev/null)" && mkdir -p ~/.gstack/projects/$SLUG
DATETIME=$(date +%Y%m%d-%H%M%S)
REPORT=~/.gstack/projects/$SLUG/test-audit-$DATETIME.md
DEFAULT_BRANCH=$(git symbolic-ref --short refs/remotes/origin/HEAD 2>/dev/null | sed 's|^origin/||')
echo "REPORT: $REPORT"
echo "DEFAULT_BRANCH: ${DEFAULT_BRANCH:-unknown}"
git ls-files | grep -cE '(^|/)(tests?|spec|__tests__)/|(^|/)test_[^/]+\.py$|_test\.(go|py|rb|ts|js|exs)$|\.(test|spec)\.[jt]sx?$|_spec\.rb$|Test\.(java|kt)$' | sed 's/^/TESTFILES:/'
ls -t ~/.gstack/projects/$SLUG/*-"$BRANCH"-eng-review-test-plan-*.md 2>/dev/null | head -1 | sed 's/^/SEED_PLAN:/'

FILES=$(git ls-files -- <scope> | grep -E '(^|/)(tests?|spec|__tests__)/|(^|/)test_[^/]+\.py$|_test\.(go|py|rb|ts|js|exs)$|\.(test|spec)\.[jt]sx?$|_spec\.rb$')
[ -n "$FILES" ] || { echo "NO_TEST_FILES"; exit 0; }
echo "$FILES" | xargs grep -L -E 'expect|assert|should|t\.(Error|Fatal|Fail)|refute|must' 2>/dev/null | sed 's/^/NO_ASSERTION:/'
echo "$FILES" | xargs grep -l -E 'readFileSync\([^)]*\.(ts|js|py|rb|go|tmpl)|toContain\(.(import|export|function) ' 2>/dev/null | sed 's/^/SOURCE_GREP:/'
echo "$FILES" | xargs grep -l -E 'Object\.keys\(|export list|exports\)\.toEqual' 2>/dev/null | sed 's/^/EXPORT_LIST:/'
echo "$FILES" | xargs grep -l -F 'gstack:test-value keep' 2>/dev/null | sed 's/^/SUPPRESSED:/'
for f in $FILES; do printf '%s %s\n' "$(tr -d '[:space:]' < "$f" | cksum | cut -d' ' -f1)" "$f"; done | sort | awk '$1==p{print "NEAR_DUPLICATE:" pf " " $2} {p=$1; pf=$2}'

{"candidates":[{"test":"...","retirement_card":{"test":"...","detects":"...","non_test_callers":"...","search_command":"...","stronger_proof":"...","history":"...","unlocks":"...","validation":"..."},"owner_boundary":"...","verdict":"retire|rewrite|extend|retain"}],"retained":[{"test":"...","contract":"..."}],"suppressed":[{"test":"...","reason":"..."}],"loc_delta":{"production":0,"test":0}}

Test Audit

When to invoke this skill

Preamble (run first)

Plan Mode Safe Operations

Skill Invocation During Plan Mode

AskUserQuestion Format

Tool resolution (read first)

When AskUserQuestion is unavailable or a call fails

Format

Handling 5+ options — split, never drop

Self-check before emitting

Artifacts Sync (skill start)

Model-Specific Behavioral Patch (claude)

Voice

Context Recovery

Writing Style (skip entirely if `EXPLAIN_LEVEL: terse` appears in the preamble echo OR the user's current message explicitly requests terse / no-explanations output)

Completeness Principle — Boil the Ocean

Confusion Protocol

Claimed Limitations Need Evidence

Context Health (soft directive)

Question Tuning (skip entirely if `QUESTION_TUNING: false`)

Completion Status Protocol

Operational Self-Improvement

Telemetry (run last)

Plan Status Footer

/test-audit: Test value sweep

Boundaries

Step 1: Scope and seeds

Step 2: Mechanical pre-filter

Step 3: Evidence

Step 4: Report and sidecar

Step 5: One question per batch

Step 6: Apply an approved batch

Handoff

When to invoke this skill

Preamble (run first)

Plan Mode Safe Operations

Skill Invocation During Plan Mode

AskUserQuestion Format

Tool resolution (read first)

When AskUserQuestion is unavailable or a call fails

Format

Handling 5+ options — split, never drop

Self-check before emitting

Artifacts Sync (skill start)

Model-Specific Behavioral Patch (claude)

Voice

Context Recovery

Writing Style (skip entirely if EXPLAIN_LEVEL: terse appears in the preamble echo OR the user's current message explicitly requests terse / no-explanations output)

Completeness Principle — Boil the Ocean

Confusion Protocol

Claimed Limitations Need Evidence

Context Health (soft directive)

Question Tuning (skip entirely if QUESTION_TUNING: false)

Completion Status Protocol

Operational Self-Improvement

Telemetry (run last)

Plan Status Footer

/test-audit: Test value sweep

Boundaries

Step 1: Scope and seeds

Step 2: Mechanical pre-filter

Step 3: Evidence

Step 4: Report and sidecar

Step 5: One question per batch

Step 6: Apply an approved batch

Handoff

Writing Style (skip entirely if `EXPLAIN_LEVEL: terse` appears in the preamble echo OR the user's current message explicitly requests terse / no-explanations output)

Question Tuning (skip entirely if `QUESTION_TUNING: false`)