Skip to content

CI Healer: implementation backlog ​

Status key: Done (implemented and tested here), Done, unverified live (implemented and tested with fakes; a live check is still needed), Blocked (needs something this environment lacks), Open (not started).

Gaps from the original review ​

IDGapStatusWhat was doneRemaining
G-1State was a JSON fileDoneSQLite ledger (persistence/ledger-store.ts), schema v1 and v2, WAL, transactionsThe Ruby reference still uses JSON, by design
G-2Standalone Ruby tool, not in src/domains/DoneTypeScript port in src/domains/ci-healer/, command nexum ci-healerRuby kept as reference
G-3Setup and verify commands ran on the hostDone, unverified liveDocker sandbox with the same hardening as nexum's ShellTool; verify has no network; preflight fails closed; host mode needs an explicit acknowledgementReal-daemon test is gated (CI_HEALER_DOCKER_TESTS=1) and was not run here: no Docker daemon. Image: the default nexum-sandbox:latest is Node-only (see B-05)
G-4Agent inherited GH_TOKEN and other credentialsDoneAllowlisted environment for the agent and repo commands; GitHub, SSH, and GPG names are always deniedFile-based credentials under HOME are not covered (see B-06)
G-5Run discovery was not paginated and kept no cursorDone, unverified live, replaced designDiscovery scans branch heads instead of a run window, so no cutoff can hide a current failure. A persisted cursor rotates through branches beyond max_branches_per_tick. The REST runs endpoint, filtered by branch, returns runs for one branch. per_page bounds each pageThis replaces the proposal's "pagination plus run cursor". The change is a judgement call: review it before merging
G-6Only nexum rpc was wiredDone, unverified liveOrdered runtime chain (nexum, opencode, agy) with circuit breakers and rate-limit cooldowns, stored in SQLite. Free-tier routing is ordered (runtime, model) entriesopencode and agy are not installed here. Their argument vectors come from the documentation (see B-03 and B-04)
G-7Stale-run check on pull_request runs unverifiedDone, verified against the real repoDiscovery compares run SHAs to the branch head. A mismatch logs the reported SHAs, and plan shows the resultResolved by B-01: GitHub reports the PR head SHA

Items from the proposal's "not included" list ​

IDItemStatusNotes
N-1CI Healer design documentDonedocs/ci-healer/DESIGN.md
N-2Implementation backlogDoneThis file
N-3Docker sandboxingDone, unverified liveSee G-3
N-4SQLite migrationDoneVersioned migrations; upgrade path tested
N-5Credential scrubbingDoneSee G-4

Live verification (remaining) ​

IDCheckBlockerProcedure
B-01headSha semantics for pull_request runsResolvedplan against shubhamtaywade82/nexum (REST API) found the PR head SHA reported for pull_request runs, and no mismatch was logged. The no runs report the branch head log would mean GitHub changed this
B-02Real-daemon sandbox behaviourNo Docker daemondocker build -t nexum-sandbox:latest docker/nexum-sandbox/, then CI_HEALER_DOCKER_TESTS=1 npx jest tests/domains/ci-healer/docker-sandbox.gated.test.ts
B-03opencode headless flagsBinary not installedRun opencode run --help on the installed version. Compare with openCodeAgent in adapters/cli-agents.ts. Check whether --format json and --model provider/model still match
B-04agy headless flags and authBinary not installedRun agy --help. The documented print mode is agy -p <prompt> --output-format json. Sign in once interactively first; a headless run without cached credentials fails
B-05Language toolchains in the sandbox imageImage is Node-onlyAdd toolchains to docker/nexum-sandbox/Dockerfile only when a target repo needs them, or set sandbox.image to a repo-specific image
B-06File-based credentials under HOMENot in scope of the env policyFor Docker mode, verify commands see no host HOME, so this is covered. For host mode, it is not covered and needs the acknowledgement. For agy, the cached sign-in is reachable by design
B-07Ollama condenserNo Ollama reachableRun with condenser.type = ollama against a running Ollama; confirm the fallback when it is stopped
B-08Merge-SHA mappingNot neededB-01 found that GitHub reports the PR head SHA, so no mapping is required. Reopen only if GitHub starts reporting merge SHAs

Known limitations (accepted, documented) ​

IDLimitationWhy acceptedMitigation
L-1The worktree's .git file points at the host repository, so git inside the container cannot resolve itMounting the host .git would expose hooks and config to repo codeVerify commands that need git history must run in host mode; most test commands do not need it
L-2Setup has network access in Docker modeDependency installation needs itSet setup_network to none for vendored dependencies
L-3Runtime CLIs' JSON output is not parsedTheir schemas are not stable contractsExit status plus the verify gate decide
L-4One failing run per head is digestedOne lineage per head keeps attempts boundedThe digest is the newest failing run; other failing workflows are not merged into it
L-5Windows and macOS paths are untestedThe build environment is LinuxDocker bind mounts with commas are rejected explicitly

Open items ​

IDItemPhase
O-1Failure classification (flaky, infrastructure, dependency, code) before repair2
O-2Webhook-triggered ticks in place of polling4
O-3Multi-repository config and per-repository execution policy5
O-4Repair history and runtime metrics (latency, tokens, cost)5
O-5Notifications and human-approval controls5
O-6Interactive dashboard5

Acceptance criteria ​

The criteria are in DESIGN.md section 11. Each gate that protects a push is covered by a test that fails when the gate is removed.

Released under the MIT License.