Drive Hermetic Build Audit from your own code
The page reads one pasted Bazel workspace configuration in the browser for free — one or more files
joined into one paste, each introduced by a # file: <path> line: .bazelversion,
.bazelrc and other rc files, MODULE.bazel, WORKSPACE, the BUILD files that
matter and the .bzl files they load — with its line numbers. It lexes Starlark strings and
comments out first, reads every top-level call (rules, macros, load, package,
module, bazel_dep, overrides, repository rules) with its attributes, reads every rc
line with its command, config and flags (quotes and \ continuations included) and follows
--config chains, so a flag that reaches every build through a config counts. Then it runs
21 deterministic checks drawn from Bazel build practice (catalogue ids BZ-01..BZ-21, plus BZ-22 for context):
whether two machines given the same commit build the same bytes (the Bazel version, checksums, commits,
absolute paths, network or the clock inside actions, the sandbox) and share cache keys (the action
environment, who writes the remote cache, plaintext endpoints, stamping, test caching, targets that switch
caching off), whether the graph is sound (glob() in deps, root targets over the whole tree,
public-by-default packages), and whether the rc files do what they look like (undefined configs,
credentials, WORKSPACE on Bazel 8, --jobs, timeouts, experimental flags). Nothing is fetched,
analysed or built: every flag is a reading of the text. Its two paid steps are available over HTTP. You
send one JSON object with task set to "audit" or "fix", the paste as
code and the page’s check as a facts string, and the model replies with one
JSON object: an audit that rates hermeticity, caching, the build graph and the configuration, answers every
flag and gives exact before/after edits per finding, or the whole fixed file(s) with a list of every change
tied to the page’s flags.
Base URL: https://api.skillsafe.ai/v1/app-api. Every call sends
Content-Type: application/json and, once you have a token,
Authorization: Bearer <token>. The paths below (/guest,
/me, /estimate, /run, /jobs/{job_id},
/run-stream) are relative to it, exactly as sdk.js calls them. A
plain-text summary of the app for agents is at /llms.txt. Both lanes run on
the gpt-terra model with a 10% publisher markup. A run needs a signed-in (user) token and
is metered: the platform places a hold on your credits when the run starts and then
charges only what the run used. /estimate is free.
The task field: two lanes
task is the first field of every body. It routes the run, and it takes exactly two values:
"audit" and "fix". Each lane has its own reply contract, and a reply never
mixes them. The skill’s instructions tell the model to do the audit lane when task is
missing or anything else (and the page’s own Bazelkit.buildInput turns any lane that is
not "fix" into "audit"), so a typo silently gets you an audit: always send one
of the two values.
task | What it does | verdict | Use it when |
|---|---|---|---|
"audit" | Reviews the configuration the way a senior build engineer reviews a Bazel workspace before tuning CI: says what the workspace builds and how its dependencies and cache are set up (workspace), rates the four areas (hermetic, cache, graph, config) as sound, fragile or broken, confirms or dismisses every page flag, adds what text rules cannot see, and gives the verbatim evidence and exact before/after edits for each finding, listed in the order to apply them — what fails or leaks first (undefined configs, credentials, unpinned downloads, branch pins, absolute paths, network in actions), then what splits or poisons the cache, then what wastes time, then hygiene — with a plan that ends in two clean CI builds and a read of the second one’s cache hits. | blocked | needs_work | ready | Before you tune CI or turn on a shared cache: does the build fail, drift between machines or leak a key, and what should change first. |
"fix" | Writes the fixed configuration: the whole of every pasted file it changes (never a new file), with every target, repository, bazel_dep and named config kept — plus a changes list tied to the page flags (each naming the target label it is in, or (file)), the verify steps (a clean build and test with the CI config, then the check that would have caught the worst problem), and what it left open for a fact or a decision only the engineer has (a checksum, a commit, an endpoint, whether a tag is still needed). | fixed | partial | When you want the files changed, not a list of findings: a configuration you already know is broken, or what is left after an audit. |
The model never invents a checksum, a commit, a registry version, an endpoint or a label. When the note or
the paste does not give one, the audit leaves the edit out and asks in questions, and the fix
lane leaves the item open. It never copies a credential it finds in the paste.
The page chains the two: after an audit, “Apply every edit and fix the rest” puts
every audit edit into the paste, switches to the fix lane on that edited paste and adds an
audit_handoff string (below) with the audit’s verdict, headline and findings. Over
the API that is a second run with task: "fix", the edited paste as code,
facts recomputed from the edited paste, and that string. The page drops the handoff as
soon as the paste is edited again.
Guards the page applies before a run and the API does not: it refuses to price or run a paste it cannot
read (“Nothing was pasted.”, “No Bazel file was found. Paste .bazelrc, MODULE.bazel or
WORKSPACE, .bazelversion and BUILD files, with a # file: <path> line above each.”, or “This
does not read as Bazel: no rc line, no rule or macro call, no load() and no .bazelversion was found.”),
and it warns before a run on a paste with no .bazelrc (“No .bazelrc was pasted: the
cache, sandbox and config checks need it.”).
One worked example per lane follows, both examples the page ships (the example buttons on the
app page; the full pastes are in /example.js, ids
monorepo and vendor), built exactly as the page builds them:
Bazelkit.buildInput(lane, Bazelkit.analyze(text), note, null), with the saved model reply for
each. The other two examples are audits that reach the other verdicts: legacy
(needs_work, 8 flags (5 medium, 3 low)) and ready (ready, 1 flag (1 low)); the
monorepo audit also carries a saved fix run of its edited files (fixed). The request bodies cut the
long code and facts strings with a visible ...; the replies are shown
whole, pretty-printed, exactly as saved. The companies, hosts, keys and paths are invented.
Worked example, task: "audit": a Python monorepo whose CI config names a config that does not exist, with an API key in .bazelrc, the sandbox off, PATH leaking into cache keys and stamping on every build
5 files: .bazelversion, .bazelrc, MODULE.bazel, services/ingest/BUILD.bazel, libs/textkit/BUILD.bazel. The page’s free check raises 8 flags (2 high, 4 medium, 2 low) (plus the BZ-22 context flag);
its own reading of the verdict is blocked. Running build-check.js on it
(below) prints 5 file(s), 5 target(s), 3 repositor(ies), 14 rc line(s), 2 config(s); 8 flags (2 high, 4 medium, 2 low); verdict_candidate blocked. The engineer’s note:
“Laptops and CI almost never share cache hits. CI runs bazel build //... --config=ci and must keep writing the remote cache; laptops should only read it. --spawn_strategy=local was added months ago while debugging one flaky test and nobody remembers why. The --stamp timestamp is only needed for release builds. What do we fix first?”
The request body (POST /run or /run-stream), idempotency key hermetic-build-audit:audit:1e712138-26f6:a1:
{
"task": "audit",
"code": "# file: .bazelversion\n7.4.1\n# file: .bazelrc\ncommon --enable_bzlmod\n\n# Everyone\nbuild --enable_platform_specific_config\nbuild --action_env=PATH\nbuild --spawn_strategy=local\nbuild --stamp\nbuild --workspace_status_command=tools/status.sh\nbuild --experimental_rem...",
"facts": "{\"workspace\":{\"bazel_version\":\"7.4.1\",\"bazelversion_file\":\"7.4.1\",\"module\":{\"name\":\"quillpath\",\"version\":\"0.1.0\"},\"bzlmod\":true,\"workspace_deps\":false,\"files\":5,\"targets\":5,\"repositories\":0,\"bazel_deps\":3,\"rc_lines\":14,\"...",
"note": "Laptops and CI almost never share cache hits. CI runs bazel build //... --config=ci and must keep writing the remote cache; laptops should only read it. --spawn_strategy=local was added months ago while debugging one flaky test and nobody remembers why. The --stamp timestamp is only needed for release builds. What do we fix first?"
}
The saved reply (output.output), verdict blocked, 8 findings, 6 edits:
{
"task": "audit",
"verdict": "blocked",
"headline": "Two high findings - an undefined --config and a committed cache credential - block every CI build outright, and several medium cache-key issues explain why laptops and CI still don't share hits.",
"workspace": "The quillpath workspace builds two Bazel-native Python packages (services/ingest and libs/textkit) via Bzlmod, with three pinned bazel_dep registry versions and no WORKSPACE dependencies, and reads/writes a grpcs remote cache split between an always-on build:remote_cache config and a build:ci override.",
"areas": [
{
"area": "hermetic",
"status": "fragile",
"note": "The version is pinned and every declared dependency uses a registry version, but the global spawn strategy override lets actions build outside the sandbox."
},
{
"area": "cache",
"status": "fragile",
"note": "A leaking PATH in the action environment, no global stop on non-CI uploads, and stamping on every build together explain why laptops and CI rarely share a hit."
},
{
"area": "graph",
"status": "sound",
"note": "Both packages use fine-grained, labelled targets with deliberate visibility; nothing here globs deps or grants blanket public access."
},
{
"area": "config",
"status": "broken",
"note": "build:ci points at an undefined --config and the shared remote cache config carries a live credential in the checked-in file."
}
],
"findings": [
{
"id": "BZ-16",
"flags": [
"F2"
],
"file": ".bazelrc",
"line": 20,
"severity": "high",
"area": "config",
"title": "build:ci chains to an undefined --config",
"evidence": "build:ci --config=remote-cache",
"why": "Any command that reaches this line - including CI's own bazel build //... --config=ci - fails with \"Config value 'remote-cache' is not defined in any .rc file\"; remote_cache (underscore) is defined four lines above and is almost certainly the intended target.",
"edits": [
{
"before": "build:ci --config=remote-cache",
"after": "build:ci --config=remote_cache"
}
]
},
{
"id": "BZ-18",
"flags": [
"F1"
],
"file": ".bazelrc",
"line": 15,
"severity": "high",
"area": "config",
"title": "Remote cache header carries a live credential",
"evidence": "build:remote_cache --remote_cache=grpcs://cache.quillpath.example:443",
"why": "The line just below this one sets --remote_header to a literal x-api-key value (not quoted here); anyone who can read the repository can call the cache service as this workspace. Rotate that value and move it out of the checked-in file - a CI secret passed on the command line, a git-ignored user.bazelrc read via try-import, or --credential_helper.",
"edits": []
},
{
"id": "BZ-07",
"flags": [
"F3",
"F6"
],
"file": ".bazelrc",
"line": 8,
"severity": "medium",
"area": "cache",
"title": "Action environment leaks the client's PATH into cache keys",
"evidence": "build --action_env=PATH",
"why": "A bare --action_env=PATH copies whichever PATH the person or machine running Bazel has into every action's cache key, so a laptop and CI compute different keys for the same action and never share a hit; --incompatible_strict_action_env fixes PATH and every other inherited variable at once, which is what a remote cache needs to be worth having.",
"edits": [
{
"before": "build --action_env=PATH",
"after": "build --incompatible_strict_action_env"
}
]
},
{
"id": "BZ-08",
"flags": [],
"file": ".bazelrc",
"line": 15,
"severity": "medium",
"area": "cache",
"title": "Nothing stops a laptop from writing the shared cache",
"evidence": "build:remote_cache --remote_cache=grpcs://cache.quillpath.example:443",
"why": "Only build:ci sets --remote_upload_local_results=true; there is no global --noremote_upload_local_results, so any machine that turns on the remote_cache config also uploads. Per the note, CI must keep writing the cache and laptops should only read it - a global --noremote_upload_local_results alongside the existing build:ci override gives exactly that split.",
"edits": [
{
"before": "# Shared cache\nbuild:remote_cache --remote_cache=grpcs://cache.quillpath.example:443",
"after": "# Shared cache\nbuild --noremote_upload_local_results\nbuild:remote_cache --remote_cache=grpcs://cache.quillpath.example:443"
}
]
},
{
"id": "BZ-10",
"flags": [
"F5"
],
"file": ".bazelrc",
"line": 10,
"severity": "medium",
"area": "cache",
"title": "Stamping runs on every build, not just releases",
"evidence": "build --stamp",
"why": "A stamped action embeds the commit and build time, so its output - and everything that links it - changes on every commit and never hits the cache. Per the note the timestamp is only needed for release builds, so --stamp belongs in a new build:release config; release builds add --config=release.",
"edits": [
{
"before": "build --stamp",
"after": "build:release --stamp"
}
]
},
{
"id": "BZ-06",
"flags": [
"F4"
],
"file": ".bazelrc",
"line": 9,
"severity": "medium",
"area": "hermetic",
"title": "Sandbox switched off for every action",
"evidence": "build --spawn_strategy=local",
"why": "Running every action outside the sandbox lets it read undeclared files or the host's own tools, so it can build different bytes on a machine whose sandbox stays on and poison the shared cache. Per the note nobody remembers which action needed this months-old override, so the safe fix is to drop the global flag; if one action still needs it, give that action a tag or execution_requirements instead of every build.",
"edits": [
{
"before": "build --spawn_strategy=local\nbuild:release --stamp",
"after": "build:release --stamp"
}
]
},
{
"id": "BZ-21",
"flags": [
"F7"
],
"file": ".bazelrc",
"line": 12,
"severity": "low",
"area": "config",
"title": "Experimental flag on every build",
"evidence": "build --experimental_remote_cache_compression",
"why": "Experimental flags can change meaning or vanish in any release, and an unrecognized one stops every command outright; whether this one is still current on Bazel 7.4.1 needs a check against that release's docs, which the paste doesn't show.",
"edits": []
},
{
"id": "BZ-20",
"flags": [
"F8"
],
"file": ".bazelrc",
"line": 17,
"severity": "low",
"area": "config",
"title": "Remote timeout set to an hour",
"evidence": "build:remote_cache --remote_timeout=3600",
"why": "At 3600 seconds, a single hung remote-cache call can block for an hour before Bazel falls back to building locally. The default of 60 seconds is usually right; this raises it a little for slow uploads without leaving a bad cache host able to stall a build for an hour.",
"edits": [
{
"before": "build:remote_cache --remote_timeout=3600",
"after": "build:remote_cache --remote_timeout=120"
}
]
}
],
"flag_answers": [
{
"flag": "F1",
"status": "confirmed",
"reason": "carried by the BZ-18 finding"
},
{
"flag": "F2",
"status": "confirmed",
"reason": "carried by the BZ-16 finding"
},
{
"flag": "F3",
"status": "confirmed",
"reason": "carried by the BZ-07 finding"
},
{
"flag": "F4",
"status": "confirmed",
"reason": "carried by the BZ-06 finding"
},
{
"flag": "F5",
"status": "confirmed",
"reason": "carried by the BZ-10 finding"
},
{
"flag": "F6",
"status": "confirmed",
"reason": "carried by the same BZ-07 finding as F3"
},
{
"flag": "F7",
"status": "confirmed",
"reason": "carried by the BZ-21 finding"
},
{
"flag": "F8",
"status": "confirmed",
"reason": "carried by the BZ-20 finding"
}
],
"plan": [
"Fix the --config=remote-cache typo on build:ci to --config=remote_cache so bazel build //... --config=ci stops failing outright.",
"Rotate the x-api-key value on the remote_header line and move it out of .bazelrc - a CI secret on the command line, a git-ignored user.bazelrc via try-import, or --credential_helper.",
"Replace build --action_env=PATH with build --incompatible_strict_action_env so every machine computes the same action keys.",
"Add a global build --noremote_upload_local_results so only build:ci --remote_upload_local_results=true writes the shared cache.",
"Move --stamp into a new build:release config so only release builds pay the never-cached cost.",
"Drop the global --spawn_strategy=local so actions run sandboxed again; give the one flaky-test action a targeted tag instead if it still needs it.",
"Confirm whether --experimental_remote_cache_compression is still valid on Bazel 7.4.1 and drop it if it has been promoted or removed.",
"Lower --remote_timeout from 3600 toward the 60s default.",
"Verify: run bazel build //... --config=ci twice on a clean CI runner and read the second run's cache hits (bazel build --profile=profile.gz then bazel analyze-profile profile.gz, or the Build Event Protocol), and run bazel aquery on one target on a laptop and on CI to compare action keys."
],
"questions": [
"Does any action still need --spawn_strategy=local (the flaky test nobody remembers), and if so which target, so it can get a tag instead of the global override?",
"Is --experimental_remote_cache_compression still experimental on Bazel 7.4.1, or has it been promoted or removed?",
"libs/textkit/BUILD.bazel globs srcs as textkit/*.py from inside libs/textkit/ itself - if the .py files live directly in that directory rather than a nested libs/textkit/textkit/ folder, the glob matches nothing and srcs is silently empty; is that pattern intentional?",
"Should build --workspace_status_command=tools/status.sh move into build:release alongside --stamp, or does something else on every build consume its output?"
],
"summary": "Fix the two high findings first: the build:ci config typo (remote-cache vs remote_cache) that fails every --config=ci build, and the committed x-api-key, which must be rotated and moved off disk. Then close the medium cache-key issues - the leaking PATH, the missing upload guard, and universal stamping - which is why laptops and CI almost never share a hit; --spawn_strategy=local looks safe to drop now that no one can say which action still needs it. The experimental flag and the hour-long timeout are hygiene, not blockers."
}
The credential finding carries no edit: it cannot be fixed without quoting the value, so the fix lane rewrites the file without it. The page applies the other edits in order and re-checks: 6 edit(s) applied, 0 did not; with every edit in place the free check raises 1 high, 0 medium and 1 low flag(s).
Worked example, task: "fix": a map-rendering workspace with an absolute local_path_override, a placeholder sha256, a git repository on a branch, curl inside a genrule, a plaintext cache and every test run three times
4 files: .bazelversion, .bazelrc, MODULE.bazel, tools/fonts/BUILD.bazel. The free check raises 7 flags (4 high, 3 medium); build-check.js prints
4 file(s), 2 target(s), 6 repositor(ies), 6 rc line(s), 1 config(s); 7 flags (4 high, 3 medium, 0 low); verdict_candidate blocked. The engineer’s note: “Fix what you can. geo_math is vendored at third_party/geo-math in this repository now. The tested commit of shape_codec is 3f9c2e1d4b6a8c0e2f4a6b8d0c2e4f6a8b0d2c4e. The cache also serves grpcs on bazel-cache.corp.lan:9092. We do not have the tzdata sha256 yet and the fonts zip has no published checksum - leave those two for us. The three runs per test were only meant for the geometry tests we suspect are flaky.”
The request body, idempotency key hermetic-build-audit:fix:212354fb-1d96:a1:
{
"task": "fix",
"code": "# file: .bazelversion\n7.4.1\n# file: .bazelrc\ncommon --enable_bzlmod\nbuild --incompatible_strict_action_env\n\nbuild:ci --remote_cache=http://bazel-cache.corp.lan:8080\nbuild:ci --remote_upload_local_results\n\ntest --runs_per_test=3\ntest --test_output=errors\n# file...",
"facts": "{\"workspace\":{\"bazel_version\":\"7.4.1\",\"bazelversion_file\":\"7.4.1\",\"module\":{\"name\":\"orbrinth_maps\",\"version\":\"1.8.0\"},\"bzlmod\":true,\"workspace_deps\":false,\"files\":4,\"targets\":2,\"repositories\":3,\"bazel_deps\":3,\"rc_lines\":...",
"note": "Fix what you can. geo_math is vendored at third_party/geo-math in this repository now. The tested commit of shape_codec is 3f9c2e1d4b6a8c0e2f4a6b8d0c2e4f6a8b0d2c4e. The cache also serves grpcs on bazel-cache.corp.lan:9092. We do not have the tzdata sha256 yet and the fonts zip has no published checksum - leave those two for us. The three runs per test were only meant for the geometry tests we suspect are flaky."
}
The saved reply, verdict partial, 2 file(s), 4 change(s), 2 open item(s):
{
"task": "fix",
"verdict": "partial",
"headline": "Vendored geo_math to a workspace-relative path, pinned shape_codec to its tested commit, moved the CI remote cache to grpcs, and scoped the 3x test reruns into a test:flaky config; the tzdata checksum and the font-fetch genrule (with its no-sandbox tag) stay open pending checksums you don't have yet.",
"files": [
{
"path": ".bazelrc",
"content": "common --enable_bzlmod\nbuild --incompatible_strict_action_env\n\nbuild:ci --remote_cache=grpcs://bazel-cache.corp.lan:9092\nbuild:ci --remote_upload_local_results\n\ntest:flaky --runs_per_test=3\ntest --test_output=errors\n"
},
{
"path": "MODULE.bazel",
"content": "module(name = \"orbrinth_maps\", version = \"1.8.0\")\n\nbazel_dep(name = \"platforms\", version = \"0.0.10\")\nbazel_dep(name = \"rules_cc\", version = \"0.0.16\")\nbazel_dep(name = \"geo_math\", version = \"0.3.0\")\nlocal_path_override(\n module_name = \"geo_math\",\n path = \"third_party/geo-math\",\n)\n\nhttp_archive = use_repo_rule(\"@bazel_tools//tools/build_defs/repo:http.bzl\", \"http_archive\")\ngit_repository = use_repo_rule(\"@bazel_tools//tools/build_defs/repo:git.bzl\", \"git_repository\")\n\nhttp_archive(\n name = \"tzdata_bundle\",\n build_file = \"//third_party:tzdata.BUILD\",\n sha256 = \"...\",\n strip_prefix = \"tzdata-2024b\",\n urls = [\"https://data.iana.org/time-zones/releases/tzdata2024b.tar.gz\"],\n)\n\ngit_repository(\n name = \"shape_codec\",\n remote = \"https://github.com/orbrinth-maps/shape-codec.git\",\n commit = \"3f9c2e1d4b6a8c0e2f4a6b8d0c2e4f6a8b0d2c4e\",\n)\n"
}
],
"changes": [
{
"file": "MODULE.bazel",
"target": "(file)",
"flags": [
"F1"
],
"change": "Changed local_path_override(geo_math)'s path from the absolute /home/dana/src/geo-math to the vendored third_party/geo-math.",
"why": "Every machine now resolves geo_math from inside the checked-out workspace instead of a directory that only exists on one developer's machine."
},
{
"file": "MODULE.bazel",
"target": "(file)",
"flags": [
"F3"
],
"change": "Replaced git_repository(shape_codec)'s branch = \"main\" with commit = \"3f9c2e1d4b6a8c0e2f4a6b8d0c2e4f6a8b0d2c4e\".",
"why": "Every fetch now resolves the exact tested revision instead of whatever main points to at fetch time, so two machines build the same code under the same cache key."
},
{
"file": ".bazelrc",
"target": "(file)",
"flags": [
"F5"
],
"change": "Changed build:ci --remote_cache from http://bazel-cache.corp.lan:8080 to grpcs://bazel-cache.corp.lan:9092.",
"why": "Cache blobs and headers now travel encrypted instead of being readable or spoofable by anyone on the path."
},
{
"file": ".bazelrc",
"target": "(file)",
"flags": [
"F6"
],
"change": "Moved --runs_per_test=3 off the shared test command into a new test:flaky config.",
"why": "The extra runs were only meant for the suspected-flaky geometry tests, so every other test stops paying the 3x cost; run the geometry tests with --config=flaky until they carry flaky = True directly."
}
],
"verify": [
"bazel build //... --config=ci and bazel test //... --config=ci on a clean checkout, including a machine that never had /home/dana/src/geo-math, to confirm geo_math still resolves via third_party/geo-math.",
"bazel fetch --config=ci //... then bazel aquery on a target depending on @shape_codec to confirm the fetched revision is 3f9c2e1d4b6a8c0e2f4a6b8d0c2e4f6a8b0d2c4e on every machine, not whatever main currently points to.",
"Run bazel build //... --config=ci twice on a clean CI runner and check the Build Event Protocol (or bazel build --profile=profile.gz then bazel analyze-profile profile.gz) for remote cache hits over grpcs://bazel-cache.corp.lan:9092 on the second run.",
"bazel test //path/to:geometry_tests --config=flaky to confirm the 3x reruns still apply there, and bazel test //... --config=ci to confirm the rest of the suite now runs each test once."
],
"open": [
{
"item": "http_archive(tzdata_bundle) still has the placeholder sha256 = \"...\"",
"flags": [
"F2"
],
"reason": "You said you don't have the real tzdata sha256 yet; fetch the archive once and paste its sha256 (or integrity) here before this can be pinned."
},
{
"item": "genrule //tools/fonts:inter_fonts still curls the Inter fonts zip at build time and keeps the no-sandbox tag",
"flags": [
"F4",
"F7"
],
"reason": "You said the fonts zip has no published checksum yet, so the curl can't become a checksummed http_file/http_archive dependency; until it does, the action still needs network access and no-sandbox has to stay or a sandboxed build would fail trying to reach the network. Once you have a checksum, declare it as a pinned repository (via use_repo_rule) and depend on its label, then drop the tag."
}
],
"questions": [],
"summary": "Fixed the four flags the note gave you facts for: geo_math now points at the vendored third_party/geo-math, shape_codec is pinned to the tested commit, the CI remote cache uses the grpcs endpoint you gave, and the 3x test reruns moved into a test:flaky config instead of applying to every test. The tzdata checksum and the fonts-zip fetch (and its no-sandbox tag) are left open because you said those two checksums aren't available yet; nothing else in the paste was touched."
}
The fixes use only what the note gives (the vendored path, the tested commit, the grpcs port); the two
checksums the note says are not known yet stay open, so the verdict is partial.
The response envelope
Responses are JSON of the form {"ok": ..., "data": ..., "error": {...}}. The page’s
SDK (sdk.js) decides success from the HTTP status, not from ok: on a
2xx it returns data; otherwise it raises an error whose message is
error.message (or the HTTP status text when the body carries none), with
status, code (error.code) and details
(error.details) attached. A body that is not JSON is treated as {}.
{ "ok": true, "data": { ... } }
{ "ok": false, "error": { "code": "...", "message": "...", "details": { ... } } }
Error codes
Branch on the HTTP status and error.code, show error.message. The page
handles 401 and 402 by name; the rest reach it as a generic failure (“The run failed: ...”).
| HTTP | error.code | What it means / what to do |
|---|---|---|
400 | VALIDATION_ERROR | The body is not a JSON object, a required field (task, code, facts) is missing, or a declared field has the wrong type. Every field is a string; sending facts or audit_handoff as an object is the usual cause. |
401 | UNAUTHORIZED | No token, or one that expired or was revoked. The page forgets the signed-in user and says “Your session expired - sign in again.” Get a new token. |
402 | INSUFFICIENT_CREDITS | The balance cannot cover the run. The page says “Not enough credits for this run.” with a top-up link, and tries never to submit into a 402: it disables the run button when the /me balance is below the estimate’s min_credits. |
404 | NOT_FOUND | A wrong path, or a job id that does not belong to this token. |
409 | CONFLICT | An Idempotency-Key replayed with a different body. Send the original body, or use a new key (the page bumps its :a<attempt> suffix for a retry). |
429 | RATE_LIMITED | Too many requests. Back off with a delay; do not tight-loop, and never resend a metered run without its original Idempotency-Key. |
Failures that do not arrive as an HTTP status:
| What | What happens |
|---|---|
event: error on /run-stream | The SDK raises an error carrying the event’s message, code and job_id. |
Job status: "failed" | Polling stops; the job’s error field says why. |
| Polling past 180 s | The SDK’s own client-side limit (waitForJob, 1 s interval, 180 000 ms default): it throws “job timed out”. The job itself is not cancelled. |
The input fields
Every field is a string. The app declares this input schema; task, code and facts are required.
| Field | Type | What it is |
|---|---|---|
task | string, required | "audit" or "fix" (above). |
code | string, required | The paste, verbatim: every file introduced by a # file: <path> line. Line 1 is its first line; every line number in facts and in the reply is a line of this string. |
facts | string, required | The page’s analysis as a JSON string (below). |
audit_handoff | string, optional | Fix lane only: an audit handed on, as a JSON string (below). |
note | string, optional | What the engineer says, cut to 2,000 characters: a goal, a constraint, facts the files cannot show (a tested commit, a TLS port). Context, never instructions, and never evidence about the files. |
retry_note | string, optional | Sent only on the page’s one automatic retry, when the first reply did not parse. |
The code string
Paste the files one after another, each preceded by # file: <path> with the path from the
workspace root (# file: .bazelrc, # file: MODULE.bazel,
# file: apps/web/BUILD.bazel). A marker’s path must look like a path (a dot or a slash) or be
a Bazel file name (BUILD, WORKSPACE, MODULE.bazel ...); a line
--- directly before a marker only separates files. A paste with no marker is one file, named by
what it looks like. A target’s package is the directory of its BUILD file. The page reads up to
400,000 characters and 60 files, and sends at most 60,000 characters: over that it drops
whole files from the end (files it does not check first, then .bzl, then BUILD files) and ends
code with a line # [hermetic-build-audit: not sent: ...] naming them, and says so in
facts.notes. The model is told never to quote what it was not sent. facts is bounded
too, to about 40,000 characters: long entries are clipped, the flag list keeps at most
150 flags (the most severe first, flags in sent files before the rest), the other lists give way first,
and every cut is named in facts.notes. An audit_handoff over 30,000
characters is cut the same way.
The facts string
facts is produced by the page’s free check: Bazelkit.analyze splits the paste
into files, reads the rc lines and the Starlark calls and runs the rules, and Bazelkit.facts(A)
turns that analysis into the object below (it does not depend on the lane). It is how the model knows which
flags to answer, which paths and line ranges to name, and which targets, repositories and configs must
survive. Its top-level keys, in the order built:
| Key | Contents |
|---|---|
workspace | bazel_version (the exact release in a pasted .bazelversion, or null), bazelversion_file (its raw value, or null when none was pasted), module (name and version from module(), or null), bzlmod (a MODULE.bazel was pasted), workspace_deps (a WORKSPACE declares more than workspace()), and counts: files, targets, repositories, bazel_deps, rc_lines, configs. |
files | Every file: path (use it verbatim), kind (version, rc, module, workspace, build, bzl, ignore, other), line and end_line (the paste lines of the file’s content, without its # file: line). |
targets | Every target (up to 300): label (//pkg:name), kind (the rule or macro), file, line, and tags when it has any. |
repositories | Every repository rule and override (up to 120): kind, name, file, line, pinned (checksum, placeholder, none, commit, tag, branch, path). |
bazel_deps | Every bazel_dep (up to 150): name, version, dev, file, line. |
rc | Every rc line (up to 200): file, line, command, config, flags (as written; a credential value shows as ***). |
configs_defined, configs_referenced | The named configs rc lines define (build:ci ...), and every value passed to --config. |
flags | Every flag the page raised: uid (F1, F2, ... in severity, file, then line order), id (BZ-01..BZ-22), sev (high | medium | low | info), file, line, text, detail, and lines (every line an aggregated flag covers, when there are several). |
verdict_candidate | The audit verdict the flags point at: blocked with any high flag, needs_work with any medium, otherwise ready. |
notes | Plain sentences about what was not read, not sent or cut to fit (empty for a paste that fits). |
The catalogue:
| Id | Flag | Area |
|---|---|---|
BZ-01 | Bazel version not pinned to an exact release | hermetic |
BZ-02 | download with no checksum, or a placeholder checksum | hermetic |
BZ-03 | git dependency pinned to a branch, or not pinned | hermetic |
BZ-04 | local repository at an absolute path on one machine | hermetic |
BZ-05 | network access or the clock inside a build action | hermetic |
BZ-06 | sandboxing switched off | hermetic |
BZ-07 | action environment leaks into cache keys | cache |
BZ-08 | every machine writes the shared remote cache | cache |
BZ-09 | remote cache or executor over a plaintext connection | cache |
BZ-10 | stamping on every build | cache |
BZ-11 | test results never cached, or every test run several times | cache |
BZ-12 | caching switched off on a target | cache |
BZ-13 | glob() in deps | graph |
BZ-14 | one coarse target over the whole tree | graph |
BZ-15 | every target public by default | graph |
BZ-16 | --config that no rc line defines | config |
BZ-17 | external dependencies still in WORKSPACE | config |
BZ-18 | credential literal in the build configuration | config |
BZ-19 | --jobs far beyond one machine with no remote executor | config |
BZ-20 | remote timeout of ten minutes or more | config |
BZ-21 | experimental flag in the shared configuration | config |
BZ-22 | context | context |
The simplest source is bazelkit.js itself: the page loads it from
/bazelkit.js, and it also runs in Node (it sets module.exports).
This script prints what the page would send as code and facts (with the
60,000-character cut applied), with the facts decoded so you can read them; the step 4 samples compact
them back into the page’s string byte for byte. Save the paste as monorepo-paste.txt and
redirect the output into monorepo-check.json:
// build-check.js -- run: node build-check.js monorepo-paste.txt > monorepo-check.json (bazelkit.js saved next to it)
const fs = require("fs");
const Bazelkit = require("./bazelkit.js");
const A = Bazelkit.analyze(fs.readFileSync(process.argv[2], "utf8"));
if (!A.ok) { console.error(A.errors.join("\n")); process.exit(1); }
const inp = Bazelkit.buildInput("audit", A, "", null); // code and facts do not depend on the lane
console.log(JSON.stringify({ code: inp.code, facts: JSON.parse(inp.facts) }, null, 2));
const real = A.flags.filter((f) => f.sev !== "info");
console.error(A.files.length + " file(s), " + A.targets.length + " target(s), " + (A.repos.length + A.deps.length) + " repositor(ies), " +
A.rc.length + " rc line(s), " + A.configs.length + " config(s); " + real.length + " flags (" + A.counts.high + " high, " +
A.counts.medium + " medium, " + A.counts.low + " low); verdict_candidate " + A.candidate);
For the monorepo example it reports 5 file(s), 5 target(s), 3 repositor(ies), 14 rc line(s), 2 config(s); 8 flags (2 high, 4 medium, 2 low); verdict_candidate blocked; for the vendor example,
4 file(s), 2 target(s), 6 repositor(ies), 6 rc line(s), 1 config(s); 7 flags (4 high, 3 medium, 0 low); verdict_candidate blocked. For both worked examples its code and facts are
exactly what the page sends.
Without the page’s check. The platform only checks that facts is a string,
so "{}" works and the model still reads code. But you lose what the check adds: with
no flags there is nothing to confirm, dismiss or close, with no files there are no
paths and line ranges to name, with no bazel_version the model has to hedge every
version-dependent edit (WORKSPACE on Bazel 8), and with no targets, repositories and
configs_defined nothing says what must survive a fix. Build it with bazelkit.js when
you can.
The audit_handoff string
Only for task: "fix", and only for the files the audit was about with every audit edit
applied. The page builds both from the normalised audit reply res
(recon.js normalize) and the analysis A of the audited paste, in
recon.js fixHandoff:
const run = Recon.applyFindings(A, res.findings, Bazelkit); // every finding's edits, in order, each on top of the last
JSON.stringify({ verdict: res.verdict, headline: res.headline,
findings: res.findings.map(f => ({ id: f.id, severity: f.severity, file: f.file, line: f.line, title: f.title, why: f.why, edits: f.edits.length })),
applied: /* edits that applied */, failed: /* edits that did not */ })
// run.A.text is the edited paste: it becomes code, and facts is recomputed from it
Each finding’s edits there is the number of edits it carried.
applyFindings applies every finding’s edits with Bazelkit.applyEdits, in the
order listed, to the finding’s file: each before must occur exactly once in that
file as the earlier edits left it (blank lines at either end are tolerated, and a blank line written inside
after is kept), or that edit is reported as failed and skipped — it is never guessed. This
script does the whole step, in the same shape as build-check.js prints:
// handoff.js -- run: node handoff.js monorepo-paste.txt monorepo-reply.txt > monorepo-fix-check.json (bazelkit.js, recon.js next to it)
const fs = require("fs");
const Bazelkit = require("./bazelkit.js");
const Recon = require("./recon.js");
const A = Bazelkit.analyze(fs.readFileSync(process.argv[2], "utf8")); // the audited paste
const res = Recon.normalize(Recon.parseResult(fs.readFileSync(process.argv[3], "utf8")), "audit");
const ho = Recon.fixHandoff(A, res); // every edit, in order
const B = Bazelkit.analyze(ho.text); // facts recomputed from the edited paste
const inp = Bazelkit.buildInput("fix", B, "", ho.handoff);
console.log(JSON.stringify({ code: inp.code, facts: JSON.parse(inp.facts), audit_handoff: inp.audit_handoff }, null, 2));
console.error(ho.applied + " edit(s) applied, " + ho.failed + " did not apply");
For the monorepo audit above it reports 6 edit(s) applied, 0 did not apply; the edited
paste raises 1 high, 0 medium and 1 low flag(s), and the page sends that fix run with the
audit’s note and the key hermetic-build-audit:fix:560de5de-2db6:a1. Its saved reply is fixed
(1 file(s), 1 change(s), 0 open item(s)).
The output contracts
The model is instructed (SKILL.md) to reply with one JSON object and nothing else
— no prose, no code fences — with every array of its lane present ([] when empty). The
run’s output text is that object. In both lanes it grounds everything in code: unpasted BUILD
and .bzl files, the remote cache, the CI runner and the machines are outside the paste, so they are asked about,
never stated as facts. It never invents a checksum, a commit, a version, a URL, a host, a port or a label; it
never writes a secret value (a line that holds a credential is never copied into the reply — the value
moves to a CI secret, a git-ignored user.bazelrc read with try-import, or
--credential_helper); and it keeps the build’s meaning: which targets exist, what they depend
on, which repositories and named configs exist. Version-dependent advice follows
facts.workspace.bazel_version. Every page flag whose sev is not info is
answered exactly once (audit lane: in flag_answers), and an info flag is never a finding.
task: "audit"
task:"audit".verdict:blocked(any finding is high) |needs_work(the worst is medium) |ready(nothing above low).headline: one sentence.workspace: one or two sentences — what the workspace builds and how its dependencies and cache are set up.areas[]: exactly four entries, in the order hermetic, cache, graph, config:{area, status, note}withstatussound|fragile|broken(brokenwhen a high finding sits in that area,fragilefor a medium one).findings[]:{id, flags[], file, line, severity, area, title, evidence, why, edits[]}.idis a catalogue id (BZ-01..BZ-21) orother;flagslists the pageuids it confirms ([]for something the page did not flag);fileis apathfromfacts.files— the file its edits change;lineis the line ofcodewith the offending text;evidenceis one to three whole consecutive lines copied from that file (never a line holding a credential).edits[]:{before, after}on that one file. Eachbeforeis a contiguous, exact slice of the file — whole lines with their indentation — that occurs exactly once in the file as it stands when the edit is applied. Findings are applied in the order listed and their edits in order, each on top of the ones before. An audit edit never creates a file;editsis[]for a low finding, a credential, a change outside the paste, or one that needs a fact or decision the note does not give.flag_answers[]:{flag, status, reason}, exactly one per non-info page flag,confirmed(a finding lists that uid) |dismissed(with the reason).plan[]: the findings in the order to apply them, ending with two clean CI builds, a read of the second one’s cache hits (--profileandbazel analyze-profile, or the Build Event Protocol) and anaquerycomparison of one action’s key on two machines.questions[]andsummary(two to four sentences, answeringnote).
task: "fix"
task:"fix".verdict:fixed(openis empty) |partial.headline: one sentence.files[]:{path, content}— every pasted file it changes, whole (every line, without the# file:line), under itspathfromfacts.files. No file is added (auser.bazelrcfor a credential is the engineer’s to create).changes[]:{file, target, flags[], change, why}, one per change.targetis the label the change is in, asfacts.targetsnames it, or(file)for an rc line, MODULE.bazel, WORKSPACE or a load;flagsare the page flags it closes. Every high and medium flag appears in a change or anopenitem.verify[]: a cleanbazel build //... --config=ciandbazel test //... --config=ci, then the check that would have caught the worst problem (remote cache hits on a second CI run,bazel fetchfailing on a wrong checksum, the sameaquerykey on two machines, actions without network via--sandbox_default_allow_network=false).open[]:{item, flags[], reason}for what it left unfixed.questions[]andsummary, as in the audit lane; with anaudit_handoff, it fixes what the audit left.
How the page reads a reply (recon.js parseResult and normalize): it strips
a code fence and parses from the first { to the last }. A reply is read as the lane
that was asked for, unless it names the other lane and carries that lane’s body (a
files array for fix, a findings array for audit). An unknown verdict
shows as needs_work or partial and counts as a disagreement; an unknown
severity becomes medium; ids like BZ7 become BZ-07;
flags keeps only F<n> values; workspace may also arrive as
job, file as path, a change’s target as
label or function and its change as what;
questions is capped at 10, plan at 30 and verify at 20. An audit reply
carries 9 sections (verdict, headline, workspace, areas, findings, flag_answers, plan, questions, summary) and a fix reply 8
(verdict, headline, files, changes, verify, open, questions, summary), which is what the page counts when it recovers a partial reply.
The page checks the reply against the files; the API does not
What comes back from the API is the model’s raw text in output.output. The page holds it to
account in the browser before showing it (recon.js reconcile, with
the facts, the code and the handoff it sent): it applies every edit or returned file to the paste and re-runs
its own checks on the result. None of that happens in the API.
- Both lanes: the reply is the lane you asked for, carries a valid verdict, and repeats no credential from the paste anywhere.
- Audit, areas: exactly four rows, hermetic, cache, graph and config once each, each with a valid status.
- Audit, every finding is anchored: a catalogue id (BZ-22 is context and never a finding), a pasted file, a line inside it, evidence found line by line in that file and on or near that line, and only non-info page flags. A medium or high finding with no edit is pointed out (except a credential, which cannot carry one).
- Audit, every flag answered once: a
confirmedflag needs a finding carrying it, adismissedone must not be carried; the verdict must fit the worst severity. - Audit, every edit put in place and re-checked: applied in the listed order with
Bazelkit.applyEdits; abeforethat is not found, is found twice, or changes nothing is a disagreement. After each finding the check runs again and no new high flag may appear; with every edit applied every confirmed flag a finding with edits carries must be gone, and every target, repository and named config must still be there. - Fix, the files: at least one, none new, none empty, none with a
# file:line; put into the paste (Bazelkit.replaceFiles) and re-checked: no high or medium flag of the paste may remain unless anopenitem names it, none may be new, and every target, repository and config survives. - Fix, the changes list: every change names a returned file, a label in the fixed files (or
(file)) and real page flags; a high or medium flag nothing names is pointed out; the verdict must fitopen, and an emptyverifyis pointed out.
recon.js also runs in Node, so you can hold a reply to exactly the page’s standard. Save
/bazelkit.js and /recon.js next to this script:
// recheck.js -- run: node recheck.js monorepo-check.json monorepo-reply.txt audit > monorepo-edited.txt (bazelkit.js, recon.js next to it)
const fs = require("fs");
const Recon = require("./recon.js"); // loads ./bazelkit.js itself
const sent = JSON.parse(fs.readFileSync(process.argv[2], "utf8")); // code, facts (and audit_handoff) you sent
const lane = process.argv[4] || "audit";
const res = Recon.normalize(Recon.parseResult(fs.readFileSync(process.argv[3], "utf8")), lane); // output.output
const r = Recon.reconcile(res, { facts: sent.facts, code: sent.code, handoff: sent.audit_handoff || null });
r.notes.forEach((n) => console.error(n.kind + ": " + n.text));
console.error(r.disagreements + " disagreement(s) in " + r.statements + " statements checked");
if (r.after) (r.after.flags || []).forEach((f) => console.error("still raised: " + f.id + " " + f.sev + " " + f.file + ":" + f.line + " " + f.text));
if (r.after && r.after.text) console.log(r.after.text.replace(/\n$/, "")); // the paste with every edit (or file) put back
For the monorepo audit reply it reports:
ok: The areas table rates hermeticity, caching, the build graph and the configuration once each.
ok: Every one of the 8 findings names a pasted file, a line inside it and evidence copied from it.
ok: The verdict (blocked) fits the worst finding.
ok: Every one of the page's 8 flags is answered once.
ok: All 6 edits applied in order, each before text found exactly once.
ok: Every confirmed flag a finding carries is gone once all edits are applied.
ok: Every target, repository and named config of the paste is still there after the edits (5 targets, 3 repositories, 2 configs).
ok: The reply repeats no credential from the paste (1 found in it).
0 disagreement(s) in 8 statements checked
still raised: BZ-18 high .bazelrc:16 --remote_header carries a literal x-api-key value
still raised: BZ-21 low .bazelrc:11 1 experimental flag on every build (--experimental_remote_cache_compression)
and for the vendor fix reply (node recheck.js vendor-check.json vendor-reply.txt fix):
ok: 2 pasted files returned whole.
ok: With the fixed files in place the free check raises no high or medium flag beyond the ones left open (0 low).
info: F2, F4, F7 left open by the reply.
ok: Every target, repository and named config is still there (2 targets, 5 repositories, 1 config).
ok: Every one of the 4 changes names a returned file, a target in the fixed files (or the file itself) and real page flags.
ok: The verdict (partial) fits the open list.
ok: The reply repeats no credential from the paste.
0 disagreement(s) in 6 statements checked
still raised: BZ-02 high MODULE.bazel:26 http_archive tzdata_bundle has a placeholder sha256 "..."
still raised: BZ-05 high tools/fonts/BUILD.bazel:43 genrule //tools/fonts:inter_fonts runs curl inside the action
still raised: BZ-06 medium tools/fonts/BUILD.bazel:44 1 target tagged no-sandbox or local (:inter_fonts)
The page does not run Bazel. After that, build and test with the CI config twice on a clean runner, read the
second run’s cache hits, and compare one action’s key on two machines, plus whatever the
reply’s plan or verify names.
Step 1 — a tiny client helper
One function per language that does what sdk.js does for every call: JSON in,
bearer token if you have one, data out on a 2xx, an error carrying
status, error.code and error.message otherwise. The later
steps assume this helper is in the same file (for Go, Java and C#, the later snippets go inside
main / the top-level statements; Java’s makeInput from step 4 goes
in the class).
BASE="https://api.skillsafe.ai/v1/app-api"
TOKEN="${SKILLSAFE_TOKEN:-YOUR_TOKEN}" # step 2
# api METHOD PATH [extra curl args...] -- prints the whole envelope; pipe to jq
api() {
method="$1"; path="$2"; shift 2
curl -sS -X "$method" "$BASE$path" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $TOKEN" "$@"
}
# success: jq '.data' failure: jq '.error | {code, message}'
import hashlib, json, os, time, urllib.parse
import requests # pip install requests
BASE = "https://api.skillsafe.ai/v1/app-api"
TOKEN = os.environ.get("SKILLSAFE_TOKEN", "YOUR_TOKEN") # step 2
class ApiError(Exception):
def __init__(self, status, code, message, details=None):
super().__init__(f"{status} {code}: {message}")
self.status, self.code, self.details = status, code, details
def api(method, path, body=None, headers=None):
h = {"Content-Type": "application/json"}
if TOKEN:
h["Authorization"] = "Bearer " + TOKEN
h.update(headers or {})
r = requests.request(method, BASE + path, headers=h,
data=None if body is None else json.dumps(body))
try:
env = r.json()
except ValueError:
env = {}
if not r.ok:
err = env.get("error") or {}
raise ApiError(r.status_code, err.get("code"), err.get("message") or r.reason, err.get("details"))
return env.get("data")
// hermetic-build-audit.mjs -- Node 18+ (global fetch), run with: node hermetic-build-audit.mjs
import { readFileSync } from "node:fs";
import { createHash } from "node:crypto";
const BASE = "https://api.skillsafe.ai/v1/app-api";
const TOKEN = "YOUR_TOKEN"; // your personal token from /tokens.html (step 2)
let token = TOKEN; // step 2 may swap in a guest token
async function api(method, path, body, headers = {}) {
const h = { "Content-Type": "application/json", ...headers };
if (token) h.Authorization = "Bearer " + token;
const res = await fetch(BASE + path, {
method,
headers: h,
body: body === undefined ? undefined : JSON.stringify(body),
});
const json = await res.json().catch(() => ({}));
if (!res.ok) {
const err = new Error((json.error && json.error.message) || res.statusText);
err.status = res.status;
err.code = json.error && json.error.code;
err.details = json.error && json.error.details;
throw err;
}
return json.data;
}
package main
import (
"bufio"
"bytes"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"fmt"
"net/http"
"net/url"
"os"
"strings"
"time"
)
const base = "https://api.skillsafe.ai/v1/app-api"
var token = os.Getenv("SKILLSAFE_TOKEN") // step 2
type apiError struct {
Status int
Code string
Message string
}
func (e *apiError) Error() string { return fmt.Sprintf("%d %s: %s", e.Status, e.Code, e.Message) }
type envelope struct {
OK bool `json:"ok"`
Data json.RawMessage `json:"data"`
Error *struct {
Code string `json:"code"`
Message string `json:"message"`
} `json:"error"`
}
func newRequest(method, path string, body any, headers map[string]string) (*http.Request, error) {
var buf []byte
if body != nil {
b, err := json.Marshal(body)
if err != nil {
return nil, err
}
buf = b
}
req, err := http.NewRequest(method, base+path, bytes.NewReader(buf))
if err != nil {
return nil, err
}
req.Header.Set("Content-Type", "application/json")
if token != "" {
req.Header.Set("Authorization", "Bearer "+token)
}
for k, v := range headers {
req.Header.Set(k, v)
}
return req, nil
}
func errorFrom(res *http.Response, env envelope) error {
e := &apiError{Status: res.StatusCode, Message: http.StatusText(res.StatusCode)}
if env.Error != nil {
e.Code = env.Error.Code
if env.Error.Message != "" {
e.Message = env.Error.Message
}
}
return e
}
// api sends body as JSON and decodes the envelope's data into out (out may be nil).
func api(method, path string, body any, headers map[string]string, out any) error {
req, err := newRequest(method, path, body, headers)
if err != nil {
return err
}
res, err := http.DefaultClient.Do(req)
if err != nil {
return err
}
defer res.Body.Close()
var env envelope
_ = json.NewDecoder(res.Body).Decode(&env)
if res.StatusCode < 200 || res.StatusCode > 299 {
return errorFrom(res, env)
}
if out != nil && len(env.Data) > 0 {
return json.Unmarshal(env.Data, out)
}
return nil
}
func main() {
// steps 2-6 go here; remove any import a step you skip no longer uses
_, _, _, _, _ = bufio.NewScanner, sha256.Sum256, hex.EncodeToString, url.PathEscape, strings.Index
_ = time.Second
}
// HermeticBuildAudit.java -- Java 17+, needs com.fasterxml.jackson.core:jackson-databind on the classpath
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import java.net.URI;
import java.net.URLEncoder;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.security.MessageDigest;
import java.util.*;
import java.util.stream.Collectors;
import java.util.stream.Stream;
public class HermeticBuildAudit {
static final String BASE = "https://api.skillsafe.ai/v1/app-api";
static String token = System.getenv().getOrDefault("SKILLSAFE_TOKEN", "YOUR_TOKEN"); // step 2
static final HttpClient HTTP = HttpClient.newHttpClient();
static final ObjectMapper JSON = new ObjectMapper();
static class ApiException extends Exception {
final int status;
final String code;
ApiException(int status, String code, String message) {
super(status + " " + code + ": " + message);
this.status = status;
this.code = code;
}
}
static HttpRequest.Builder request(String method, String path, Object body, Map<String, String> headers) throws Exception {
HttpRequest.Builder b = HttpRequest.newBuilder(URI.create(BASE + path))
.header("Content-Type", "application/json")
.method(method, body == null ? HttpRequest.BodyPublishers.noBody()
: HttpRequest.BodyPublishers.ofString(JSON.writeValueAsString(body)));
if (token != null && !token.isEmpty()) b.header("Authorization", "Bearer " + token);
if (headers != null) headers.forEach(b::header);
return b;
}
static ApiException errorFrom(int status, JsonNode env) {
JsonNode err = env.path("error");
return new ApiException(status, err.path("code").asText(null), err.path("message").asText("HTTP " + status));
}
static JsonNode api(String method, String path, Object body, Map<String, String> headers) throws Exception {
HttpResponse<String> res = HTTP.send(request(method, path, body, headers).build(), HttpResponse.BodyHandlers.ofString());
JsonNode env;
try { env = JSON.readTree(res.body()); } catch (Exception e) { env = JSON.createObjectNode(); }
if (env == null) env = JSON.createObjectNode();
if (res.statusCode() / 100 != 2) throw errorFrom(res.statusCode(), env);
return env.path("data");
}
public static void main(String[] args) throws Exception {
// steps 2-6 go here (makeInput from step 4 goes next to api())
}
}
require "digest"
require "json"
require "net/http"
require "uri"
BASE = "https://api.skillsafe.ai/v1/app-api"
$token = ENV.fetch("SKILLSAFE_TOKEN", "YOUR_TOKEN") # step 2
class ApiError < StandardError
attr_reader :status, :code
def initialize(status, code, message)
super("#{status} #{code}: #{message}")
@status = status
@code = code
end
end
def build_request(method, path, body = nil, headers = {})
uri = URI(BASE + path)
req = Net::HTTPGenericRequest.new(method, !body.nil?, true, uri.request_uri)
req["Content-Type"] = "application/json"
req["Authorization"] = "Bearer #{$token}" if $token && !$token.empty?
headers.each { |k, v| req[k] = v }
req.body = JSON.generate(body) unless body.nil?
[uri, req]
end
def api(method, path, body = nil, headers = {})
uri, req = build_request(method, path, body, headers)
res = Net::HTTP.start(uri.host, uri.port, use_ssl: true) { |h| h.request(req) }
env = (JSON.parse(res.body) rescue {})
env = {} unless env.is_a?(Hash)
unless res.is_a?(Net::HTTPSuccess)
err = env["error"] || {}
raise ApiError.new(res.code.to_i, err["code"], err["message"] || res.message)
end
env["data"]
end
<?php
// needs the curl extension
const BASE = "https://api.skillsafe.ai/v1/app-api";
$TOKEN = getenv("SKILLSAFE_TOKEN") ?: "YOUR_TOKEN"; // step 2
class ApiError extends Exception {
public $status;
public $apiCode;
public function __construct($status, $code, $message) {
parent::__construct("$status $code: $message");
$this->status = $status;
$this->apiCode = $code;
}
}
function headers_for(array $extra = []): array {
global $TOKEN;
$h = ["Content-Type: application/json"];
if ($TOKEN) $h[] = "Authorization: Bearer $TOKEN";
foreach ($extra as $k => $v) $h[] = "$k: $v";
return $h;
}
function api(string $method, string $path, $body = null, array $headers = []) {
$ch = curl_init(BASE . $path);
curl_setopt_array($ch, [
CURLOPT_CUSTOMREQUEST => $method,
CURLOPT_HTTPHEADER => headers_for($headers),
CURLOPT_RETURNTRANSFER => true,
]);
if ($body !== null) curl_setopt($ch, CURLOPT_POSTFIELDS, json_encode($body, JSON_UNESCAPED_SLASHES | JSON_UNESCAPED_UNICODE));
$raw = curl_exec($ch);
if ($raw === false) throw new RuntimeException(curl_error($ch));
$status = curl_getinfo($ch, CURLINFO_RESPONSE_CODE);
curl_close($ch);
$env = json_decode($raw);
if (!is_object($env)) $env = new stdClass();
if ($status < 200 || $status > 299) {
$e = $env->error ?? new stdClass();
throw new ApiError($status, $e->code ?? null, $e->message ?? "HTTP $status");
}
return $env->data ?? null;
}
// Program.cs -- .NET 6+ console app (implicit usings on) with top-level statements; steps 2-6 (with the MakeInput local function) go ABOVE this class
using System.Text;
using System.Text.Json;
using System.Text.Json.Nodes;
static class Api
{
public const string Base = "https://api.skillsafe.ai/v1/app-api";
public static string Token = Environment.GetEnvironmentVariable("SKILLSAFE_TOKEN") ?? "YOUR_TOKEN"; // step 2
public static readonly HttpClient Http = new HttpClient();
public class ApiException : Exception
{
public int Status;
public string? Code;
public ApiException(int status, string? code, string message) : base($"{status} {code}: {message}")
{
Status = status;
Code = code;
}
}
public static HttpRequestMessage Request(string method, string path, object? body = null, IDictionary<string, string>? headers = null)
{
var req = new HttpRequestMessage(new HttpMethod(method), Base + path);
if (body != null) req.Content = new StringContent(JsonSerializer.Serialize(body), Encoding.UTF8, "application/json");
if (!string.IsNullOrEmpty(Token)) req.Headers.TryAddWithoutValidation("Authorization", "Bearer " + Token);
if (headers != null) foreach (var kv in headers) req.Headers.TryAddWithoutValidation(kv.Key, kv.Value);
return req;
}
public static ApiException ErrorFrom(HttpResponseMessage res, JsonNode? env)
{
var err = env?["error"];
return new ApiException((int)res.StatusCode, err?["code"]?.GetValue<string>(),
err?["message"]?.GetValue<string>() ?? res.ReasonPhrase ?? "error");
}
public static async Task<JsonNode?> Call(string method, string path, object? body = null, IDictionary<string, string>? headers = null)
{
using var req = Request(method, path, body, headers);
using var res = await Http.SendAsync(req);
var text = await res.Content.ReadAsStringAsync();
JsonNode? env = null;
try { env = JsonNode.Parse(text); } catch (JsonException) { }
if (!res.IsSuccessStatusCode) throw ErrorFrom(res, env);
return env?["data"];
}
}
Step 2 — a token
Two kinds work:
-
Personal — needed for a run. Both lanes are metered in credits on
gpt-terra, so a run needs a signed-in (user) token (the page says “The audit run is metered, so it needs an account.”, or the same of the fix run) unless the estimate sayssponsor_enabled. Sign in and copy your token from /tokens.html, then export it asSKILLSAFE_TOKEN(the samples fall back to the placeholderYOUR_TOKEN). -
Guest — enough for
/meand/estimate.POST /guestwith{"slug": "hermetic-build-audit"}and noAuthorizationheader, exactly assdk.jsguest()does. The response’sdatacarriestokenandguest_id. Reuse a token you already hold rather than minting a new one each time.
# personal token (needed for a run): copy it from /tokens.html, then
# export SKILLSAFE_TOKEN=... and skip the rest of this block
# guest token (enough for /me and /estimate): no Authorization header on this one call
TOKEN=$(curl -sS -X POST "$BASE/guest" \
-H "Content-Type: application/json" \
-d '{"slug":"hermetic-build-audit"}' | jq -r '.data.token')
echo "guest token: ${TOKEN:0:12}..."
# personal token (needed for a run): set SKILLSAFE_TOKEN (from /tokens.html) and skip this
if TOKEN in ("", "YOUR_TOKEN"):
TOKEN = "" # no Authorization header on /guest
guest = api("POST", "/guest", {"slug": "hermetic-build-audit"})
TOKEN = guest["token"]
print("guest", guest.get("guest_id"))
// personal token (needed for a run): paste it from /tokens.html into TOKEN (step 1) and skip this
if (!token || token === "YOUR_TOKEN") {
token = ""; // no Authorization header on /guest
const guest = await api("POST", "/guest", { slug: "hermetic-build-audit" });
token = guest.token;
console.log("guest", guest.guest_id);
}
// inside main -- personal token (needed for a run): set SKILLSAFE_TOKEN (from /tokens.html) and skip this
if token == "" {
var guest struct {
Token string `json:"token"`
GuestID string `json:"guest_id"`
}
if err := api("POST", "/guest", map[string]string{"slug": "hermetic-build-audit"}, nil, &guest); err != nil {
panic(err)
}
token = guest.Token
fmt.Println("guest", guest.GuestID)
}
// inside main -- personal token (needed for a run): set SKILLSAFE_TOKEN (from /tokens.html) and skip this
if (token.isEmpty() || token.equals("YOUR_TOKEN")) {
token = ""; // no Authorization header on /guest
JsonNode guest = api("POST", "/guest", Map.of("slug", "hermetic-build-audit"), null);
token = guest.path("token").asText();
System.out.println("guest " + guest.path("guest_id").asText());
}
# personal token (needed for a run): set SKILLSAFE_TOKEN (from /tokens.html) and skip this
if $token.empty? || $token == "YOUR_TOKEN"
$token = "" # no Authorization header on /guest
guest = api("POST", "/guest", { slug: "hermetic-build-audit" })
$token = guest["token"]
puts "guest #{guest["guest_id"]}"
end
// personal token (needed for a run): set SKILLSAFE_TOKEN (from /tokens.html) and skip this
if ($TOKEN === "" || $TOKEN === "YOUR_TOKEN") {
$TOKEN = ""; // no Authorization header on /guest
$guest = api("POST", "/guest", ["slug" => "hermetic-build-audit"]);
$TOKEN = $guest->token;
echo "guest {$guest->guest_id}\n";
}
// top-level statements (above the Api class) -- personal token (needed for a run): set SKILLSAFE_TOKEN (from /tokens.html) and skip this
if (Api.Token == "" || Api.Token == "YOUR_TOKEN")
{
Api.Token = ""; // no Authorization header on /guest
var guest = await Api.Call("POST", "/guest", new { slug = "hermetic-build-audit" });
Api.Token = guest!["token"]!.GetValue<string>();
Console.WriteLine($"guest {guest["guest_id"]}");
}
Step 3 — GET /me
Who the token belongs to and what it can spend. sdk.js documents the shape as
{subject_type, subject_id, credits, profile?}. The page treats the caller as signed
in only when subject_type is "user", shows credits as the
balance (1 credit = $0.0001, so it divides by 10 000 for the dollar figure), and compares it
with the estimate in step 4 before it lets you run.
api GET /me | jq '.data // .error'
me = api("GET", "/me")
print(me["subject_type"], me.get("subject_id"), me.get("credits"))
signed_in = me["subject_type"] == "user"
const me = await api("GET", "/me");
console.log(me.subject_type, me.subject_id, me.credits);
const signedIn = me.subject_type === "user";
var me struct {
SubjectType string `json:"subject_type"`
SubjectID any `json:"subject_id"`
Credits any `json:"credits"`
}
if err := api("GET", "/me", nil, nil, &me); err != nil {
panic(err)
}
fmt.Println(me.SubjectType, me.SubjectID, me.Credits)
signedIn := me.SubjectType == "user"
_ = signedIn
JsonNode me = api("GET", "/me", null, null);
System.out.println(me.path("subject_type").asText() + " " + me.path("subject_id").asText() + " " + me.path("credits"));
boolean signedIn = me.path("subject_type").asText().equals("user");
me = api("GET", "/me")
puts [me["subject_type"], me["subject_id"], me["credits"]].join(" ")
signed_in = me["subject_type"] == "user"
$me = api("GET", "/me");
echo $me->subject_type, " ", $me->subject_id ?? "", " ", $me->credits ?? "", "\n";
$signedIn = $me->subject_type === "user";
var me = await Api.Call("GET", "/me");
Console.WriteLine($"{me?["subject_type"]} {me?["subject_id"]} {me?["credits"]}");
var signedIn = me?["subject_type"]?.GetValue<string>() == "user";
Step 4 — build the input, then POST /estimate
First a small makeInput that does what buildInput does: task,
then code as the paste the page would send, facts as a compact JSON string,
the note when it is not blank, and — for the fix lane only — the
audit_handoff string when the file carries one. It reads the file
build-check.js (or handoff.js) printed. The samples build the
audit input of the monorepo example from monorepo-check.json; step 6 reuses
makeInput for the vendor (fix) example. Then price it. The body of
/estimate is the same input you will send to /run; no job is created and
nothing is charged — /estimate is free. From the response the page reads:
| Field | Meaning |
|---|---|
hold_credits | A reservation, not a price: the worst case the platform holds while the run is in flight. The page shows it as “reserves N credits ($...) for the audit run” and adds that you are charged only for what the run actually uses, usually far less. What you pay is the job’s charged_credits after settlement (step 5). A balance below the hold is allowed, but the page warns that the reply may be cut short (truncated: true). |
min_credits | The smallest balance that can start a run. Below it the page disables the run button and names the shortfall, because the call would be a 402. |
sponsor_enabled | When true, a caller who is not signed in may run; otherwise the page asks them to sign in. |
warnings | Input warnings. They do not refuse the call; the page shows them next to the price (“input warnings: ...”), so check them before you run. |
The page re-prices whenever the paste, the note or the lane change. The amounts grow with the size
of code and facts, so read them from the call rather than hard-coding them.
# monorepo-check.json: printed by build-check.js (see "The facts string").
# (.facts | tojson) turns the facts into ONE compact string, as JSON.stringify does.
BODY=$(jq -c --arg note "Laptops and CI almost never share cache hits. CI runs bazel build //... --config=ci and must keep writing the remote cache; laptops should only read it. --spawn_strategy=local was added months ago while debugging one flaky test and nobody remembers why. The --stamp timestamp is only needed for release builds. What do we fix first?" \
'{task: "audit", code, facts: (.facts | tojson), note: $note}' monorepo-check.json)
# No page check at hand? facts: "{}" is accepted as a string, but the reply then has no page flags to answer.
# Fix lane after an audit: read handoff.js's output instead and add its string:
# '{task: "fix", code, facts: (.facts | tojson), note: $note} + (if .audit_handoff then {audit_handoff} else {} end)'
api POST /estimate -d "$BODY" \
| jq '.error // (.data | {hold_credits, min_credits, sponsor_enabled, warnings})'
def make_input(task, check_path, note=None):
"""The body the page sends: task, code, facts as a JSON STRING, an optional note, and
(fix lane only) the audit_handoff string -- every value a string."""
with open(check_path, encoding="utf-8") as f:
check = json.load(f) # build-check.js (or handoff.js) output
inp = {
"task": task,
"code": check["code"], # the paste as the page sends it
"facts": json.dumps(check["facts"], separators=(",", ":"), ensure_ascii=False), # a STRING, like JSON.stringify
}
n = (note or "").strip()
if n:
inp["note"] = n[:2000] # optional; the page caps it at 2000 chars
if task == "fix" and check.get("audit_handoff"):
inp["audit_handoff"] = check["audit_handoff"] # optional; already a JSON string
return inp
inp = make_input("audit", "monorepo-check.json",
"Laptops and CI almost never share cache hits. CI runs bazel build //... --config=ci and must keep writing the remote cache; laptops should only read it. --spawn_strategy=local was added months ago while debugging one flaky test and nobody remembers why. The --stamp timestamp is only needed for release builds. What do we fix first?")
est = api("POST", "/estimate", inp)
print("hold", est.get("hold_credits"), "min", est.get("min_credits"),
"sponsored", est.get("sponsor_enabled"), "warnings", est.get("warnings") or [])
// The body the page sends: task, code, facts as a JSON STRING, an optional note,
// and (fix lane only) the audit_handoff string -- every value a string.
function makeInput(task, checkPath, note) {
const check = JSON.parse(readFileSync(checkPath, "utf8")); // build-check.js (or handoff.js) output
const input = {
task,
code: check.code, // the paste as the page sends it
facts: JSON.stringify(check.facts), // a STRING: the page's free check
};
const n = (note || "").trim();
if (n) input.note = n.length > 2000 ? n.slice(0, 2000) : n; // optional; max 2000 chars
if (task === "fix" && check.audit_handoff) input.audit_handoff = check.audit_handoff; // optional
return input;
}
const input = makeInput("audit", "monorepo-check.json",
"Laptops and CI almost never share cache hits. CI runs bazel build //... --config=ci and must keep writing the remote cache; laptops should only read it. --spawn_strategy=local was added months ago while debugging one flaky test and nobody remembers why. The --stamp timestamp is only needed for release builds. What do we fix first?");
const est = await api("POST", "/estimate", input);
console.log("hold", est.hold_credits, "min", est.min_credits,
"sponsored", est.sponsor_enabled, "warnings", est.warnings || []);
// makeInput builds the body the page sends: task, code, facts as a JSON STRING,
// an optional note and (fix lane only) the audit_handoff string.
makeInput := func(task, checkPath, note string) map[string]any {
raw, err := os.ReadFile(checkPath) // build-check.js (or handoff.js) output
if err != nil {
panic(err)
}
var check struct {
Code string `json:"code"`
Facts json.RawMessage `json:"facts"`
AuditHandoff string `json:"audit_handoff"`
}
if err := json.Unmarshal(raw, &check); err != nil {
panic(err)
}
var facts bytes.Buffer
if err := json.Compact(&facts, check.Facts); err != nil { // keeps key order; travels as a STRING
panic(err)
}
in := map[string]any{"task": task, "code": check.Code, "facts": facts.String()}
if n := []rune(strings.TrimSpace(note)); len(n) > 0 {
if len(n) > 2000 {
n = n[:2000] // optional; the page caps it at 2000 chars
}
in["note"] = string(n)
}
if task == "fix" && check.AuditHandoff != "" {
in["audit_handoff"] = check.AuditHandoff // optional; already a JSON string
}
return in
}
input := makeInput("audit", "monorepo-check.json",
"Laptops and CI almost never share cache hits. CI runs bazel build //... --config=ci and must keep writing the remote cache; laptops should only read it. --spawn_strategy=local was added months ago while debugging one flaky test and nobody remembers why. The --stamp timestamp is only needed for release builds. What do we fix first?")
var est struct {
HoldCredits any `json:"hold_credits"`
MinCredits any `json:"min_credits"`
SponsorEnabled bool `json:"sponsor_enabled"`
Warnings []any `json:"warnings"`
}
if err := api("POST", "/estimate", input, nil, &est); err != nil {
panic(err)
}
fmt.Printf("hold %v min %v sponsored %v warnings %v\n", est.HoldCredits, est.MinCredits, est.SponsorEnabled, est.Warnings)
// ---- add to the HermeticBuildAudit class, next to api(): the body the page sends
static Map<String, Object> makeInput(String task, String checkPath, String note) throws Exception {
JsonNode check = JSON.readTree(Files.readString(Path.of(checkPath))); // build-check.js (or handoff.js) output
Map<String, Object> in = new LinkedHashMap<>();
in.put("task", task);
in.put("code", check.path("code").asText()); // the paste as the page sends it
in.put("facts", JSON.writeValueAsString(check.path("facts"))); // a STRING, compact
String n = note == null ? "" : note.strip();
if (!n.isEmpty()) in.put("note", n.length() > 2000 ? n.substring(0, 2000) : n); // optional; max 2000 chars
if (task.equals("fix") && check.hasNonNull("audit_handoff")) in.put("audit_handoff", check.get("audit_handoff").asText());
return in;
}
// ---- inside main
Map<String, Object> input = makeInput("audit", "monorepo-check.json",
"Laptops and CI almost never share cache hits. CI runs bazel build //... --config=ci and must keep writing the remote cache; laptops should only read it. --spawn_strategy=local was added months ago while debugging one flaky test and nobody remembers why. The --stamp timestamp is only needed for release builds. What do we fix first?");
JsonNode est = api("POST", "/estimate", input, null);
System.out.println("hold " + est.path("hold_credits") + " min " + est.path("min_credits")
+ " sponsored " + est.path("sponsor_enabled").asBoolean(false) + " warnings " + est.path("warnings"));
# The body the page sends: task, code, facts as a JSON STRING, an optional note,
# and (fix lane only) the audit_handoff string -- every value a string.
def make_input(task, check_path, note = nil)
check = JSON.parse(File.read(check_path, encoding: "UTF-8")) # build-check.js (or handoff.js) output
input = {
task: task,
code: check["code"], # the paste as the page sends it
facts: JSON.generate(check["facts"]) # a STRING, compact
}
n = note.to_s.strip
input[:note] = n[0, 2000] unless n.empty? # optional; max 2000 chars
input[:audit_handoff] = check["audit_handoff"] if task == "fix" && !check["audit_handoff"].to_s.empty?
input
end
input = make_input("audit", "monorepo-check.json",
"Laptops and CI almost never share cache hits. CI runs bazel build //... --config=ci and must keep writing the remote cache; laptops should only read it. --spawn_strategy=local was added months ago while debugging one flaky test and nobody remembers why. The --stamp timestamp is only needed for release builds. What do we fix first?")
est = api("POST", "/estimate", input)
puts "hold #{est["hold_credits"]} min #{est["min_credits"]} " \
"sponsored #{est["sponsor_enabled"]} warnings #{(est["warnings"] || []).inspect}"
// The body the page sends: task, code, facts as a JSON STRING, an optional note,
// and (fix lane only) the audit_handoff string -- every value a string.
function make_input(string $task, string $checkPath, string $note = ""): array {
// decode to objects (not arrays) so an empty {} stays {} when re-encoded
$check = json_decode(file_get_contents($checkPath)); // build-check.js (or handoff.js) output
$input = [
"task" => $task,
"code" => $check->code, // the paste as the page sends it
"facts" => json_encode($check->facts, JSON_UNESCAPED_SLASHES | JSON_UNESCAPED_UNICODE), // a STRING
];
$n = trim($note);
if ($n !== "") $input["note"] = mb_substr($n, 0, 2000); // optional; max 2000 chars
if ($task === "fix" && !empty($check->audit_handoff)) $input["audit_handoff"] = $check->audit_handoff;
return $input;
}
$input = make_input("audit", "monorepo-check.json",
"Laptops and CI almost never share cache hits. CI runs bazel build //... --config=ci and must keep writing the remote cache; laptops should only read it. --spawn_strategy=local was added months ago while debugging one flaky test and nobody remembers why. The --stamp timestamp is only needed for release builds. What do we fix first?");
$est = api("POST", "/estimate", $input);
echo "hold ", $est->hold_credits ?? "-", " min ", $est->min_credits ?? "-",
" sponsored ", json_encode($est->sponsor_enabled ?? false),
" warnings ", json_encode($est->warnings ?? []), "\n";
// The body the page sends: task, code, facts as a JSON STRING, an optional note,
// and (fix lane only) the audit_handoff string -- every value a string.
static Dictionary<string, string> MakeInput(string task, string checkPath, string? note = null)
{
var compact = new JsonSerializerOptions { Encoder = System.Text.Encodings.Web.JavaScriptEncoder.UnsafeRelaxedJsonEscaping };
var check = JsonNode.Parse(File.ReadAllText(checkPath))!; // build-check.js (or handoff.js) output
var input = new Dictionary<string, string>
{
["task"] = task,
["code"] = check["code"]!.GetValue<string>(), // the paste as the page sends it
["facts"] = check["facts"]!.ToJsonString(compact), // a STRING, compact
};
var n = (note ?? "").Trim();
if (n.Length > 0) input["note"] = n.Length > 2000 ? n[..2000] : n; // optional; max 2000 chars
var handoff = check["audit_handoff"]?.GetValue<string>();
if (task == "fix" && !string.IsNullOrEmpty(handoff)) input["audit_handoff"] = handoff;
return input;
}
var input = MakeInput("audit", "monorepo-check.json",
"Laptops and CI almost never share cache hits. CI runs bazel build //... --config=ci and must keep writing the remote cache; laptops should only read it. --spawn_strategy=local was added months ago while debugging one flaky test and nobody remembers why. The --stamp timestamp is only needed for release builds. What do we fix first?");
var est = await Api.Call("POST", "/estimate", input);
Console.WriteLine($"hold {est?["hold_credits"]} min {est?["min_credits"]} " +
$"sponsored {est?["sponsor_enabled"]} warnings {est?["warnings"]?.ToJsonString() ?? "[]"}");
Step 5 — POST /run, then poll the job (audit)
Send the same input to /run with an Idempotency-Key header, using a
personal token. Its data carries a job_id. Poll
GET /jobs/{job_id} the way sdk.js waitForJob does: every
second until status is succeeded or failed, giving up on
the client side after 180 seconds. On the finished job, read:
output.output— the model’s reply as a string: the JSON object from the output contract of the lane you asked for. Parse it yourself (from the first{to the last}). The samples print the verdict and headline, each area’s status, each finding’sid,file,line,severityand number of edits, then each flag answer. Keep the text:recheck.jsreads it.truncated—truewhen the reply was cut short because the balance ran out. The page then closes the cut-off JSON, shows what arrived as a partial result (“N of 9 sections recovered” for an audit, of 8 for a fix run), and says it was cut short by the available balance.charged_credits— the real cost of the run, as opposed to the estimate’shold_creditsreservation. The page re-reads/meafterwards for the new balance.error— on afailedjob.
Idempotency key. Resending a request with the same key must not start a second
paid run, so derive the key from the input, not from the clock — and include the lane, so an
audit and a fix run of the same paste never share a key. The page uses
hermetic-build-audit:<lane>:<inputHash>:a<attempt> (app.js
run), where <lane> is audit or fix,
<inputHash> is a 32-bit djb2 hash of JSON.stringify(input) in hex, a
dash, and the string’s length in hex (app.js inputHash, over the
input without retry_note), and <attempt> is 1
for a normal run and 2 for the automatic reformat retry (which also adds
retry_note). For the two worked examples that is
hermetic-build-audit:audit:1e712138-26f6:a1 (monorepo audit) and hermetic-build-audit:fix:212354fb-1d96:a1
(vendor fix); the fix run of the monorepo audit handed on (below) is hermetic-build-audit:fix:560de5de-2db6:a1. The format is the page’s
convention, not a requirement: any stable, unique string does. The samples below use the lane and a
SHA-256 prefix.
The page itself uses /run-stream (step 6), not /run; both take the
same body and header.
KEY="hermetic-build-audit:audit:$(printf '%s' "$BODY" | shasum -a 256 | cut -c1-16):a1"
JOB=$(api POST /run -H "Idempotency-Key: $KEY" -d "$BODY" | jq -r '.data.job_id')
for i in $(seq 180); do # 1 s interval, 180 s client-side limit
J=$(api GET "/jobs/$JOB")
S=$(printf '%s' "$J" | jq -r '.data.status')
if [ "$S" = "succeeded" ] || [ "$S" = "failed" ]; then break; fi
sleep 1
done
printf '%s' "$J" | jq '.data | {status, truncated, charged_credits, error}'
printf '%s' "$J" | jq -r '.data.output.output' \
| jq '{verdict, headline, areas: [(.areas // [])[] | {area, status}],
findings: [(.findings // [])[] | {id, file, line, severity, edits: ((.edits // []) | length)}],
flag_answers: [(.flag_answers // [])[] | {flag, status}]}'
body = json.dumps(inp)
key = "hermetic-build-audit:audit:" + hashlib.sha256(body.encode()).hexdigest()[:16] + ":a1"
job_id = api("POST", "/run", inp, {"Idempotency-Key": key})["job_id"]
deadline = time.time() + 180 # 1 s interval, 180 s client-side limit
while True:
job = api("GET", "/jobs/" + urllib.parse.quote(job_id, safe=""))
if job["status"] in ("succeeded", "failed"):
break
if time.time() > deadline:
raise TimeoutError("job timed out")
time.sleep(1)
if job["status"] == "failed":
raise SystemExit(f"run failed: {job.get('error')}")
print("truncated:", job.get("truncated") is True, "charged:", job.get("charged_credits"))
text = (job.get("output") or {}).get("output") or ""
reply = json.loads(text[text.index("{"): text.rindex("}") + 1])
print(reply["verdict"], "-", reply["headline"])
for s in reply.get("areas", []):
print("area", s["area"], s["status"])
for f in reply.get("findings", []):
print(f["id"], f["file"], f.get("line"), f["severity"], len(f.get("edits") or []), "edit(s)")
for a in reply.get("flag_answers", []):
print("answer", a["flag"], a["status"])
const key = "hermetic-build-audit:audit:" +
createHash("sha256").update(JSON.stringify(input)).digest("hex").slice(0, 16) + ":a1";
const { job_id } = await api("POST", "/run", input, { "Idempotency-Key": key });
let job;
const deadline = Date.now() + 180000; // 1 s interval, 180 s client-side limit
for (;;) {
job = await api("GET", "/jobs/" + encodeURIComponent(job_id));
if (job.status === "succeeded" || job.status === "failed") break;
if (Date.now() > deadline) throw new Error("job timed out");
await new Promise((r) => setTimeout(r, 1000));
}
if (job.status === "failed") throw new Error("run failed: " + JSON.stringify(job.error));
console.log("truncated:", job.truncated === true, "charged:", job.charged_credits);
const text = (job.output && job.output.output) || "";
const reply = JSON.parse(text.slice(text.indexOf("{"), text.lastIndexOf("}") + 1));
console.log(reply.verdict, "-", reply.headline);
for (const s of reply.areas || []) console.log("area", s.area, s.status);
for (const f of reply.findings || []) console.log(f.id, f.file, f.line, f.severity, (f.edits || []).length, "edit(s)");
for (const a of reply.flag_answers || []) console.log("answer", a.flag, a.status);
inputJSON, _ := json.Marshal(input)
sum := sha256.Sum256(inputJSON)
key := "hermetic-build-audit:audit:" + hex.EncodeToString(sum[:8]) + ":a1"
var started struct {
JobID string `json:"job_id"`
}
if err := api("POST", "/run", input, map[string]string{"Idempotency-Key": key}, &started); err != nil {
panic(err)
}
type jobT struct {
Status string `json:"status"`
Output *struct {
Output string `json:"output"`
} `json:"output"`
Truncated bool `json:"truncated"`
ChargedCredits any `json:"charged_credits"`
Error any `json:"error"`
}
var job jobT
deadline := time.Now().Add(180 * time.Second) // 1 s interval, 180 s client-side limit
for {
job = jobT{}
if err := api("GET", "/jobs/"+url.PathEscape(started.JobID), nil, nil, &job); err != nil {
panic(err)
}
if job.Status == "succeeded" || job.Status == "failed" {
break
}
if time.Now().After(deadline) {
panic("job timed out")
}
time.Sleep(time.Second)
}
if job.Status == "failed" {
panic(fmt.Sprintf("run failed: %v", job.Error))
}
fmt.Println("truncated:", job.Truncated, "charged:", job.ChargedCredits)
text := ""
if job.Output != nil {
text = job.Output.Output
}
var reply map[string]any
if a, b := strings.Index(text, "{"), strings.LastIndex(text, "}"); a >= 0 && b > a {
if err := json.Unmarshal([]byte(text[a:b+1]), &reply); err != nil {
panic(err)
}
}
fmt.Println(reply["verdict"], "-", reply["headline"])
areas, _ := reply["areas"].([]any)
for _, e := range areas {
s, _ := e.(map[string]any)
fmt.Println("area", s["area"], s["status"])
}
findings, _ := reply["findings"].([]any)
for _, e := range findings {
f, _ := e.(map[string]any)
edits, _ := f["edits"].([]any)
fmt.Println(f["id"], f["file"], f["line"], f["severity"], len(edits), "edit(s)")
}
answers, _ := reply["flag_answers"].([]any)
for _, e := range answers {
a, _ := e.(map[string]any)
fmt.Println("answer", a["flag"], a["status"])
}
byte[] digest = MessageDigest.getInstance("SHA-256").digest(JSON.writeValueAsBytes(input));
String key = "hermetic-build-audit:audit:" + HexFormat.of().formatHex(digest, 0, 8) + ":a1";
String jobId = api("POST", "/run", input, Map.of("Idempotency-Key", key)).path("job_id").asText();
JsonNode job;
long deadline = System.currentTimeMillis() + 180_000; // 1 s interval, 180 s client-side limit
while (true) {
job = api("GET", "/jobs/" + URLEncoder.encode(jobId, StandardCharsets.UTF_8), null, null);
String s = job.path("status").asText();
if (s.equals("succeeded") || s.equals("failed")) break;
if (System.currentTimeMillis() > deadline) throw new RuntimeException("job timed out");
Thread.sleep(1000);
}
if (job.path("status").asText().equals("failed")) throw new RuntimeException("run failed: " + job.path("error"));
System.out.println("truncated: " + job.path("truncated").asBoolean(false) + ", charged: " + job.path("charged_credits"));
String text = job.path("output").path("output").asText("");
JsonNode reply = JSON.readTree(text.substring(text.indexOf('{'), text.lastIndexOf('}') + 1));
System.out.println(reply.path("verdict").asText() + " - " + reply.path("headline").asText());
for (JsonNode s : reply.path("areas")) {
System.out.println("area " + s.path("area").asText() + " " + s.path("status").asText());
}
for (JsonNode f : reply.path("findings")) {
System.out.println(f.path("id").asText() + " " + f.path("file").asText() + " " + f.path("line") + " "
+ f.path("severity").asText() + " " + f.path("edits").size() + " edit(s)");
}
for (JsonNode a : reply.path("flag_answers")) {
System.out.println("answer " + a.path("flag").asText() + " " + a.path("status").asText());
}
key = "hermetic-build-audit:audit:#{Digest::SHA256.hexdigest(JSON.generate(input))[0, 16]}:a1"
job_id = api("POST", "/run", input, { "Idempotency-Key" => key })["job_id"]
deadline = Time.now + 180 # 1 s interval, 180 s client-side limit
job = nil
loop do
job = api("GET", "/jobs/#{URI.encode_www_form_component(job_id)}")
break if %w[succeeded failed].include?(job["status"])
raise "job timed out" if Time.now > deadline
sleep 1
end
raise "run failed: #{job["error"].inspect}" if job["status"] == "failed"
puts "truncated: #{job["truncated"] == true}, charged: #{job["charged_credits"]}"
text = (job["output"] || {})["output"].to_s
reply = JSON.parse(text[text.index("{")..text.rindex("}")])
puts "#{reply["verdict"]} - #{reply["headline"]}"
(reply["areas"] || []).each { |s| puts "area #{s["area"]} #{s["status"]}" }
(reply["findings"] || []).each { |f| puts "#{f["id"]} #{f["file"]} #{f["line"].inspect} #{f["severity"]} #{(f["edits"] || []).size} edit(s)" }
(reply["flag_answers"] || []).each { |a| puts "answer #{a["flag"]} #{a["status"]}" }
$key = "hermetic-build-audit:audit:" . substr(hash("sha256", json_encode($input)), 0, 16) . ":a1";
$jobId = api("POST", "/run", $input, ["Idempotency-Key" => $key])->job_id;
$deadline = time() + 180; // 1 s interval, 180 s client-side limit
while (true) {
$job = api("GET", "/jobs/" . rawurlencode($jobId));
if (in_array($job->status, ["succeeded", "failed"], true)) break;
if (time() > $deadline) throw new RuntimeException("job timed out");
sleep(1);
}
if ($job->status === "failed") throw new RuntimeException("run failed: " . json_encode($job->error ?? null));
echo "truncated: ", json_encode(($job->truncated ?? false) === true), ", charged: ", $job->charged_credits ?? "-", "\n";
$text = $job->output->output ?? "";
$reply = json_decode(substr($text, strpos($text, "{"), strrpos($text, "}") - strpos($text, "{") + 1));
echo $reply->verdict, " - ", $reply->headline, "\n";
foreach ($reply->areas ?? [] as $s) echo "area ", $s->area, " ", $s->status, "\n";
foreach ($reply->findings ?? [] as $f) echo $f->id, " ", $f->file, " ", json_encode($f->line ?? null), " ", $f->severity, " ", count($f->edits ?? []), " edit(s)\n";
foreach ($reply->flag_answers ?? [] as $a) echo "answer ", $a->flag, " ", $a->status, "\n";
var digest = System.Security.Cryptography.SHA256.HashData(Encoding.UTF8.GetBytes(JsonSerializer.Serialize(input)));
var key = "hermetic-build-audit:audit:" + Convert.ToHexString(digest, 0, 8).ToLowerInvariant() + ":a1";
var started = await Api.Call("POST", "/run", input, new Dictionary<string, string> { ["Idempotency-Key"] = key });
var jobId = started!["job_id"]!.GetValue<string>();
JsonNode job;
var deadline = DateTime.UtcNow.AddSeconds(180); // 1 s interval, 180 s client-side limit
while (true)
{
job = (await Api.Call("GET", "/jobs/" + Uri.EscapeDataString(jobId)))!;
var s = job["status"]?.GetValue<string>();
if (s == "succeeded" || s == "failed") break;
if (DateTime.UtcNow > deadline) throw new TimeoutException("job timed out");
await Task.Delay(1000);
}
if (job["status"]?.GetValue<string>() == "failed") throw new Exception($"run failed: {job["error"]?.ToJsonString()}");
var truncated = job["truncated"] is JsonValue t && t.TryGetValue(out bool tb) && tb;
Console.WriteLine($"truncated: {truncated}, charged: {job["charged_credits"]}");
var text = job["output"]?["output"]?.GetValue<string>() ?? "";
var reply = JsonNode.Parse(text[text.IndexOf('{')..(text.LastIndexOf('}') + 1)])!;
Console.WriteLine($"{reply["verdict"]} - {reply["headline"]}");
foreach (var s in reply["areas"]?.AsArray() ?? new JsonArray())
Console.WriteLine($"area {s?["area"]} {s?["status"]}");
foreach (var f in reply["findings"]?.AsArray() ?? new JsonArray())
Console.WriteLine($"{f?["id"]} {f?["file"]} {f?["line"]?.ToJsonString() ?? "null"} {f?["severity"]} {f?["edits"]?.AsArray().Count ?? 0} edit(s)");
foreach (var a in reply["flag_answers"]?.AsArray() ?? new JsonArray())
Console.WriteLine($"answer {a?["flag"]} {a?["status"]}");
Step 6 — POST /run-stream (Server-Sent Events, fix)
The alternative to step 5, and what the page uses: same kind of body, same
Idempotency-Key header, one request. Run one or the other for a given input, not
both. To show the second lane, the samples build the fix input of the
vendor example (with its own key) and stream it; to stream the audit input instead, pass the
step 4 input and key. The response is text/event-stream. sdk.js splits
it into frames on a blank line, takes the event name from an event: line (default
message) and concatenates the data: lines, skips frames with no data or
with data that is not JSON, and handles these events:
| Event | Data | What the SDK does |
|---|---|---|
delta | {text} | Passes text to onDelta. |
job | job info | Passes it to onJob (the page uses it to advance its progress display). |
done | the final payload: {job_id, status, charged_credits, output} | Keeps it as the result. |
pending | a payload | Treated exactly like done. If it carries a job_id and no output, poll that job as in step 5. |
error | {message, code, job_id} | After the stream ends, raises an error with those fields. |
If the response is not text/event-stream, it is an ordinary JSON envelope:
the SDK returns its data (an idempotent replay returns this way) or raises the
error on a non-2xx status. Handle both shapes.
From the final payload the page reads output.output (falling back to the
concatenated delta text), truncated and charged_credits.
Browsers receive progress ticks rather than deltas from this endpoint, so the page relies on the
final payload; if the stream fails part-way, it tries to recover a partial reply from whatever
delta text arrived. The samples print the verdict and headline, each returned file’s path and
size, each change (target, flags and change), each
verification step, each open item and each question. To put the files into the paste and re-check
them the way the page’s “Put every file into the paste” does, keep the reply text
and run recheck.js with the lane fix. To fix code you audited, read handoff.js’s output instead of
vendor-check.json: makeInput then carries its audit_handoff.
# the fix worked example: vendor-check.json from build-check.js
# (for audited code, read handoff.js's output instead; the + (...) adds its audit_handoff)
FBODY=$(jq -c --arg note "Fix what you can. geo_math is vendored at third_party/geo-math in this repository now. The tested commit of shape_codec is 3f9c2e1d4b6a8c0e2f4a6b8d0c2e4f6a8b0d2c4e. The cache also serves grpcs on bazel-cache.corp.lan:9092. We do not have the tzdata sha256 yet and the fonts zip has no published checksum - leave those two for us. The three runs per test were only meant for the geometry tests we suspect are flaky." \
'{task: "fix", code, facts: (.facts | tojson), note: $note} + (if .audit_handoff then {audit_handoff} else {} end)' \
vendor-check.json)
FKEY="hermetic-build-audit:fix:$(printf '%s' "$FBODY" | shasum -a 256 | cut -c1-16):a1"
# -N turns off buffering; the whole stream lands in $STREAM
STREAM=$(api POST /run-stream -N -H "Idempotency-Key: $FKEY" -d "$FBODY")
# frames look like:
# event: job
# data: {...}
#
# event: done
# data: {"job_id":"...","status":"...","charged_credits":...,"output":{"output":"{\"task\":\"fix\",...}"}}
# the final reply text, from the done (or pending) frame:
REPLY=$(printf '%s\n' "$STREAM" \
| awk '/^event:/{ev=$2} /^data:/ && (ev=="done" || ev=="pending"){sub(/^data: */,""); print}' \
| jq -r '.output.output')
# an idempotent replay answers with a plain JSON envelope instead: REPLY=$(printf '%s' "$STREAM" | jq -r '.data.output.output')
printf '%s' "$REPLY" | jq '{verdict, headline,
files: [(.files // [])[] | {path, characters: (.content | length)}],
changes: [(.changes // [])[] | {target, flags, change}], verify, open, questions}'
# the fix worked example: vendor-check.json from build-check.js
# (for audited code, pass handoff.js's output instead: make_input adds its audit_handoff)
fix_inp = make_input("fix", "vendor-check.json",
"Fix what you can. geo_math is vendored at third_party/geo-math in this repository now. The tested commit of shape_codec is 3f9c2e1d4b6a8c0e2f4a6b8d0c2e4f6a8b0d2c4e. The cache also serves grpcs on bazel-cache.corp.lan:9092. We do not have the tzdata sha256 yet and the fonts zip has no published checksum - leave those two for us. The three runs per test were only meant for the geometry tests we suspect are flaky.")
fix_key = "hermetic-build-audit:fix:" + hashlib.sha256(json.dumps(fix_inp).encode()).hexdigest()[:16] + ":a1"
h = {"Content-Type": "application/json", "Authorization": "Bearer " + TOKEN, "Idempotency-Key": fix_key}
done, failure, streamed = None, None, ""
with requests.post(BASE + "/run-stream", headers=h, data=json.dumps(fix_inp), stream=True) as r:
if "text/event-stream" not in r.headers.get("content-type", ""):
try:
env = r.json()
except ValueError:
env = {}
if not r.ok:
err = env.get("error") or {}
raise ApiError(r.status_code, err.get("code"), err.get("message") or r.reason)
done = env.get("data")
else:
event, data = "message", ""
for line in r.iter_lines(decode_unicode=True):
if line == "": # a blank line ends a frame
if data:
try:
payload = json.loads(data)
except ValueError:
payload = None
if event == "delta" and payload is not None:
streamed += payload.get("text") or ""
elif event == "job":
print("job", payload)
elif event in ("done", "pending"):
done = payload
elif event == "error":
failure = payload
event, data = "message", ""
elif line.startswith("event:"):
event = line[6:].strip()
elif line.startswith("data:"):
data += line[5:].strip()
if failure:
raise SystemExit(f"{failure.get('code')}: {failure.get('message')} (job {failure.get('job_id')})")
done = done or {}
full = (done.get("output") or {}).get("output") or streamed
print("truncated:", done.get("truncated") is True, "charged:", done.get("charged_credits"))
result = json.loads(full[full.index("{"): full.rindex("}") + 1])
print(result["verdict"], "-", result["headline"])
for f in result.get("files", []):
print("file", f["path"], len(f.get("content") or ""), "characters")
for c in result.get("changes", []):
print("change", c.get("target"), ",".join(c.get("flags") or []) or "-", "-", c.get("change"))
for step in result.get("verify", []):
print("verify", step)
for o in result.get("open", []):
print("open", o["item"], "-", o.get("reason"))
for q in result.get("questions", []):
print("question", q)
// the fix worked example: vendor-check.json from build-check.js
// (for audited code, pass handoff.js's output instead: makeInput adds its audit_handoff)
const fixInput = makeInput("fix", "vendor-check.json",
"Fix what you can. geo_math is vendored at third_party/geo-math in this repository now. The tested commit of shape_codec is 3f9c2e1d4b6a8c0e2f4a6b8d0c2e4f6a8b0d2c4e. The cache also serves grpcs on bazel-cache.corp.lan:9092. We do not have the tzdata sha256 yet and the fonts zip has no published checksum - leave those two for us. The three runs per test were only meant for the geometry tests we suspect are flaky.");
const fixKey = "hermetic-build-audit:fix:" +
createHash("sha256").update(JSON.stringify(fixInput)).digest("hex").slice(0, 16) + ":a1";
const res = await fetch(BASE + "/run-stream", {
method: "POST",
headers: { "Content-Type": "application/json", Authorization: "Bearer " + token, "Idempotency-Key": fixKey },
body: JSON.stringify(fixInput),
});
let done = null, failure = null, streamed = "";
if (!(res.headers.get("content-type") || "").includes("text/event-stream")) {
const json = await res.json().catch(() => ({}));
if (!res.ok) throw new Error((json.error && json.error.message) || res.statusText);
done = json.data;
} else {
const reader = res.body.getReader();
const decoder = new TextDecoder();
let buffer = "";
for (;;) {
const chunk = await reader.read();
if (chunk.done) break;
buffer += decoder.decode(chunk.value, { stream: true });
let idx;
while ((idx = buffer.indexOf("\n\n")) >= 0) {
const frame = buffer.slice(0, idx);
buffer = buffer.slice(idx + 2);
let event = "message", dataStr = "";
for (const line of frame.split("\n")) {
if (line.startsWith("event:")) event = line.slice(6).trim();
else if (line.startsWith("data:")) dataStr += line.slice(5).trim();
}
if (!dataStr) continue;
let data;
try { data = JSON.parse(dataStr); } catch { continue; }
if (event === "delta") streamed += data.text || "";
else if (event === "job") console.log("job", data);
else if (event === "done" || event === "pending") done = data;
else if (event === "error") failure = data;
}
}
}
if (failure) throw new Error(`${failure.code}: ${failure.message} (job ${failure.job_id})`);
const full = (done && done.output && done.output.output) || streamed;
console.log("truncated:", !!(done && done.truncated), "charged:", done && done.charged_credits);
const result = JSON.parse(full.slice(full.indexOf("{"), full.lastIndexOf("}") + 1));
console.log(result.verdict, "-", result.headline);
for (const f of result.files || []) console.log("file", f.path, (f.content || "").length, "characters");
for (const c of result.changes || []) console.log("change", c.target, (c.flags || []).join(",") || "-", "-", c.change);
for (const step of result.verify || []) console.log("verify", step);
for (const o of result.open || []) console.log("open", o.item, "-", o.reason);
for (const q of result.questions || []) console.log("question", q);
// the fix worked example: vendor-check.json from build-check.js
// (for audited code, pass handoff.js's output instead: makeInput adds its audit_handoff)
fixInput := makeInput("fix", "vendor-check.json",
"Fix what you can. geo_math is vendored at third_party/geo-math in this repository now. The tested commit of shape_codec is 3f9c2e1d4b6a8c0e2f4a6b8d0c2e4f6a8b0d2c4e. The cache also serves grpcs on bazel-cache.corp.lan:9092. We do not have the tzdata sha256 yet and the fonts zip has no published checksum - leave those two for us. The three runs per test were only meant for the geometry tests we suspect are flaky.")
fixJSON, _ := json.Marshal(fixInput)
fixSum := sha256.Sum256(fixJSON)
fixKey := "hermetic-build-audit:fix:" + hex.EncodeToString(fixSum[:8]) + ":a1"
sreq, err := newRequest("POST", "/run-stream", fixInput, map[string]string{"Idempotency-Key": fixKey})
if err != nil {
panic(err)
}
sres, err := http.DefaultClient.Do(sreq)
if err != nil {
panic(err)
}
defer sres.Body.Close()
var done, failure map[string]any
var streamed strings.Builder
if !strings.Contains(sres.Header.Get("Content-Type"), "text/event-stream") {
var env envelope
_ = json.NewDecoder(sres.Body).Decode(&env)
if sres.StatusCode < 200 || sres.StatusCode > 299 {
panic(errorFrom(sres, env))
}
_ = json.Unmarshal(env.Data, &done)
} else {
sc := bufio.NewScanner(sres.Body)
sc.Buffer(make([]byte, 64*1024), 16*1024*1024) // the done frame can be long
event, data := "message", ""
for sc.Scan() {
line := sc.Text()
switch {
case line == "": // a blank line ends a frame
if data != "" {
var p map[string]any
if json.Unmarshal([]byte(data), &p) == nil {
switch event {
case "delta":
if t, ok := p["text"].(string); ok {
streamed.WriteString(t)
}
case "job":
fmt.Println("job", p)
case "done", "pending":
done = p
case "error":
failure = p
}
}
}
event, data = "message", ""
case strings.HasPrefix(line, "event:"):
event = strings.TrimSpace(line[6:])
case strings.HasPrefix(line, "data:"):
data += strings.TrimSpace(line[5:])
}
}
}
if failure != nil {
panic(fmt.Sprintf("%v: %v (job %v)", failure["code"], failure["message"], failure["job_id"]))
}
full := streamed.String()
if out, ok := done["output"].(map[string]any); ok {
if s, ok := out["output"].(string); ok && s != "" {
full = s
}
}
fmt.Println("truncated:", done["truncated"] == true, "charged:", done["charged_credits"])
var result map[string]any
if a, b := strings.Index(full, "{"), strings.LastIndex(full, "}"); a >= 0 && b > a {
if err := json.Unmarshal([]byte(full[a:b+1]), &result); err != nil {
panic(err)
}
}
fmt.Println(result["verdict"], "-", result["headline"])
files, _ := result["files"].([]any)
for _, e := range files {
f, _ := e.(map[string]any)
c, _ := f["content"].(string)
fmt.Println("file", f["path"], len([]rune(c)), "characters")
}
changes, _ := result["changes"].([]any)
for _, e := range changes {
c, _ := e.(map[string]any)
var flags []string
if fs, ok := c["flags"].([]any); ok {
for _, fl := range fs {
flags = append(flags, fmt.Sprint(fl))
}
}
fl := strings.Join(flags, ",")
if fl == "" {
fl = "-"
}
fmt.Println("change", c["target"], fl, "-", c["change"])
}
verify, _ := result["verify"].([]any)
for _, st := range verify {
fmt.Println("verify", st)
}
open, _ := result["open"].([]any)
for _, e := range open {
o, _ := e.(map[string]any)
fmt.Println("open", o["item"], "-", o["reason"])
}
questions, _ := result["questions"].([]any)
for _, q := range questions {
fmt.Println("question", q)
}
// the fix worked example: vendor-check.json from build-check.js
// (for audited code, pass handoff.js's output instead: makeInput adds its audit_handoff)
Map<String, Object> fixInput = makeInput("fix", "vendor-check.json",
"Fix what you can. geo_math is vendored at third_party/geo-math in this repository now. The tested commit of shape_codec is 3f9c2e1d4b6a8c0e2f4a6b8d0c2e4f6a8b0d2c4e. The cache also serves grpcs on bazel-cache.corp.lan:9092. We do not have the tzdata sha256 yet and the fonts zip has no published checksum - leave those two for us. The three runs per test were only meant for the geometry tests we suspect are flaky.");
byte[] fixDigest = MessageDigest.getInstance("SHA-256").digest(JSON.writeValueAsBytes(fixInput));
String fixKey = "hermetic-build-audit:fix:" + HexFormat.of().formatHex(fixDigest, 0, 8) + ":a1";
HttpRequest sreq = request("POST", "/run-stream", fixInput, Map.of("Idempotency-Key", fixKey)).build();
HttpResponse<Stream<String>> sres = HTTP.send(sreq, HttpResponse.BodyHandlers.ofLines());
JsonNode done = null, failure = null;
StringBuilder streamed = new StringBuilder();
if (!sres.headers().firstValue("content-type").orElse("").contains("text/event-stream")) {
JsonNode env = JSON.readTree(sres.body().collect(Collectors.joining("\n")));
if (env == null) env = JSON.createObjectNode();
if (sres.statusCode() / 100 != 2) throw errorFrom(sres.statusCode(), env);
done = env.path("data");
} else {
String event = "message";
StringBuilder data = new StringBuilder();
Iterator<String> it = sres.body().iterator();
while (it.hasNext()) {
String line = it.next();
if (line.isEmpty()) { // a blank line ends a frame
if (data.length() > 0) {
JsonNode p = null;
try { p = JSON.readTree(data.toString()); } catch (Exception ignored) { }
if (p != null) {
switch (event) {
case "delta": streamed.append(p.path("text").asText("")); break;
case "job": System.out.println("job " + p); break;
case "done": case "pending": done = p; break;
case "error": failure = p; break;
default: break;
}
}
}
event = "message";
data.setLength(0);
} else if (line.startsWith("event:")) event = line.substring(6).trim();
else if (line.startsWith("data:")) data.append(line.substring(5).trim());
}
}
if (failure != null) {
throw new RuntimeException(failure.path("code").asText() + ": " + failure.path("message").asText()
+ " (job " + failure.path("job_id").asText() + ")");
}
String full = done == null ? "" : done.path("output").path("output").asText("");
if (full.isEmpty()) full = streamed.toString();
System.out.println("truncated: " + (done != null && done.path("truncated").asBoolean(false))
+ ", charged: " + (done == null ? "-" : done.path("charged_credits").toString()));
JsonNode result = JSON.readTree(full.substring(full.indexOf('{'), full.lastIndexOf('}') + 1));
System.out.println(result.path("verdict").asText() + " - " + result.path("headline").asText());
for (JsonNode f : result.path("files")) {
System.out.println("file " + f.path("path").asText() + " " + f.path("content").asText("").codePointCount(0, f.path("content").asText("").length()) + " characters");
}
for (JsonNode c : result.path("changes")) {
List<String> flags = new ArrayList<>();
c.path("flags").forEach(fl -> flags.add(fl.asText()));
System.out.println("change " + c.path("target").asText() + " " + (flags.isEmpty() ? "-" : String.join(",", flags))
+ " - " + c.path("change").asText());
}
for (JsonNode step : result.path("verify")) System.out.println("verify " + step.asText());
for (JsonNode o : result.path("open")) System.out.println("open " + o.path("item").asText() + " - " + o.path("reason").asText());
for (JsonNode q : result.path("questions")) System.out.println("question " + q.asText());
# the fix worked example: vendor-check.json from build-check.js
# (for audited code, pass handoff.js's output instead: make_input adds its audit_handoff)
fix_input = make_input("fix", "vendor-check.json",
"Fix what you can. geo_math is vendored at third_party/geo-math in this repository now. The tested commit of shape_codec is 3f9c2e1d4b6a8c0e2f4a6b8d0c2e4f6a8b0d2c4e. The cache also serves grpcs on bazel-cache.corp.lan:9092. We do not have the tzdata sha256 yet and the fonts zip has no published checksum - leave those two for us. The three runs per test were only meant for the geometry tests we suspect are flaky.")
fix_key = "hermetic-build-audit:fix:#{Digest::SHA256.hexdigest(JSON.generate(fix_input))[0, 16]}:a1"
uri, req = build_request("POST", "/run-stream", fix_input, { "Idempotency-Key" => fix_key })
done = failure = nil
streamed = +""
Net::HTTP.start(uri.host, uri.port, use_ssl: true, read_timeout: 300) do |http|
http.request(req) do |res|
unless res["content-type"].to_s.include?("text/event-stream")
env = (JSON.parse(res.body) rescue {})
unless res.is_a?(Net::HTTPSuccess)
raise ApiError.new(res.code.to_i, env.dig("error", "code"), env.dig("error", "message") || res.message)
end
done = env["data"]
next
end
buf = +""
res.read_body do |chunk|
buf << chunk
while (i = buf.index("\n\n"))
frame = buf.slice!(0, i + 2)
event = "message"
data = +""
frame.split("\n").each do |line|
if line.start_with?("event:") then event = line[6..].strip
elsif line.start_with?("data:") then data << line[5..].strip
end
end
next if data.empty?
payload = (JSON.parse(data) rescue nil)
next if payload.nil?
case event
when "delta" then streamed << payload["text"].to_s
when "job" then puts "job #{payload.inspect}"
when "done", "pending" then done = payload
when "error" then failure = payload
end
end
end
end
end
raise "#{failure["code"]}: #{failure["message"]} (job #{failure["job_id"]})" if failure
done ||= {}
full = (done["output"] || {})["output"].to_s
full = streamed if full.empty?
puts "truncated: #{done["truncated"] == true}, charged: #{done["charged_credits"]}"
result = JSON.parse(full[full.index("{")..full.rindex("}")])
puts "#{result["verdict"]} - #{result["headline"]}"
(result["files"] || []).each { |f| puts "file #{f["path"]} #{f["content"].to_s.length} characters" }
(result["changes"] || []).each do |c|
flags = (c["flags"] || []).join(",")
puts "change #{c["target"]} #{flags.empty? ? "-" : flags} - #{c["change"]}"
end
(result["verify"] || []).each { |step| puts "verify #{step}" }
(result["open"] || []).each { |o| puts "open #{o["item"]} - #{o["reason"]}" }
(result["questions"] || []).each { |q| puts "question #{q}" }
// the fix worked example: vendor-check.json from build-check.js
// (for audited code, pass handoff.js's output instead: make_input adds its audit_handoff)
$fixInput = make_input("fix", "vendor-check.json",
"Fix what you can. geo_math is vendored at third_party/geo-math in this repository now. The tested commit of shape_codec is 3f9c2e1d4b6a8c0e2f4a6b8d0c2e4f6a8b0d2c4e. The cache also serves grpcs on bazel-cache.corp.lan:9092. We do not have the tzdata sha256 yet and the fonts zip has no published checksum - leave those two for us. The three runs per test were only meant for the geometry tests we suspect are flaky.");
$fixKey = "hermetic-build-audit:fix:" . substr(hash("sha256", json_encode($fixInput)), 0, 16) . ":a1";
$done = null; $failure = null; $streamed = ""; $buf = ""; $ctype = "";
$handle = function (string $event, string $data) use (&$done, &$failure, &$streamed) {
$p = json_decode($data);
if ($p === null) return; // not JSON: skipped, as sdk.js does
switch ($event) {
case "delta": $streamed .= $p->text ?? ""; break;
case "job": echo "job ", json_encode($p), "\n"; break;
case "done": case "pending": $done = $p; break;
case "error": $failure = $p; break;
}
};
$ch = curl_init(BASE . "/run-stream");
curl_setopt_array($ch, [
CURLOPT_POST => true,
CURLOPT_HTTPHEADER => headers_for(["Idempotency-Key" => $fixKey]),
CURLOPT_POSTFIELDS => json_encode($fixInput, JSON_UNESCAPED_SLASHES | JSON_UNESCAPED_UNICODE),
CURLOPT_HEADERFUNCTION => function ($ch, $line) use (&$ctype) {
if (stripos($line, "content-type:") === 0) $ctype = strtolower(trim(substr($line, 13)));
return strlen($line);
},
CURLOPT_WRITEFUNCTION => function ($ch, $chunk) use (&$buf, &$ctype, $handle) {
$buf .= $chunk; // collects the response in memory
if (strpos($ctype, "text/event-stream") === false) return strlen($chunk); // plain JSON: parse at the end
while (($i = strpos($buf, "\n\n")) !== false) {
$frame = substr($buf, 0, $i);
$buf = substr($buf, $i + 2);
$event = "message"; $data = "";
foreach (explode("\n", $frame) as $line) {
if (strpos($line, "event:") === 0) $event = trim(substr($line, 6));
elseif (strpos($line, "data:") === 0) $data .= trim(substr($line, 5));
}
if ($data !== "") $handle($event, $data);
}
return strlen($chunk);
},
]);
if (curl_exec($ch) === false) throw new RuntimeException(curl_error($ch));
$status = curl_getinfo($ch, CURLINFO_RESPONSE_CODE);
curl_close($ch);
if (strpos($ctype, "text/event-stream") === false) {
$env = json_decode($buf) ?? new stdClass();
if ($status < 200 || $status > 299) {
throw new ApiError($status, $env->error->code ?? null, $env->error->message ?? "HTTP $status");
}
$done = $env->data ?? null;
}
if ($failure) throw new RuntimeException(($failure->code ?? "") . ": " . ($failure->message ?? "") . " (job " . ($failure->job_id ?? "") . ")");
$full = $done->output->output ?? "";
if ($full === "") $full = $streamed;
echo "truncated: ", json_encode(($done->truncated ?? false) === true), ", charged: ", $done->charged_credits ?? "-", "\n";
$result = json_decode(substr($full, strpos($full, "{"), strrpos($full, "}") - strpos($full, "{") + 1));
echo $result->verdict, " - ", $result->headline, "\n";
foreach ($result->files ?? [] as $f) echo "file ", $f->path, " ", mb_strlen($f->content ?? ""), " characters\n";
foreach ($result->changes ?? [] as $c) {
$flags = implode(",", $c->flags ?? []);
echo "change ", $c->target ?? "", " ", $flags === "" ? "-" : $flags, " - ", $c->change ?? "", "\n";
}
foreach ($result->verify ?? [] as $step) echo "verify ", $step, "\n";
foreach ($result->open ?? [] as $o) echo "open ", $o->item, " - ", $o->reason ?? "", "\n";
foreach ($result->questions ?? [] as $q) echo "question ", $q, "\n";
// the fix worked example: vendor-check.json from build-check.js
// (for audited code, pass handoff.js's output instead: MakeInput adds its audit_handoff)
var fixInput = MakeInput("fix", "vendor-check.json",
"Fix what you can. geo_math is vendored at third_party/geo-math in this repository now. The tested commit of shape_codec is 3f9c2e1d4b6a8c0e2f4a6b8d0c2e4f6a8b0d2c4e. The cache also serves grpcs on bazel-cache.corp.lan:9092. We do not have the tzdata sha256 yet and the fonts zip has no published checksum - leave those two for us. The three runs per test were only meant for the geometry tests we suspect are flaky.");
var fixDigest = System.Security.Cryptography.SHA256.HashData(Encoding.UTF8.GetBytes(JsonSerializer.Serialize(fixInput)));
var fixKey = "hermetic-build-audit:fix:" + Convert.ToHexString(fixDigest, 0, 8).ToLowerInvariant() + ":a1";
using var sreq = Api.Request("POST", "/run-stream", fixInput, new Dictionary<string, string> { ["Idempotency-Key"] = fixKey });
using var sres = await Api.Http.SendAsync(sreq, HttpCompletionOption.ResponseHeadersRead);
JsonNode? done = null, failure = null;
var streamed = new StringBuilder();
if (sres.Content.Headers.ContentType?.MediaType != "text/event-stream")
{
JsonNode? env = null;
try { env = JsonNode.Parse(await sres.Content.ReadAsStringAsync()); } catch (JsonException) { }
if (!sres.IsSuccessStatusCode) throw Api.ErrorFrom(sres, env);
done = env?["data"];
}
else
{
using var reader = new StreamReader(await sres.Content.ReadAsStreamAsync());
var ev = "message";
var data = new StringBuilder();
string? line;
while ((line = await reader.ReadLineAsync()) != null)
{
if (line.Length == 0) // a blank line ends a frame
{
if (data.Length > 0)
{
JsonNode? p = null;
try { p = JsonNode.Parse(data.ToString()); } catch (JsonException) { }
if (p != null)
{
switch (ev)
{
case "delta": streamed.Append(p["text"]?.GetValue<string>() ?? ""); break;
case "job": Console.WriteLine($"job {p.ToJsonString()}"); break;
case "done": case "pending": done = p; break;
case "error": failure = p; break;
}
}
}
ev = "message";
data.Clear();
}
else if (line.StartsWith("event:")) ev = line[6..].Trim();
else if (line.StartsWith("data:")) data.Append(line[5..].Trim());
}
}
if (failure != null) throw new Exception($"{failure["code"]}: {failure["message"]} (job {failure["job_id"]})");
var full = done?["output"]?["output"]?.GetValue<string>();
if (string.IsNullOrEmpty(full)) full = streamed.ToString();
var cut = done?["truncated"] is JsonValue tv && tv.TryGetValue(out bool cutShort) && cutShort;
Console.WriteLine($"truncated: {cut}, charged: {done?["charged_credits"]}");
var result = JsonNode.Parse(full[full.IndexOf('{')..(full.LastIndexOf('}') + 1)])!;
Console.WriteLine($"{result["verdict"]} - {result["headline"]}");
foreach (var f in result["files"]?.AsArray() ?? new JsonArray())
Console.WriteLine($"file {f?["path"]} {(f?["content"]?.GetValue<string>() ?? "").Length} characters");
foreach (var c in result["changes"]?.AsArray() ?? new JsonArray())
{
var flags = string.Join(",", (c?["flags"]?.AsArray() ?? new JsonArray()).Select(fl => fl?.GetValue<string>()));
Console.WriteLine($"change {c?["target"]} {(flags == "" ? "-" : flags)} - {c?["change"]}");
}
foreach (var step in result["verify"]?.AsArray() ?? new JsonArray()) Console.WriteLine($"verify {step}");
foreach (var o in result["open"]?.AsArray() ?? new JsonArray()) Console.WriteLine($"open {o?["item"]} - {o?["reason"]}");
foreach (var q in result["questions"]?.AsArray() ?? new JsonArray()) Console.WriteLine($"question {q}");
What the page does that you now do
- Reads the paste in the browser and builds
codeandfactsfrom it (bazelkit.js), joining dropped files under# file:lines and cutting a large paste on whole files with a# [hermetic-build-audit: not sent: ...]line; you send the same two strings. - Prices every input with
/estimateand refuses to run belowmin_credits, or on a paste with no Bazel file it can read. - Sends each attempt with an input- and lane-derived
Idempotency-Key, and retries once withretry_notewhen a reply does not parse. - Closes a cut-off JSON reply and shows what arrived when
truncatedis true or the stream ends early. - Hands an audit to the fix lane as
audit_handoff, with every edit applied to the paste first, and only while that edited paste is unchanged. - Checks the reply against the files (
recon.js), in the browser: files, lines, evidence, every edit applied in order and the free check re-run after each finding, flag answers and verdict; the fixed files put back, re-checked and matched against the changes list and the open items; every target, repository and config still there; and repeated credentials.recheck.jsabove runs the same checks for you.
Or skip all of this and use Hermetic Build Audit in the browser: the check, the flags, the repository, rc-line and target inventories, putting an edit or the fixed files back into the paste, and the Markdown, CSV and JSON exports are free.