ablative docs

Go to

↑↓ move↵ openesc close
 Stack Notebook ablative.com.au

Operational HTTP API

Inspect dead letters, unserved queues, failed recovery and releases; drain or stop workers.

Open .md

Dispatch and recovery

POST /outbox/dead-letters
Body: workflow_id UUID string. Requires the deployment-wide deploy grant and a commissioned durable outbox. Returns dead_letters ordered by ordinal, including dispatch_key, activity_type, namespace, task_queue, node, attempt, failure_delivered and redrivable.
GET /queues/unserved
Namespace-filtered list of unserved addresses. Each row names the queue, action, reason, detail, policy, worker counts, workers_at_capacity, unserved_for_ms and waiting dispatches with workflow/activity ids.
GET /workflows/unrecoverable
Namespace-filtered list of runs startup recovery failed to load. Rows carry workflow_id, namespace, workflow_type, reason, observed_at and remedy. A recovered run clears from this process’s set.
GET /changelog
Authenticated JSON with markdown containing the changelog compiled into the running binary.
GET /update-status
Authenticated JSON: installed, latest_known and checked_at. The latter two are null until a manual check completes. This read serves the saved result without making a network request.

Replace the example workflow UUID with the workflow being diagnosed.

curl --fail-with-body -sS -X POST "${AION_HTTP_ENDPOINT:-http://127.0.0.1:8080}/outbox/dead-letters" \
  -H "authorization: Bearer $AION_TOKEN" \
  -H 'content-type: application/json' \
  --data-binary '{"workflow_id":"00000000-0000-0000-0000-000000000001"}'

A redrivable row has a failure that was still undelivered to workflow history. A delivered failure has already changed the workflow’s recorded decisions. Inspect that history before choosing a redrive through the cluster command API.

curl --fail-with-body -sS -X GET "${AION_HTTP_ENDPOINT:-http://127.0.0.1:8080}/queues/unserved" \
  -H "authorization: Bearer $AION_TOKEN"
curl --fail-with-body -sS -X GET "${AION_HTTP_ENDPOINT:-http://127.0.0.1:8080}/workflows/unrecoverable" \
  -H "authorization: Bearer $AION_TOKEN"
curl --fail-with-body -sS -X GET "${AION_HTTP_ENDPOINT:-http://127.0.0.1:8080}/changelog" \
  -H "authorization: Bearer $AION_TOKEN"
curl --fail-with-body -sS -X GET "${AION_HTTP_ENDPOINT:-http://127.0.0.1:8080}/update-status" \
  -H "authorization: Bearer $AION_TOKEN"

For an unrecoverable run, restore its pinned package and plan recovery, or cancel it when termination is intended. An empty list means this process currently records no failed recoveries visible to this caller.

Drain or stop a worker

POST /workers/{worker_id}/drain
Requires deploy grant. Body is empty. Fences new assignment, sends a graceful drain and returns HTTP 202 with worker_id and state draining or deregistered.
POST /workers/{worker_id}/stop
Requires deploy grant. Body is empty. Starts drain; after drain.timeout_seconds, force-deregisters a worker still present and fails its pending tasks. HTTP 202 state is stopping or deregistered.
Unknown worker
A worker id absent from the live registry returns not-found. Read connected workers to obtain the numeric registration id.
Managed process control
GET /workers/managed reads supervised process status. POST /workers/managed/{name}/start, /stop and /restart change desired process state; these mutations also require the enabled deploy surface.

Replace the numeric worker id with a live registration id. Draining can close that registration once its work settles. Read connected workers again before sending another control; an id whose worker disconnected returns not_found.

curl --fail-with-body -sS -X POST "${AION_HTTP_ENDPOINT:-http://127.0.0.1:8080}/workers/42/drain" \
  -H "authorization: Bearer $AION_TOKEN"
curl --fail-with-body -sS -X POST "${AION_HTTP_ENDPOINT:-http://127.0.0.1:8080}/workers/42/stop" \
  -H "authorization: Bearer $AION_TOKEN"

These calls control task admission and worker-loss handling. Use managed worker controls when you need the server to stop a supervised OS process.

Use namespace creation, agent interventions and worker deployment records for those resources.

Dispatch inspection
crates/aion-server/src/api/http/outbox.rs and queues.rs define request/response shapes and grant filters.
Worker control
crates/aion-server/src/api/http/workers.rs:23–108 defines drain and stop. Stop force-deregisters the registration after its deadline.
Recovery and release reads
crates/aion-server/src/api/http/unrecoverable.rs, changelog.rs and update_status.rs define these read contracts.