Commands
| Command | What it does |
|---|---|
suncly attest <card-url> --sandbox | Evaluate the agent at the card URL: fetch and hash the card, draft or reuse a contract, record the approval, run, judge, decide (flag), sign, write the report folder. |
suncly demo | Start the bundled honest and lying mock agents and evaluate both. --runs (default 3), --reports-dir, --json. |
suncly verify <report-folder> | Check the signature, card hash, decision, transcript hashes and counts of a report folder. --public-key overrides the embedded key. |
suncly keys init [--new] | Create the local Ed25519 deployment key. attest and demo create one on first use if none exists. |
suncly db migrate, suncly db check | Apply and check the Postgres schema (seven tables, eight enums, triggers) when DATABASE_URL is set. |
suncly doctor [card-url] | Check Python, the deployment key, the store configuration and, optionally, that a card is reachable and parsable. |
--home DIR before any command moves the whole state folder (store, transcripts, keys). -h prints help; --version the version.
suncly attest options (Proposed)
| Option | Meaning |
|---|---|
--sandbox | Declares the endpoint a sandbox or dry-run endpoint. Required: without it nothing runs (DR-006). Suncly cannot verify a sandbox. |
--runs N | Repetitions per test case. Default 5 (SUNCLY_RUNS). |
--budget N | Budget in attempts. Default 2 × planned runs. One attempt costs 1, retries included; the attestation ends failed when reached. |
--approve-as ID | Approves the drafted contract as ID without a prompt; ID becomes approved_by. Without it the CLI shows the draft and asks; with no terminal, it refuses. |
--contract FILE | Uses a contract file instead of the drafter. The imported contract becomes a new version and still needs approval. |
--export-draft FILE | Writes the draft contract to FILE and stops. Nothing runs. |
--owner, --risk-level | agent.owner and agent.risk_level (low, medium, high), recorded on first sight of the card URL. Defaults: unspecified and high. The risk level does not change the outcome yet. |
--reports-dir DIR | Where the report folder goes. Default ./suncly-reports. |
--json | Prints a machine-readable result (see below). Progress goes to stderr. |
--debug | Shows tracebacks. |
An attestation started from the CLI has trigger manual, also when CI calls the CLI.
Exit codes
| Code | Meaning |
|---|---|
0 | The attestation completed: decided (flag) and signed. Not an approval. |
1 | Suncly itself failed. |
2 | Wrong arguments. |
3 | Refused to start: no --sandbox, no approval, unusable card or contract. |
4 | The attestation ended failed: the budget stopped it, or the card could not be re-fetched. |
5 | The attestation ended invalidated: the card changed while it ran. |
6 | suncly verify or suncly doctor found a problem. |
Because every decision is flag in this version, no pipeline should gate a release on the exit code alone. Archive the report folder and route it to a reviewer.
Contract file (Proposed)
A hand-written or exported contract: one JSON document with the test cases of one contract for one card. Field names are the data model's. The file carries no approval fields: approval is recorded by --approve-as or the prompt.
{
"suncly_contract_file": 1,
"card_hash": "sha256:…",
"agent_name": "Order Status Agent",
"skills_without_test_case": [],
"test_cases": [
{
"skill_id": "order-status",
"kind": "skill",
"input": {"text": "Where is order 1234?"},
"criteria": {
"final_state": "TASK_STATE_COMPLETED",
"latency_limit_ms": 10000,
"response_present": true,
"output_modes": ["text/plain"],
"required_fields": ["/artifacts/0/parts/0/text"],
"response_schema": {"type": "object", "required": ["artifacts"]},
"model_checks": [],
"accept_direct_message": false
}
}
]
}card_hashmust equal the hash of the fetched card, or the file is refused.- Every declared skill needs a test case, or must be listed under
skills_without_test_caseto acknowledge that it stays untested. inputis{"text": …}or{"parts": […]}with A2A Part objects.criteriakeys:final_state(a terminal task state, defaultTASK_STATE_COMPLETED);latency_limit_ms(required);response_present;output_modes(media types every output part must use;nulldisables the check);required_fields(JSON pointers that must exist and be non-empty in the final response);response_schema(a JSON Schema the final response must satisfy);model_checks(criteria for Layer 2, which does not exist yet, so any entry makes the runinconclusive);accept_direct_message(a direct Message reply counts as a completed task). Unknown keys are refused, so a typo can never silently weaken a check.
Settings and environment
Resolved in this order: built-in defaults, then SUNCLY_HOME/config.toml, then environment variables. A blank value is the same as an unset one.
| Variable | Default | Meaning |
|---|---|---|
SUNCLY_HOME | ~/.suncly | State folder: file store, transcripts, deployment keys. |
DATABASE_URL | unset | When set, the Postgres store is used instead of the file store. |
SUNCLY_REPORTS_DIR | ./suncly-reports | One report folder per attestation. |
SUNCLY_RUNS | 5 | Repetitions per test case. |
SUNCLY_BUDGET_LIMIT | 2 × planned runs | Budget in attempts. |
SUNCLY_RUN_TIMEOUT_S | 30 | Per-run timeout; the Runner process is killed 10 s after it. |
SUNCLY_LATENCY_LIMIT_MS | 10000 | Latency limit written into drafted criteria. |
SUNCLY_MAX_RETRIES | 2 | Retries per run, under the same run key. |
SUNCLY_CONCURRENCY | 4 | Concurrent runs. |
SUNCLY_POLL_INTERVAL_S | 0.5 | Seconds between GetTask polls. |
SUNCLY_CARD_TIMEOUT_S, SUNCLY_CARD_MAX_BYTES | 10, 1000000 | Card fetch limits. |
SUNCLY_MAX_TEST_CASES_PER_SKILL | 3 | Drafted test cases per skill, one per declared example up to this cap. |
SUNCLY_AGENT_AUTHORIZATION | unset | The agent credential. Read only by the Runner process; redacted from every transcript. |
Machine-readable output
suncly attest … --json prints one JSON object with exit_code, exit_code_meaning (“0 means completed and signed, not approved”), kind (completed, failed, invalidated or draft_exported), attestation, decision, results, not_tested and report_dir. suncly verify … --json prints ok and the list of checks.