Skip to content
Command reference

One command does the work. Five keep it honest.

The CLI parses, prompts and prints; every decision lives in the core library the future HTTP API will call too. Options marked Proposed implement proposals for open questions and may change when the founders decide them.

Commands

CommandWhat it does
suncly attest <card-url> --sandboxEvaluate the agent at the card URL: fetch and hash the card, draft or reuse a contract, record the approval, run, judge, decide (flag), sign, write the report folder.
suncly demoStart the bundled honest and lying mock agents and evaluate both. --runs (default 3), --reports-dir, --json.
suncly verify <report-folder>Check the signature, card hash, decision, transcript hashes and counts of a report folder. --public-key overrides the embedded key.
suncly keys init [--new]Create the local Ed25519 deployment key. attest and demo create one on first use if none exists.
suncly db migrate, suncly db checkApply and check the Postgres schema (seven tables, eight enums, triggers) when DATABASE_URL is set.
suncly doctor [card-url]Check Python, the deployment key, the store configuration and, optionally, that a card is reachable and parsable.

--home DIR before any command moves the whole state folder (store, transcripts, keys). -h prints help; --version the version.

suncly attest options (Proposed)

OptionMeaning
--sandboxDeclares the endpoint a sandbox or dry-run endpoint. Required: without it nothing runs (DR-006). Suncly cannot verify a sandbox.
--runs NRepetitions per test case. Default 5 (SUNCLY_RUNS).
--budget NBudget in attempts. Default 2 × planned runs. One attempt costs 1, retries included; the attestation ends failed when reached.
--approve-as IDApproves the drafted contract as ID without a prompt; ID becomes approved_by. Without it the CLI shows the draft and asks; with no terminal, it refuses.
--contract FILEUses a contract file instead of the drafter. The imported contract becomes a new version and still needs approval.
--export-draft FILEWrites the draft contract to FILE and stops. Nothing runs.
--owner, --risk-levelagent.owner and agent.risk_level (low, medium, high), recorded on first sight of the card URL. Defaults: unspecified and high. The risk level does not change the outcome yet.
--reports-dir DIRWhere the report folder goes. Default ./suncly-reports.
--jsonPrints a machine-readable result (see below). Progress goes to stderr.
--debugShows tracebacks.

An attestation started from the CLI has trigger manual, also when CI calls the CLI.

Exit codes

CodeMeaning
0The attestation completed: decided (flag) and signed. Not an approval.
1Suncly itself failed.
2Wrong arguments.
3Refused to start: no --sandbox, no approval, unusable card or contract.
4The attestation ended failed: the budget stopped it, or the card could not be re-fetched.
5The attestation ended invalidated: the card changed while it ran.
6suncly verify or suncly doctor found a problem.

Because every decision is flag in this version, no pipeline should gate a release on the exit code alone. Archive the report folder and route it to a reviewer.

Contract file (Proposed)

A hand-written or exported contract: one JSON document with the test cases of one contract for one card. Field names are the data model's. The file carries no approval fields: approval is recorded by --approve-as or the prompt.

{
  "suncly_contract_file": 1,
  "card_hash": "sha256:…",
  "agent_name": "Order Status Agent",
  "skills_without_test_case": [],
  "test_cases": [
    {
      "skill_id": "order-status",
      "kind": "skill",
      "input": {"text": "Where is order 1234?"},
      "criteria": {
        "final_state": "TASK_STATE_COMPLETED",
        "latency_limit_ms": 10000,
        "response_present": true,
        "output_modes": ["text/plain"],
        "required_fields": ["/artifacts/0/parts/0/text"],
        "response_schema": {"type": "object", "required": ["artifacts"]},
        "model_checks": [],
        "accept_direct_message": false
      }
    }
  ]
}
  • card_hash must equal the hash of the fetched card, or the file is refused.
  • Every declared skill needs a test case, or must be listed under skills_without_test_case to acknowledge that it stays untested.
  • input is {"text": …} or {"parts": […]} with A2A Part objects.
  • criteria keys: final_state (a terminal task state, default TASK_STATE_COMPLETED); latency_limit_ms (required); response_present; output_modes (media types every output part must use; null disables the check); required_fields (JSON pointers that must exist and be non-empty in the final response); response_schema (a JSON Schema the final response must satisfy); model_checks (criteria for Layer 2, which does not exist yet, so any entry makes the run inconclusive); accept_direct_message (a direct Message reply counts as a completed task). Unknown keys are refused, so a typo can never silently weaken a check.

Settings and environment

Resolved in this order: built-in defaults, then SUNCLY_HOME/config.toml, then environment variables. A blank value is the same as an unset one.

VariableDefaultMeaning
SUNCLY_HOME~/.sunclyState folder: file store, transcripts, deployment keys.
DATABASE_URLunsetWhen set, the Postgres store is used instead of the file store.
SUNCLY_REPORTS_DIR./suncly-reportsOne report folder per attestation.
SUNCLY_RUNS5Repetitions per test case.
SUNCLY_BUDGET_LIMIT2 × planned runsBudget in attempts.
SUNCLY_RUN_TIMEOUT_S30Per-run timeout; the Runner process is killed 10 s after it.
SUNCLY_LATENCY_LIMIT_MS10000Latency limit written into drafted criteria.
SUNCLY_MAX_RETRIES2Retries per run, under the same run key.
SUNCLY_CONCURRENCY4Concurrent runs.
SUNCLY_POLL_INTERVAL_S0.5Seconds between GetTask polls.
SUNCLY_CARD_TIMEOUT_S, SUNCLY_CARD_MAX_BYTES10, 1000000Card fetch limits.
SUNCLY_MAX_TEST_CASES_PER_SKILL3Drafted test cases per skill, one per declared example up to this cap.
SUNCLY_AGENT_AUTHORIZATIONunsetThe agent credential. Read only by the Runner process; redacted from every transcript.

Machine-readable output

suncly attest … --json prints one JSON object with exit_code, exit_code_meaning (“0 means completed and signed, not approved”), kind (completed, failed, invalidated or draft_exported), attestation, decision, results, not_tested and report_dir. suncly verify … --json prints ok and the list of checks.

Pilot

Run the first evaluation with us.

Suncly is in pilot. If your team approves A2A agents by hand today, tell us about one agent and one sandbox, and we will run the first evaluation together.

Or write to team@suncly.com.