Skip to content
Ledgerline Help

OpenAPI 3.1.0 · version 1.0.0

API reference

Four routes: ask a question and stream a cited answer, hand a conversation to a person, rate an answer, and read the evaluation history. Ledgerline is a fictional company and the docs are fictional. The API serves the help site and its embeddable widget on the same origin, so it sends no CORS headers; call it from a server. Every response carries an X-Request-Id header, and every failure has the same JSON body.

openapi.jsonhttps://help.phoenixcodes.org

post/api/chat

Ask a question

Searches the docs, and answers from the passages found, citing them as [1], [2] and so on.

Order of work: rate limit, retrieval, the confidence gate (below the threshold no model is called and the answer is the standard decline), the daily token budget, then the model.

The response is a stream in the Vercel AI SDK UI message format (server-sent events). It carries the answer as text deltas and two data parts: data-meta (a ChatMeta, sent first with the retrieval confidence and again at the end with the outcome) and data-citations (the numbered passages the model was shown, a Citation per passage).

A visitor cookie (ll_vid, HttpOnly) is set on the first call so a conversation can continue. Ask follow-ups by sending the returned conversationId back.

Request body (JSON)

messagestringrequired
1 to 500 charactersThe question. Whitespace at either end is trimmed before the length is checked.
conversationIdstring (uuid)
Continue a conversation. Omit to start one; the id comes back in ChatMeta.
surface"site" | "widget"
default "site"Where the question was asked. The widget's visitor cookie is partitioned for use in an iframe.

Example

curl -N 'https://help.phoenixcodes.org/api/chat' \
  -H 'Content-Type: application/json' \
  -d '{"message":"How do I verify a webhook signature?","surface":"site"}'

Responses

  • 200The answer, streamed.text/event-stream · string
  • 400The request is not valid. message says which field and why.application/json · Error
  • 429Too many requests from this client. Wait retryAfterSeconds.application/json · Error
  • 503A dependency (search, the model, or the database) is unavailable, or on /api/chat the daily token budget is used up (budget_exhausted, with resetsAt). escalate is true when a person should take over.application/json · Error

post/api/tickets

Talk to a human

Opens a support ticket for the admin queue. No email is sent anywhere: the ticket is a row a person reads in /admin. At most 5 per hour per client.

Request body (JSON)

namestringrequired
1 to 80 characters
emailstring (email)required
up to 200 characters
summarystringrequired
10 to 2000 characters
conversationIdstring (uuid)
The conversation the ticket is about.

Example

curl 'https://help.phoenixcodes.org/api/tickets' \
  -H 'Content-Type: application/json' \
  -d '{"name":"Ada","email":"ada@example.com","summary":"I asked about crypto payouts and the docs have no answer. Can a person help?","conversationId":"6f1c1f2e-8a0e-4c58-9a57-3a2f4a8b7c10"}'

Responses

  • 201The ticket was created.application/json · Ticket
  • 400The request is not valid. message says which field and why.application/json · Error
  • 429Too many requests from this client. Wait retryAfterSeconds.application/json · Error
  • 503A dependency (search, the model, or the database) is unavailable, or on /api/chat the daily token budget is used up (budget_exhausted, with resetsAt). escalate is true when a person should take over.application/json · Error

post/api/feedback

Rate an answer

A thumbs up (1) or down (-1) on an assistant answer, with an optional comment. One rating per answer: a second one replaces the first. messageId is the one in the ChatMeta of that answer. At most 30 per 10 minutes per client.

Request body (JSON)

messageIdstring (uuid)required
rating1 | -1required
1 is a thumbs up, -1 a thumbs down.
commentstring
up to 1000 characters

Example

curl 'https://help.phoenixcodes.org/api/feedback' \
  -H 'Content-Type: application/json' \
  -d '{"messageId":"0b9d2c1a-5e44-4e0f-b6d2-13f1d6a6f5ee","rating":-1,"comment":"The answer skipped the retry limit."}'

Responses

  • 200The rating was saved.application/json · object
  • 400The request is not valid. message says which field and why.application/json · Error
  • 404There is no answer with that id.application/json · Error
  • 429Too many requests from this client. Wait retryAfterSeconds.application/json · Error
  • 503A dependency (search, the model, or the database) is unavailable, or on /api/chat the daily token budget is used up (budget_exhausted, with resetsAt). escalate is true when a person should take over.application/json · Error

get/api/eval/runs

Evaluation runs

The runs of pnpm eval, newest first: retrieval, declining, answer quality, speed, tokens and cost. These are the numbers on the "How well it works" page, /eval. Responses are cached for a minute at the edge. At most 60 per 10 minutes per client.

Query parameters

limitinteger
1 to 30 · default 10How many runs to return, newest first.
resultsboolean
default falseInclude the per-question results (each question, its answer and the judge's note). Large: leave it off for a history chart.

Example

curl 'https://help.phoenixcodes.org/api/eval/runs?limit=5'

Responses

  • 200The runs. Empty before the first run has been recorded.application/json · object
  • 400The request is not valid. message says which field and why.application/json · Error
  • 429Too many requests from this client. Wait retryAfterSeconds.application/json · Error
  • 503A dependency (search, the model, or the database) is unavailable, or on /api/chat the daily token budget is used up (budget_exhausted, with resetsAt). escalate is true when a person should take over.application/json · Error

Schemas

The shapes the routes above return. ChatMeta and Citation travel inside the chat stream as the data-meta and data-citations parts.

ChatMeta

The data-meta part of the chat stream.

conversationIdstring (uuid)required
messageIdstring (uuid)required
Id of the assistant message; what feedback refers to.
confidencenumberrequired
0 to 1Retrieval confidence.
thresholdnumberrequired
0 to 1
gateDeclinedbooleanrequired
The confidence gate stopped the request before any model call.
declinedbooleanrequired
The answer is the standard "not in the docs" decline (by the gate or the model).
escalatebooleanrequired
Offer to hand over to a person.
relatedobject[]
Closest pages, offered with a decline when the match was not hopeless.
provider"anthropic" | "workers-ai"
modelstring
promptTokensinteger
completionTokensinteger
costUsdnumber
ttftMsinteger
Milliseconds from request to the first token.
latencyMsinteger

Citation

One passage the model was shown. The answer cites a subset by its number, [n].

nintegerrequired
at least 1
chunkIdstring (uuid)required
docSlugstringrequired
Open it at /docs/{docSlug}?chunk={chunkId}.
breadcrumbstringrequired
excerptstringrequired

Ticket

idstring (uuid)required
refstringrequired
Short reference to quote to a person.
status"open"required
createdAtstring (date-time)required

Error

Every failure that is not a stream has this body.

errorobjectrequired
error.code"invalid_request" | "rate_limited" | "budget_exhausted" | "upstream_unavailable" | "unauthorized" | "not_found" | "internal_error"required
error.messagestringrequired
error.requestIdstringrequired
error.retryAfterSecondsinteger
error.resetsAtstring (date-time)
budget_exhausted only: when the daily budget resets (UTC).
escalateboolean
Offer to hand over to a person.

EvalRun

One run of pnpm eval.

idstring (uuid)required
createdAtstring (date-time)required
gitShastring | nullrequired
Short commit the run was made from; a trailing -dirty means uncommitted changes.
modelstring | nullrequired
Model that wrote the answers (provider/model id).
judgeModelstring | nullrequired
Model that scored faithfulness and correctness.
questionCountintegerrequired
All questions in the run.
outOfScopeCountintegerrequired
How many of them the docs do not answer.
confidenceThresholdnumber | nullrequired
CONFIDENCE_THRESHOLD in force for the run.
answerPromptVersionstring | nullrequired
judgePromptVersionstring | nullrequired
retrievalobjectrequired
Whether the right document was retrieved, over the questions the docs answer.
retrieval.hitAt1numberrequired
0 to 1
retrieval.hitAt3numberrequired
0 to 1
retrieval.hitAt8numberrequired
0 to 1
retrieval.mrrnumberrequired
0 to 1Mean reciprocal rank.
decliningobjectrequired
declining.declineAccuracynumber | nullrequired
0 to 1Out-of-scope questions that ended in a decline, by the gate or the model.
declining.gateDeclineAccuracynumber | nullrequired
0 to 1Out-of-scope questions the confidence gate stopped before any model call.
declining.falseDeclineRatenumber | nullrequired
0 to 1Questions the docs answer that were declined anyway.
qualityobjectrequired
quality.faithfulnessnumber | nullrequired
1 to 5Mean judge score: is every claim in the answer supported by the passages.
quality.correctnessnumber | nullrequired
1 to 5Mean judge score against the reference answer.
quality.citationRatenumber | nullrequired
0 to 1Answers whose [n] markers all map to a retrieved passage.
speedobjectrequired
Milliseconds, measured by the caller of the route.
speed.latencyP50Msintegerrequired
speed.latencyP95Msintegerrequired
speed.ttftP50Msinteger | nullrequired
Time to the first streamed token.
speed.ttftP95Msinteger | nullrequired
tokensobjectrequired
tokens.totalintegerrequired
tokens.promptintegerrequired
tokens.completionintegerrequired
costUsdnumberrequired
Billed cost of the run in US dollars.
resultsEvalQuestionResult[]
Per-question detail. Present only when results=true.

EvalQuestionResult

idstringrequired
kind"gold" | "out_of_scope"required
Gold questions have an answer in the docs.
questionstringrequired
expectedDocSlugsstring[]required
retrievedDocSlugsstring[]required
firstHitRankinteger | nullrequired
1-based rank of the first expected document.
confidencenumberrequired
gateDeclinedbooleanrequired
declinedbooleanrequired
passbooleanrequired
answerstringrequired
validCitationsintegerrequired
totalCitationsintegerrequired
faithfulnessinteger | nullrequired
1 to 5
correctnessinteger | nullrequired
1 to 5
judgeNotestring | nullrequired
latencyMsintegerrequired
ttftMsinteger
promptTokensintegerrequired
completionTokensintegerrequired
providerstring
modelstring
costUsdnumber
errorstring