OpenAPI 3.1.0 · version 1.0.0
API reference
Four routes: ask a question and stream a cited answer, hand a conversation to a person, rate an answer, and read the evaluation history. Ledgerline is a fictional company and the docs are fictional. The API serves the help site and its embeddable widget on the same origin, so it sends no CORS headers; call it from a server. Every response carries an X-Request-Id header, and every failure has the same JSON body.
openapi.jsonhttps://help.phoenixcodes.org
post/api/chat
Ask a question
Searches the docs, and answers from the passages found, citing them as [1], [2] and so on.
Order of work: rate limit, retrieval, the confidence gate (below the threshold no model is called and the answer is the standard decline), the daily token budget, then the model.
The response is a stream in the Vercel AI SDK UI message format (server-sent events). It carries the answer as text deltas and two data parts: data-meta (a ChatMeta, sent first with the retrieval confidence and again at the end with the outcome) and data-citations (the numbered passages the model was shown, a Citation per passage).
A visitor cookie (ll_vid, HttpOnly) is set on the first call so a conversation can continue. Ask follow-ups by sending the returned conversationId back.
Request body (JSON)
messagestringrequired- 1 to 500 charactersThe question. Whitespace at either end is trimmed before the length is checked.
conversationIdstring (uuid)- Continue a conversation. Omit to start one; the id comes back in ChatMeta.
surface"site" | "widget"- default "site"Where the question was asked. The widget's visitor cookie is partitioned for use in an iframe.
Example
curl -N 'https://help.phoenixcodes.org/api/chat' \
-H 'Content-Type: application/json' \
-d '{"message":"How do I verify a webhook signature?","surface":"site"}'Responses
- 200The answer, streamed.text/event-stream · string
- 400The request is not valid.
messagesays which field and why.application/json · Error - 429Too many requests from this client. Wait
retryAfterSeconds.application/json · Error - 503A dependency (search, the model, or the database) is unavailable, or on /api/chat the daily token budget is used up (
budget_exhausted, withresetsAt).escalateis true when a person should take over.application/json · Error
post/api/tickets
Talk to a human
Opens a support ticket for the admin queue. No email is sent anywhere: the ticket is a row a person reads in /admin. At most 5 per hour per client.
Request body (JSON)
namestringrequired- 1 to 80 characters
emailstring (email)required- up to 200 characters
summarystringrequired- 10 to 2000 characters
conversationIdstring (uuid)- The conversation the ticket is about.
Example
curl 'https://help.phoenixcodes.org/api/tickets' \
-H 'Content-Type: application/json' \
-d '{"name":"Ada","email":"ada@example.com","summary":"I asked about crypto payouts and the docs have no answer. Can a person help?","conversationId":"6f1c1f2e-8a0e-4c58-9a57-3a2f4a8b7c10"}'Responses
- 201The ticket was created.application/json · Ticket
- 400The request is not valid.
messagesays which field and why.application/json · Error - 429Too many requests from this client. Wait
retryAfterSeconds.application/json · Error - 503A dependency (search, the model, or the database) is unavailable, or on /api/chat the daily token budget is used up (
budget_exhausted, withresetsAt).escalateis true when a person should take over.application/json · Error
post/api/feedback
Rate an answer
A thumbs up (1) or down (-1) on an assistant answer, with an optional comment. One rating per answer: a second one replaces the first. messageId is the one in the ChatMeta of that answer. At most 30 per 10 minutes per client.
Request body (JSON)
messageIdstring (uuid)requiredrating1 | -1required- 1 is a thumbs up, -1 a thumbs down.
commentstring- up to 1000 characters
Example
curl 'https://help.phoenixcodes.org/api/feedback' \
-H 'Content-Type: application/json' \
-d '{"messageId":"0b9d2c1a-5e44-4e0f-b6d2-13f1d6a6f5ee","rating":-1,"comment":"The answer skipped the retry limit."}'Responses
- 200The rating was saved.application/json · object
- 400The request is not valid.
messagesays which field and why.application/json · Error - 404There is no answer with that id.application/json · Error
- 429Too many requests from this client. Wait
retryAfterSeconds.application/json · Error - 503A dependency (search, the model, or the database) is unavailable, or on /api/chat the daily token budget is used up (
budget_exhausted, withresetsAt).escalateis true when a person should take over.application/json · Error
get/api/eval/runs
Evaluation runs
The runs of pnpm eval, newest first: retrieval, declining, answer quality, speed, tokens and cost. These are the numbers on the "How well it works" page, /eval. Responses are cached for a minute at the edge. At most 60 per 10 minutes per client.
Query parameters
limitinteger- 1 to 30 · default 10How many runs to return, newest first.
resultsboolean- default falseInclude the per-question results (each question, its answer and the judge's note). Large: leave it off for a history chart.
Example
curl 'https://help.phoenixcodes.org/api/eval/runs?limit=5'Responses
- 200The runs. Empty before the first run has been recorded.application/json · object
- 400The request is not valid.
messagesays which field and why.application/json · Error - 429Too many requests from this client. Wait
retryAfterSeconds.application/json · Error - 503A dependency (search, the model, or the database) is unavailable, or on /api/chat the daily token budget is used up (
budget_exhausted, withresetsAt).escalateis true when a person should take over.application/json · Error
Schemas
The shapes the routes above return. ChatMeta and Citation travel inside the chat stream as the data-meta and data-citations parts.
ChatMeta
The data-meta part of the chat stream.
conversationIdstring (uuid)requiredmessageIdstring (uuid)required- Id of the assistant message; what feedback refers to.
confidencenumberrequired- 0 to 1Retrieval confidence.
thresholdnumberrequired- 0 to 1
gateDeclinedbooleanrequired- The confidence gate stopped the request before any model call.
declinedbooleanrequired- The answer is the standard "not in the docs" decline (by the gate or the model).
escalatebooleanrequired- Offer to hand over to a person.
relatedobject[]- Closest pages, offered with a decline when the match was not hopeless.
provider"anthropic" | "workers-ai"modelstringpromptTokensintegercompletionTokensintegercostUsdnumberttftMsinteger- Milliseconds from request to the first token.
latencyMsinteger
Citation
One passage the model was shown. The answer cites a subset by its number, [n].
nintegerrequired- at least 1
chunkIdstring (uuid)requireddocSlugstringrequired- Open it at /docs/{docSlug}?chunk={chunkId}.
breadcrumbstringrequiredexcerptstringrequired
Ticket
idstring (uuid)requiredrefstringrequired- Short reference to quote to a person.
status"open"requiredcreatedAtstring (date-time)required
Error
Every failure that is not a stream has this body.
errorobjectrequirederror.code"invalid_request" | "rate_limited" | "budget_exhausted" | "upstream_unavailable" | "unauthorized" | "not_found" | "internal_error"requirederror.messagestringrequirederror.requestIdstringrequirederror.retryAfterSecondsintegererror.resetsAtstring (date-time)- budget_exhausted only: when the daily budget resets (UTC).
escalateboolean- Offer to hand over to a person.
EvalRun
One run of pnpm eval.
idstring (uuid)requiredcreatedAtstring (date-time)requiredgitShastring | nullrequired- Short commit the run was made from; a trailing -dirty means uncommitted changes.
modelstring | nullrequired- Model that wrote the answers (provider/model id).
judgeModelstring | nullrequired- Model that scored faithfulness and correctness.
questionCountintegerrequired- All questions in the run.
outOfScopeCountintegerrequired- How many of them the docs do not answer.
confidenceThresholdnumber | nullrequired- CONFIDENCE_THRESHOLD in force for the run.
answerPromptVersionstring | nullrequiredjudgePromptVersionstring | nullrequiredretrievalobjectrequired- Whether the right document was retrieved, over the questions the docs answer.
retrieval.hitAt1numberrequired- 0 to 1
retrieval.hitAt3numberrequired- 0 to 1
retrieval.hitAt8numberrequired- 0 to 1
retrieval.mrrnumberrequired- 0 to 1Mean reciprocal rank.
decliningobjectrequireddeclining.declineAccuracynumber | nullrequired- 0 to 1Out-of-scope questions that ended in a decline, by the gate or the model.
declining.gateDeclineAccuracynumber | nullrequired- 0 to 1Out-of-scope questions the confidence gate stopped before any model call.
declining.falseDeclineRatenumber | nullrequired- 0 to 1Questions the docs answer that were declined anyway.
qualityobjectrequiredquality.faithfulnessnumber | nullrequired- 1 to 5Mean judge score: is every claim in the answer supported by the passages.
quality.correctnessnumber | nullrequired- 1 to 5Mean judge score against the reference answer.
quality.citationRatenumber | nullrequired- 0 to 1Answers whose [n] markers all map to a retrieved passage.
speedobjectrequired- Milliseconds, measured by the caller of the route.
speed.latencyP50Msintegerrequiredspeed.latencyP95Msintegerrequiredspeed.ttftP50Msinteger | nullrequired- Time to the first streamed token.
speed.ttftP95Msinteger | nullrequiredtokensobjectrequiredtokens.totalintegerrequiredtokens.promptintegerrequiredtokens.completionintegerrequiredcostUsdnumberrequired- Billed cost of the run in US dollars.
resultsEvalQuestionResult[]- Per-question detail. Present only when
results=true.
EvalQuestionResult
idstringrequiredkind"gold" | "out_of_scope"required- Gold questions have an answer in the docs.
questionstringrequiredexpectedDocSlugsstring[]requiredretrievedDocSlugsstring[]requiredfirstHitRankinteger | nullrequired- 1-based rank of the first expected document.
confidencenumberrequiredgateDeclinedbooleanrequireddeclinedbooleanrequiredpassbooleanrequiredanswerstringrequiredvalidCitationsintegerrequiredtotalCitationsintegerrequiredfaithfulnessinteger | nullrequired- 1 to 5
correctnessinteger | nullrequired- 1 to 5
judgeNotestring | nullrequiredlatencyMsintegerrequiredttftMsintegerpromptTokensintegerrequiredcompletionTokensintegerrequiredproviderstringmodelstringcostUsdnumbererrorstring