What happens to a request
In theory, everyone answering questions is connected to a websocket and will be asked live the questions that arrive at the API. Their answers are assembled into the response. That’s the goal, and it works across a small group, but it’s almost certainly bound to break if too many people visit. I look forward to finding out the strange new ways this can break.
Some software posts a request to /v1/systemone. Up to five available people get the question, with one live assignment per answering page. Each person answers once. When the panel has answered, the caller gets the average. New visitors don’t make an existing panel bigger.
A request has 30 seconds from the moment it arrives, including any time it spends waiting in line. At the deadline the caller gets the average of whatever answers are in. If nobody answered, it gets a 504. You can still answer late and compare with Jev, but the caller never sees it.
Only visible answering pages count. If you skip, switch tabs, or close the page, someone else can take your place; an answer you already sent still counts. If the panel stays incomplete, the caller gets the available answers at the deadline.
Expired or interrupted drafts stay in your tab for up to a minute, with at most three unfinished late cards. Late answers and their comparisons are local to you. If answering capacity is full, the API returns 503 with a retry delay. Requests must fit within 2,400 bytes and ask at most 3 questions. Humans have 30 seconds.
The request
A request has state, the thing to judge (text or JSON), and questions, keyed by an ID the caller picks. Each question has a type, instructions, and, for Choice and Score, criteria.
{
"model": "jev-latest",
"state": "Subject: Charged twice\nHi, I see two $49 charges for the same order. Could you refund the extra charge? The product itself works great.",
"questions": {
"happy": { "type": "noul", "instructions": "Is the customer happy?" },
"team": {
"type": "choice",
"instructions": "Which team should handle this email?",
"criteria": {
"billing": "Payments and refunds",
"technical": "Problems using the product",
"sales": "Questions about buying"
}
},
"urgency": {
"type": "score",
"instructions": "How soon does this email need attention?",
"criteria": ["Can wait a few days", "Needs attention today", "Needs attention now"]
}
}
}
The three question types
Noul
A yes/no question answered as a probability. noul runs from 0 (definitely no) to 1 (definitely yes), and 0.5 means you can’t tell. There’s no separate confidence. The name comes from a Bernoulli trial.
Choice
Pick between named options. Each bar fills independently from 0 to 100%. Your fullest bar is your confidence. Each bar divided by the total is its probability, so probabilities always sums to 1. choice is the option with the highest probability, and ties go to the first one.
Two full bars give 50/50 with confidence 1: you’re sure it’s one of the two. Two half-full bars give the same 50/50 with confidence 0.5: you’re unsure about everything. Leaving every bar empty isn’t allowed.
Good questions include an “unknown” or “none of these” option. Without one, the answer is forced into an option that doesn’t fit.
Score
A position on an ordered scale. The first level is 0, the next 1, and so on. The answer can land between levels: 1.75 is most of the way from “today” to “now”. legend maps each level to its label. probabilities splits the weight between the two nearest levels, and confidence is higher when the answer sits right on a level.
The response
Here’s what the caller gets back if one person answers 0.8 for happy, fills billing all the way and technical halfway, and slides urgency to 1.75. The response follows the TypeSafe System One API, with model set to reverse-horse and zero token usage.
{
"model": "reverse-horse",
"answers": {
"happy": { "type": "noul", "noul": 0.8 },
"team": {
"type": "choice",
"choice": "billing",
"probabilities": { "billing": 0.6666666666666666, "technical": 0.3333333333333333, "sales": 0 },
"confidence": 1
},
"urgency": {
"type": "score",
"score": 1.75,
"legend": { "0": "Can wait a few days", "1": "Needs attention today", "2": "Needs attention now" },
"probabilities": { "0": 0, "1": 0.25, "2": 0.75 },
"confidence": 0.4881404928570853
}
},
"usage": { "input_tokens": 0, "output_tokens": 0 }
}
When several people answer
Noul values, scores, probabilities, and confidence are averaged. For Choice, each person’s probabilities count equally, however full their bars were, and the option with the highest average wins.
Matching Jev
Pink markers show what Jev answered on the same control. Your answer turns green when it matches and pink when it doesn’t. A match means the same choice, the same side of 0.5 for Noul, or the same nearest level for Score (halfway rounds up).
The Answer page always compares against saved Jev answers, for practice rounds and for any live request that matches one. To ask the real Jev, use Ask Jev too on the Question page after your answer comes back. It connects your OpenRouter key, which stays in your browser, and calls Jev from your browser on your credits.
Your pay stub
After you submit, the button tells you your hourly pay. Don’t get excited. You’re paid exactly what Jev would have charged to answer the same request, stretched over the time it took you.
Jev on OpenRouter costs $0.042 per million input tokens, and output is free. Every request carries about 233 tokens of overhead, plus roughly one token for every three characters of state and questions. A typical practice round comes to around 350 tokens, or about $0.000015. That’s a thousandth and a half of a cent. Take 10 seconds to answer it and you’re earning about half a cent an hour. Take the full 30 and you’d better not have rent due.
The token count is an estimate. TypeSafe doesn’t publish Jev’s tokenizer, so I sent every practice round to the real Jev, read back the usage it reported, and fit a line through it. The estimate lands within about 65 tokens of the real bill. Close enough to ruin your day.
How much slower you are
Where the clock was, a finished card tells you how many times slower than Jev you were. It’s gray because it isn’t a scolding. It’s just the math.
I measured it the boring way: I timed real requests for all 119 practice rounds from my laptop through OpenRouter, twice. The median round trip was 190 to 199 milliseconds, network and all, so the horse uses 0.2 seconds. The slowest single call took 740 milliseconds, which is still faster than you can read the question. Answer in 10 seconds and you were 50x slower. Use the whole 30 and it’s 150x.
Your time starts when the request arrives and stops when you hit submit. Paused practice doesn’t count against you. Training has no clock, so there it starts when the card appears.