Jev

Why Choose Jev?

Jev approaches AI integration from a different direction than conventional language models. A normal LLM is optimized to generate a sequence of tokens, even when the application ultimately wants something much simpler such as a category, risk level, routing destination, or yes/no probability. Developers then have to constrain, parse, validate, and interpret that generated response.

Jev removes the text-generation stage from this class of workflow. The application defines the possible decisions in advance, sends the relevant state, and receives values that can immediately participate in ordinary program logic.

That distinction matters most when AI is buried several layers inside a production system. A chatbot response can tolerate flexible wording; an internal branch such as send_to_human, retry, or approve_candidate benefits much more from a narrow interface whose possible outputs are known before the request is made.

Jev should therefore be evaluated less like another chatbot model and more like a probabilistic software primitive. The surrounding application remains responsible for policy, side effects, thresholds, and deterministic rules, while the model handles judgments that are difficult to express with traditional conditionals.

Core Workflow

The most effective architecture is to begin with a piece of application state and decompose the desired decision into small independent questions.

For example, an agent processing an incoming request might need to determine intent, estimate risk, decide whether a specialist tool is relevant, and assess whether enough information is available. Those judgments do not necessarily belong in one large prompt. Jev's design encourages developers to express them independently and combine the results afterward in code.

This separation has an important operational benefit: business policy stays outside the model. A team can modify the weight attached to a score, raise the threshold required for an expensive action, or disable an automation path without rewriting the underlying evaluation question.

A practical flow usually looks like this:

1. Build a compact state containing only the information needed for the decision.
2. Decompose the workflow into atomic judgments.
3. Evaluate independent judgments together when they share the same state.
4. Inspect the returned probabilities or confidence signals rather than relying only on the top result.
5. Apply deterministic application rules to decide whether to act, request clarification, call another model, or escalate to a human.
6. Store outcomes and periodically compare them with labeled production examples.

The last step is particularly important. Confidence is useful only when it is calibrated against the consequences of mistakes in the actual product. A threshold suitable for choosing a documentation category is not automatically suitable for approving an irreversible action.

Use Cases

One of the clearest applications is model routing. Instead of sending every request to an expensive reasoning model, Jev can evaluate the request first and decide whether deterministic code, a lightweight handler, a specialist model, a larger reasoning model, or human review is appropriate. This turns model selection into an explicit part of the application architecture.

Another strong pattern is agent control flow. Agent systems frequently need decisions such as whether to use a tool, which tool to try, whether the result is sufficient, whether a retry is justified, or whether the task should stop. These are narrower than code generation and can be represented as bounded decisions rather than another open-ended generation call.

Jev can also sit after a generative model as an evaluation layer. Instead of asking the original model to both generate an answer and judge its own answer inside the same prompt, an application can independently evaluate properties such as relevance, citation support, possible prompt injection, or policy risk before accepting the result.

For retrieval systems, the same architecture can be used to filter or rerank retrieved material before paying for a larger answering model. Each candidate can be judged against explicit criteria, with application code deciding how many passages survive to the next stage.

High-volume back-office workflows are another natural fit. Support triage, lead qualification, document routing, catalog classification, moderation queues, and similar systems frequently contain hundreds of small fuzzy decisions but comparatively little need for generated prose.

Comparison to Alternatives

General-purpose LLM APIs can reproduce many Jev workflows using JSON schemas, tool calling, constrained decoding, or structured-output modes. The important comparison is therefore not whether another model can classify text; most modern models can. The question is which execution model better fits the application.

A general LLM remains more appropriate when a single request needs to move fluidly between reasoning, extraction, explanation, code generation, and natural-language output. Keeping everything inside one model can also simplify an early prototype where latency and token cost are secondary concerns.

Jev becomes more interesting when the workload has already been decomposed into stable software decisions. At that point, repeatedly invoking a generative model mainly to produce a small enum or score can be unnecessary overhead.

The two approaches are also complementary. A production agent can use Jev for routing and gating while reserving a larger reasoning model for the smaller percentage of requests that actually require deliberation or generation. In that architecture, Jev is not replacing the reasoning model; it is reducing how often the expensive part of the stack needs to run.

Best Configuration

Treat question design as interface design rather than prompt writing. A useful question should correspond to one decision your program knows how to consume. If the application cannot explain what it will do with the answer, the question is probably too vague.

Avoid collapsing several independent dimensions into one subjective score. For example, a product-review workflow will usually be easier to tune if relevance, evidence quality, policy risk, and commercial intent are evaluated independently. Application code can then combine them according to current business priorities.

Choice sets should also include realistic escape routes. If every incoming request must be forced into one of three categories even when none applies, the application creates false certainty. Depending on the workflow, an explicit other, unknown, or manual-review path can be more useful than trying to eliminate ambiguity through prompt wording.

Confidence thresholds should be attached to actions rather than globally to the model. Low-impact routing can tolerate a lower threshold, while irreversible or high-cost operations should demand stronger evidence, explicit confirmation, or another verification step.

For production deployments, maintain a labeled evaluation set drawn from real traffic. Measure both decision accuracy and the behavior of confidence buckets. The useful question is not simply whether the model is accurate overall, but whether cases that the system intends to automate are accurate enough at the selected threshold.

Migration Notes

Teams migrating an existing LLM classifier should resist copying the old prompt verbatim. Long prompts often contain output-format instructions, JSON examples, warnings against invalid values, parsing conventions, and other scaffolding that existed only because the previous model generated strings. Much of that belongs in the schema or application code instead.

A good migration process starts by identifying the actual decisions hidden inside the prompt. Convert each independent judgment into a bounded question, preserve deterministic business logic in code, and retain the existing LLM as a fallback during evaluation.

Run both implementations against historical examples before changing production routing. Cases where they disagree are particularly valuable because they expose ambiguous labels, missing categories, poorly defined rubrics, or decisions that actually require deeper reasoning.

Migration does not have to be all-or-nothing. A practical first deployment is often one narrow classifier or verification step with an existing deterministic fallback. Additional decisions can move to Jev only after the team has measured their behavior on real traffic.

Finally, applications should pin or monitor model behavior rather than assuming jev-latest will remain identical indefinitely. TypeSafe's customer terms allow the service and API to evolve, and decision thresholds that were calibrated against one model version should be rechecked after meaningful model changes.