Many apps need a gate: a point where something gets approved, held for review, or rejected. User posts, uploaded profiles, refund requests, signups that look suspicious. Doing this by hand doesn't scale; doing it with a pile of keyword rules misses context. This post walks through a gating pattern built on typed AI decisions, where the model returns a probability instead of an essay.
The three-lane gate
The core idea is simple. Instead of a binary allow/block, use three lanes:
| Probability that the item violates policy | Action |
|---|---|
| Low (e.g., < 0.2) | Auto-approve |
| Middle (0.2 – 0.8) | Human review queue |
| High (> 0.8) | Auto-hold or reject |
The thresholds are yours to set. The important part is that the middle band exists, so the model only acts alone when it's confident.
Step 1: Write questions as narrow yes/no checks
One broad question ("Is this bad?") is hard to tune. Several narrow ones are easier:
- "Does this post contain a link to an external shop?" → yes/no
- "Is the text harassing a specific person?" → yes/no
- "Is this request asking for a refund outside the stated window?" → yes/no
Narrow questions give you clearer probabilities, and you can set different thresholds for each based on how costly a mistake is.
Step 2: Send state, not just text
Context changes the answer. Include what the decision depends on: the content, the user's account age, the category it was posted in, your policy summary. A typed decision model like Jev — TypeSafe AI's decision model, documented on jevmodel.org — takes app state plus typed questions together and returns a probability per yes/no question.
Step 3: Map probabilities to lanes in code
Pseudo-code (check the docs for the actual request and response format):
p = result["contains_harassment"].probability
if p > 0.8:
hold(item, reason="harassment")
elif p > 0.2:
queue_for_review(item, score=p)
else:
approve(item)
Store the probability with the item. Reviewers can sort the queue by it and handle the most likely problems first.
Step 4: Calibrate with real data
Thresholds pulled from thin air will be wrong. Calibrate:
- Gather a sample of past items with known outcomes (approved/removed).
- Run them through the model.
- For each candidate threshold, count false approvals and false holds.
- Choose thresholds that match your tolerance — a kids' platform and a developer forum will choose very differently.
Re-run this when your policy or content mix changes.
Step 5: Keep humans in the loop
Automated gates fail in ways that matter to real people. Good practice:
- Give users a way to appeal a hold.
- Log the decision, probability and question so you can explain outcomes.
- Sample auto-approved items regularly to catch drift.
- Never let a model be the only check for high-stakes decisions (legal, safety, financial) without review.
Step 6: Add a score for prioritization
Besides yes/no, a score question can rank items: "How likely is this to need urgent attention, 1–5?" Combine it with your gate so the review queue isn't first-in-first-out but most-important-first.
Why not just use a chat model?
You can, but you'll spend effort forcing it into a fixed format, parsing its replies, and inferring confidence from wording. A model that returns typed outputs gives you a number to threshold directly. If you need to send a written explanation to the user, generate it afterward with a general LLM, once the decision is made.
Practical notes and costs
Jev is closed and hosted, so plan for network latency and service dependency in your design, and decide a fallback lane (usually "send to review") if the call fails.
On pricing, jevmodel.org is usage-based: $0.042 per 1M input tokens, output free. The Playground includes 10,000 free input tokens (100,000 after signing in), and a $9.90 one-time starter pack gives 15M input tokens that never expire. There's no unlimited free tier, so estimate your daily volume before you switch a gate on.
Summary
A good content gate has three lanes, narrow questions, calibrated thresholds and a human for the uncertain middle. Working with probabilities instead of free-text answers makes each of those pieces explicit and easier to tune.
Disclosure: jevmodel.org is an independent guide and access site for the Jev model. It is not affiliated with or endorsed by TypeSafe AI.
Top comments (1)
Some comments may only be visible to logged-in visitors. Sign in to view all comments.