Jev Browser Use

Jev Browser Use finding Airtable API examples, with the live browser, scores, and action history visible.

Describe a task. Jev picks every next move in a live cloud browser you watch in an iframe.

Setup

Two keys, both required:

Without them the page says which one is missing and asks nothing, rather than failing on the first click. /api/config reports the gap.

How the demo works

Val Town runs the app, Kernel supplies the remote browser, and Jev chooses the next action from extracted page text and controls. The live view is read-only: you can watch the browser and action log while the agent works. Action history scrolls within a bounded panel, following new steps unless you scroll up to inspect earlier ones.

Before Run, use Include site navigation (on by default) to choose whether surrounding page controls are eligible. Turn it off to focus on content; Back remains available. The setting is fixed for that run.

Each session saves actions and observed outcomes in SQLite. Returning to the same page state excludes previously attempted controls; repeating the same typed action is also blocked.

When a route stops being useful, Jev can choose Back if the previous browser entry has untried controls. Returning keeps explored routes excluded, so it can try an alternative. Back consumes a normal step and verifies the history entry and URL without waiting for DOMContentLoaded.

A run ends on completion, stuck, Stop, or a step/time limit. The app then closes the remote browser. See Scoring for how Jev evaluates controls and decides whether the goal is reached.

The loop

Rendering mermaid diagram...

Each step reads page text and controls, applies the navigation setting, and filters tried actions. Jev chooses a click, typing, Back, or a stop decision (goal reached or stuck).

The demo UI requests one step at a time and displays each result. Stop and step/time limits also end the run.

Scoring

Jev uses the task, page text, recent outcomes, and each control's label, destination, and region. After repeat filtering, up to 60 controls enter scoring. With Include site navigation on, 45 slots go to content and 15 are shared by navigation, headers, footers, sidebars, and search, with unused slots borrowed by the other group. With it off, only content controls are eligible and can use all 60 slots. Region is context, not a score bonus or penalty; the same guidance applies to every task.

StageValueMeaning and use
1 — assess each controlgoal (0–1)Does activating it lead directly to the page or thing the task names?
1 — assess each controlcloser (0–1)Is it a useful intermediate step toward the task?
1 — shortlistgoal + (1 - goal) * closerCode combines both estimates and keeps up to 24 finalists. A strong answer to either question helps a control survive.
2 — choosenextJev compares the finalists and chooses one action, eligible Back, completion, or stuck. It need not choose the highest stage-1 score.
2 — check completiongoal_reached (0–1)Does the current page already satisfy the task? Success requires both an explicit completion choice and a score of at least 0.6.

The action-log percentage is Jev's probability for its stage-2 choice, not the combined stage-1 score or a measured task-success rate. The UI also shows the model's separate confidence value. These are model estimates.

Why two stages

Stage 1 is a filter, not a ranking you should trust. On a fixture where the goal link sits beside a plausible neighbour, stage 1 ranks the neighbour first:

0.708  goal 0.21  closer 0.63  Gustave Eiffel
0.626  goal 0.22  closer 0.52  Search Wikipedia
0.622  goal 0.38  closer 0.39  Eiffel Tower      ← the goal

That's fine. Stage 1 only has to keep the goal in the finalist set, and stage 2 — which sees up to 24 finalists side by side — picks the Eiffel Tower. Two atomic questions beat one compound one: the first version asked a single "does this move us closer?", which conflates the goal with a step toward it, and it clicked the biographer. Same cost either way, since questions in a request are evaluated in parallel.

Run it yourself: tests/decide.ts needs no browser.

Background

Everything in this section is from Steve of Val Town, who got this up and running — thanks, Steve.

The thing I went looking for and didn't find

TypeSafe's docs have no computer-use guide. docs.typesafe.ai/demos.md lists exactly one demo (Smart Home Assistant). The wikiracing demo that looks like computer use only exists as a launch-blog demo — no docs page, no code, no walkthrough.

What the blog does say, in its "Nuance" sections, is the whole spec:

Jev supports a cardinality up to 255. For the higher cardinality choices, we do a 2 stage-system of scoring independently then making an explicit choice, hence the occasional slowdown.

The demo is on structured state as a data structure with text, not on images (yet…)

So Jev never sees the screen, and it never emits an action. It has no string output at all — only choice, score, noul. "Browser use with Jev" therefore can't be a model emitting tool calls. It's:

code builds a closed set out of the page → Jev ranks and chooses within it → code does the clicking.

That constraint is the design, not a workaround. Every action is a value Jev was handed, so it cannot invent a selector, hallucinate a URL, or make a type error.

Three bugs a real page found

The perceive() snippet was measured against a live Wikipedia article, not reasoned about — see @stevekrouse/browser-tester's tests/jev-perceive.ts, which runs the exact snippet. All three were real:

  1. Text came from document.body, which on Wikipedia is mostly furniture — "Jump to content Main menu Search Appearance Donate Create account". Jev was being asked to navigate a page whose text was the chrome. Now it reads the content root (article, #mw-content-text, …) and only falls back to the body.
  2. A candidate's centre could sit below the fold. The measured page had one at y=519 in a 513px viewport — an element whose top is on screen can have its centre off it, and the centre is what gets clicked. Centres are now required to be inside the viewport.
  3. Nothing below the fold was reachable at all. Every candidate is now tagged with data-jev in the live DOM, so act() can scroll it into view and re-read where it actually is before clicking. That took the candidate count on the same page from 10 to 195.

Limits, honestly

  • No extracted answer output. Page text informs navigation and completion decisions, but the demo does not return a written answer or structured data. Reaching a pricing page, for example, does not produce a price report.
  • Typing is the weak spot. Jev can't write text, so harvestTexts() pulls quoted phrases out of your task and Jev chooses which to type. A task with no quoted phrase can't type anything. This is the one place the no-strings constraint genuinely costs capability rather than buying safety.
  • Back is bounded recovery. Jev can choose Back when the actual previous browser history entry has saved, untried non-editable controls. It returns one entry, counts toward the step budget, and keeps explored routes excluded. Failed or repeated Back attempts from the same state are blocked. If the previous entry is unknown or exhausted, Back is not offered; the agent does not jump over it to older pages. Back restores navigation, not submitted forms or other side effects. If no useful action remains, the run reports "stuck" rather than claiming success.
  • History sees extracted state. It compares the URL, extracted text, and control identities, not screenshots. Dynamic content can make a revisited page look new; meaningful changes outside the extracted content can be missed.
  • Selection is still bounded. Region budgets keep both content and surrounding controls represented; they do not guarantee that every relevant control on a large page reaches scoring.
  • Run limits. Up to 12 steps or 180 seconds per run, with five concurrent demo sessions by default. Use Stop to release a session when finished testing. Wikiracing can need more steps than this budget allows.

Testing

Refresh the demo and start a fresh run to exercise the full history and Back behavior. Existing sessions may lack the navigation snapshots needed to offer Back. Watch for a different choice after revisiting a page, an explicit "go back to …" entry when recovery is selected, and "stuck" when no useful action remains. Back is conditional; it will not appear on every run.

Regression scripts can be run individually with Val Town's run-file tool:

  • tests/navigation-setting.ts: 12 checks for content-only filtering, full content quotas, Back eligibility, defaults, and per-session SQLite persistence. Creates no browser.

  • tests/candidates.ts: 15 checks for balanced quotas, borrowing unused slots, ordering, region context, and history filtering before allocation. Uses model stubs and creates no browser.

  • tests/regions-browser.ts: 13 checks with generic long-page fixtures and a live Val Town pricing navigation regression. Covers footer controls after 320 content links and content after 320 navigation links. Makes live Jev calls and deletes its fixture browser.

  • tests/history.ts: 38 checks covering repeat filtering, Back eligibility, invalid choices, and SQLite persistence. Uses model stubs and creates no browser.

  • tests/layout-browser.ts: 15 checks with mocked run APIs for layout, scrolling, auto-follow, and mobile overflow. Creates and deletes one fixture browser; makes no model calls.

  • tests/back-browser.ts: 14 checks in a real Kernel browser against fixture pages, including returning to an untried route, rejecting stale Back decisions, same-document navigation, and returning before DOMContentLoaded on a page with a blocked script. Makes no model calls and deletes its test browser in a finally block.

  • tests/decide.ts: the original decision fixture; makes live Jev calls without creating a browser.

The candidate, history, and region/browser suites passed after the region changes. The live pricing check selected Pricing from navigation and reached the pricing page using the general scoring instructions. It checks navigation, not extraction of the cheapest plan's price. The 14 Back checks passed after removing the load-event wait; the 15 layout checks also passed. These checks do not establish success on every task or a full Wikipedia race.

Files

See AGENTS.md for the file map.