jev-browser-use

Describe a task. Jev picks every next move in a live cloud browser you watch in an iframe.

File map

PathTypePurpose
index.tshttpHono app: shell, assets, GET /source, GET /api/config, POST /api/session, POST /api/step, POST /api/session/:id/stop
decide.tsscriptThe Jev half: two-stage score-then-choose over the page's controls
candidates.tsscriptShared 75/25 content/surrounding allocation with unused-slot borrowing
tests/candidates.tsscriptGeneral allocation, scoring context, and history-filter regressions
tests/navigation-setting.tsscriptNavigation toggle filtering, quotas, Back eligibility, and persistence
tests/regions-browser.tsscriptLong-page region fixtures and live pricing navigation; deletes its browser
browser-use.tsscriptThe Kernel half: perceive() reads the page as data, act() clicks/types
kernel.tsscriptKernel REST client — browsers, live view, computer controls
sessions.tsscriptLive-browser registry, Kernel limits, and per-session SQLite history
history.tsscriptPage fingerprints, action identities, repeat filtering, snapshots, and Back eligibility
tests/history.tsscriptDeterministic loop, Back eligibility, completion, and persistence checks
tests/back-browser.tsscriptLive Kernel Back regression; creates and deletes one fixture browser
rate-limit.tsscriptBest-effort in-memory per-IP + global burst control, bucketed per endpoint
typesafe.tsscriptMinimal TypeSafe (System One) REST client + answer types
tests/decide.tsscriptRuns decide() against a fixture page state — no browser needed
frontend/root.tsxscriptHTML shell (Hono JSX); stamps immutable asset URLs
frontend/index.tsxscriptReact entrypoint
frontend/components/App.tsxscriptState machine, controls, step log, ?present=1 mode
frontend/components/Stage.tsxscriptThe square stage: task, live view iframe, caption, score strip
frontend/lib/api.tsscriptTyped client for the val's own API
frontend/favicon.svgfileIcon

The loop

perceive()  →  decide()  →  act()  →  repeat
   Kernel        Jev         Kernel

POST /api/step runs exactly one turn, and the browser drives the loop by calling it repeatedly. That keeps each request short and lets the UI render every step as it lands — which is the whole point of the demo.

Notes

  • Jev cannot emit an action. It has no string output at all — only choice, score, noul. So this is not a model emitting tool calls: decide.ts builds a closed set out of the page, Jev ranks and chooses within it, and ordinary code does the clicking. That's the constraint the whole design hangs on, and it's why the model can't hallucinate a selector or invent a URL.
  • Two stages, per TypeSafe's own wikiracing demo ("Jev supports a cardinality up to 255... for the higher cardinality choices, we do a 2 stage-system of scoring independently then making an explicit choice"). Stage 1 asks two noul questions per candidate — a filter, it never picks. Stage 2 is one choice over the top FINALISTS, eligible __back__, __done__, and __stuck__. Constants live at the top of decide.ts.
  • Regions are context, not a relevance rule. Keep content, navigation, header, footer, sidebar, and search controls. Shared allocation reserves 45 content / 15 surrounding scoring slots and borrows unused slots; history filtering comes first. Collection also balances its 250 slots (187/63), scanning up to 250 eligible controls per group so late footer controls can survive a long content section. Scoring instructions are general across tasks.
  • Navigation is a per-run preference. includeNavigation defaults to true, is validated by /api/session, and persists as sessions.include_navigation. The checkbox is locked during a run; presets do not override it. When false, perceive() filters to content before fingerprints and snapshots, with all 250 collection slots and 60 scoring slots available to content. Back remains independent of this region filter. This uses DOM regions, not control names.
  • History is server-owned. history.ts fingerprints the URL, extracted text, and sorted control identities. sessions.ts persists attempts in SQLite; the next observation records the outcome. Repeated non-editable controls in the same state are filtered before scoring. Exact repeated typed actions stop before execution. Invalid or exhausted choices report stuck, not success.
  • Back is a normal, budgeted action. Perception reads Chromium history entry IDs via CDP. Back is offered only when the immediately previous entry has a saved snapshot with untried non-editable controls. Execution rechecks both entry IDs and the destination URL, calls CDP navigateToHistoryEntry, and polls for the expected entry and URL for up to eight seconds. It does not wait for DOMContentLoaded; same-document history is supported. Back attempts are saved before execution; failures and repeats are excluded. Older sessions without navigation snapshots cannot offer Back until a known parent snapshot exists. Browser navigation does not undo side effects.
  • Jev never sees the screen. It gets { url, title, visible_text, candidates } as text, plus the goal and recent actions. No screenshots — matching the blog's note that the Doom demo runs "on structured state as a data structure with text, not on images (yet…)".
  • Typing is the one thing Jev can't source. It can't write text, so harvestTexts() pulls quoted phrases out of the user's own task and Jev chooses which to type. A task with no quoted phrase simply can't type.
  • Kiosk mode is load-bearing, not cosmetic. It removes the browser chrome so the window is the page viewport, which makes getBoundingClientRect() coordinates equal the screen coordinates the computer controls click. With chrome in the way every click lands a toolbar-height too high.
  • Clicking always moves the cursor first, via moveMouse with Kernel's default Bézier path. It's not just for the video — hover-sensitive menus ignore a click at coordinates the cursor never visited.
  • rate-limit.ts is burst control, bucketed per endpoint so a burst of steps can't exhaust the allowance for starting a browser: 6/min per IP and 60/min globally on /api/session, 90/min and 300/min on /api/step.
  • The browser is the thing that can run away, so the limits are aimed at it — a browser keeps running after the request that made it returns.
    • MAX_SESSION_SECONDS (180) caps one browser's life. MAX_STEPS bounds the work a run does, not the time it takes — steps are what keep a browser alive, so a client looping on /api/step could otherwise hold one open indefinitely.
    • sweepStaleSessions closes browsers whose clients walked away: nothing else notices them, because /api/step is what keeps a run moving. It runs opportunistically on session creation, before the live count is taken, so an abandoned browser can't hold a concurrency slot forever. Known gap: if nobody starts a session, the row sits live until the next one does. Kernel's 240s idle timeout still deletes the browser itself, so it's a stale row rather than a browser that outlives its welcome.
    • MAX_LIVE_SESSIONS is 5, matching Kernel's Developer-tier ceiling of 5 concurrent browsers. Above that the sixth createBrowser is refused by Kernel rather than by this guard, so raise it only with a Kernel plan that allows more. Override with MAX_LIVE_SESSIONS; a value that isn't a positive integer falls back to 5 rather than to NaN, which would disable the cap entirely.
  • Browsers must be deleted, not abandoned. finish() closes the session row and deletes the browser at Kernel. It's called on done, on stop, on step-limit, on the session time cap, and on error. The client also stops the session in a finally, so a loop that exits early can't leave a browser running. Kernel's timeout_seconds (240s idle) is the backstop.
  • data-* hooks for Playwright: data-stage, data-task, data-caption, data-status, data-scores, data-live-view, data-log, data-log-entry, data-task-input, data-url-input, data-include-navigation, data-run, data-stop.
  • ?present=1 renders only the square stage, sized to the viewport. That's the mode to screen-record.
  • Env: TYPESAFE_API_KEY (https://console.typesafe.ai/settings/keys) and KERNEL_API_KEY (https://dashboard.onkernel.com/settings/api-keys). Both are required; /api/config reports which are missing and the page says so up front.