Public
Hosted GitHub <-> Val Town sync service prototype
Val Town is a collaborative website to build and scale JavaScript apps.
Deploy APIs, crons, & store data – all from the browser, and deployed in milliseconds.

githubsync

Hosted GitHub <-> Val Town sync service prototype. Connect a val to a GitHub repo; we keep them in sync.

How it works

Each connection pairs one val with one repo and names exactly one side the source of truth:

  • GitHub as source: a push webhook mirrors added/modified/removed files into the val. Force pushes and 20+-commit pushes fall back to a full tree sync (including deletes).
  • Val Town as source: a 1-minute cron diffs the val's files against the repo HEAD by computing git blob shas locally and pushes one commit per change set. No diff, no commit.

Onboarding flow

  1. Sign in with Val Town (/auth/login, std/oauth with the project_rw scope).
  2. Install the shared GitHub App: /github/install redirects to https://github.com/apps/<GH_APP_SLUG>/installations/new. After install, GitHub redirects back to /github/setup?installation_id=..., which records the installation. A pasted GitHub token is the labeled dev fallback.
  3. Pick a val, a repo, a branch, and a source of truth; Connect a repo runs the initial backfill and shows a status page with recent events and a Resync button.

The shared GitHub App

There is ONE GitHub App that all users install (not one per user). Its credentials live in GH_APP_* env vars, which are the single source of truth (lib/app.ts -> sharedApp()). Installation tokens for sync are minted with installationToken(sharedApp(), installation_id) and the webhook HMAC is verified against GH_APP_WEBHOOK_SECRET.

Creating a brand-new app (operator-only, rarely needed) still lives behind /github/new-app + /github/manifest-callback, guarded by DEV_SECRET. After creating one, copy its credentials into the GH_APP_* env vars to promote it.

Dev mode

POST /dev/connect with Authorization: Bearer $DEV_SECRET and JSON {val, repo, direction, vt_token, gh_token, branch?, register_webhook?} creates a connection without the browser flows. Token values of the form env:NAME resolve from this val's environment at use time and never land in the database.

Environment variables

  • OAUTH_STATE_ENCRYPTION_KEY - required by std/oauth
  • DEV_SECRET - bearer token guarding /dev/* and the operator-only /github/new-app
  • VT_DEV_TOKEN, GITHUB_DEV_TOKEN - dev-mode credentials referenced as env: tokens
  • GH_APP_ID, GH_APP_SLUG, GH_APP_CLIENT_ID, GH_APP_CLIENT_SECRET, GH_APP_WEBHOOK_SECRET, GH_APP_PEM - the shared GitHub App (single source of truth)

File map

  • index.tsx - HTTP entrypoint: oauth middleware + routes + dashboard
  • cron.ts - 1-minute sweep of Val-Town-as-source connections
  • db.ts - SQLite schema (users, apps, installations, connections, events) + queries
  • lib/app.ts - the shared GitHub App from GH_APP_* env vars (sharedApp(), installUrl())
  • lib/github.ts - GitHub API, app JWT (PKCS#1 pem via node:crypto), blob sha, webhook registration
  • lib/valtown.ts - Val Town v2 API: list/read/upsert/delete val files
  • lib/creds.ts - env-token resolution, per-connection credential lookup, dev auth
  • sync/ghToVal.ts - incremental (webhook payload) and full (tree) repo-to-val sync
  • sync/valToGh.ts - blob-sha diff, one-commit val-to-repo push
  • routes/ - webhook, dev API, connections CRUD, GitHub App install/setup (+ operator create)
  • views/ - Layout, Landing, Dashboard, Status (Hono JSX + Tailwind)

Known limits (prototype)

  • Text files only; >80k-char files skipped (Val Town file API limit); only one branch per connection.
  • New val files arrive as type script/file; trigger types need setting once by hand.
  • OAuth access tokens stored at connect time are not yet refreshed when they expire; the refresh-token flow is the next step. Dev-mode env: tokens do not expire.
  • The webhook on a repo is not removed when a connection is deleted.

TODO (productionizing — handle later)

  • Rename the GitHub App to something canonical. It's currently the auto-generated valtown-sync-707f8d. Rename in the GitHub UI at github.com/settings/apps/valtown-sync-707f8d to "Val Town" (or "Val Town Sync" if the global name is taken — GitHub App names are globally unique). Renaming changes the slug, so update the GH_APP_SLUG env var afterward (or do the live-slug item below). Existing installations survive a rename — they're tied to the numeric app id, not the slug.
  • Transfer the app to the val-town GitHub org. It currently lives under a personal account; for canonical ownership use app settings → Transfer ownership.
  • Read the app slug live instead of pinning GH_APP_SLUG. Have sharedApp() / the install route fetch the slug from GET /app (via the app JWT) so a rename never needs an env-var change. Cache it to avoid an API call per request.
  • Implement OAuth access-token refresh. Val Town access tokens stored at connect time aren't refreshed on expiry, so browser-created connections break when the token lapses. Use the stored refresh token against Val Town's token endpoint. (Dev-mode env: tokens don't expire, which is why testing doesn't hit this.)
  • Scope dev connections to the dev user. listConnections() returns user_id = 'dev' rows to every signed-in user, and the access check treats them as shared — convenient for testing, but must be removed before the URL is shared.
  • Deregister the repo webhook on connection delete (and on app uninstall) so orphaned hooks don't accumulate.