Public
pomdtr
code_search_is_easy
HTTP
Code Search is Easy Earlier this week, Tom MacWright posted Code Search is Hard . He describes the research he his doing to improve the code search experience of Val Town . It was a great read, and you might have seen it trending on Hacker News . As Val Town's most active user (behind Steve Krouse, one of the founders of Val Town), I for sure agree with Tom that the search feature needs improvements. But while reading his post, I immediately thought of a different approach to the problem. And a few hours later, Val Town Search was born. Do things that don't scale How does this new shiny search engine work? Well, it's quite simple. I wrote a Deno script that fetches all vals from the Val Town API. #!/usr/bin/env -S deno run -A
import * as path from "https://deno.land/std/path/mod.ts";
const dir = path.join(import.meta.dirname!, "..", "vals");
const blocklist = Deno.readTextFileSync(
path.join(import.meta.dirname!, "blocklist.txt")
)
.split("\n")
.map((line) => line.trim())
.filter((line) => line.length > 0);
let url = `https://api.val.town/v1/search/vals?limit=100&query=+`;
const vals = [];
while (true) {
console.log("fetching", url);
const resp = await fetch(url);
if (!resp.ok) {
console.error(resp.statusText);
Deno.exit(1);
}
const { data, links } = await resp.json();
vals.push(...data);
if (!links.next) {
break;
}
url = links.next;
}
Deno.removeSync(dir, { recursive: true });
Deno.mkdirSync(dir, { recursive: true });
for (const val of vals) {
const slug = `${val.author.username}/${val.name}`;
if (blocklist.includes(slug)) {
console.log("skipping", slug);
continue;
}
const userDir = path.join(dir, val.author.username);
Deno.mkdirSync(userDir, { recursive: true });
Deno.writeTextFileSync(path.join(userDir, `${val.name}.tsx`), val.code);
} I pushed the data to a Github Repository (now private) I added a Github Action that runs the script every hour to refresh the data. #!/usr/bin/env -S deno run -A
import * as path from "https://deno.land/std/path/mod.ts";
const dir = path.join(import.meta.dirname!, "..", "vals");
const blocklist = Deno.readTextFileSync(
path.join(import.meta.dirname!, "blocklist.txt")
)
.split("\n")
.map((line) => line.trim())
.filter((line) => line.length > 0);
let url = `https://api.val.town/v1/search/vals?limit=100&query=+`;
const vals = [];
while (true) {
console.log("fetching", url);
const resp = await fetch(url);
if (!resp.ok) {
console.error(resp.statusText);
Deno.exit(1);
}
const { data, links } = await resp.json();
vals.push(...data);
if (!links.next) {
break;
}
url = links.next;
}
Deno.removeSync(dir, { recursive: true });
Deno.mkdirSync(dir, { recursive: true });
for (const val of vals) {
const slug = `${val.author.username}/${val.name}`;
if (blocklist.includes(slug)) {
console.log("skipping", slug);
continue;
}
const userDir = path.join(dir, val.author.username);
Deno.mkdirSync(userDir, { recursive: true });
Deno.writeTextFileSync(path.join(userDir, `${val.name}.tsx`), val.code);
} I created a simple frontend on top of the Github Search API that allows you to search the data. It's hosted on Val Town (obviously). That was it. I didn't have to build a complex search engine, I just used the tools that were available to me. Is this a scalable solution for Val Town? Probably not. Am I abusing the Github API? Maybe. Does it work better than the current search feature of Val Town? Absolutely! I hope that the val.town engineers will come up with a search feature that will put my little project to shame. But for now, you won't find a better way to search for vals than Val Town Search . PS: This post was written / is served from Val Town
1
stevekrouse
ssr_react_mini_starter
HTTP
Starter App for ssr_react_mini You need to export four things: loader - runs on any GET request, on the server. it accepts the Request and returns the props of your React compnent. action - runs on the server on any non-GET, ie POST, PUT, DELETE, or <form> s submit Component - your React component. it's initially server-rendered and then client-hydrated default - you should mostly leave this line alone This is framework is bleeding-edge. You'll need to read the code of the framework itself (it's very short) to understand what it's doing. If you have questions or comments, please comment below on this val! (or any of these vals)
2
Updated: July 25, 2024