std/blob

References

Referenced 38 times
Fork
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
import { blob } from "https://esm.town/v/std/blob?v=10";
function isJSON(input: string | null | undefined) {
if (!input || input === null) return false;
try {
JSON.parse(input);
return true;
} catch (e) {
return false;
}
}
function processInputHtml(html: string) {
let output = html;
// strip out backticks which gpt loves generating
output = output.replaceAll("```html", "");
output = output.replaceAll("```", "");
return output;
}
function formatPage(key: string, data: string) {
return `
${data}
<br />
<br />
hosted by
<a href="https://www.val.town/v/jamiedubs/valtownGeocities" target="_blank">
valtownGeocities
</a>
| key: ${key}
`;
}
export const valtownGeocities = async (req: Request) => {
const searchParams = new URL(req.url).searchParams;
const format = searchParams.get("format") ?? "html";
const key = searchParams.get("key") ?? "default";
// if (!key) throw new Error("missing ?key=");
console.log("hello", { format, key });
let data;
const oldData = await blob.getJSON(key);
if (req.method == "GET") {
data = { data: oldData };
} else if (req.method == "POST") {
const rawData = await req.text();
console.log("received rawData...", rawData);

A simple Rest API that allows for you GPT to save and recall snippets of data (memories). You can read my blog post explaining it in detail here: xkonti.tech

Demonstration

First conversation:

FirstConversation.png

What GPT sent do the API:

{
  "name": "Life Essence and Biological Processes",
  "description": "Explore the role of life essence in enabling biological processes, with a focus on how dimensional interference, particularly from the Incorporeal dimension, facilitates the existence of these processes. This exploration should delve into the complex interplay between the Physical and Incorporeal dimensions, examining how life essence acts as a crucial element in the emergence and sustenance of life in the Physical dimension.",
  "summary": "Expanding on the role of life essence in biological processes, emphasizing the interaction between the Physical and Incorporeal dimensions.",
  "reason": "To deepen the understanding of how life essence, influenced by dimensional interference, is essential for biological processes in the novel's universe."
}

Separate conversation somewhere in the future:

Second Conversation.png

Setup

There are several steps to set up the API:

  • deploy and configure the API
  • create the API key for your GPT
  • add an action for the API in you GPT
  • add prompt section to your GPT so that it can use it properly

Deploying the API on Val Town

Deploy your own memory API. You can fork the following Val to do it: https://www.val.town/v/xkonti/memoryApiExample

In the code configure the appropriate values:

  • apiName the name of your API - used in the Privacy Policy (eg. Memory API)
  • contactEmail - the email to provide for contact in the Privacy Policy (eg. some@email.com)
  • lastPolicyUpdate - the date the Privacy Policy was last updated (eg. 2023-11-28)
  • blobKeyPrefix - the prefix for the blob storage keys used by your API - more info below (eg. gpt:memories:)
  • apiKeyPrefix - the prefix for you API Keys secrets - more info below (eg. GPTMEMORYAPI_KEY_)

Create API keys

The Memory API is designed to serve multiple GPTs at the same time. Each GPT should have it's own unique name and API key.

The name is used for identifying the specific GPT and appended to both:

  • blobKeyPrefix- to maintain separate memory storage from other GPTs
  • apiKeyPrefix - to maintain separate API key for each GPT
  1. Please pick a unique alphanumeric name for your GPT. For example personaltrainer.
  2. Generate some alphanumeric API key for your GPT. For example Wrangle-Chapped-Monkhood4-Domain-Suspend
  3. Add a new secret to your Val.town secrets storage. The Key should be the picked name prefixed by apiKeyPrefix. Using the default it would be GPTMEMORYAPI_KEY_personaltrainer. The value of the secret should be the API key itself.

The memories of the GPT will be stored in the blob storage under the key blobKeyPrefix + name, for example: gpt:memories:personaltrainer.

Adding GPT action

  1. Add a new action in your GPT.
  2. Get the OpenAPI spefication by calling the /openapi endpoint of your API
  3. Change all <APIURL> instances within the specification to the url of your deployed API. For example https://xkonti-memoryapiexample.web.val.run
  4. Set the authentication method to basic and provide a base64 encoded version of the <name>:<apiKey>. For example: personaltrainer:Wrangle-Chapped-Monkhood4-Domain-Suspend -> cGVyc29uYWx0cmFpbmVyOldyYW5nbGUtQ2hhcHBlZC1Nb25raG9vZDQtRG9tYWluLVN1c3BlbmQ=
  5. Add the link to the privacy policy, which is the /privacy endpoint of your API. For example: https://xkonti-memoryapiexample.web.val.run/privacy

Adding the prompt section

To make your GPT understand the usage of your new action you should include usage instruction in your prompt. Here's an example of such instructions section:

# Long-term memory

At some point the user might ask you to do something with "memory". Things like "remember", "save to memory", "forget", "update memory", etc. Please use corresponding actions to achieve those tasks. User might also ask you to perform some task with the context of your "memory" - in that case fetch all memories before proceeding with the task. The memories should be formed in a clear and purely informative language, void of unnecessary adjectives or decorative language forms. An exception to that rule might be a case when the language itself is the integral part of information (snippet of writing to remember for later, examples of some specific language forms, quotes, etc.).

Structure of a memory:
- name - a simple name of the memory to give it context at a glance
- description - a detailed description of the thing that should be remembered. There is no length limit.
- summary - a short summary of the memory. This should be formed in a way that will allow for ease of understanding which memories to retrieve in full detail just by reading the list of summaries. If there are some simple facts that have to be remembered and are the main point of the memory they should be included in the summary. The summary should also be written in a compressed way with all unnecessary words skipped if possible (more like a set of keywords or a Google Search input).
- reason - the reason for the remembering - this should give extra information about the situation in which the memory was requested to be saved.

The memory accessed through those actions is a long-term memory persistent between various conversations with the user. You can assume that there already are many memories available for retrieval.

In some situations you might want to save some information to your memory for future recall. Do it in situations where you expect that some important details of the conversation might be lost and should be preserved.

Analogously you can retrieve memories at any point if the task at hand suggests the need or there isn't much information about the subject in your knowledge base.
Readme
Fork
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
import * as uuid from "https://deno.land/std/uuid/mod.ts";
import { blob } from "https://esm.town/v/std/blob";
import { getPolicy } from "https://esm.town/v/xkonti/memoryApiPolicy";
import { Hono } from "npm:hono@3";
export const handleMemoryApiRequest = async (
req: Request,
apiName: string,
contactEmail: string,
lastPolicyUpdate: string,
blobKeyPrefix: string,
apiKeyPrefix: string,
) => {
// ==== HELPERS ====
const getMemoriesKey = (key: string): string => {
return `${blobKeyPrefix}${key}`;
};
const getMemories = async (key: string) => {
return await blob.getJSON(getMemoriesKey(key), []);
};
const setMemories = async (memories: any, key: string) => {
await blob.setJSON(getMemoriesKey(key), memories);
};
const verifyRequest = (c): { memoriesKey: string; error: any } => {
// Verify API key coming as a Bearer header
const authHeader = c.req.headers.get("Authorization");
if (!authHeader || !authHeader.startsWith("Basic ")) {
console.error("Missing or invalid authorization header");
return { memoriesKey: "", error: c.text("Unauthorized", 401) };
}
// Extract and decode the base64 encoded credentials
const base64Credentials = authHeader.split(" ")[1];
const credentials = atob(base64Credentials);
// Split into user and password
const [key, token] = credentials.split(":");
if (key == null || key === "") {
console.error("No memory key in authorization header");
return { memoriesKey: "", error: c.text("Forbidden", 403) };
}
const expectedKey = Deno.env.get(apiKeyPrefix + key) ?? null;
if (token !== expectedKey) {
console.error("Invalid API KEY header");
return { memoriesKey: "", error: c.text("Forbidden", 403) };
}
return { memoriesKey: key, error: null };

Uses instructor and open ai (with gpt-4-turbo) to process any content into a notion database entry.

Use addToNotion with any database id and content.

await addToNotion(
  "DB_ID_GOES_HERE",
  "CONTENT_GOES HERE"//"for example: $43.28 ordered malai kofta and kadhi (doordash) [me and mom] jan 3 2024"
);

Prompts are created based on your database name, database description, property name, property type, property description, and if applicable, property options (and their descriptions).

Supports: checkbox, date, multi_select, number, rich_text, select, status, title, url, email

  • Uses NOTION_API_KEY, OPENAI_API_KEY stored in env variables and uses Valtown blob storage to store information about the database.
  • Use get_notion_db_info to use the stored blob if exists or create one, use get_and_save_notion_db_info to create a new blob (and replace an existing one if exists).
Readme
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
import { blob } from "https://esm.town/v/std/blob";
import process from "node:process";
import Instructor from "npm:@instructor-ai/instructor";
import { Client } from "npm:@notionhq/client";
import OpenAI from "npm:openai";
import { z } from "npm:zod";
const NOTION_API_KEY = process.env.NOTION_API_KEY;
const notion = new Client({
auth: NOTION_API_KEY,
});
const MAGIC_AI_FILL_SYMB = "✨";
const supported_notion_props = {
"checkbox": "boolean",
"date": "date",
"multi_select": "array_enum",
"number": "number",
"rich_text": "string",
"select": "enum",
"status": "enum",
"title": "string",
"url": "string_url",
"email": "string_email",
};
const oai = new OpenAI({
apiKey: process.env.OPENAI_API_KEY ?? undefined,
});
const client = Instructor({
client: oai,
mode: "TOOLS",
});
function createPrompt(title, description, properties) {
let prompt =
"You are processing content into a database. Based on the title of the database, its properties, their types and options, and any existing descriptions, infer appropriate values for the fields:\n";
prompt += `Database Title: ${title}\n`;
if (description) {
prompt += `Database Description: ${description}\n\n`;
} else {
prompt += "\n";
}
prompt += "Properties (with types and options where applicable.):\n";
Object.keys(properties).forEach(key => {

A simple website using Hono, Twind and HTMX.

Hono is a tiny web server library. Now with JSX!

Twind is a tiny Tailwind replacement

HTMX is a tiny way to add interactivity

Readme
Fork
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
/** @jsx jsx */
import { FC, jsx } from "https://deno.land/x/hono@v3.11.7/middleware.ts";
import { Hono } from "https://deno.land/x/hono@v3.11.7/mod.ts";
import { blob } from "https://esm.town/v/std/blob";
const app = new Hono();
const TopBar: FC = () => (
<div class="w-full p-4 flex font-bold place-content-between flex-row">
<a href="/">⚙️ Control Panel</a>
<a href="https://www.val.town/v/wilhelm/HTHTMX">&lt;/&gt;</a>
</div>
);
const Layout: FC = ({ children, title = "Control Panel" }) => (
<html lang="en" hidden>
<head>
<link
rel="icon"
href="data:image/svg+xml,<svg xmlns=%22http://www.w3.org/2000/svg%22 viewBox=%220 0 100 100%22><text y=%22.9em%22 font-size=%2290%22>⚙️</text></svg>"
/>
<title>{title}</title>
<script src="https://unpkg.com/htmx.org@1.9.9"></script>
<script type="module" src="https://cdn.skypack.dev/twind/shim"></script>
</head>
<body class="h-screen flex flex-col bg-gray-600 text-gray-800">
<TopBar />
{children}
</body>
</html>
);
const Toggle = ({ on }) => (
<div id="toggleSection">
<div class="font-bold text-xl flex-col">
{on ? "It's so on" : "You should turn it on"}
</div>
<div class="flex items-center justify-center p-2">
<input
hx-put="/toggle"
hx-swap="outerHTML"
hx-target="#toggleSection"
type="checkbox"
name="toggle"
checked={on}
/>
</div>
</div>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
import { fetchPelotonData } from "https://esm.town/v/andreterron/fetchPelotonData";
import { blob } from "https://esm.town/v/std/blob?v=3";
export let myPelotonWorkouts = async () => {
let pelotonCache = await blob.getJSON("pelotonCache");
if (Date.now() > pelotonCache.time + 8 * 60 * 1000) {
const value = await fetchPelotonData();
pelotonCache = {
value,
time: Date.now(),
};
await blob.setJSON("pelotonCache", pelotonCache);
}
return pelotonCache.value;
};

Comments (just add water)

A self-contained comments system Val. Just fork this val and you have a complete (but extremely minimal) comment system!

Call on the front-end using:

const MY_FORKED_VAL_URL = "https://vez-comments.web.val.run";

const getComments = async () => {
  const response = await fetch(MY_FORKED_VAL_URL);
  const json = await response.json();
  return json;
};

const addComment = async (str) => {
  try {
    const response = await fetch(MY_FORKED_VAL_URL, {
      method: "POST",
      body: JSON.stringify(str),
    });

    if (response.status >= 400 && response.status < 600) {
      /* error */
      return false;
    } else {
      /* success */
      return true;
    }
  } catch (e) {
    /* error */
    return false;
  }
};
Readme
Fork
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
import { blob } from "https://esm.town/v/std/blob?v=10";
import { email } from "https://esm.town/v/std/email?v=9";
import { Hono } from "npm:hono@3.9.2";
const KEY = import.meta.url.split("?")[0];
export async function addComment(str) {
const comments = await blob.getJSON(KEY) as Array<string> ?? [];
comments.push(str);
await blob.setJSON(KEY, comments);
await email({ text: "New Comment Alert!: " + str });
}
export async function getComments() {
return await blob.getJSON(KEY) as Array<string> ?? [];
}
const app = new Hono();
app.get("/", async (c) => c.json(await getComments()));
app.post("/", async (c) => {
const str = await c.req.json();
await addComment(str);
return c.text("Added comment!", 200);
});
export default app.fetch;
1
2
3
4
import { blob } from "https://esm.town/v/std/blob?v=3";
await blob.setJSON("myKey", { foo: "bar" });
export let blobDemo = await blob.getJSON("myKey");
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
/** @jsxImportSource https://esm.sh/hono@3.9.2/jsx **/
import { sqlite } from "https://esm.town/v/std/sqlite";
export async function sqlite_admin_tables() {
let data = await sqlite.execute(`
SELECT
name
FROM
sqlite_schema
WHERE
type ='table' AND
name NOT LIKE 'sqlite_%';`);
let tables = data.rows;
return (
<div>
<h1>SQLite Admin</h1>
<h2>Tables</h2>
<ul>
{tables.map(t => (
<li>
<a href={t.toString()}>{t.toString()}</a>
</li>
))}
</ul>
</div>
);
}
1
2
3
4
5
6
7
import { blob } from "https://esm.town/v/std/blob";
let calText = await blob.getJSON("calText") ?? [];
calText.push("a");
await blob.setJSON("calText", calText);
console.log(await blob.getJSON("calText"));
export var apricotBedbug = true;
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
import { email as sendEmail } from "https://esm.town/v/std/email?v=11";
interface VerificationEmailParams {
emailAddress: string;
html: string;
}
export async function sendVerificationEmail({ emailAddress, html }: VerificationEmailParams) {
try {
// email a confirmation link to the subscriber
await sendEmail({
to: emailAddress,
from: {
name: "Pete Millspaugh",
email: "petermillspaugh.sendVerificationEmail@valtown.email",
},
replyTo: "pete@petemillspaugh.com",
subject: "Confirm your subscription to petemillspaugh.com",
html,
});
// email myself a success notification
await sendEmail({
subject: `${emailAddress} subscribed to petemillspaugh.com`,
text: `A notification for ${emailAddress} verification should come in any minute now.`,
});
} catch (error) {
const { name, message, stack } = error;
await sendEmail({
subject: `Error sending email verification to ${emailAddress}`,
html: `
<pre>
<code>
${name}
${message}
${stack}
</code>
</pre>
`,
});
}
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
import { blob } from "https://esm.town/v/std/blob";
import { Hero } from "https://esm.town/v/todepond/Hero";
import process from "node:process";
export async function setHeroes(heroes: Hero[], originalHeroes: Hero[], password: string) {
if (password !== process.env.FAME_ADMIN_PASSWORD) {
return { success: false, error: "Wrong password" };
}
const actualOriginalHeroes = await blob.getJSON("heroes");
if (JSON.stringify(actualOriginalHeroes) !== JSON.stringify(originalHeroes)) {
return { success: false, error: "Conflict" };
}
await blob.setJSON("heroes", heroes);
return { success: true };
}
Fork
1
2
3
4
5
6
7
8
9
10
11
12
13
const output = await replicate.run(
"sczhou/codeformer:7de2ea26c616d5bf2245ad0d5e24f0ff9a6204578a5c876db53142edd9d2cd56",
{
input: {
image: "https://replicate.delivery/pbxt/IngEkQmZiZ3whtbkNAiOIdCsYAWVMkmoIBJnw7t2TPgvJn5S/photo.jpg",
upscale: 1,
face_upsample: true,
background_enhance: true,
codeformer_fidelity: 0.7
}
}
);
console.log(output);
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
import { blob } from "https://esm.town/v/std/blob";
import { decrypt } from "https://esm.town/v/todepond/decrypt";
import { Supporter } from "https://esm.town/v/todepond/Supporter";
import { getSupporters } from "https://esm.town/v/todepond/getSupporters";
import process from "node:process";
export async function getVotes() {
const supporters = await getSupporters(process.env.FAME_ADMIN_PASSWORD);
const votes = {
newFractal: 0,
snakesInSnakesInSnakes: 0,
thisIsATode: 0,
};
for (const supporter of supporters) {
if (supporter.votes === undefined) {
continue;
}
if (supporter.votes.newFractal) {
votes.newFractal += 1;
}
if (supporter.votes.snakesInSnakesInSnakes) {
votes.snakesInSnakesInSnakes += 1;
}
if (supporter.votes.thisIsATode) {
votes.thisIsATode += 1;
}
}
return votes;
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
export type Hero = {
name: string;
tier: Tier;
flavour: Flavour;
supporter: number;
};
export type Tier =
| "froggy"
| "flappy"
| "beepy";
export type Flavour =
| "fire"
| "water"
| "air"
| "sand"
| "wood"
| "flower"
| "pink sand"
| "metal"
| "poison"
| "leaf"
| "void"
| "cloud";

Bluesky keyword alerts

Custom notifications for when you, your company, or anything you care about is mentioned on Bluesky.

1. Query

Specify your queries in the queries variable.

Bluesky doesn't support boolean OR yet so we do a separate search for each keyword.

2. Notification

Below I'm sending these mentions to a private channel in our company Discord, but you can customize that to whatever you want, @std/email, Slack, Telegram, whatever.

Readme
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
import { blob } from "https://esm.town/v/std/blob";
import { discordWebhook } from "https://esm.town/v/stevekrouse/discordWebhook";
import { fetchJSON } from "https://esm.town/v/stevekrouse/fetchJSON";
import { searchBlueskyPosts } from "https://esm.town/v/stevekrouse/searchBlueskyPosts";
const encounteredIDs_KEY = "bluesky_encounteredIDs";
const queries = ["val town", "val.town"];
export const blueskyAlert = async () => {
let posts = (await Promise.all(queries.map(searchBlueskyPosts))).flat();
// filter for new posts
let encounteredIDs = await blob.getJSON(encounteredIDs_KEY) ?? [];
let newPosts = posts.filter((post) => !encounteredIDs.includes(post.tid));
await blob.setJSON(encounteredIDs_KEY, [
...encounteredIDs,
...newPosts.map((post) => post.tid),
]);
if (newPosts.length === 0) return;
// format
const content = posts.map(
post => `https://bsky.app/profile/${post.user.handle}/post/${post.tid.split("/")[1]}`,
).join(
"\n",
);
// notify
await discordWebhook({
url: Deno.env.get("mentionsDiscord"),
content,
});
return newPosts;
};