Log inSign Up
Avatar

dvsj

Just hangin' around, lurking. :)
4 public vals
Joined April 2, 2024

image.png

You know how when you paste a URL in Twitter or Slack it shows you a nice preview? This val gives you that data.
Given a URL, this will return metadata about the website like title, description, imageURL, image as base64 etc.

Sample input - paste this in your URL bar

https://dvsj-GetWebsiteMetadata.web.val.run?targetURL=https://dvsj.in
https://dvsj-GetWebsiteMetadata.web.val.run?targetURL=<your-target-url-here>

Sample output:

{
   status: 200,
   url: "https://dvsj.in",
   title: "Dav-is-here ➜",
   description: "Davis' not-so-secret stash",
   imgUrl: "https://www.dvsj.in/cover-picture.png",
   imgData: "data:image/png;base64,qwertyblahblah"
}

FAQ:
Why is imgData sent when imgUrl is already present?
Because you shouldn't hotlink images from 3rd parties. Store the base64 image on your server and use it in your app.
It's unfair to use their server bandwidth and could be a security issue for you if they change the content of the link later.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
// Although you probably want this, you can take a peek at the implementation at https://www.val.town/v/dvsj/getOpengraphMetadata too.
import getOpengraphMetadata from "https://esm.town/v/dvsj/getOpengraphMetadata";
export default async function(req: Request): Promise<Response> {
// First extract query param from the URL
const url = new URL(req.url);
// People forget capitalization all the time. Let's go easy on them and check a few queryParam keys. :)
const targetUrlKeys = [
"targetURL",
"TargetURL",
"targetUrl",
"TargetUrl",
"Targeturl",
"targeturl",
"GIMME_THE_META_DAMMIT",
];
let targetURL = null;
for (let i = 0; i < targetUrlKeys.length; i++) {
targetURL = url.searchParams.get(targetUrlKeys[i]);
if (targetURL != null) {
break;
}
}
// URL isn't present. Oopsie!
if (targetURL == null || targetURL.trim() == "") {
return Response.json({
"error":
"targetURL is missing in query params. If you want to get the metadata for `https://dvsj.in`, call this function in this format: `https://fn-url?targetURL=https://dvsj.in`",
});
}
// Let's go!
return Response.json(await getOpengraphMetadata(targetURL));
}
2

Add an email entry option to your static website/blog. Easy peasy. 🚀

newsletter.png

PoV: You just hacked together a portfolio website or launched a blog as a static website. Some people who visit might be interested in hearing more from you. ❤️ But you don't want to get lost building your backend, API, DB or fancy apps like SubstandardStack or MailMachineGun for people to sign up to your newsletter. 😩

All you want is a simple input box on your website - when someone types their email, username or social link in and submits it, you want to be notified.

psst...do you want another one that uses the DB instead of email so you can look up all entries at once? Let me know and I'll get cooking!

Quickstart

Call the val URL with data in the query param userContact . That's it!

// Format
`https://<val_url>?userContact=<mandatory_primary_contact>`

// Examples
`https://dvsj-subscribeToNewsletter.web.val.run?userContact=dav.is@zohomail.in`
`https://dvsj-subscribeToNewsletter.web.val.run?userContact=CatalanCabbage`

Bonus

Have extra data apart from email?

Pass any encoded data in the queryParam userData, will be included in the email. It's optional.

// Format
`https://<val_url>?userContact=<mandatory_primary_contact>&userData=<optional_any_data>`

//Examples
`https://dvsj-subscribeToNewsletter.web.val.run?userContact=dav.is@zohomail.in&userData={"time": "2/2/1969", "twitter": "https://twitter.com/dvsj_in"}`

// Note: All values should be URL encoded. Example:
let userData = {"time": "2/2/1969", "twitter": "https://twitter.com/dvsj_in"}
let encodedUserData = encodeURIComponent(userData) //This should go in the query param

Want bot protection?

Add a simple question to your website, like "okay, so what's one minus one?".
In the val, set isBotProtectionOn = true and botProtectionAnswer="0".
When you call the val, include the encoded user's answer to the bot question as botProtection query param.
Answer will be compared with botProtectionAnswer; if the answer is wrong, the request is rejected.

// Format
`https://<val_url>?userContact=<mandatory_primary_contact>&userData=<optional_any_data>&botProtection=<answer>`

//Examples
`https://dvsj-subscribeToNewsletter.web.val.run?userContact=dav.is@zohomail.in&botProtection=123`

Add it to your website

Want to add it to your site but get a headstart coding it? Use this ChatGPT prompt to get code for your website!

I'm building a simple form submission component. It should a submit button and these 2 input boxes: 
1. "userContact" to get the user's email (mandatory)
2. "userData" to get a custom message from the user (optional)

On clicking the submit button: 
1. Both input values should be encoded using "encodeURIComponent" 
2. A GET URL should be built in this format with query params. Include userData query param only if input is not null or empty.
`https://dvsj-subscribeToNewsletter.web.val.run?userContact=<encodedUserContact>&userData=<encodedUserData>`
3. The GET URL should be called and result printed in the console.

I'm using React, so make it a react component.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
import { email } from "https://esm.town/v/std/email?v=11";
// You can turn this on if needed. Check the readme!
const isBotProtectionOn = false;
const botProtectionAnswer = "0";
export default async function(req: Request): Promise<Response> {
// Get the data from URL's query params
const url = new URL(req.url);
let botProtectionInput = url.searchParams.get("botProtection");
let userContact = url.searchParams.get("userContact");
let userData = url.searchParams.get("userData");
// Check if query params are valid
const areInputsValid = validateInputs(botProtectionInput, userContact);
if (!areInputsValid.isValid) {
return Response.json({
success: false,
msg: areInputsValid.errorMsg,
});
}
// And send email!
await sendEmail(userContact, userData);
return Response.json({ ok: true });
}
async function sendEmail(userContact, userData) {
userContact = decodeURIComponent(userContact);
// Email body should have userData part only if it's present in query params
let userDataMsg = "";
if (userData != null) {
userData = decodeURIComponent(userData);
userDataMsg = `User data was: ${userData} \n`;
}
// Yay!
const subject = `You've got a new subscriber ${userContact}! 🎉`;
const text = `User with contact ${userContact} has signed up to your newsletter. \n ${userDataMsg} Let's goo! 🥂`;
await email({ subject, text });
}
function validateInputs(botProtectionInput, userContact) {
let isValid = true;
let errorMsg = "";
// Validate bot protection input only if it's turned on
if (isBotProtectionOn) {
botProtectionInput = decodeURIComponent(botProtectionInput);
if (botProtectionInput == null || botProtectionInput != botProtectionAnswer) {
isValid = false;
errorMsg += "botProtection failed, expected *** but answer was " + botProtectionInput + ". Bad bot! ";
}
}
// userContact is mandatory
if (userContact == null || userContact.trim() == "") {
isValid = false;
errorMsg +=
"userContact is missing. It should be a queryParam in the URL, like https://<val_url>?userContact=<user_contact> where user_contact is email, username, social link etc. Eg: https://<val_url>?userContact=dav.is@zohomail.in";
}
return { isValid, errorMsg };
}
0

Pass a URL to ping a website and see if response is a 200. If it isn't, you get an email with the error response received.

Tip: Make a new, separate cron val to call this isMyWebsiteDown() periodically to check the website's health every 15 minutes or so.
Your separate cron val's code will look something like this:

//Import this val
import isMyWebsiteDown from "https://esm.town/v/dvsj/isMyWebsiteDown";

//Add all the websites you want to check
isMyWebsiteDown(`https://dvsj.in`);
isMyWebsiteDown(`https://blog.dvsj.in`);

export default {};
Fork
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
import { email } from "https://esm.town/v/std/email?v=11";
import { fetch } from "https://esm.town/v/std/fetch";
export default async (URL) => {
const [date, time] = new Date().toISOString().split("T");
let ok = true;
let reason: string;
try {
const res = await fetch(URL);
if (res.status !== 200) {
reason = `(status code: ${res.status})`;
ok = false;
}
} catch (e) {
reason = `couldn't fetch: ${e}`;
ok = false;
}
if (ok) {
console.log(`Website up (${URL})`);
} else {
const subject = `Website down (${URL})`;
const text = `At ${date} ${time} (UTC), ${URL} was down (reason: ${reason}).`;
console.log(subject);
console.log(text);
await email({ subject, text });
}
};
11
0
Fork
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
// Might look complicated, don't be scared! Just follow the "//", I'll walk you through it.
import { Buffer } from "node:buffer";
import jsdom from "npm:jsdom";
const JSDOM = jsdom.JSDOM;
export default async function getMetadata(url) {
// Do we have a URL?
if (url == null || url.trim() == "") {
return { "status": 400, "error_msg": "Valid URL is required as input. URL was empty." };
}
let resp = await fetch(url);
let text = await resp.text();
// We parse the page as HTML and then pick the data we want.
// Right now we have the page as a looong string, let's convert it into a HTML document
let frag = new JSDOM(text).window.document;
// Get the data we want and send
let imgUrl = getImageUrl(frag, url);
let imgData = await getImageDataFromUrl(imgUrl);
let title = getTitle(frag);
let description = getDescription(frag);
return { "status": 200, "url": url, title, description, imgUrl, imgData };
}
function getImageUrl(frag, url) {
let imgUrl = "";
let selectors = [
"meta[property=\"og:image:secure_url\"]",
"meta[property=\"og:image:url\"]",
"meta[property=\"og:image\"]",
"meta[name=\"twitter:image:src\"]",
"meta[property=\"twitter:image:src\"]",
"meta[name=\"twitter:image\"]",
"meta[property=\"twitter:image\"]",
"meta[itemprop=\"image\"]",
];
// Get image from the HTML fragment
let element;
for (let i = 0; i < selectors.length; i++) {
element = frag.querySelector(selectors[i]);
if (!imgUrl && element && element.content) {
imgUrl = element.content;
}
}
// Still not present? Try to get the image of the author and use it instead
element = frag.querySelector("img[alt*=\"author\" i]");
if (!imgUrl && element && element.src) {
imgUrl = element.getAttribute("src");
}
// Still not present? Well let's take ANY visible image from the page.
// You leave me no choice, my friend.
element = frag.querySelector("img[src]:not([aria-hidden=\"true\"])");
if (!imgUrl && element && element.src) {
imgUrl = element.getAttribute("src");
}
if (imgUrl !== "") {
// Some img src URLs are relative. In that case, convert to absolute.
// https://stackoverflow.com/a/44547904/12415069
imgUrl = new URL(imgUrl, url).href;
}
return imgUrl;
}
async function getImageDataFromUrl(url) {
const response = await fetch(url);
const base64data = Buffer.from(await response.arrayBuffer()).toString("base64");
return "data:image/png;base64," + base64data;
}
function getTitle(frag) {
let element;
let title = "";
let selectors = [
"meta[property=\"og:title\"]",
"meta[name=\"twitter:title\"]",
"meta[property=\"twitter:title\"]",
];
// Get image from the HTML fragment
for (let i = 0; i < selectors.length; i++) {
element = frag.querySelector(selectors[i]);
if (element && element.content) {
title = element.content;
return title;
}
}
// Not present? Take the text that shows up in the browser tab
element = frag.querySelector("title");
if (element && (element.innerText || element.text || element.textContent)) {
title = element.innerText || element.text || element.textContent;
return title;
}
return title;
}
2
0
Next