Public
Like
1
scrape-website
Val Town is a collaborative website to build and scale JavaScript apps.
Deploy APIs, crons, & store data – all from the browser, and deployed in miliseconds.
Please see the file scrape.tsx on how to scrape a website with a real browser through TestingBot.
Make sure to define your TestingBot key (testingbot_key) and secret (testingbot_secret) as environment variables.
Simply provide a URL and selector to scrape a specific part of a page.
For example:
console.log("Scrape result:", await ScrapeWebsite("https://en.wikipedia.org/wiki/OpenAI", "p:nth-of-type(2)"));
More options can be specified, such as injecting headers, emulating media, disabling javascript and more. Please see the Scrape Documentation for an overview of all available options.