Test runner

to be able to run a number of tests (e.g. on a different val). check the references for seeing how it is used. It is extracted into a val to avoid having all that clutter in the same val as your tests.

Each test is a named function (which can be async), the name is used as the name for the test.

  • the input passed as the first argument is passed to each test, great for importing assertion methods, stubs, fixed values, ... everything that you do not mutate during a test
  • if a function is async (it returns a promise) there is a timeout of 2 seconds before the test is marked as failed.
  • all tests are called in the declared order, but async tests run in parallel afterwards, so don't assume any order
  • if a test starts with skip it is not executed
  • if a test fails it throws the output, so it appears in the read box below the val and the evaluation/run is marked red
  • if all tests pass it returns the output, so it appears in the grey box and the evaluation/run is marked green.

Note: If you are using the test runner to store the result in that val, as described above, it is considered a "JSON val" and has a run button, but it also means that another of your vals could update the val with just any other (JSON) state. Alternatively you can define a function val that calls the test runner and have a separete val to keep the curretn test results, but it means after updating the tests you need to fest save that val and then reevaluate to val storing the test state.

Readme
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
import { getRaw } from "https://esm.town/v/karfau/getRaw";
import { sleep } from "https://esm.town/v/stevekrouse/sleep?v=1";
export async function testRunner<
Input extends {
val?: Ref;
},
>(input: Input, ...tests: Test<Input>[]) {
let reportHeader = "";
try {
const [userHandle, valName] = input.val || [];
if (userHandle && valName) {
const version = await getRaw([
userHandle,
valName,
], "version");
if (typeof version === "number") {
reportHeader = `Testing @${userHandle}.${valName} v${version} ${new Date().toISOString()}:\n`;
}
}
}
catch {
}
let failedCount = 0;
let skipCount = 0;
let start = Date.now();
const result = (await Promise.all(tests.map(async (t, index) => {
const name = t.name.replaceAll(/_/g, " ");
if (name.startsWith("skip")) {
skipCount++;
return `${index + 1} [${name}]`;
}
const tstart = Date.now();
try {
const p = t(input);
if (p && p instanceof Promise) {
await Promise.race([
p,
sleep(2000).then(() => {
throw "async test did not complete in time";
}),
]);
}
return `${index + 1} ${name}: passed in ${Date.now() - tstart}ms`;
}
catch (error) {
failedCount++;
return `${index + 1} ${name}: failed after ${Date.now() - tstart}ms\n ${
/^assert/i.test(error.toString()) ? error : error.stack
}`;
👆 This is a val. Vals are TypeScript snippets of code, written in the browser and run on our servers. Create scheduled functions, email yourself, and persist small pieces of data — all from the browser.
Comments
karfau avatar

v1 tracks ms each test needs and for all tests and replaces _ in test names with spaces in the output.

karfau avatar

v2,v3 improve output to separate test result from name

karfau avatar

v4 adds the error stack to the output if the Error is not an "AssertionError" (protytpe name check)

karfau avatar

v5 fixes a regression in v4 and switches to a regexpr to see i the error is an assertion error

karfau avatar

v6 avoids duplicate output for errors

karfau avatar

v10 handle skip prefix v7-v9 debugging reference issue

kajgod avatar

hey, karfau, I'm interested to try the tester, but when I click references, it says "This val has not been referenced in public vals or your vals."

can you make at least one or two examples as public vals?

karfau avatar

@kajgod Just did that and added a note about two possible usage approaches in the readme.