Hourly discovery, cross-source duplicate checks, English translation and Telegram posting on Val Town.

• Sxodim: https://sxodim.com/almaty
• Topbilet: https://topbilet.kz/en/afisha
• Kassir.kz: https://kassir.kz/en/almaty

Ticketon is not enabled because test requests were blocked. GoVibester was assessed but not enabled because sampled entries included roundups and missing venue data.

hourly.ts -> pipeline.ts -> collection.ts/providers.ts -> SQLite -> matching.ts/dedup_store.ts -> english.ts -> telegram.ts

The existing 0 * * * * schedule runs at minute zero of each hour, including overnight. The destination defaults to @almaty_expats, using @almaty_expats_bot. At most one message attempt per clock hour, shared across all sources. An empty or unverified queue means no message.

Each run reads all three listing pages, then up to two new or due-for-refresh detail pages per source (six total). Previously checked details are due again after 12 hours. New providers can be added to the registry without another publisher or database. Each source has independent errors and timeouts; one unavailable site does not abort collection from the others. The chosen event is rechecked before publication.

Sources are allowlisted. Requests identify the crawler, enforce size/time limits, limit concurrency per host, and check robots.txt. Missing robots files (404/410) are treated as absent rules; 403, 429, server errors, invalid responses and unreachable policies block that source. Crawl delays over one second defer that source pending a slower adapter. Robots rules are not a republication licence. Only short factual metadata is used, credited and linked; full descriptions and poster images are not copied.

Only Almaty events starting in the next 14 days, with at least one hour's notice. Cancellation/sold-out information is used when present in the parsed metadata. Article publication dates and unrelated recommendations are never used as event dates. Topbilet currently collects the explicitly dated next performance on each detail page, not all hidden schedule dates. Kassir transport-only and multi-day-pass add-ons are excluded. The collection is deliberately bounded, not an exhaustive crawl of each catalogue.

Source URL + performance date still identifies the original source record. Cross-source matching then compares normalized names, local date/time, venue/address, available coordinates and numbered sessions. Normalization covers punctuation, marketing words, Cyrillic-to-Latin transliteration, and a small explicit venue-alias dictionary. It is not a universal semantic translator.

• Clear matching name + venue + exact start minute: one canonical event, other source rows marked duplicate.
• Plausible or conflicting matches: secondary rows marked review and excluded from automatic publication.
• Different days, showtimes more than 15 minutes apart, or conflicting numbered sessions remain separate.
• Prices are not used as identity, and conflicting prices are not averaged or invented.
• Already posted, sending or uncertain-delivery records take precedence and are never rewritten by matching.

All original source rows and URLs remain in events. event_matches records the canonical ID, duplicate/review decision, comparison score and reason. Scores are heuristic similarity values, not calibrated probabilities. Existing queued and recently published history is included, so adding another source does not reset posting history.

Matching is conservative but not perfect. Different translations, abbreviations, unnamed venues, entirely different titles or dates can escape detection. Review rows require human inspection; there is no review dashboard or alert yet. A future refresh may resolve a match. The matcher fails closed above 1,000 relevant records rather than silently checking only part of the queue.

events: original metadata, translation cache, queue/posted/duplicate/review state and Telegram message ID.

event_matches: cross-source relationships and reasons.

source_checks: inspected URLs, timestamps and source errors.

outbox: hour reservations, delivery status and message IDs.

locks: prevents concurrent publishers. runs: execution summaries retained for 30 days, including per-source counts and duplicate/review counts.

Uncertain Telegram sends are held, never automatically retried. Translations only cover title, venue and category. Date/time and price formatting is deterministic. Address numbers must survive translation. English listing text does not imply an English-language performance. Published posts are not automatically edited for later cancellations.

Keep TELEGRAM_BOT_TOKEN only in encrypted Val Town environment variables, never code or logs. It has already been configured for this deployment. The bot must retain channel administrator permission to post.

Optional TELEGRAM_CHAT_ID overrides the default channel, TELEGRAM_THREAD_ID selects a forum topic, and PUBLISH_ENABLED=false pauses messages while collection continues. No webhook is needed for outbound posting. No public HTTP endpoint can trigger a send.

Run tests_multi.ts for baseline and cross-source matching/parser tests. These use fixtures, send no messages and write no production data. preview.ts performs live discovery and saves real queued events, but never sends to Telegram. Running hourly.ts is a production run and may send a message.

Inspect source counts and errors in runs. Inspect held records with:

SELECT e.id, e.url, e.state, m.canonical_id, m.score, m.reason
FROM events e LEFT JOIN event_matches m ON m.event_id=e.id
WHERE e.state IN ('duplicate','review');

Incoming bot commands, community moderation, subscriber acquisition, failure alerts, cancellation updates and a review UI are outside this version.

https://docs.val.town/guides/telegram https://docs.val.town/reference/std/sqlite/usage https://docs.val.town/vals/cron https://core.telegram.org/bots/api#sendmessage https://www.rfc-editor.org/rfc/rfc9309.html