This project fetches three different IANA data files related to top-level domains each night, and stores copies of each in source control. They are small data, so it's ok.

The idea is to be able to see at a glance, things like:

• How many delegated TLDs are there?
• Of these, how many are country-code TLDs vs. generics?
• How many have RDAP servers? (all the generics are supposed to have them, but for the ccTLDs it's entirely optional)
• Are there ccTLD RDAP servers that aren't in the IANA bootstrap file?

And later on, we'll add some lightweight monitoring to stay ahead of issues like this.

• What's RDAP? See: https://en.wikipedia.org/wiki/Registration_Data_Access_Protocol
• What are ccTLDs? See: https://en.wikipedia.org/wiki/Country_code_top-level_domain
• What are gTLDs? See: https://en.wikipedia.org/wiki/Generic_top-level_domain
• Which files are we keeping an eye on here?
  • The RDAP "bootstrap" file: https://data.iana.org/rdap/dns.json
  • The "all TLDs" txt file: https://data.iana.org/TLD/tlds-alpha-by-domain.txt
  • The Root Zone DB html file, which alas doesn't appear to be available in a friendlier format: https://www.iana.org/domains/root/db

In the fixture files, the IDN ccTLDs are:

• xn--kpry57d is 台灣 (tw) - via
• xn--2scrj9c is ಭಾರತ (in) - via
• xn--4dbrk0ce is ישראל (il) - via
• xn--flw351e is 谷歌 ("Google" in Chinese) - via
• xn--wgbh1c is .مصر (eg) - via

We're able to check for changes prior to downloading the files:

• The RDAP bootstrap file has both ETag and Last-Modified headers
• The TLD list file has both ETag and Last-Modified headers
• The root zone db (html) has a Cache-Control header

• Nightly data updates
• A unified, generated tlds.json file that includes all the data we need, for an eventual web interface
• An HTTP + web interface
• Add the ccTLD RDAP servers from overrides.py
• Lightweight "monitoring" checks for the ccTLD RDAP servers, which don't have the same SLAs as the generics

2025-11-11:

• Project creation
• Fixtures for all three data files
• Download & last-updated checks for all three files
• Downloaded file validation, parsing, and tests for all three files
• CLI, for understanding and munging the data, and prototyping future interfaces