# Palo: a census of the South African web

> Palo, by Dirišwago, finds websites under .za, visits them, and records how they link to one another. It publishes counts, graph measures and a record for every site it holds. The data is CC BY 4.0; credit "Palo, Dirišwago (palo.dirishwago.co.za)".

As of the analysis on 17 October 2026, 16:22 SAST: 250,119 sites found, of which 192,065 active, 45,353 dead, 10,820 blocked and 1,881 unavailable; 433,357 links between .za sites. An analysis runs about every two hours.

## Start here

- [Census summary](https://palo.dirishwago.co.za/api/summary): every census-wide figure, with definitions, methods and limitations (plain text)
- A site's record: https://palo.dirishwago.co.za/explore/{domain}.md, for example [domains.co.za](https://palo.dirishwago.co.za/explore/domains.co.za.md), [status.domains.co.za](https://palo.dirishwago.co.za/explore/status.domains.co.za.md), [sacoronavirus.co.za](https://palo.dirishwago.co.za/explore/sacoronavirus.co.za.md). It gives the site's status, what it declares about itself, its place in the link graph, and the .za sites it links to and from, 50 a page with a link to the next page; each linked site is a link to its own record
- A name the census does not hold answers 404 with a short explanation. If it is a well-formed .za name with an address in DNS, Palo queues it, and its record appears once a crawler has visited it

## The census over time

- [Sites by status](https://palo.dirishwago.co.za/census/status.md): Every .za site in the census, counted by what happened when Palo last visited it.
- [Islands and unrecognised sites](https://palo.dirishwago.co.za/census/connection.md): Of the active sites in the census, the share with no links to or from any other site in the census (islands), and the share that link out but have no links in (unrecognised).
- [Largest connected group](https://palo.dirishwago.co.za/census/largest-group.md): The largest group of sites in which each can reach every other by following links, as a share of the active sites in the census.
- [Links per active site](https://palo.dirishwago.co.za/census/links.md): The links between .za sites in the census, divided by the number of active sites.
- [Links that leave .za](https://palo.dirishwago.co.za/census/links-leaving.md): Of the links from active sites to other domains, the share that go to domains outside .za. Each site and destination domain count once, however many pages carry the link.
- [Timeline, CSV](https://palo.dirishwago.co.za/api/timeline.csv): every figure, per South African day (also as [JSON](https://palo.dirishwago.co.za/api/timeline))
- [Events](https://palo.dirishwago.co.za/api/events): changes to how the census counts or what it covers, so a step in a figure is not read as a change in the .za web itself (JSON)

## Questions across the census

Each answers in Markdown, 50 sites a page with a link to the next; each site links to its record.

- Search names and the titles sites declare, for every word in any order: https://palo.dirishwago.co.za/census/search.md?q={words}, for example [clinic](https://palo.dirishwago.co.za/census/search.md?q=clinic)
- [Sites by zone](https://palo.dirishwago.co.za/census/zones.md) (co.za, gov.za, ac.za …), and lists by zone, status, declared language and schema.org type: https://palo.dirishwago.co.za/census/sites.md?zone={zone}&status={status}&language={code}&type={type}&sort={order}, for example [active gov.za sites](https://palo.dirishwago.co.za/census/sites.md?zone=gov.za&status=active)
- Rankings of the analysed graph: [PageRank](https://palo.dirishwago.co.za/census/top/pagerank.md), [PageRank without footer-only links](https://palo.dirishwago.co.za/census/top/pagerank-body.md), [PageRank between registered names](https://palo.dirishwago.co.za/census/top/pagerank-between-names.md), [Betweenness](https://palo.dirishwago.co.za/census/top/betweenness.md), [Links in](https://palo.dirishwago.co.za/census/top/links-in.md)
- The shortest chain of links between two sites, both ways: https://palo.dirishwago.co.za/census/path.md?from={domain}&to={domain}

## Bulk data

- [Bulk data](https://palo.dirishwago.co.za/data.md): every site and every link as gzipped CSV, written nightly from one snapshot, with a description of each column: [sites.csv.gz](https://palo.dirishwago.co.za/data/sites.csv.gz), [links.csv.gz](https://palo.dirishwago.co.za/data/links.csv.gz), and their row counts, sizes and checksums in [manifest.json](https://palo.dirishwago.co.za/data/manifest.json)

## Prompts

- [Prompts](https://palo.dirishwago.co.za/prompts.md): questions people can ask an AI assistant about the census, each with the census links that answer it; for people at https://palo.dirishwago.co.za/prompts

## Reports

Dated reports and updates; each keeps its figures as of publication.

- [How the analysis changed](https://palo.dirishwago.co.za/reports/2026-10-16-how-the-analysis-changed.md) (update, 16 October 2026): On 16 October 2026 Palo moved its graph analysis to a faster library. Betweenness is now exact, PageRank is now fully worked out, and reach was corrected for more than 32,000 sites. What changed, and by how much.
- [Malformed names removed from the census](https://palo.dirishwago.co.za/reports/2026-10-15-malformed-names-removed.md) (update, 15 October 2026): On 15 October 2026 Palo removed 4,434 entries that were not single website names, most of them counted as dead. What they were, and how the figures moved.
- [Reading the census, for people and AI assistants](https://palo.dirishwago.co.za/reports/2026-10-15-reading-the-census.md) (update, 15 October 2026): Every site in the census now has a record, the census can be searched, listed and ranked, and all of it can be read by AI assistants as well as people. Hlahlatha was retired. What changed, and where to find it.
- [How the counting changed](https://palo.dirishwago.co.za/reports/2026-10-13-how-the-counting-changed.md) (update, 13 October 2026): In October 2026 Palo changed how it decides whether a site is alive, read a new source of domains, and began recording more about each site. What changed, and by how much.
- [The .za web, September 2026](https://palo.dirishwago.co.za/reports/2026-10-13-the-za-web-september-2026.md) (report, 13 October 2026): The essay that opened Palo until October 2026: what the census found under its first counting rules.

## Notes

- The census holds the .za sites Palo has found so far and is unlikely to be complete.
- Only links between .za sites in the census are in the graph; links from outside .za, social media and search engines are not observed.
- Figures before 11 October 2026 were counted under earlier rules and are not directly comparable with later ones; see the events.
- [About Palo](https://palo.dirishwago.co.za/about.md): how it finds and visits sites, and what it records