Checklist
Machine-readability checklist.
Twenty questions in four areas. Most can be answered in an afternoon with a browser, your logs and your edge settings.
Search engines, AI crawlers and assistants all read your site before they quote it, summarise it or send anyone to it. This checklist covers the four areas we score in a machine-readability assessment. Use it on your own site. Any question you cannot answer is worth a look. We ran it on our own site too.
Discoverable
Can crawlers and assistants find your pages, and are they allowed to fetch them?
- Which AI crawlers does your robots.txt allow, and was that a decision?Many sites block or allow them by accident, through an old default or an edge setting without a clear owner.
- Does your edge or firewall block bots that robots.txt allows?Bot protection in front of the site can override robots.txt without anyone noticing. Check those rules as well as the file.
- Is there a sitemap, and does it list only pages that return 200?Redirects, missing pages and pages marked noindex in a sitemap waste crawl budget and blur what you want read.
- Do important pages have a canonical address that points to themselves?Language versions, campaign parameters and trailing-slash variants split one page into several in the eyes of a machine.
- Can you see in your logs which machines read your site and how often?Without this, the rest of the list cannot be measured.
Renderable
Is the content in the HTML a machine receives, or does it appear only after scripts run?
- Fetch a key page with JavaScript off. Is the main text there?Many crawlers do not run scripts. What they do not receive, they cannot read.
- Are prices, specifications and availability in the first HTML response?Product data loaded after the page renders is not seen by crawlers that skip scripts, which includes several AI crawlers.
- Are tabs, accordions and "read more" sections present in the HTML?Content can be visually hidden and still be read; content that loads only after a click usually is not.
- How long does the first byte take for a crawler from another continent?Slow origins get fewer pages crawled. Measure from outside your own region and cache.
- Do cookie banners or login walls cover content that should be public?An overlay a person can dismiss may be the only thing a machine ever sees.
Structured
Can a machine tell what each part of a page means?
- Does each page have one h1 and a heading order that matches its content?Headings are the cheapest structure there is, and assistants use them to cut pages into answers.
- Do you publish schema.org data for your main types: organisation, product, article, FAQ?Check it with a validator. Wrong structured data can mislead more than none.
- Are tables real tables and lists real lists?Layout built from generic boxes loses the relationships a machine needs to read a specification.
- Do images that carry information have text alternatives?A chart without alt text or a caption is empty to a language model.
- Is the same fact stated the same way everywhere?Two prices or two names for one product make machines guess, and they often guess wrong.
Addressable
Can one item be fetched by a stable address, without the rest of the site?
- Does every product, document and article have a permanent address?Content that exists only inside a search result or a filter state cannot be cited.
- Do old addresses redirect to their replacements after a redesign or migration?Broken links throw away what crawlers learned about your site over years.
- Can a machine fetch your content through an API or a feed as well as HTML?A content API, RSS or a product feed is often the cleanest source for an assistant.
- Do you say which content machines may use, and how?Licensing and usage signals are new and not settled. Decide your position before others decide it for you.
- Do permissions hold in every place your content is copied to?Check search indexes, feeds and APIs, not only the page itself.
Want it measured instead?
We run these checks against your site, score each area and tell you what to change first.