Quality
Spider
Also called search spider.
Spider is a crawler that walks links, often from a search engine. The name comes from crawling the web.
How it is measured
It starts from known URLs, reads the links, queues them and visits them in turn. It identifies itself in the user-agent and usually respects robots.txt.
Spiders that do not run JavaScript never appear in Tarsier. They appear in server logs.
Worked example
A bookshop's log shows a search spider fetching 14,000 URLs in a week, 9,000 of which are filter combinations like ?colour=red&size=9. Only 600 are real products.
The owner adds noindex rules to the filters and the spider spends its time on the 600.
How it differs
A spider follows links. A crawler is any automated fetcher. Bot traffic is the umbrella. A good spider is one you want, such as a search engine.
Common errors
Blocking search spiders. Letting infinite filters trap them. Believing the user-agent. Ignoring crawl budget. Counting spider fetches as visits.
In practice
Check your logs for which URLs spiders hit most and fix any trap.