Screaming Frog

Screaming Frog SEO Spider is a website crawler for technical SEO audits, developed by the UK search marketing agency Screaming Frog. It crawls sites to identify SEO issues, warnings and opportunities affecting search performance and user experience, covering errors, redirects, metadata, duplicate content, directives, site structure and more.

The tool runs on Windows, macOS and Linux and audits over 300 SEO issues. Key capabilities include broken link and redirect analysis, page title and meta description checks, XML sitemap generation, JavaScript rendering via integrated Chromium, site architecture visualisations, crawl scheduling and comparison, custom extraction with XPath and regex, structured data and accessibility validation, and integrations with Google Analytics, Search Console, PageSpeed Insights and Looker Studio. It also supports crawling with OpenAI and Gemini, and the company offers a separate Log File Analyser for server log analysis.

SEO Spider is aimed at SEO professionals and agencies. A free version covers core crawling with a 500-URL limit; a paid licence removes the limit and unlocks advanced features, licensed per user on an annual basis with renewal required, and bulk discounts apply for purchases of five or more licences.