io.github.puneetindersingh/open-seo-crawler

What it mainly works on, as our AI model read its tool descriptions: files and code on the machine it runs on.

3 of 13 tools are described as changing something (creating, editing, deleting, sending or running).

• start_crawlon 1 other serverdescribed as changing data

  “Crawl a website with the same engine and defaults as the app (1500 pages, 5 workers, 0.4 s delay per host, depth 10, robots.txt respected; then sitemap analysis, near-duplicate content and the AI cra…”

  Publisher declares: read-only no · destructive no · open-world yes

  18 inputs, 1 required

  • urlstringrequired“Start URL, e.g.”
  • max_pagesinteger“Page cap (default 1500; 5000 or more means no cap).”
  • max_depthinteger“Click depth limit (default 10).”
  • max_workersinteger“Parallel fetches (default 5).”
  • crawl_delaynumber“Seconds between requests to one host (default 0.4).”
  • render_jsboolean“Render pages in a headless browser (for JS sites; 3 to 5 times slower).”
  • compare_no_jsboolean“With render_js: also fetch plain HTML and report what only appears with JS.”
  • ignore_robotsboolean“Crawl URLs robots.txt disallows.”
  • ignore_noindexboolean“Follow links on noindex pages too.”
  • include_patternsstring“Only crawl matching URLs, one robots.txt-style pattern per line (* any text, $ end).”
  • exclude_patternsstring“Skip matching URLs, one robots.txt-style pattern per line.”
  • user_agentstring“Blank for Chrome, a preset (chrome-mobile, googlebot, googlebot-mobile, bingbot, firefox, safari) or a full user-agent string.”
  • solve_challengesboolean“Open a browser window to pass a bot challenge if one appears (default true).”
  • sitemapboolean“Compare the crawl with the XML sitemap afterwards (default true).”
  • near_dupboolean“Find near-duplicate pages afterwards (default true).”
  • near_dup_thresholdnumber“Similarity percent for near duplicates (default 90).”
  • bot_checkboolean“Run the AI and search crawler readability check afterwards (default true).”
  • bot_sampleinteger“Pages in the readability sample (default 10).”
• crawl_statuson 2 other serversreads or searches

  “Progress of a crawl started in this session: state (crawling, sitemap, near_dup, bot_check, saving, done, stopped, error), pages so far, queue, and the saved crawl name once done.”

  Publisher declares: read-only yes

  1 input

  • jobstring“Job id from start_crawl (default: the latest).”
• stop_crawldescribed as changing data

  “Stop a running crawl like the Stop button.”

  Publisher declares: read-only no · destructive no

  1 input

  • jobstring“Job id (default: the latest).”
• list_crawlson 2 other serversreads or searches

  “Saved crawls from the last 30 days, newest first (crawls from the app and from here).”

  Publisher declares: read-only yes

  2 inputs

  • sitestring“Only crawls whose start URL or name contains this.”
  • limitinteger“Default 20.”
• crawl_summaryreads or searches

  “What the Summary tab shows, as data: pages, status codes, indexable pages, every issue grouped into errors, warnings and info with page counts and sample URLs, duplicates, redirect chains, sitemap ga…”

  Publisher declares: read-only yes

  2 inputs

  • crawlstring“Which saved crawl: "latest" (default), a crawl file name from list_crawls, a job id from start_crawl, or part of the site name.”
  • sample_urlsinteger“Sample URLs per issue (default 5).”
• get_pageson 11 other serversreads or searches

  “Rows of a saved crawl, filtered and paged.”

  Publisher declares: read-only yes

  10 inputs

  • crawlstring“Which saved crawl: "latest" (default), a crawl file name from list_crawls, a job id from start_crawl, or part of the site name.”
  • issuestring
  • severitystring
  • statusstring“A code such as 404, or a class such as 3xx.”
  • url_containsstring
  • indexableboolean
  • fieldsstring“Comma-separated field names, or "all".”
  • limitinteger“Default 50.”
  • offsetinteger
  • sortstring“Default: most issues first.”
• get_pageon 100 other serversreads or searches

  “Everything the crawl holds for one URL (meta, headings, canonical, hreflang, schema types, links, images, issues) plus the internal links pointing at it.”

  Publisher declares: read-only yes

  2 inputs, 1 required

  • crawlstring“Which saved crawl: "latest" (default), a crawl file name from list_crawls, a job id from start_crawl, or part of the site name.”
  • urlstringrequired
• get_briefon 20 other serversreads or searches

  “The app's AI brief for a crawl: a Markdown report grouped by severity with capped URL lists and a page table, under an audit prompt.”

  Publisher declares: read-only yes

  3 inputs

  • crawlstring“Which saved crawl: "latest" (default), a crawl file name from list_crawls, a job id from start_crawl, or part of the site name.”
  • page_tableboolean“Include the page table (default true).”
  • page_limitinteger“Rows in the page table (default 200).”
• compare_crawlsreads or searches

  “Diff two saved crawls of a site: pages added, removed and changed, and how the issue counts moved.”

  Publisher declares: read-only yes

  2 inputs, 2 required

  • beforestringrequired“The older crawl (same forms as crawl elsewhere).”
  • afterstringrequired“The newer crawl.”
• bot_checkreads or searches

  “Fetch pages as Googlebot, Bingbot, AI search, AI assistant and training crawlers and compare each with what a browser gets: blocked, CDN challenge, noindex, redirected, thinner or different content.”

  Publisher declares: read-only yes · open-world yes

  3 inputs, 1 required

  • urlsarrayrequired
  • delaynumber“Seconds between requests (default 0.4).”
  • renderboolean“Also render the browser baseline with JS (default true).”
• check_robotson 2 other serversreads or searches

  “Fetch and parse a site's robots.txt: rules, sitemaps, and which search and AI crawlers it blocks.”

  Publisher declares: read-only yes · open-world yes

  1 input, 1 required

  • urlstringrequired
• detect_cmsreads or searches

  “Identify the CMS behind a URL, with the crawl settings and exclude patterns the app suggests for it.”

  Publisher declares: read-only yes · open-world yes

  1 input, 1 required

  • urlstringrequired

Each quoted sentence is the first sentence of the publisher’s own description. The kind of tool and the “changes data” mark are our AI model’s reading of that description, not something we tested: two differently worded runs had to agree, otherwise no label is shown, and nobody has checked them by hand. A tool without the “changes data” mark is described as only reading, searching or calculating, or the two runs did not agree. No tool was called. Read the publisher’s full description before relying on it. “Publisher declares” lists the hints the server itself sends with the tool (MCP annotations: read-only, destructive, idempotent, open-world); they are the publisher's claims, not checked by us. Inputs are the parameters in the tool's own input schema: name, declared type, whether it is required, and the first sentence of its description. Read 2026-10-23.

The registry verifies a package once, when it is published, and does not re-check it. A difference below is about the listing, not a judgement of the publisher.