Crawl Tools

Crawl tools identify the dataset behind an SEO analysis. Always use list_crawls and a returned crawl_uid; never use an internal ID or an external crawler run ID.

Read-only crawl tools

list_crawls

Lists crawls for a known project. Use status and timestamps to select a completed, relevant dataset.

get_crawl_status

Checks the current state of one known crawl. Useful when a user asks whether a requested crawl has finished.

get_crawl_summary

Returns a concise crawl-level summary. Use it to orient an analysis before retrieving detailed records.

get_on_page_overview

Returns aggregated on-page distributions and issue context. Pair it with record tools before making page-specific claims.

compare_crawls

Compares two crawl UIDs. Confirm comparable scope and settings before treating a difference as a regression.

State-changing crawl tool

start_crawl

Starts a crawl only when the user explicitly requests it. If the tool requires confirm=true, the user must confirm that exact crawl and configuration first.

Select the right crawl

List crawls for [PROJECT_UID].
Prefer completed crawls. Show crawl UID, status, start time, completion time,
and enough scope information for me to choose. Do not start a new crawl.

Compare two crawl datasets

Compare crawl [BEFORE_CRAWL_UID] with [AFTER_CRAWL_UID].
Separate expected changes, improvements, regressions, and uncertain deltas.
Verify major findings with page-level records.

Common mistakes

• Using the newest crawl even when it is incomplete.
• Comparing crawls with materially different limits or settings.
• Treating an overview score as page-level proof.
• Starting a fresh crawl when a completed dataset already answers the question.
• Omitting crawl timestamps and UIDs from the final report.