Rival
Jun 14 2026, 22:05 UTClive
MatrixAPI InsightsDemo
SCAN HISTORY · HIGH

fastCRW — Scan History

Every fetch, parse, and diff recorded for fastCRW. Filter by surface to focus the timeline; each row captures what was found, what changed, and the fields returned.

Scans · 14d
160
Changed
150
Surfaces
4
BY SURFACE

Filter

160 scans
All160blog34docs42homepage42pricing42
MOST RECENT FIRST

Timeline

last change Jun 14, 06:44
Jun 14, 2026
12 changed · 12 scans
06:44
Jun 14
BLOG+1

The primary topics list was updated, with one new topic added and several existing topics rephrased for clarity and additional context. All recent post URLs were also cleaned to remove an unusual suffix.

post frequency
daily
primary topics
Web Scraping & APIs, AI Agents & RAG Pipelines, Comparisons & Alternatives (Firecrawl, Scrapy), Developer Tools & Infra…
recent post urls
/blog/convert-typescript-python-browser-automation, /blog/migrate-from-scrapy-to-fastcrw, /blog/mcp-vs-rest-api-agents,…
developer focused
true
recent post dates
Jun 12, 2026, Jun 12, 2026, Jun 11, 2026, Jun 11, 2026, Jun 10, 2026, Jun 10, 2026, Jun 9, 2026, Jun 9, 2026, Jun 8, 20…
recent post titles
Port a TypeScript Scraper to Python: Skip the Rewrite, Migrating from Scrapy to fastCRW: A Practical Guide (2026), MCP …
06:44
Jun 14
DOCS+1

The 'last_update_date' field was updated from 'null' to an empty string, indicating a change in the date value.

sections
Documentation, API Reference, MCP Server, Changelog
has sdk docs
true
has tutorials
false
last update date
null
has api reference
true
06:44
Jun 14
PRICING+1

A new "Forever Free" tier has been introduced, offering self-hosting with an AGPL-3.0 license, unlimited requests and concurrency. Minor adjustments were also made to the "Free" tier's billing period and the overall `free_tier_limits` description.

tiers
{"cta":"Contact Sales","name":"Enterprise","price":"Custom","features":["Volume credit rates","Dedicated infrastructure…
has free tier
true
free tier limits
500 credits
pricing transparent
false
06:43
Jun 14
HOMEPAGE+1

The page's positioning now highlights fastCRW as an open-source alternative to Firecrawl and Tavily for AI agents. Key differentiators have been significantly revised, with many old, detailed descriptions replaced by more concise or updated feature statements, including new points on unified endpoints, transparent benchmarks, and a free-tier pricing message.

sub tagline
You keep your pipeline, your data, and your budget when you leave. The engine is AGPL-3.0, the benchmark is public, and…
primary cta url
/pricing
primary tagline
The web data API with no exit cost
primary cta text
500 Free Credits — Get Started
nav primary items
Playground, Docs, Blog, Pricing, Skills, Resources, GitHub, Sign in, Get Started Free
secondary cta text
null
06:27
Jun 14
BLOG+1

The primary topics list was updated with new phrasing and a new category, while two older topics were removed. The list of recent blog posts was truncated to the 10 most recent, and the URLs for these posts are now provided as a single, concatenated string.

post frequency
daily
primary topics
Web Scraping & Data Extraction, AI Agents & RAG Pipelines, API Comparisons & Alternatives, Developer Tools & Infrastruc…
recent post urls
/blog/convert-typescript-python-browser-automation股东;/blog/migrate-from-scrapy-to-fastcrw股东;/blog/mcp-vs-rest-api-agent…
developer focused
true
recent post dates
Jun 12, 2026, Jun 12, 2026, Jun 11, 2026, Jun 11, 2026, Jun 10, 2026, Jun 10, 2026, Jun 9, 2026, Jun 9, 2026, Jun 8, 20…
recent post titles
Port a TypeScript Scraper to Python: Skip the Rewrite, Migrating from Scrapy to fastCRW: A Practical Guide (2026), MCP …
06:26
Jun 14
DOCS+1

The `has_tutorials` property was changed to `false`, indicating that tutorials are no longer present. The `last_update_date` string value was also updated from `.null` to `null`.

sections
Documentation, API Reference, MCP Server, Changelog
has sdk docs
true
has tutorials
false
last update date
null
has api reference
true
06:26
Jun 14
PRICING+1

The free tier's limits description has been updated and its billing period is now explicitly defined as "one-time / lifetime". Additionally, the overall pricing transparency status has changed to false.

tiers
{"cta":"Contact Sales","name":"Enterprise","price":"Custom","features":["Volume credit rates","Dedicated infrastructure…
has free tier
true
free tier limits
500 credits, 2 concurrency
pricing transparent
false
06:26
Jun 14
HOMEPAGE+1

The page's calls to action have been updated, with the primary call to action now emphasizing '500 Free Credits' and linking to pricing, and the secondary CTA removed. Many new developer-focused differentiators have been added, highlighting Rust, self-hosting, MCP support, and advanced AI agent capabilities, while some older general selling points were removed. The overall messaging is now more explicitly tailored for AI agents.

sub tagline
You keep your pipeline, your data, and your budget when you leave. The engine is AGPL-3.0, the benchmark is public, and…
primary cta url
/pricing
primary tagline
The web data API with no exit cost
primary cta text
500 Free Credits — Get Started
nav primary items
Playground, Docs, Blog, Pricing, Skills, Resources, GitHub, Sign in, Get Started Free
secondary cta text
null
06:17
Jun 14
BLOG+1

The blog content has significantly expanded with numerous new posts covering CRW updates, competitor comparisons, advanced web scraping tutorials in multiple languages, and engineering deep dives. The site's primary topics have been completely revised to reflect this broader and more detailed coverage.

post frequency
daily
primary topics
Web Scraping Tools & Techniques, AI Agents & RAG Pipelines, API Integrations & Alternatives, Rust Infrastructure & Perf…
recent post urls
/blog/convert-typescript-python-browser-automation, /blog/migrate-from-scrapy-to-fastcrw, /blog/mcp-vs-rest-api-agents,…
developer focused
true
recent post dates
Jun 12, 2026, Jun 12, 2026, Jun 11, 2026, Jun 11, 2026, Jun 10, 2026, Jun 10, 2026, Jun 9, 2026, Jun 9, 2026, Jun 8, 20…
recent post titles
Port a TypeScript Scraper to Python: Skip the Rewrite, Migrating from Scrapy to fastCRW: A Practical Guide (2026), MCP …
06:16
Jun 14
DOCS+1

Tutorials are now available. Several documentation sections, including 'Quick: copy this and run', 'Other endpoints', 'Install', and 'Resources', have been removed.

sections
Documentation, API Reference, MCP Server, Changelog
has sdk docs
true
has tutorials
true
last update date
.null
has api reference
true
06:16
Jun 14
PRICING+1

The pricing transparency status has changed to true. The Enterprise plan now has a monthly billing period, and the Free plan's billing period is now null.

tiers
{"cta":"Contact Sales","name":"Enterprise","price":"Custom","features":["Volume credit rates","Dedicated infrastructure…
has free tier
true
free tier limits
500 credits, one-time / lifetime
pricing transparent
true
06:16
Jun 14
HOMEPAGE+1

The page significantly expanded its list of key differentiators, introducing new features like interactive scraping actions, stealth anti-detection, detailed media parsing, and more explicit pricing details like free credits. Some previous technical details about the Rust build, self-hosting cost, and specific feature supports were removed or condensed, and the positioning statement was updated to highlight 'map' functionality and a 'free tier.'

sub tagline
You keep your pipeline, your data, and your budget when you leave. The engine is AGPL-3.0, the benchmark is public, and…
primary cta url
/register
primary tagline
The web data API with no exit cost
primary cta text
Get Started Free
nav primary items
Playground, Docs, Blog, Pricing, Skills, Resources, GitHub, Sign in, Get Started Free
secondary cta text
500 Free Credits — Get Started
Jun 13, 2026
11 changed · 12 scans
06:47
Jun 13
BLOG+1

The primary topics covered on the blog page were updated to provide more descriptive and detailed categories for content related to web scraping for AI agents, API comparisons, engineering, tutorials, and cost optimization.

post frequency
daily
primary topics
Web Scraping for AI Agents and RAG Pipelines, API and Tool Comparisons and Alternatives, Engineering and Architecture f…
recent post urls
/blog/convert-typescript-python-browser-automation, /blog/migrate-from-scrapy-to-fastcrw, /blog/mcp-vs-rest-api-agents,…
developer focused
true
recent post dates
Jun 12, 2026, Jun 12, 2026, Jun 11, 2026, Jun 11, 2026, Jun 10, 2026, Jun 10, 2026, Jun 9, 2026, Jun 9, 2026, Jun 8, 20…
recent post titles
Port a TypeScript Scraper to Python: Skip the Rewrite, Migrating from Scrapy to fastCRW: A Practical Guide (2026), MCP …
06:46
Jun 13
DOCS+1

New sections 'Quick: copy this and run', 'Other endpoints', 'Install', and 'Resources' have been added. The 'last_update_date' value was also updated from '.null' to 'null'.

sections
Documentation, API Reference, MCP Server, Changelog, Quick: copy this and run, Other endpoints, Install, Resources
has sdk docs
true
has tutorials
false
last update date
null
has api reference
true
06:46
Jun 13
PRICING+1

The primary change is a shift from annual to monthly billing for most paid tiers. Additionally, the 'pricing_transparent' flag was set to false, and minor adjustments were made to the free and enterprise tier details.

tiers
{"cta":"Contact Sales","name":"Enterprise","price":"Custom","features":["Volume credit rates","Dedicated infrastructure…
has free tier
true
free tier limits
500 credits, one-time / lifetime
pricing transparent
false
06:45
Jun 13
HOMEPAGE+1

The page now features a more concise list of key differentiators, with several verbose technical and feature descriptions removed or condensed. A new secondary call to action was added, and primary CTA, social proof, positioning, and target audience statements were updated.

sub tagline
You keep your pipeline, your data, and your budget when you leave. The engine is AGPL-3.0, the benchmark is public, and…
primary cta url
/register
primary tagline
The web data API with no exit cost
primary cta text
Get Started Free
nav primary items
Playground, Docs, Blog, Pricing, Skills, Resources, GitHub, Sign in, Get Started Free
secondary cta text
See pricing
06:28
Jun 13
BLOG+1

The primary topics were updated with rephrased descriptions, the removal of two topics, and the addition of a new topic focusing on cost optimization.

post frequency
daily
primary topics
AI Agents & RAG Pipelines, Web Scraping Techniques & APIs, Performance & Rust Engineering, Competitor Comparisons & Alt…
recent post urls
/blog/convert-typescript-python-browser-automation, /blog/migrate-from-scrapy-to-fastcrw, /blog/mcp-vs-rest-api-agents,…
developer focused
true
recent post dates
Jun 12, 2026, Jun 12, 2026, Jun 11, 2026, Jun 11, 2026, Jun 10, 2026, Jun 10, 2026, Jun 9, 2026, Jun 9, 2026, Jun 8, 20…
recent post titles
Port a TypeScript Scraper to Python: Skip the Rewrite, Migrating from Scrapy to fastCRW: A Practical Guide (2026), MCP …
06:27
Jun 13
DOCSNO CHANGE

No change since last scan.

sections
Documentation, API Reference, MCP Server, Changelog
has sdk docs
true
has tutorials
false
last update date
.null
has api reference
true
06:27
Jun 13
PRICING+1

Key changes include a shift to annual billing for all paid tiers, an update to the 'pricing_transparent' flag to true, and clarification in the free tier limits that credits are not reset monthly. Additionally, the number formatting for credit amounts in feature descriptions has been updated.

tiers
{"cta":"Contact Sales","name":"Enterprise","price":"Custom","features":["Volume credit rates","Dedicated infrastructure…
has free tier
true
free tier limits
500 credits, one-time / lifetime — not a monthly reset
pricing transparent
true
06:26
Jun 13
HOMEPAGE+1

The page has significantly expanded its feature set and technical details, emphasizing a "Developer First" and "Local-first by design" positioning, with new capabilities for AI agents and clearer explanations of its Rust core and open-source compatibility. The target audience is broadened, and a secondary call to action was removed.

sub tagline
You keep your pipeline, your data, and your budget when you leave. The engine is AGPL-3.0, the benchmark is public, and…
primary cta url
/pricing
primary tagline
The web data API with no exit cost
primary cta text
500 Free Credits — Get Started
nav primary items
Playground, Docs, Blog, Pricing, Skills, Resources, GitHub, Sign in, Get Started Free
secondary cta text
null
06:13
Jun 13
BLOG+1

The primary topics section was updated with new, expanded, and rephrased categories, including a new 'Data Extraction & Processing' topic. Two new recent posts, 'Port a TypeScript Scraper to Python' and 'Migrating from Scrapy to fastCRW', were added, replacing two older posts.

post frequency
daily
primary topics
AI Agents & RAG Pipelines, Web Scraping Technologies & APIs, Performance & Infrastructure (Rust, self-hosting), Tool Co…
recent post urls
/blog/convert-typescript-python-browser-automation, /blog/migrate-from-scrapy-to-fastcrw, /blog/mcp-vs-rest-api-agents,…
developer focused
true
recent post dates
Jun 12, 2026, Jun 12, 2026, Jun 11, 2026, Jun 11, 2026, Jun 10, 2026, Jun 10, 2026, Jun 9, 2026, Jun 9, 2026, Jun 8, 20…
recent post titles
Port a TypeScript Scraper to Python: Skip the Rewrite, Migrating from Scrapy to fastCRW: A Practical Guide (2026), MCP …
06:12
Jun 13
DOCS+1

The 'Quickstart', 'Install', and 'Resources' sections have been removed, and tutorials are no longer available.

sections
Documentation, API Reference, MCP Server, Changelog
has sdk docs
true
has tutorials
false
last update date
.null
has api reference
true
06:12
Jun 13
PRICING+1

The billing period for the Scale, Growth, Standard, and Hobby pricing tiers has been updated from annual to monthly, and the description for free tier limits was slightly revised.

tiers
{"cta":"Contact Sales","name":"Enterprise","price":"Custom","features":["Volume credit rates","Dedicated infrastructure…
has free tier
true
free tier limits
500 credits, one-time / lifetime
pricing transparent
false
06:11
Jun 13
HOMEPAGE+1

The page's messaging has been streamlined with several verbose key differentiators replaced by more concise points or removed. New differentiators highlight built-in stealth, a self-contained binary, and clarified pricing. CTA buttons and URLs were updated, and the positioning statement was rephrased.

sub tagline
You keep your pipeline, your data, and your budget when you leave. The engine is AGPL-3.0, the benchmark is public, and…
primary cta url
/pricing
primary tagline
The web data API with no exit cost
primary cta text
500 Free Credits — Get Started
nav primary items
Playground, Docs, Blog, Pricing, Skills, Resources, GitHub, Sign in, Get Started Free
secondary cta text
Get Started Free
Jun 12, 2026
10 changed · 10 scans
06:42
Jun 12
BLOG+1

The page's primary content categorization has been updated and refined, with new, more specific topics introduced. The list of recent blog posts has been significantly shortened to feature only the newest 10 articles, including two new ones.

post frequency
daily
primary topics
AI Agent and RAG Web Data, Web Scraping Tools and APIs, Rust and Engineering for Scraping, Developer Integrations and U…
recent post urls
/blog/mcp-vs-rest-api-agents, /blog/weaviate-fastcrw-semantic-search, /blog/langgraph-web-aware-rag-latency, /blog/octo…
developer focused
true
recent post dates
Jun 11, 2026, Jun 11, 2026, Jun 10, 2026, Jun 10, 2026, Jun 9, 2026, Jun 9, 2026, Jun 8, 2026, Jun 8, 2026, Jun 7, 2026…
recent post titles
MCP vs REST API: Why Agents Prefer crw-mcp, Weaviate + fastCRW: Semantic Search From Web, LangGraph Web-Aware RAG at Lo…
06:41
Jun 12
DOCS+1

New sections for Quickstart, Install, and Resources have been added, and tutorials support has been enabled.

sections
Documentation, API Reference, MCP Server, Changelog, Quickstart, Install, Resources
has sdk docs
true
has tutorials
true
last update date
.null
has api reference
true
06:41
Jun 12
PRICING+1

The pricing model has shifted from monthly to annual billing for most paid tiers, and the company no longer states that its pricing is transparent. The description of the free tier's limits and billing period was also slightly updated.

tiers
{"cta":"Contact Sales","name":"Enterprise","price":"Custom","features":["Volume credit rates","Dedicated infrastructure…
has free tier
true
free tier limits
500 credits, one-time / lifetime — not a monthly reset
pricing transparent
false
06:40
Jun 12
HOMEPAGE+1

The page introduces several new developer-focused features and clarifications, such as detailed technical differentiators, Firecrawl migration guidance, and expanded pricing comparisons. It also updates its primary call to action and rewords its competitive positioning.

sub tagline
You keep your pipeline, your data, and your budget when you leave.<br/> The engine is <a href="https://github.com/us/cr…
primary cta url
/register
primary tagline
The web data API with<br/><span>no exit cost</span>
primary cta text
Get Started Free
nav primary items
Playground, Docs, Blog, Pricing, Skills, Resources, GitHub, Sign in, Get Started Free
secondary cta text
500 Free Credits — Get Started
06:24
Jun 12
DOCS+1

No significant changes detected; only key order differs.

sections
Documentation, API Reference, MCP Server, Changelog
has sdk docs
true
has tutorials
false
last update date
null
has api reference
true
06:24
Jun 12
PRICING+1

The "Forever Free (Self-Hosted)" pricing tier has been removed. The "Free" tier and overall free tier limits now explicitly state "lifetime" availability for its 500 credits.

tiers
{"cta":"Contact Sales","name":"Enterprise","price":"Custom","features":["Volume credit rates","Dedicated infrastructure…
has free tier
true
free tier limits
500 credits, one-time / lifetime
pricing transparent
true
06:23
Jun 12
HOMEPAGE+1

The page underwent significant rephrasing and expansion of key differentiators, adding technical details about the Rust core and self-hosting benefits. The number of core capabilities was reduced from eight to five, and specific differentiators like Firecrawl compatibility and MCP support were removed from the prominent list. The primary CTA URL and text were updated.

sub tagline
You keep your pipeline, your data, and your budget when you leave. The engine is AGPL-3.0, the benchmark is public, and…
primary cta url
/pricing
primary tagline
The web data API with no exit cost
primary cta text
500 Free Credits — Get Started
nav primary items
Playground, Docs, Blog, Pricing, Skills, Resources, GitHub, Sign in, Get Started Free
secondary cta text
.null
06:09
Jun 12
DOCS+1

The documentation structure has been updated, with a new "Documentation" section replacing "Quickstart," "Installation," and "Resources." Additionally, tutorials are no longer available.

sections
Documentation, API Reference, MCP Server, Changelog
has sdk docs
true
has tutorials
false
last update date
null
has api reference
true
06:09
Jun 12
PRICING+1

A new 'Forever Free (Self-Hosted)' tier has been introduced, and the free tier limits description was updated. Additionally, minor formatting changes were made to credit amounts in features across several existing tiers, and the Enterprise tier's billing period was adjusted.

tiers
{"cta":"Contact Sales","name":"Enterprise","price":"Custom","features":["Volume credit rates","Dedicated infrastructure…
has free tier
true
free tier limits
500 credits, one-time
pricing transparent
true
06:08
Jun 12
HOMEPAGE+1

The competitor page significantly revamped its key differentiators, consolidating verbose descriptions into concise statements, and explicitly highlighting new capabilities like 'Browse', 'Parse', and 'Monitor' endpoints, while also expanding its stated target audience.

sub tagline
You keep your pipeline, your data, and your budget when you leave.<br/> The engine is <a href="https://github.com/us/cr…
primary cta url
/register
primary tagline
The web data API with<br/><span>no exit cost</span>
primary cta text
Get started — 500 free credits
nav primary items
Playground, Docs, Blog, Pricing, Skills, Resources, GitHub, Sign in, Get Started Free
secondary cta text
See pricing
Jun 11, 2026
11 changed · 11 scans
06:44
Jun 11
DOCS+1

The site's documentation structure has been updated, introducing new sections like Quickstart, Installation, and Resources, and adding tutorials. The general "Documentation" entry has been removed.

sections
Quickstart, API Reference, MCP Server, Installation, Resources, Changelog
has sdk docs
true
has tutorials
true
last update date
null
has api reference
true
06:44
Jun 11
PRICING+1

The Enterprise tier's billing period was updated to 'custom', and credit amounts in feature descriptions for several tiers were reformatted with thousands separators.

tiers
{"cta":"Contact Sales","name":"Enterprise","price":"Custom","features":["Volume credit rates","Dedicated infrastructure…
has free tier
true
free tier limits
500 credits, one-time / lifetime
pricing transparent
true
06:43
Jun 11
HOMEPAGE+1

The page has significantly refined its marketing language, specifically repositioning itself as an 'Open source Firecrawl + Tavily alternative' for 'AI Agents'. Key differentiators have been overhauled, with many specific feature descriptions replaced by consolidated 'Yes.' statements that provide more technical detail, new pricing comparisons, and more explicit Firecrawl compatibility details. The primary call to action has also been updated to direct users to registration.

sub tagline
You keep your pipeline, your data, and your budget when you leave. The engine is AGPL-3.0, the benchmark is public, and…
primary cta url
/register
primary tagline
The web data API with no exit cost
primary cta text
Get started — 500 free credits
nav primary items
Playground, Docs, Blog, Pricing, Skills, Resources, GitHub, Sign in, Get Started Free
secondary cta text
See pricing
06:26
Jun 11
BLOG+1

The blog significantly expanded its content, adding many new developer-focused posts and a new primary topic on self-hosting. Existing primary topics were rephrased to reflect broader categories.

post frequency
daily
primary topics
Web Scraping and Data Extraction, AI Agents and RAG Pipelines, API Comparisons and Alternatives, Developer Tools and En…
recent post urls
/blog/langgraph-web-aware-rag-latency, /blog/octoparse-alternative, /blog/cursor-fastcrw-mcp-plugin, /blog/sitemap-to-c…
developer focused
true
recent post dates
Jun 10, 2026, Jun 10, 2026, Jun 9, 2026, Jun 9, 2026, Jun 8, 2026, Jun 8, 2026, Jun 7, 2026, Jun 7, 2026, Jun 6, 2026, …
recent post titles
LangGraph Web-Aware RAG at Lower Latency, Octoparse Alternative for Developers: From No-Code GUI to a Real API (2026), …
06:25
Jun 11
DOCS+1

Several navigation links, including GitHub, Issues, and the License, have been removed from the sections, and the `has_tutorials` flag is now false, indicating tutorials are no longer available.

sections
Documentation, API Reference, MCP Server, Changelog
has sdk docs
true
has tutorials
false
last update date
null
has api reference
true
06:25
Jun 11
PRICING+1

Most paid tiers shifted from annual to monthly billing. The Enterprise and Free tiers also had their billing period and per-unit descriptions simplified to 'null'.

tiers
{"cta":"Contact Sales","name":"Enterprise","price":"Custom","features":["Volume credit rates","Dedicated infrastructure…
has free tier
true
free tier limits
500 credits, one-time / lifetime
pricing transparent
true
06:24
Jun 11
HOMEPAGE+1

The competitor page has significantly expanded its feature set, adding detailed developer-facing capabilities like PDF/DOCX parsing, interactive actions, built-in anti-detection, single-binary deployment, and advanced AI agent integrations (MCP, structured JSON extraction). Existing key differentiators have been elaborated, and the primary call-to-action and positioning statement have been revised.

sub tagline
You keep your pipeline, your data, and your budget when you leave. The engine is AGPL-3.0, the benchmark is public, and…
primary cta url
/pricing
primary tagline
The web data API with no exit cost
primary cta text
500 Free Credits — Get Started
nav primary items
Playground, Docs, Blog, Pricing, Skills, Resources, GitHub, Sign in, Get Started Free
secondary cta text
See pricing
06:18
Jun 11
BLOG+1

The blog's primary topics have been entirely revised to reflect new focus areas. Additionally, two new blog posts were added to the recent posts list, with corresponding updates to their URLs and dates, replacing two older entries.

post frequency
daily
primary topics
Web Scraping APIs and Tools, AI Agents and RAG Pipelines, Rust and Go Web Scraping, Scraping Tool Comparisons, Develope…
recent post urls
/blog/langgraph-web-aware-rag-latency, /blog/octoparse-alternative, /blog/cursor-fastcrw-mcp-plugin, /blog/sitemap-to-c…
developer focused
true
recent post dates
Jun 10, 2026, Jun 10, 2026, Jun 9, 2026, Jun 9, 2026, Jun 8, 2026, Jun 8, 2026, Jun 7, 2026, Jun 7, 2026, Jun 6, 2026, …
recent post titles
LangGraph Web-Aware RAG at Lower Latency, Octoparse Alternative for Developers: From No-Code GUI to a Real API (2026), …
06:18
Jun 11
DOCS+1

New sections for GitHub, Issues, and License (AGPL-3.0) have been added. Tutorials are now also available.

sections
Documentation, API Reference, MCP Server, Changelog, GitHub, Issues, License (AGPL-3.0)
has sdk docs
true
has tutorials
true
last update date
.null
has api reference
true
06:17
Jun 11
PRICING+1

The explicit 'Self-Host (AGPL)' pricing tier has been removed, and all managed cloud plans now default to an annual billing period. Additionally, pricing transparency has been enabled, and free tier details have been updated.

tiers
{"cta":"Contact Sales","name":"Enterprise","price":"Custom","features":["Volume credit rates","Dedicated infrastructure…
has free tier
true
free tier limits
500 credits, one-time / lifetime
pricing transparent
true
06:17
Jun 11
HOMEPAGE+1

The page updated its navigation with new sign-in/get started options. Key differentiators were significantly revised, removing detailed feature descriptions in favor of concise points, and adding focus on integrations and pricing. The main call to action, social proof, positioning statement, and target audience were also updated.

sub tagline
You keep your pipeline, your data, and your budget when you leave.<br/> The engine is <a href="https://github.com/us/cr…
primary cta url
/register
primary tagline
The web data API with<br/><span>no exit cost</span>
primary cta text
Get Started Free
nav primary items
Playground, Docs, Blog, Pricing, Skills, Resources, GitHub, Sign in, Get Started Free
secondary cta text
See pricing
Jun 10, 2026
11 changed · 12 scans
06:56
Jun 10
BLOG+1

The blog's primary topics have been completely revised to focus on AI agents, LLM integration, and Rust infrastructure. The list of recent blog posts displayed on the page has also been significantly truncated.

post frequency
daily
primary topics
Web Scraping for AI Agents, LLM Integration & Costs, API Comparison and Alternatives, Rust Engineering & Infrastructure…
recent post urls
/blog/cursor-fastcrw-mcp-plugin, /blog/sitemap-to-crawl-large-sites, /blog/managed-llm-search-api-costs, /blog/stateles…
developer focused
true
recent post dates
Jun 9, 2026, Jun 9, 2026, Jun 8, 2026, Jun 8, 2026, Jun 7, 2026, Jun 7, 2026, Jun 6, 2026, Jun 6, 2026, Jun 5, 2026, Ju…
recent post titles
Cursor + fastCRW: Live Web Context via MCP, Sitemap to Crawl: Optimized Discovery at Scale, Managed LLM Search API Cost…
06:55
Jun 10
DOCS+1

Tutorials are no longer available. The 'last_update_date' field value was updated from '.null' to an empty string.

sections
Documentation, API Reference, MCP Server, Changelog
has sdk docs
true
has tutorials
false
last update date
.null
has api reference
true
06:55
Jun 10
PRICING+1

A new 'Self-Host (AGPL)' pricing tier has been introduced. Additionally, details for the 'Enterprise' and 'Free' tiers, and the 'free_tier_limits' description have been updated.

tiers
{"cta":"Contact Sales","name":"Enterprise","price":"Custom","features":["Volume credit rates","Dedicated infrastructure…
has free tier
true
free tier limits
500 credits, one-time / lifetime — not a monthly reset, 2 concurrency
pricing transparent
false
06:54
Jun 10
HOMEPAGE+1

The page updated its key differentiators, adding new technical capabilities like media parsing, smart waiting, anti-detection, and structured JSON extraction, while consolidating existing points. The target audience and social proof summary were updated, and the positioning statement expanded to emphasize self-hosting and free tiers.

sub tagline
You keep your pipeline, your data, and your budget when you leave. The engine is AGPL-3.0, the benchmark is public, and…
primary cta url
/register
primary tagline
The web data API with no exit cost
primary cta text
Get started — 500 free credits
nav primary items
Playground, Docs, Blog, Pricing, Skills, Resources, GitHub
secondary cta text
See pricing
06:39
Jun 10
BLOG+1

The blog content has significantly expanded with many new posts on web scraping techniques across different languages, AI agent integrations, RAG pipelines, and detailed comparisons of various scraping and search APIs. The primary topics were also updated to reflect this broader and more specific developer-focused content.

post frequency
daily
primary topics
Web Scraping APIs & Libraries, AI Agents & RAG Pipelines, Competitor Comparisons (Firecrawl, Tavily, Exa, Crawl4AI), En…
recent post urls
/blog/cursor-fastcrw-mcp-plugin, /blog/sitemap-to-crawl-large-sites, /blog/managed-llm-search-api-costs, /blog/stateles…
developer focused
true
recent post dates
Jun 9, 2026, Jun 9, 2026, Jun 8, 2026, Jun 8, 2026, Jun 7, 2026, Jun 7, 2026, Jun 6, 2026, Jun 6, 2026, Jun 5, 2026, Ju…
recent post titles
Cursor + fastCRW: Live Web Context via MCP, Sitemap to Crawl: Optimized Discovery at Scale, Managed LLM Search API Cost…
06:38
Jun 10
DOCSNO CHANGE

No change since last scan.

sections
Documentation, API Reference, MCP Server, Changelog
has sdk docs
true
has tutorials
true
last update date
.null
has api reference
true
06:38
Jun 10
PRICING+1

The pricing transparency status has changed to false. Additionally, the 'per_unit' field for Enterprise and Free tiers is now an empty string, and the 'billing_period' for the Enterprise tier has been updated to 'monthly'.

tiers
{"cta":"Contact Sales","name":"Enterprise","price":"Custom","features":["Volume credit rates","Dedicated infrastructure…
has free tier
true
free tier limits
500 credits, one-time / lifetime
pricing transparent
false
06:37
Jun 10
HOMEPAGE+1

The primary call to action text and navigation items have been updated. The key differentiators list has been significantly refined with several new benefits, while the overall positioning statement and target audience now explicitly emphasize being an open-source alternative for AI agents.

sub tagline
You keep your pipeline, your data, and your budget when you leave. The engine is AGPL-3.0, the benchmark is public, and…
primary cta url
/register
primary tagline
The web data API with no exit cost
primary cta text
Get started — 500 free credits
nav primary items
Playground, Docs, Blog, Pricing, Skills, Resources, GitHub
secondary cta text
See pricing
06:12
Jun 10
BLOG+1

Two new articles, 'Cursor + fastCRW: Live Web Context via MCP' and 'Sitemap to Crawl: Optimized Discovery at Scale,' were added to the blog. The primary topics for blog posts have been completely re-categorized, and many older blog posts are no longer listed in the 'recent posts' section.

post frequency
daily
primary topics
Web Scraping for AI Agents, RAG Pipelines & LLM Integration, Rust Infrastructure & Performance, API Comparisons & Alter…
recent post urls
/blog/cursor-fastcrw-mcp-plugin, /blog/sitemap-to-crawl-large-sites, /blog/managed-llm-search-api-costs, /blog/stateles…
developer focused
true
recent post dates
Jun 9, 2026, Jun 9, 2026, Jun 8, 2026, Jun 8, 2026, Jun 7, 2026, Jun 7, 2026, Jun 6, 2026, Jun 6, 2026, Jun 5, 2026, Ju…
recent post titles
Cursor + fastCRW: Live Web Context via MCP, Sitemap to Crawl: Optimized Discovery at Scale, Managed LLM Search API Cost…
06:11
Jun 10
DOCS+1

The service now includes support for tutorials. A minor adjustment was also made to the format of the `last_update_date` field.

sections
Documentation, API Reference, MCP Server, Changelog
has sdk docs
true
has tutorials
true
last update date
.null
has api reference
true
06:11
Jun 10
PRICING+1

The 'Free' tier's billing period was updated from 'one-time / lifetime' to 'one-time'.

tiers
{"cta":"Contact Sales","name":"Enterprise","price":"Custom","features":["Volume credit rates","Dedicated infrastructure…
has free tier
true
free tier limits
500 credits, one-time / lifetime
pricing transparent
true
06:10
Jun 10
HOMEPAGE+1

The page updated its core messaging and key differentiators, providing more detailed technical explanations for its features and highlighting new capabilities like Firecrawl compatibility, while also streamlining navigation and calls to action.

sub tagline
You keep your pipeline, your data, and your budget when you leave. The engine is AGPL-3.0, the benchmark is public, and…
primary cta url
/register
primary tagline
The web data API with no exit cost
primary cta text
Get Started Free
nav primary items
Playground, Docs, Blog, Pricing, Skills, GitHub
secondary cta text
See pricing
Jun 9, 2026
10 changed · 12 scans
06:42
Jun 9
BLOG+88

The blog content has significantly expanded with 88 new posts, and the primary topics have been entirely rephrased to reflect updated focus areas on scraping APIs, AI agents, RAG pipelines, and cost models.

post frequency
daily
primary topics
Web Scraping APIs and Tools, AI Agents and RAG Pipelines, Rust and Go Engineering for Scraping, Scraping Cost and Prici…
recent post urls
/blog/managed-llm-search-api-costs, /blog/stateless-architecture-request-model, /blog/best-llm-ready-web-data-apis, /bl…
developer focused
true
recent post dates
Jun 8, 2026, Jun 8, 2026, Jun 7, 2026, Jun 7, 2026, Jun 6, 2026, Jun 6, 2026, Jun 5, 2026, Jun 5, 2026, Jun 4, 2026, Ju…
recent post titles
Managed LLM Search API Costs: Capped DeepSeek, Why a Stateless Request Model Beats Sessions, LLM-Ready Web Data APIs: 2…
06:41
Jun 9
DOCS+1

The competitor page now explicitly states that tutorials are not available by setting 'has_tutorials' to false. Additionally, the 'last_update_date' field has been normalized from ".null" to "null".

sections
Documentation, API Reference, MCP Server, Changelog
has sdk docs
true
has tutorials
false
last update date
null
has api reference
true
06:41
Jun 9
PRICINGNO CHANGE

No change since last scan.

tiers
{"cta":"Contact Sales","name":"Enterprise","price":"Custom","features":["Volume credit rates","Dedicated infrastructure…
has free tier
true
free tier limits
500 credits, one-time / lifetime
pricing transparent
true
06:40
Jun 9
HOMEPAGE+1

The page's primary and sub-taglines received minor formatting updates, and call-to-action details were revised. The key differentiators list was substantially expanded and rephrased to highlight new developer-focused features, integration support, and specific Rust-based advantages, while old descriptive items were removed.

sub tagline
You keep your pipeline, your data, and your budget when you leave.<br/> The engine is <a href="https://github.com/us/cr…
primary cta url
/register
primary tagline
The web data API with<br/><span>no exit cost</span>
primary cta text
Get Started Free
nav primary items
Playground, Docs, Blog, Pricing, Skills, Resources, GitHub
secondary cta text
500 Free Credits — Get Started
06:24
Jun 9
BLOG+1

The primary blog topics have been re-categorized and simplified. Additionally, the lists of recent blog post URLs, titles, and dates have been significantly shortened, with 76 posts no longer being listed.

post frequency
daily
primary topics
Web Scraping, AI Agents and LLMs, Rust Infrastructure, API Comparisons and Alternatives, Technical Tutorials (Python, G…
recent post urls
/blog/managed-llm-search-api-costs, /blog/stateless-architecture-request-model, /blog/best-llm-ready-web-data-apis, /bl…
developer focused
true
recent post dates
Jun 8, 2026, Jun 8, 2026, Jun 7, 2026, Jun 7, 2026, Jun 6, 2026, Jun 6, 2026, Jun 5, 2026, Jun 5, 2026, Jun 4, 2026, Ju…
recent post titles
Managed LLM Search API Costs: Capped DeepSeek, Why a Stateless Request Model Beats Sessions, LLM-Ready Web Data APIs: 2…
06:24
Jun 9
DOCSNO CHANGE

No change since last scan.

sections
Documentation, API Reference, MCP Server, Changelog
has sdk docs
true
has tutorials
true
last update date
.null
has api reference
true
06:24
Jun 9
PRICING+1

The "Forever Free" tier has been removed. Additionally, the "Free" tier's billing period is now explicitly "one-time / lifetime".

tiers
{"cta":"Contact Sales","name":"Enterprise","price":"Custom","features":["Volume credit rates","Dedicated infrastructure…
has free tier
true
free tier limits
500 credits, one-time / lifetime
pricing transparent
true
06:24
Jun 9
HOMEPAGE+1

The page significantly restructured its key differentiators, removing many specific points while expanding others to be more descriptive. Additionally, dedicated social proof and target audience statements were removed, and the secondary CTA text was cleared.

sub tagline
You keep your pipeline, your data, and your budget when you leave. The engine is AGPL-3.0, the benchmark is public, and…
primary cta url
/pricing
primary tagline
The web data API with no exit cost
primary cta text
500 Free Credits — Get Started
nav primary items
Playground, Docs, Blog, Pricing, Skills, Resources, GitHub, Sign in, Get Started Free
secondary cta text
null
06:10
Jun 9
BLOG+85

The blog content has significantly expanded with 85 new posts covering advanced web scraping and AI agent integration topics. The primary topics were also updated to reflect this broader, more specialized content.

post frequency
daily
primary topics
AI Agent Web Scraping, RAG Pipeline Integration, Web Scraping API Comparisons & Benchmarking, Rust & Self-Hosted Infras…
recent post urls
/blog/managed-llm-search-api-costs, /blog/stateless-architecture-request-model, /blog/best-llm-ready-web-data-apis, /bl…
developer focused
true
recent post dates
Jun 8, 2026, Jun 8, 2026, Jun 7, 2026, Jun 7, 2026, Jun 6, 2026, Jun 6, 2026, Jun 5, 2026, Jun 5, 2026, Jun 4, 2026, Ju…
recent post titles
Managed LLM Search API Costs: Capped DeepSeek, Why a Stateless Request Model Beats Sessions, LLM-Ready Web Data APIs: 2…
06:09
Jun 9
DOCS+1

Tutorials support has been added to the competitor page. Additionally, the string value of the `last_update_date` field was slightly modified.

sections
Documentation, API Reference, MCP Server, Changelog
has sdk docs
true
has tutorials
true
last update date
.null
has api reference
true
06:08
Jun 9
PRICING+1

A new 'Forever Free' tier for self-hosting has been added, offering AGPL-3.0 licensing and unlimited requests. Minor data representation changes occurred in the 'Enterprise' and 'Free' tiers for non-applicable fields.

tiers
{"cta":"Contact Sales","name":"Enterprise","price":"Custom","features":["Volume credit rates","Dedicated infrastructure…
has free tier
true
free tier limits
500 credits, one-time / lifetime
pricing transparent
true
06:08
Jun 9
HOMEPAGE+1

The website underwent a content refresh, simplifying main taglines, updating the primary call to action, and reorganizing navigation links. The list of key differentiators was significantly condensed, shifting from verbose descriptions to a more concise bullet-point format, with new emphasis on 'Zero configuration' and 'Predictable pricing'.

sub tagline
You keep your pipeline, your data, and your budget when you leave. The engine is AGPL-3.0, the benchmark is public, and…
primary cta url
/pricing
primary tagline
The web data API with no exit cost
primary cta text
500 Free Credits — Get Started
nav primary items
Playground, Docs, Blog, Pricing, Skills, Resources, GitHub, Sign in, Get Started Free
secondary cta text
null
Jun 8, 2026
11 changed · 12 scans
06:41
Jun 8
BLOG+1

The blog's primary topics have been significantly updated, replacing detailed categories like 'Web Scraping APIs' and 'AI Agents and RAG Pipelines' with broader classifications such as 'Web Scraping', 'AI Agents and LLMs', 'Tool Comparisons', 'Engineering and Architecture', and 'Tutorials and Quickstarts'. The structure of the primary topics list has been entirely redefined.

post frequency
daily
primary topics
Web Scraping, AI Agents and LLMs, Tool Comparisons, Engineering and Architecture, Tutorials and Quickstarts
recent post urls
/blog/best-llm-ready-web-data-apis, /blog/smolagents-fastcrw-integration, /blog/ci-web-scraping-github-actions, /blog/f…
developer focused
true
recent post dates
Jun 7, 2026, Jun 7, 2026, Jun 6, 2026, Jun 6, 2026, Jun 5, 2026, Jun 5, 2026, Jun 4, 2026, Jun 3, 2026, Jun 3, 2026, Ju…
recent post titles
LLM-Ready Web Data APIs: 2026 Buyer's Guide, Smolagents + fastCRW: Web Grounding, Zero Bloat, Scheduled Web Scraping in…
06:41
Jun 8
DOCS+1

The documentation structure was updated, removing specific sections like 'Introduction', 'Quickstart', and 'Installation' and adding a general 'Documentation' section. Additionally, tutorials are no longer listed as available.

sections
Documentation, API Reference, MCP Server, Changelog
has sdk docs
true
has tutorials
false
last update date
null
has api reference
true
06:41
Jun 8
PRICING+1

Pricing transparency has been enabled, and most self-serve plans have shifted their default billing period from yearly to monthly. The Enterprise tier's per_unit description was updated, and the Free tier's limits description was shortened.

tiers
{"cta":"Contact Sales","name":"Enterprise","price":"Custom","features":["Volume credit rates","Dedicated infrastructure…
has free tier
true
free tier limits
500 credits, one-time / lifetime
pricing transparent
true
06:40
Jun 8
HOMEPAGE+1

The page underwent a significant content overhaul, particularly in its 'key differentiators' which are now much more detailed, including technical specifics, pricing comparisons, and explicit integrations. Navigation items and call-to-action texts were also updated, and the target audience was refined to focus on production AI teams.

sub tagline
You keep your pipeline, your data, and your budget when you leave.<br/> The engine is AGPL-3.0, the benchmark is public…
primary cta url
/register
primary tagline
The web data API with<br/><span>no exit cost</span>
primary cta text
Get Started Free
nav primary items
Playground, Docs, Blog, Pricing, Skills, Alternatives, Benchmarks, Use Cases, Integrations, GitHub, Sign in, Get Starte…
secondary cta text
Sign in
06:25
Jun 8
BLOG+1

The primary topics of the blog have been rephrased to provide more descriptive titles and include specific examples of tools and broader categories for infrastructure, comparisons, and pricing models.

post frequency
daily
primary topics
Web Scraping APIs and Tools, AI Agents and RAG Pipelines, Self-hosting and Infrastructure (Rust, Go, Python), Compariso…
recent post urls
/blog/best-llm-ready-web-data-apis, /blog/smolagents-fastcrw-integration, /blog/ci-web-scraping-github-actions, /blog/f…
developer focused
true
recent post dates
Jun 7, 2026, Jun 7, 2026, Jun 6, 2026, Jun 6, 2026, Jun 5, 2026, Jun 5, 2026, Jun 4, 2026, Jun 3, 2026, Jun 3, 2026, Ju…
recent post titles
LLM-Ready Web Data APIs: 2026 Buyer's Guide, Smolagents + fastCRW: Web Grounding, Zero Bloat, Scheduled Web Scraping in…
06:25
Jun 8
DOCS+1

The page now includes Introduction, Quickstart, and Installation sections, and has enabled tutorials functionality. The format of the last update date was also adjusted.

sections
Introduction, Quickstart, API Reference, MCP Server, Installation, Changelog
has sdk docs
true
has tutorials
true
last update date
null
has api reference
true
06:25
Jun 8
PRICING+1

Self-serve paid plans now use yearly billing instead of monthly. The free tier limits description was also updated to clarify that credits are a one-time allocation.

tiers
{"cta":"Contact Sales","name":"Enterprise","price":"Custom","features":["Volume credit rates","Dedicated infrastructure…
has free tier
true
free tier limits
500 credits, one-time / lifetime — not a monthly reset
pricing transparent
false
06:24
Jun 8
HOMEPAGE+1

The page was updated with new navigation and CTA elements, and the 'key differentiators' section was largely rewritten to use more concise bullet points, replacing many verbose descriptions with shorter, keyword-driven phrases. The social proof and target audience descriptions were also refined.

sub tagline
You keep your pipeline, your data, and your budget when you leave. The engine is AGPL-3.0, the benchmark is public, and…
primary cta url
/register
primary tagline
The web data API with no exit cost
primary cta text
Get started — 500 free credits
nav primary items
Playground, Docs, Blog, Pricing, Skills, Resources, GitHub, Sign in, Get Started Free
secondary cta text
See pricing
06:11
Jun 8
BLOG+1

The blog content has been updated with two new recent posts and a significant restructuring of its primary topics. Many older blog posts are also no longer listed, as the displayed number of recent posts has been reduced to 10.

post frequency
daily
primary topics
Web Scraping APIs, AI Agents & RAG Pipelines, Rust & Go Infrastructure, Competitor Comparisons, Technical Tutorials, AP…
recent post urls
/blog/best-llm-ready-web-data-apis, /blog/smolagents-fastcrw-integration, /blog/ci-web-scraping-github-actions, /blog/f…
developer focused
true
recent post dates
Jun 7, 2026, Jun 7, 2026, Jun 6, 2026, Jun 6, 2026, Jun 5, 2026, Jun 5, 2026, Jun 4, 2026, Jun 3, 2026, Jun 3, 2026, Ju…
recent post titles
LLM-Ready Web Data APIs: 2026 Buyer's Guide, Smolagents + fastCRW: Web Grounding, Zero Bloat, Scheduled Web Scraping in…
06:10
Jun 8
DOCSNO CHANGE

No change since last scan.

sections
Documentation, API Reference, MCP Server, Changelog
has sdk docs
true
has tutorials
false
last update date
.null
has api reference
true
06:10
Jun 8
PRICING+1

The 'Forever Free' tier has been removed. The 'Free' tier's billing period was updated, and the overall pricing transparency status was changed from true to false.

tiers
{"cta":"Contact Sales","name":"Enterprise","price":"Custom","features":["Volume credit rates","Dedicated infrastructure…
has free tier
true
free tier limits
500 credits, one-time / lifetime
pricing transparent
false
06:09
Jun 8
HOMEPAGE+1

The page features updated navigation, social proof, and target audience descriptions. Several key differentiators were rephrased or consolidated for conciseness, while new details about output formats and self-hosting costs were added.

sub tagline
You keep your pipeline, your data, and your budget when you leave. The engine is AGPL-3.0, the benchmark is public, and…
primary cta url
/pricing
primary tagline
The web data API with no exit cost
primary cta text
500 Free Credits — Get Started
nav primary items
Playground, Docs, Blog, Pricing, Skills, Resources, GitHub
secondary cta text
null
Jun 7, 2026
10 changed · 11 scans
06:38
Jun 7
DOCS+1

The availability of tutorials has been disabled, and the format of the 'last_update_date' string has been slightly altered.

sections
Documentation, API Reference, MCP Server, Changelog
has sdk docs
true
has tutorials
false
last update date
.null
has api reference
true
06:38
Jun 7
PRICING+1

A new 'Forever Free' self-hosted tier has been introduced, emphasizing AGPL-3.0 licensing and unlimited usage. Minor changes include updating the 'Enterprise' plan's billing period to 'Custom' and adding dollar signs to top-up credit amounts on several paid tiers.

tiers
{"cta":"Contact Sales","name":"Enterprise","price":"Custom","features":["Volume credit rates","Dedicated infrastructure…
has free tier
true
free tier limits
500 credits, one-time / lifetime
pricing transparent
true
06:38
Jun 7
HOMEPAGE+1

The webpage content was significantly reorganized, primarily consolidating numerous individual key differentiator points into fewer, more comprehensive statements. New navigation options were added, and other sections saw minor text updates and removals.

sub tagline
You keep your pipeline, your data, and your budget when you leave. The engine is AGPL-3.0, the benchmark is public, and…
primary cta url
/pricing
primary tagline
The web data API with no exit cost
primary cta text
500 Free Credits — Get Started
nav primary items
Playground, Docs, Blog, Pricing, Skills, Resources, GitHub, Sign in, Get Started Free
secondary cta text
:null
06:23
Jun 7
BLOG+1

The blog significantly expanded its content with numerous new posts covering web scraping techniques, AI agent integration, RAG pipeline construction, and detailed comparisons with various competitor tools. The primary content categories were also streamlined and generalized.

post frequency
daily
primary topics
Web Scraping, AI Agents, RAG Pipelines, Tool Comparisons and Alternatives, Engineering and Architecture, Developer Tuto…
recent post urls
/blog/ci-web-scraping-github-actions, /blog/firecrawl-pricing-explained, /blog/go-web-scraping-quickstart-crw, /blog/se…
developer focused
true
recent post dates
Jun 6, 2026, Jun 6, 2026, Jun 5, 2026, Jun 5, 2026, Jun 4, 2026, Jun 3, 2026, Jun 3, 2026, Jun 2, 2026, Jun 2, 2026, Ju…
recent post titles
Scheduled Web Scraping in GitHub Actions With CRW (2026), Firecrawl Pricing Explained (2026): Credits, Tiers, and the H…
06:22
Jun 7
DOCSNO CHANGE

No change since last scan.

sections
Documentation, API Reference, MCP Server, Changelog
has sdk docs
true
has tutorials
true
last update date
null
has api reference
true
06:22
Jun 7
PRICING+1

The pricing transparency flag is now true. The Enterprise tier's billing period is now explicitly monthly, and the '$' symbol has been removed from top-up credit amounts across several paid tiers.

tiers
{"cta":"Contact Sales","name":"Enterprise","price":"Custom","features":["Volume credit rates","Dedicated infrastructure…
has free tier
true
free tier limits
500 credits, one-time / lifetime
pricing transparent
true
06:21
Jun 7
HOMEPAGE+1

The website underwent significant content expansion, particularly detailing its API endpoints, AI agent integration capabilities, and features like structured JSON extraction. Pricing information, public benchmarks, and Firecrawl compatibility were also greatly clarified.

sub tagline
You keep your pipeline, your data, and your budget when you leave. The engine is AGPL-3.0, the benchmark is public, and…
primary cta url
/pricing
primary tagline
The web data API with no exit cost
primary cta text
500 Free Credits — Get Started
nav primary items
Playground, Docs, Blog, Pricing, Skills, Resources, GitHub
secondary cta text
null
06:05
Jun 7
BLOG+1

Two new blog posts about web scraping with GitHub Actions and Firecrawl pricing were added, replacing older content in the recent posts list. Additionally, the primary topics of the blog were updated with more descriptive categories.

post frequency
daily
primary topics
Web Scraping for AI Agents and RAG Pipelines, Comparison of Scraping Tools and APIs (e.g., Firecrawl, Tavily), Engineer…
recent post urls
/blog/ci-web-scraping-github-actions, /blog/firecrawl-pricing-explained, /blog/go-web-scraping-quickstart-crw, /blog/se…
developer focused
true
recent post dates
Jun 6, 2026, Jun 6, 2026, Jun 5, 2026, Jun 5, 2026, Jun 4, 2026, Jun 3, 2026, Jun 3, 2026, Jun 2, 2026, Jun 2, 2026, Ju…
recent post titles
Scheduled Web Scraping in GitHub Actions With CRW (2026), Firecrawl Pricing Explained (2026): Credits, Tiers, and the H…
06:05
Jun 7
DOCS+1

No significant changes were detected between the previous and current versions of the page.

sections
Documentation, API Reference, MCP Server, Changelog
has sdk docs
true
has tutorials
true
last update date
null
has api reference
true
06:05
Jun 7
PRICING+1

The site updated its global pricing transparency status to false, and adjusted the billing period definitions for the Enterprise and Free tiers.

tiers
{"cta":"Contact Sales","name":"Enterprise","price":"Custom","features":["Volume credit rates","Dedicated infrastructure…
has free tier
true
free tier limits
500 credits, one-time / lifetime
pricing transparent
false
06:04
Jun 7
HOMEPAGE+1

The page updated its primary call to action, removed the secondary CTA, and refined the target audience and social proof. Several detailed descriptions for key differentiators, such as media parsing and interactive scraping actions, were removed or shortened.

sub tagline
You keep your pipeline, your data, and your budget when you leave. The engine is AGPL-3.0, the benchmark is public, and…
primary cta url
/pricing
primary tagline
The web data API with no exit cost
primary cta text
500 Free Credits — Get Started
nav primary items
Playground, Docs, Blog, Pricing, Skills, Resources, GitHub, Sign in, Get Started Free
secondary cta text
null
Jun 6, 2026
10 changed · 12 scans
06:43
Jun 6
BLOG+1

The blog's primary topics were updated to be more concise and focused on developer-relevant areas such as AI agents, RAG pipelines, Rust infrastructure, and API comparisons.

post frequency
daily
primary topics
Web Scraping, AI Agents, RAG Pipelines, Rust Infrastructure, API Comparisons, Developer Tutorials
recent post urls
/blog/go-web-scraping-quickstart-crw, /blog/search-index-vs-live-web, /blog/web-scraping-in-go, /blog/credit-multiplier…
developer focused
true
recent post dates
Jun 5, 2026, Jun 5, 2026, Jun 4, 2026, Jun 3, 2026, Jun 3, 2026, Jun 2, 2026, Jun 2, 2026, Jun 1, 2026, Jun 1, 2026, Ma…
recent post titles
CRW Go Quickstart (2026): Scrape, Crawl, and Search With the HTTP API, Search Index vs Live Web: Agents Need Both, Web …
06:43
Jun 6
DOCSNO CHANGE

No change since last scan.

sections
Documentation, API Reference, MCP Server, Changelog
has sdk docs
true
has tutorials
true
last update date
null
has api reference
true
06:42
Jun 6
PRICING+1

The 'Forever Free' self-hosted tier, which provided an AGPL-3.0 licensed offering, has been removed from the pricing plans.

tiers
{"cta":"Contact Sales","name":"Enterprise","price":"Custom","features":["Volume credit rates","Dedicated infrastructure…
has free tier
true
free tier limits
500 credits, one-time / lifetime
pricing transparent
true
06:42
Jun 6
HOMEPAGE+1

The page significantly expanded its list of key differentiators, providing more detailed explanations for existing features and introducing several new capabilities related to reliability, self-hosting, and integrations. The primary call to action, social proof summary, positioning statement, and target audience have also been revised for conciseness or added detail.

sub tagline
You keep your pipeline, your data, and your budget when you leave. The engine is AGPL-3.0, the benchmark is public, and…
primary cta url
/pricing
primary tagline
The web data API with no exit cost
primary cta text
Get Started
nav primary items
Playground, Docs, Blog, Pricing, Skills, Resources, GitHub, Sign in, Get Started Free
secondary cta text
See pricing
06:26
Jun 6
BLOG+1

The primary blog topics have been updated to reflect a more detailed focus on AI/LLM web scraping workflows and technical engineering aspects.

post frequency
daily
primary topics
Web Scraping for AI/LLM Workflows, Technical Web Scraping Engineering, Web Scraping Tool Comparisons & Alternatives, Pr…
recent post urls
/blog/go-web-scraping-quickstart-crw, /blog/search-index-vs-live-web, /blog/web-scraping-in-go, /blog/credit-multiplier…
developer focused
true
recent post dates
Jun 5, 2026, Jun 5, 2026, Jun 4, 2026, Jun 3, 2026, Jun 3, 2026, Jun 2, 2026, Jun 2, 2026, Jun 1, 2026, Jun 1, 2026, Ma…
recent post titles
CRW Go Quickstart (2026): Scrape, Crawl, and Search With the HTTP API, Search Index vs Live Web: Agents Need Both, Web …
06:26
Jun 6
DOCSNO CHANGE

No change since last scan.

sections
Documentation, API Reference, MCP Server, Changelog
has sdk docs
true
has tutorials
true
last update date
null
has api reference
true
06:26
Jun 6
PRICING+1

A new "Forever Free" self-hosted tier has been introduced, and the existing "Free" tier's details, including its billing period and limits, have been clarified.

tiers
{"cta":"Contact Sales","name":"Enterprise","price":"Custom","features":["Volume credit rates","Dedicated infrastructure…
has free tier
true
free tier limits
500 credits (one-time / lifetime, not a monthly reset), 2 concurrency
pricing transparent
true
06:25
Jun 6
HOMEPAGE+1

The page updates primary and secondary calls to action, refines several existing key differentiators, removes others, and introduces new technical features like media parsing, smart waiting, interactive actions, stealth anti-detection, and structured JSON extraction. The target audience description was also expanded to explicitly include AI agents.

sub tagline
You keep your pipeline, your data, and your budget when you leave. The engine is AGPL-3.0, the benchmark is public, and…
primary cta url
/register
primary tagline
The web data API with no exit cost
primary cta text
Get Started Free
nav primary items
Playground, Docs, Blog, Pricing, Skills, Resources, GitHub, Sign in, Get Started Free
secondary cta text
See pricing
06:10
Jun 6
BLOG+1

The blog's primary topics have been updated to reflect a more concise and expanded focus, including Rust infrastructure and API comparisons. Two new blog posts have been added to the recent posts list, displacing two older entries.

post frequency
daily
primary topics
Web Scraping, AI Agents, RAG Pipelines, Rust Infrastructure, API Comparisons and Alternatives, Developer Tutorials
recent post urls
/blog/go-web-scraping-quickstart-crw, /blog/search-index-vs-live-web, /blog/web-scraping-in-go, /blog/credit-multiplier…
developer focused
true
recent post dates
Jun 5, 2026, Jun 5, 2026, Jun 4, 2026, Jun 3, 2026, Jun 3, 2026, Jun 2, 2026, Jun 2, 2026, Jun 1, 2026, Jun 1, 2026, Ma…
recent post titles
CRW Go Quickstart (2026): Scrape, Crawl, and Search With the HTTP API, Search Index vs Live Web: Agents Need Both, Web …
06:10
Jun 6
DOCS+1

No significant changes detected between the two versions; only key order differs.

sections
Documentation, API Reference, MCP Server, Changelog
has sdk docs
true
has tutorials
true
last update date
null
has api reference
true
06:09
Jun 6
PRICING+1

The "Forever Free (Self-Hosted)" tier has been removed, and the site now indicates transparent pricing. Minor adjustments were made to the "Enterprise" and "Free" tier details.

tiers
{"cta":"Contact Sales","name":"Enterprise","price":"Custom","features":["Volume credit rates","Dedicated infrastructure…
has free tier
true
free tier limits
500 credits, one-time / lifetime
pricing transparent
true
06:09
Jun 6
HOMEPAGE+1

The primary call to action, social proof, and target audience descriptions were updated. The extensive list of key differentiators was condensed into a shorter, more concise set of points, removing many detailed technical and feature explanations.

sub tagline
You keep your pipeline, your data, and your budget when you leave. The engine is AGPL-3.0, the benchmark is public, and…
primary cta url
/pricing
primary tagline
The web data API with no exit cost
primary cta text
500 Free Credits — Get Started
nav primary items
Playground, Docs, Blog, Pricing, Skills, Resources, GitHub, Sign in, Get Started Free
secondary cta text
null
Jun 5, 2026
12 changed · 12 scans
06:43
Jun 5
BLOG+1

The primary topics list for blog content has been updated, including rephrased existing topics, the addition of 'Web Scraping Tutorials,' and the removal of 'Self-hosting and cost optimization for scraping' and 'LLM integration with web data.'

post frequency
daily
primary topics
Web Scraping Technologies, AI Agents and RAG Pipelines, Tool and Competitor Comparisons, Scraping Engineering and Perfo…
recent post urls
/blog/web-scraping-in-go, /blog/credit-multiplier-traps-explained, /blog/url-mapping-vs-sitemap-discovery, /blog/diffbo…
developer focused
true
recent post dates
Jun 4, 2026, Jun 3, 2026, Jun 3, 2026, Jun 2, 2026, Jun 2, 2026, Jun 1, 2026, Jun 1, 2026, May 31, 2026, May 30, 2026, …
recent post titles
Web Scraping in Go (2026): Goroutines, Backpressure, and When to Stop Building It Yourself, Credit Multiplier Traps in …
06:43
Jun 5
DOCS+1

Tutorials have been added to the competitor page.

sections
Documentation, API Reference, MCP Server, Changelog
has sdk docs
true
has tutorials
true
last update date
null
has api reference
true
06:42
Jun 5
PRICING+1

A new "Forever Free (Self-Hosted)" plan was introduced, offering a zero-cost self-hosted option. Cloud plans from Hobby to Scale now default to monthly billing instead of annual, and the "Standard" plan's "is_self_serve" property was replaced with "is_unit".

tiers
{"cta":"Contact Sales","name":"Enterprise","price":"Custom","features":["Volume credit rates","Dedicated infrastructure…
has free tier
true
free tier limits
500 credits, one-time / lifetime
pricing transparent
false
06:42
Jun 5
HOMEPAGE+1

The page underwent a significant content overhaul, updating its product positioning to 'Open source Firecrawl + Tavily alternative' for AI agents. Many key differentiators were expanded or added to provide more technical detail on Rust-based performance, pricing benefits, open-source nature, Firecrawl compatibility, and advanced AI agent integrations.

sub tagline
You keep your pipeline, your data, and your budget when you leave. The engine is AGPL-3.0, the benchmark is public, and…
primary cta url
/pricing
primary tagline
The web data API with no exit cost
primary cta text
Get Started
nav primary items
Playground, Docs, Blog, Pricing, Skills, Resources, GitHub, Sign in, Get Started Free
secondary cta text
null
06:26
Jun 5
BLOG+1

The primary blog post topics were significantly updated, including the addition of 'LLM integration with web data', the removal of 'Practical Web Scraping Tutorials', and the rephrasing of several existing topics.

post frequency
daily
primary topics
Web scraping techniques and tools, AI agents and RAG pipelines, Comparison of scraping tools and APIs, Engineering and …
recent post urls
/blog/web-scraping-in-go, /blog/credit-multiplier-traps-explained, /blog/url-mapping-vs-sitemap-discovery, /blog/diffbo…
developer focused
true
recent post dates
Jun 4, 2026, Jun 3, 2026, Jun 3, 2026, Jun 2, 2026, Jun 2, 2026, Jun 1, 2026, Jun 1, 2026, May 31, 2026, May 30, 2026, …
recent post titles
Web Scraping in Go (2026): Goroutines, Backpressure, and When to Stop Building It Yourself, Credit Multiplier Traps in …
06:26
Jun 5
DOCS+1

The 'has_tutorials' flag is now false, indicating tutorials are no longer present, and the 'last_update_date' field has been updated from 'null' to an empty string.

sections
Documentation, API Reference, MCP Server, Changelog
has sdk docs
true
has tutorials
false
last update date
null
has api reference
true
06:25
Jun 5
PRICING+1

The pricing transparency flag has been updated to false, and the billing period for most paid plans is now annual instead of monthly. The Free plan's billing period also changed from 'one-time' to null.

tiers
{"cta":"Contact Sales","name":"Enterprise","price":"Custom","features":["Volume credit rates","Dedicated infrastructure…
has free tier
true
free tier limits
500 credits, one-time / lifetime
pricing transparent
false
06:25
Jun 5
HOMEPAGE+1

The page significantly expanded its detailed feature set (`key_differentiators`), introducing more specific scraping capabilities like media parsing, smart waiting, interactive actions, and unified search/scrape API calls, alongside new integrations and agent-focused MCP tools. General feature statements were replaced with granular descriptions, while some navigation links were removed and marketing texts updated.

sub tagline
You keep your pipeline, your data, and your budget when you leave. The engine is AGPL-3.0, the benchmark is public, and…
primary cta url
/register
primary tagline
The web data API with no exit cost
primary cta text
Get Started Free
nav primary items
Playground, Docs, Blog, Pricing, Skills, Resources
secondary cta text
AGPL-3.0 open engine
06:10
Jun 5
BLOG+1

Three new blog posts have been added to the recent list, and two new primary topic categories were introduced. Several existing primary topics were updated or consolidated, and one primary topic was removed.

post frequency
daily
primary topics
Web Scraping Technologies & APIs, AI Agent & RAG Pipeline Integration, Web Scraping Tool Comparisons, Engineering & Per…
recent post urls
/blog/web-scraping-in-go, /blog/credit-multiplier-traps-explained, /blog/url-mapping-vs-sitemap-discovery, /blog/diffbo…
developer focused
true
recent post dates
Jun 4, 2026, Jun 3, 2026, Jun 3, 2026, Jun 2, 2026, Jun 2, 2026, Jun 1, 2026, Jun 1, 2026, May 31, 2026, May 30, 2026, …
recent post titles
Web Scraping in Go (2026): Goroutines, Backpressure, and When to Stop Building It Yourself, Credit Multiplier Traps in …
06:10
Jun 5
DOCS+1

The navigation sections for GitHub, Issues, and License (AGPL-3.0) have been removed from the site.

sections
Documentation, API Reference, MCP Server, Changelog
has sdk docs
true
has tutorials
true
last update date
null
has api reference
true
06:10
Jun 5
PRICING+1

The pricing transparency flag is now true, and the Free tier's billing period has been clarified as 'one-time'. Additionally, numeric formatting for credit and top-up amounts in paid tier features was updated.

tiers
{"cta":"Contact Sales","name":"Enterprise","price":"Custom","features":["Volume credit rates","Dedicated infrastructure…
has free tier
true
free tier limits
500 credits, one-time / lifetime
pricing transparent
true
06:09
Jun 5
HOMEPAGE+1

The page now emphasizes more specific features like structured JSON extraction, advanced anti-detection, and local-first scraping. Key differentiators were updated, and the target audience was narrowed.

sub tagline
You keep your pipeline, your data, and your budget when you leave. The engine is AGPL-3.0, the benchmark is public, and…
primary cta url
/register
primary tagline
The web data API with no exit cost
primary cta text
Get started — 500 free credits
nav primary items
Playground, Docs, Blog, Pricing, Skills, Resources, GitHub, Sign in, Get Started Free
secondary cta text
See pricing
Jun 4, 2026
8 changed · 9 scans
06:39
Jun 4
DOCS+1

The page now includes links to GitHub, issues, and licensing information in its sections, and the last update date format has been adjusted.

sections
Documentation, API Reference, MCP Server, Changelog, GitHub, Issues, License (AGPL-3.0)
has sdk docs
true
has tutorials
true
last update date
null
has api reference
true
06:38
Jun 4
PRICING+1

The pricing model has shifted from annual to monthly billing for most paid tiers, and the overall pricing transparency status is now set to false. Additionally, the free tier limits description and the Free tier's billing period have been updated.

tiers
{"cta":"Contact Sales","name":"Enterprise","price":"Custom","features":["Volume credit rates","Dedicated infrastructure…
has free tier
true
free tier limits
500 credits, one-time / lifetime
pricing transparent
false
06:38
Jun 4
HOMEPAGE+1

The primary call-to-action was updated to highlight free credits and direct users to registration, and a new secondary CTA for pricing was introduced. The navigation menu gained "Sign in" and "Get Started Free" links. The list of key differentiators was significantly revised, focusing on specific technical advantages like local-first hosting, AGPL-3.0 engine auditability, and built-in browser rendering, while removing many broader feature descriptions.

sub tagline
You keep your pipeline, your data, and your budget when you leave. The engine is AGPL-3.0, the benchmark is public, and…
primary cta url
/register
primary tagline
The web data API with no exit cost
primary cta text
Get started — 500 free credits
nav primary items
Playground, Docs, Blog, Pricing, Skills, Resources, GitHub, Sign in, Get Started Free
secondary cta text
See pricing
06:23
Jun 4
DOCS+1

Tutorials support has been added, and the last_update_date field's value has changed.

sections
Documentation, API Reference, MCP Server, Changelog
has sdk docs
true
has tutorials
true
last update date
null
has api reference
true
06:22
Jun 4
PRICING+1

The pricing structure has been updated, primarily changing the billing period for most paid tiers from monthly to annual. Additionally, the Enterprise and Free tiers have adjustments to their billing and unit descriptions, and the free tier limits explanation was expanded.

tiers
{"cta":"Contact Sales","name":"Enterprise","price":"Custom","features":["Volume credit rates","Dedicated infrastructure…
has free tier
true
free tier limits
500 credits, one-time / lifetime — not a monthly reset
pricing transparent
true
06:22
Jun 4
HOMEPAGE+1

The website's navigation menu was streamlined, and several key differentiators were rephrased or replaced with new, more concise descriptions focusing on developer benefits like 'Local-first by design' and 'Rust core'. The primary call-to-action text and social proof summary were also simplified.

sub tagline
You keep your pipeline, your data, and your budget when you leave. The engine is AGPL-3.0, the benchmark is public, and…
primary cta url
/pricing
primary tagline
The web data API with no exit cost
primary cta text
Get Started
nav primary items
Playground, Docs, Blog, Pricing, Skills, Resources, GitHub
secondary cta text
null
06:06
Jun 4
DOCSNO CHANGE

No change since last scan.

sections
Documentation, API Reference, MCP Server, Changelog
has sdk docs
true
has tutorials
false
last update date
null
has api reference
true
06:06
Jun 4
PRICING+1

The 'Forever Free' tier has been removed, and pricing transparency is now enabled. The Free tier's 'per_unit' and 'billing_period' descriptions were updated.

tiers
{"cta":"Contact Sales","name":"Enterprise","price":"Custom","features":["Volume credit rates","Dedicated infrastructure…
has free tier
true
free tier limits
500 credits, one-time / lifetime
pricing transparent
true
06:05
Jun 4
HOMEPAGE+1

The website underwent a significant update, expanding its navigation with new sections like 'Alternatives' and 'Benchmarks', and elaborating on various key product differentiators to provide more detail on features like local-first operation, anti-detection, and browser rendering. The call to action, social proof, and overall positioning statement were also updated to reflect a more comprehensive offering targeting AI agents and teams.

sub tagline
You keep your pipeline, your data, and your budget when you leave. The engine is AGPL-3.0, the benchmark is public, and…
primary cta url
/pricing
primary tagline
The web data API with no exit cost
primary cta text
500 Free Credits — Get Started
nav primary items
Playground, Docs, Blog, Pricing, Skills, Alternatives, Benchmarks, Use Cases, Integrations, GitHub, Sign in, Get Starte…
secondary cta text
See pricing
Jun 3, 2026
11 changed · 11 scans
06:36
Jun 3
DOCS+1

The availability of tutorials changed from true to false, and links for GitHub, Issues, and License (AGPL-3.0) were removed from the sections.

sections
Documentation, API Reference, MCP Server, Changelog
has sdk docs
true
has tutorials
false
last update date
null
has api reference
true
06:36
Jun 3
PRICING+1

The pricing page updated credit number formatting across several paid tiers. Key changes include revised billing period and per_unit values for the free tiers, a change in the pricing_transparent flag, and a more concise free_tier_limits description.

tiers
{"cta":"Contact Sales","name":"Enterprise","price":"Custom","features":["Volume credit rates","Dedicated infrastructure…
has free tier
true
free tier limits
500 credits, one-time / lifetime; 2 concurrency
pricing transparent
false
06:35
Jun 3
HOMEPAGE+1

The primary call to action has been updated to emphasize a free tier and a direct registration link. Several key product differentiators were added or rephrased, highlighting open-source self-hosting, cost-effectiveness, built-in browser rendering, and native MCP support.

sub tagline
You keep your pipeline, your data, and your budget when you leave. The engine is AGPL-3.0, the benchmark is public, and…
primary cta url
/register
primary tagline
The web data API with no exit cost
primary cta text
Get Started Free
nav primary items
Playground, Docs, Blog, Pricing, Skills, Resources, GitHub, Sign in, Get Started Free
secondary cta text
null
06:20
Jun 3
BLOG+76

The blog content has significantly expanded with 76 new posts. The primary topics have been updated to reflect a more refined focus on AI agent web scraping, RAG pipelines, Rust architecture, and tool comparisons.

post frequency
daily
primary topics
AI Agent Web Scraping, RAG Pipeline Integration, Rust & Web Scraping Architecture, Scraping Tool Comparisons, Self-Host…
recent post urls
/blog/diffbot-vs-fastcrw, /blog/migrate-ruby-scraper-to-go, /blog/firecrawl-vs-fastcrw, /blog/web-scraping-in-java, /bl…
developer focused
true
recent post dates
Jun 2, 2026, Jun 2, 2026, Jun 1, 2026, Jun 1, 2026, May 31, 2026, May 30, 2026, May 30, 2026, May 29, 2026, May 28, 202…
recent post titles
Diffbot vs fastCRW: CV Extraction or LLM JSON, Ruby to Go: Rewriting Legacy Scrapers for Speed, Firecrawl vs fastCRW: T…
06:19
Jun 3
DOCS+1

New links for GitHub, Issues, and the project license have been added to the sections. Additionally, the `last_update_date` value was corrected.

sections
Documentation, API Reference, MCP Server, Changelog, GitHub, Issues, License (AGPL-3.0)
has sdk docs
true
has tutorials
true
last update date
null
has api reference
true
06:19
Jun 3
PRICING+1

A new 'Forever Free' self-hosted tier was added, providing an AGPL-3.0 licensed option with unlimited usage at no cost. The free tier limits description was also clarified, and credit number formatting was updated across several managed plans.

tiers
{"cta":"Contact Sales","name":"Enterprise","price":"Custom","features":["Volume credit rates","Dedicated infrastructure…
has free tier
true
free tier limits
500 credits, one-time / lifetime — not a monthly reset
pricing transparent
true
06:19
Jun 3
HOMEPAGE+1

The website's key differentiators have been significantly rephrased and condensed for conciseness, often with new short titles for features. Minor updates include removing a navigation item and shortening the social proof.

sub tagline
You keep your pipeline, your data, and your budget when you leave. The engine is AGPL-3.0, the benchmark is public, and…
primary cta url
/pricing
primary tagline
The web data API with no exit cost
primary cta text
500 Free Credits — Get Started
nav primary items
Playground, Docs, Blog, Pricing, Skills, Resources, GitHub, Sign in
secondary cta text
null
06:04
Jun 3
BLOG+1

The blog added two new posts and significantly revised its primary topic categorization, introducing more specific categories and rephrasing an existing one.

post frequency
daily
primary topics
Web Scraping Technologies, AI Agents and RAG Pipelines, API Comparisons and Alternatives, Self-Hosting and Infrastructu…
recent post urls
/blog/diffbot-vs-fastcrw, /blog/migrate-ruby-scraper-to-go, /blog/firecrawl-vs-fastcrw, /blog/web-scraping-in-java, /bl…
developer focused
true
recent post dates
Jun 2, 2026, Jun 2, 2026, Jun 1, 2026, Jun 1, 2026, May 31, 2026, May 30, 2026, May 30, 2026, May 29, 2026, May 28, 202…
recent post titles
Diffbot vs fastCRW: CV Extraction or LLM JSON, Ruby to Go: Rewriting Legacy Scrapers for Speed, Firecrawl vs fastCRW: T…
06:03
Jun 3
DOCS+1

The two versions of the competitor page are identical; no significant changes were detected.

sections
Documentation, API Reference, MCP Server, Changelog
has sdk docs
true
has tutorials
true
last update date
.null
has api reference
true
06:03
Jun 3
PRICING+1

The 'Forever Free' tier has been removed. The pricing transparency status is now true, and the free tier limits description has been simplified. The 'Enterprise' tier's per-unit pricing is now explicitly 'Volume credit rates'.

tiers
{"cta":"Contact Sales","name":"Enterprise","price":"Custom","features":["Volume credit rates","Dedicated infrastructure…
has free tier
true
free tier limits
500 credits, one-time / lifetime
pricing transparent
true
06:02
Jun 3
HOMEPAGE+1

The page was significantly updated with many new and detailed developer-facing features, including support for document parsing, interactive browser actions, Model Context Protocol (MCP) integrations, and structured JSON extraction. Existing differentiators were expanded with more specific technical and pricing details, while the target audience was slightly narrowed.

sub tagline
You keep your pipeline, your data, and your budget when you leave. The engine is AGPL-3.0, the benchmark is public, and…
primary cta url
/pricing
primary tagline
The web data API with no exit cost
primary cta text
500 Free Credits — Get Started
nav primary items
Playground, Docs, Blog, Pricing, Skills, Resources, GitHub, Sign in, Get Started Free
secondary cta text
null
Jun 2, 2026
11 changed · 12 scans
06:44
Jun 2
BLOG+1

The blog's primary topics have been updated with more descriptive categories, and a substantial number of new blog posts have been published, significantly expanding the available content on web scraping, AI agents, and Rust infrastructure.

post frequency
daily
primary topics
Web Scraping Tools and APIs (Comparisons & Alternatives), AI Agents and RAG Pipelines (Integration & Use Cases), Web Sc…
recent post urls
/blog/firecrawl-vs-fastcrw, /blog/web-scraping-in-java, /blog/firecrawl-scrape-endpoint-deep-dive, /blog/crw-vs-tavily-…
developer focused
true
recent post dates
Jun 1, 2026, Jun 1, 2026, May 31, 2026, May 30, 2026, May 30, 2026, May 29, 2026, May 28, 2026, May 27, 2026, May 27, 2…
recent post titles
Firecrawl vs fastCRW: The Honest 1:1 Comparison (2026), Web Scraping in Java (2026): JSoup, the JVM Footprint Tax, and …
06:42
Jun 2
DOCSNO CHANGE

No change since last scan.

sections
Documentation, API Reference, MCP Server, Changelog
has sdk docs
true
has tutorials
true
last update date
.null
has api reference
true
06:42
Jun 2
PRICING+1

A new "Forever Free" tier has been introduced, the pricing transparency status has changed to false, and the free tier limits description has been clarified. The "Standard" tier also has a minor property name change from "is_self_serve" to "is_unit".

tiers
{"cta":"Contact Sales","name":"Enterprise","price":"Custom","features":["Volume credit rates","Dedicated infrastructure…
has free tier
true
free tier limits
500 credits, one-time / lifetime — not a monthly reset
pricing transparent
false
06:41
Jun 2
HOMEPAGE+1

The page updated calls to action and expanded technical descriptions for existing key differentiators. Several new, detailed technical features were added, such as built-in stealth, Rust architecture benefits, and unified credit pooling, while some previous differentiator bullet points were either removed or integrated into more comprehensive descriptions. The target audience was also broadened.

sub tagline
You keep your pipeline, your data, and your budget when you leave. The engine is AGPL-3.0, the benchmark is public, and…
primary cta url
/pricing
primary tagline
The web data API with no exit cost
primary cta text
500 Free Credits — Get Started
nav primary items
Playground, Docs, Blog, Pricing, Skills, Resources, GitHub, Sign in, Get Started Free
secondary cta text
null
06:26
Jun 2
BLOG+1

Five primary topics were rephrased to be more descriptive or broader in scope. Additionally, an unusual prefix was removed from several recent post URLs.

post frequency
daily
primary topics
AI Agents and RAG Pipelines, Web Scraping APIs and Libraries, Performance and Engineering (Rust-focused), Competitor Co…
recent post urls
/blog/firecrawl-vs-fastcrw, /blog/web-scraping-in-java, /blog/firecrawl-scrape-endpoint-deep-dive, /blog/crw-vs-tavily-…
developer focused
true
recent post dates
Jun 1, 2026, Jun 1, 2026, May 31, 2026, May 30, 2026, May 30, 2026, May 29, 2026, May 28, 2026, May 27, 2026, May 27, 2…
recent post titles
Firecrawl vs fastCRW: The Honest 1:1 Comparison (2026), Web Scraping in Java (2026): JSoup, the JVM Footprint Tax, and …
06:25
Jun 2
DOCS+1

The `last_update_date` field was updated from a placeholder string to an empty string.

sections
Documentation, API Reference, MCP Server, Changelog
has sdk docs
true
has tutorials
true
last update date
.null
has api reference
true
06:25
Jun 2
PRICING+1

The Standard tier's 'is_self_serve' property was removed and an 'is_unit' property was added. The Free tier's 'billing_period' was changed to 'one-time'.

tiers
{"cta":"Contact Sales","name":"Enterprise","price":"Custom","features":["Volume credit rates","Dedicated infrastructure…
has free tier
true
free tier limits
500 credits, one-time / lifetime
pricing transparent
true
06:24
Jun 2
HOMEPAGE+1

The primary call to action text and URL were updated, and the stated target audience is more concise. Several new key differentiators focusing on a single binary, MCP tools, clean markdown output, and metadata inclusion were added, while a number of detailed feature descriptions were removed or shortened.

sub tagline
You keep your pipeline, your data, and your budget when you leave. The engine is AGPL-3.0, the benchmark is public, and…
primary cta url
/register
primary tagline
The web data API with no exit cost
primary cta text
Get Started Free
nav primary items
Playground, Docs, Blog, Pricing, Skills, Resources, GitHub, Sign in, Get Started Free
secondary cta text
See pricing
06:10
Jun 2
BLOG+1

The list of recent blog posts was updated with two new entries, replacing two older ones. The primary topics were reworded, and an unexpected prefix was added to several existing post URLs.

post frequency
daily
primary topics
Web Scraping APIs, AI Agents and RAG Pipelines, Technical Comparisons of Scraping Tools, Rust Engineering for Web Scrap…
recent post urls
/blog/firecrawl-vs-fastcrw, 品種/blog/web-scraping-in-java, 品種/blog/firecrawl-scrape-endpoint-deep-dive, 品種/blog/crw-vs-t…
developer focused
true
recent post dates
Jun 1, 2026, Jun 1, 2026, May 31, 2026, May 30, 2026, May 30, 2026, May 29, 2026, May 28, 2026, May 27, 2026, May 27, 2…
recent post titles
Firecrawl vs fastCRW: The Honest 1:1 Comparison (2026), Web Scraping in Java (2026): JSoup, the JVM Footprint Tax, and …
06:09
Jun 2
DOCS+1

Tutorials have been added.

sections
Documentation, API Reference, MCP Server, Changelog
has sdk docs
true
has tutorials
true
last update date
.null
has api reference
true
06:09
Jun 2
PRICING+1

The 'Forever Free (Self-Hosted)' pricing tier was removed, and the Free tier's billing period was changed to include 'lifetime'.

tiers
{"cta":"Contact Sales","name":"Enterprise","price":"Custom","features":["Volume credit rates","Dedicated infrastructure…
has free tier
true
free tier limits
500 credits, one-time / lifetime
pricing transparent
true
06:09
Jun 2
HOMEPAGE+1

The page introduced new navigation items and a secondary call to action. Key differentiators were updated with new points highlighting AI agent compatibility, performance, and cost benefits, while others were rephrased for conciseness. Some specific technical details were removed from the differentiators list, and the social proof and target audience statements were also updated.

sub tagline
You keep your pipeline, your data, and your budget when you leave. The engine is AGPL-3.0, the benchmark is public, and…
primary cta url
/pricing
primary tagline
The web data API with no exit cost
primary cta text
500 Free Credits — Get Started
nav primary items
Playground, Docs, Blog, Pricing, Skills, Resources, GitHub, Sign in, Get Started Free
secondary cta text
See pricing
Jun 1, 2026
12 changed · 12 scans
06:36
Jun 1
BLOG+1

The primary topics listed on the page have been rephrased to improve clarity and consistency.

post frequency
daily
primary topics
AI Agent Web Scraping, RAG & LLM Data Pipelines, Web Scraping API Comparisons, Rust & Performance Engineering, Self-Hos…
recent post urls
/blog/firecrawl-scrape-endpoint-deep-dive, /blog/crw-vs-tavily-exa-perplexity-search-answer-api, /blog/firecrawl-sdk-ba…
developer focused
true
recent post dates
May 31, 2026, May 30, 2026, May 30, 2026, May 29, 2026, May 28, 2026, May 27, 2026, May 27, 2026, May 27, 2026, May 25,…
recent post titles
Firecrawl /scrape Deep Dive: Formats, JS Rendering, and the Compatible Way to Call It, CRW vs Tavily vs Exa vs Perplexi…
06:36
Jun 1
DOCS+1

The navigation sections for GitHub, Issues, and License (AGPL-3.0) have been removed.

sections
Documentation, API Reference, MCP Server, Changelog
has sdk docs
true
has tutorials
false
last update date
.null
has api reference
true
06:36
Jun 1
PRICING+1

The pricing page now indicates transparent pricing, with simplifications to the free tier limits description and clearer billing period specifications for the Enterprise, Free, and Forever Free (Self-Hosted) tiers.

tiers
{"cta":"Contact Sales","name":"Enterprise","price":"Custom","features":["Volume credit rates","Dedicated infrastructure…
has free tier
true
free tier limits
500 credits, one-time / lifetime
pricing transparent
true
06:35
Jun 1
HOMEPAGE+1

The competitor page has introduced several new advanced web scraping features, including media parsing, interactive actions, and built-in stealth, while also streamlining its key differentiators for conciseness. Minor changes include updates to the primary call to action, navigation items, and target audience description.

sub tagline
You keep your pipeline, your data, and your budget when you leave. The engine is AGPL-3.0, the benchmark is public, and…
primary cta url
/pricing
primary tagline
The web data API with no exit cost
primary cta text
500 Free Credits — Get Started
nav primary items
Playground, Docs, Blog, Pricing, Skills, Resources, GitHub
secondary cta text
null
06:21
Jun 1
BLOG+1

The blog updated its primary topics to be more specific and significantly reduced the number of recent blog posts displayed from 82 to 10.

post frequency
daily
primary topics
Web Scraping for AI Agents, RAG Pipeline Integration, Web Scraping Tool Comparisons, Engineering & Architecture (Rust, …
recent post urls
/blog/firecrawl-scrape-endpoint-deep-dive, /blog/crw-vs-tavily-exa-perplexity-search-answer-api, /blog/firecrawl-sdk-ba…
developer focused
true
recent post dates
May 31, 2026, May 30, 2026, May 30, 2026, May 29, 2026, May 28, 2026, May 27, 2026, May 27, 2026, May 27, 2026, May 25,…
recent post titles
Firecrawl /scrape Deep Dive: Formats, JS Rendering, and the Compatible Way to Call It, CRW vs Tavily vs Exa vs Perplexi…
06:20
Jun 1
DOCS+1

New navigation links for GitHub, Issues, and License (AGPL-3.0) have been added to the site sections.

sections
Documentation, API Reference, MCP Server, Changelog, GitHub, Issues, License (AGPL-3.0)
has sdk docs
true
has tutorials
false
last update date
.null
has api reference
true
06:20
Jun 1
PRICING+1

The 'Free' tier now explicitly includes 'lifetime' in its billing period and specifies '2 concurrency' and 'Community support' in its limits, while the 'Self-Hosted' tier's name was reordered and its billing period also updated to 'one-time / lifetime'.

tiers
{"cta":"Contact Sales","name":"Enterprise","price":"Custom","features":["Volume credit rates","Dedicated infrastructure…
has free tier
true
free tier limits
500 credits, one-time / lifetime — not a monthly reset; 2 concurrency; Community support
pricing transparent
false
06:19
Jun 1
HOMEPAGE+1

The page introduced more explicit pricing and migration details, particularly for Firecrawl compatibility and structured data extraction. Several key feature descriptions were either condensed into shorter bullet points or entirely removed, while calls to action were updated.

sub tagline
You keep your pipeline, your data, and your budget when you leave. The engine is <a href="https://github.com/us/crw">AG…
primary cta url
/register
primary tagline
The web data API with<br/><span>no exit cost</span>
primary cta text
Get Started Free
nav primary items
Playground, Docs, Blog, Pricing, Skills, Resources, GitHub, Sign in, Get Started Free
secondary cta text
500 Free Credits — Get Started
06:05
Jun 1
BLOG+1

The blog has undergone a substantial expansion, adding numerous new posts that delve into advanced web scraping for AI agents, RAG pipelines, and specific feature deep dives. Concurrently, the primary topics list has been refined to better categorize the evolving content and highlight key areas of focus.

post frequency
daily
primary topics
AI Agent & RAG Pipeline Integration, Web Scraping & Crawling Technologies, Competitor & Tool Comparisons, Engineering &…
recent post urls
/blog/firecrawl-scrape-endpoint-deep-dive, /blog/crw-vs-tavily-exa-perplexity-search-answer-api, /blog/firecrawl-sdk-ba…
developer focused
true
recent post dates
May 31, 2026, May 30, 2026, May 30, 2026, May 29, 2026, May 28, 2026, May 27, 2026, May 27, 2026, May 27, 2026, May 25,…
recent post titles
Firecrawl /scrape Deep Dive: Formats, JS Rendering, and the Compatible Way to Call It, CRW vs Tavily vs Exa vs Perplexi…
06:04
Jun 1
DOCS+1

The 'has_tutorials' flag changed from true to false, and the 'last_update_date' value was updated from "null" to ".null".

sections
Documentation, API Reference, MCP Server, Changelog
has sdk docs
true
has tutorials
false
last update date
.null
has api reference
true
06:04
Jun 1
PRICING+1

A new 'Self-Hosted (Forever Free)' tier has been added, and the billing period for all existing managed cloud paid tiers has shifted from annual to monthly. The `pricing_transparent` flag was changed to `false`, and the free tier limits description was updated for clarity.

tiers
{"cta":"Contact Sales","name":"Enterprise","price":"Custom","features":["Volume credit rates","Dedicated infrastructure…
has free tier
true
free tier limits
500 credits, one-time / lifetime — not a monthly reset
pricing transparent
false
06:04
Jun 1
HOMEPAGE+1

The page significantly expanded its list of key differentiators, providing much more detailed descriptions for existing features and introducing several new features related to AI agent integration, structured data extraction, and cost. Minor changes were made to CTA text, URLs, and the positioning/target audience statements.

sub tagline
You keep your pipeline, your data, and your budget when you leave. The engine is AGPL-3.0, the benchmark is public, and…
primary cta url
/pricing
primary tagline
The web data API with no exit cost
primary cta text
500 Free Credits — Get Started
nav primary items
Playground, Docs, Blog, Pricing, Skills, Resources, GitHub, Sign in, Get Started Free
secondary cta text
Get Started Free
MatrixAPI InsightsDemo
Powered by Tabstack
Rival · open source competitive intelligencegithub.com/tessak22/rival