AI Crawler Checker — Robots.txt Policy for Six Bots vs ImportYeti Scraper (Apify) - US import records and bills of lading
AI Crawler Checker — Robots.txt Policy for Six Bots is an MCP server for web search and scraping. ImportYeti Scraper (Apify) - US import records and bills of lading is a paid MCP server for web search and scraping. The table puts their facts side by side; each name links to its full write-up.
- ImportYeti Scraper (Apify) - US import records and bills of lading is open source; AI Crawler Checker — Robots.txt Policy for Six Bots is not.
- ImportYeti Scraper (Apify) - US import records and bills of lading is best for find who supplies a competitor and from which country.
- In short
- Check declared robots.txt permissions for six AI crawlers at URL pathsImportYeti US import records and company profiles as JSON, billed per row
- Kind
- MCP serverMCP server
- Pricing
- Paid
- Platforms
- API
- Runs
- RemotelyRemotely
- GitHub stars
- 0
- Last commit
- Nov 2026
- Open source
Full write-ups
AI Crawler Checker — Robots.txt Policy for Six Bots vs ImportYeti Scraper (Apify) - US import records and bills of lading: questions
What is the difference between AI Crawler Checker — Robots.txt Policy for Six Bots and ImportYeti Scraper (Apify) - US import records and bills of lading?
AI Crawler Checker — Robots.txt Policy for Six Bots is an MCP server for web search and scraping. ImportYeti Scraper (Apify) - US import records and bills of lading is a paid MCP server for web search and scraping. ImportYeti Scraper (Apify) - US import records and bills of lading is open source; AI Crawler Checker — Robots.txt Policy for Six Bots is not.