# machine_internet

> machine_internet is an open-source MCP server for web search and scraping. Turns websites with hidden internal APIs into MCP servers, without an official API. It is available on CLI, self-hosted and API.

- **Kind:** MCP server
- **Pricing:** Open source
- **Platforms:** CLI, self-hosted, API
- **Repository:** https://github.com/chetanty/machine_internet
- **Categories:** [web search and scraping](https://ai-powered.com/categories/web-search-and-scraping)
- **Listing updated:** 2026-11-18

## About

The README says Machine Internet loads a page in a headless browser, records the JSON calls the page makes to its own backend, and uses an LLM to infer endpoints. Those endpoints are then condensed into a small set of named tools that an MCP client can call directly against the live site.

The workflow is a single command: pointing it at a URL produces a server that Claude Desktop, Cursor or another MCP client can run. It first checks for a published OpenAPI spec and only sniffs traffic when none exists. Auth can be supplied through environment variables for bearer tokens, API keys, basic auth or OAuth2 client credentials.

It suits developers who want structured access to data on sites that offer no API, such as directories and search pages built as single-page apps. The README is explicit that server-rendered pages and sites with bot protection often fail, and that generated tools can break when the site changes its frontend.

## Features

- **Hidden API discovery:** Records same-site JSON requests made by a page in headless Chromium and infers their endpoints.
- **OpenAPI spec detection:** Probes common spec locations first and parses a real spec when one exists.
- **Condensed agent tools:** Collapses inferred endpoints into at most 15 verb_noun tools with descriptions for agents.
- **Local and remote serving:** Runs as a stdio server by default or over SSE, with hosted examples available as remote endpoints.
- **Terminal tool calls:** Lets you list, inspect and call generated tools directly from the command line.
- **Auth via environment variables:** Reads bearer, API key, basic and OAuth2 credentials from environment variables at runtime.

## Good for

- Give an AI assistant search access to a site that has no public API
- Prototype data access to a web app before requesting an official integration
- Pull structured results from a directory or search page into an agent workflow
- Explore which internal endpoints a single-page app uses

## FAQ

### Does it need an API key?

Serving the examples needs no key. Wrapping a new site needs one LLM API key, and the README names Gemini, Anthropic and OpenAI as options.

### Which sites does it work on?

The README says it works on sites that load data with browser-side API calls, such as most single-page apps and search pages. It does not work on server-rendered pages or sites that block headless browsers.

### How is it installed?

With uv via uvx machine-internet, or with pip install machine-internet, including a direct install from GitHub.

### What license is it under?

Apache 2.0, according to the README.

### Is machine_internet free?

Yes. machine_internet is open source, so you can use and self-host it for free.

### What are the best alternatives to machine_internet?

The closest alternatives to machine_internet are Scrapling MCP Server, MarkUDown, WebCrawlerAPI, ScrapeNinja MCP by usefulapi and ScrapeOwl MCP by usefulapi.

## Where this comes from

- Description: written from the project's README.

## Alternatives

- [Scrapling MCP Server](https://ai-powered.com/mcp/scrapling-mcp-server): Scrapling MCP Server is an open-source MCP server for web search and scraping.
- [MarkUDown](https://ai-powered.com/mcp/markudown): MarkUDown is a freemium MCP server for web search and scraping.
- [WebCrawlerAPI](https://ai-powered.com/mcp/webcrawlerapi): WebCrawlerAPI is a paid MCP server for web search and scraping.
- [ScrapeNinja MCP by usefulapi](https://ai-powered.com/mcp/scrapeninja-mcp-by-usefulapi): ScrapeNinja MCP by usefulapi is a freemium MCP server for web search and scraping.
- [ScrapeOwl MCP by usefulapi](https://ai-powered.com/mcp/scrapeowl-mcp-by-usefulapi): ScrapeOwl MCP by usefulapi is a freemium MCP server for web search and scraping.
- [Browserless](https://ai-powered.com/mcp/browserless): Browserless is a freemium MCP server for browser automation.
- [OpenGraph.io MCP Server](https://ai-powered.com/mcp/opengraph-io-mcp-server): OpenGraph.io MCP Server is a freemium MCP server for web search and scraping.
- [MCP Browser](https://ai-powered.com/mcp/mcp-browser-3): MCP Browser is an open-source MCP server for browser automation.

---
Source: https://ai-powered.com/mcp/machine-internet
