AI Agent Search: No API Key, Results Reranked Locally

Giving an AI agent web search usually means connecting a third-party search API: paying, applying for a key, tuning parameters. mcpsurfr takes a different route: search requests go directly to multiple engines, and results are reranked locally. It is a locally running MCP server (Model Context Protocol, through which AI models connect to external tools and data) that feeds relevant web content into the LLM's context.

See the workflow:

mcpsurfr workflow

The LLM initiates a query; ddgs in auto mode queries several search engines at once and skips blocked ones; MobileBERT performs local reranking on NumPy; then it fetches the top-ranked pages and keeps the main text.

Results are returned in two steps: Tier 1 selects pages based on summaries; Tier 2 splits pages into passages and performs BM25 keyword matching, returning only relevant passages and source URLs, about 800 tokens. When there are too few or too many matches, it moves to Tier 3 to rank passages semantically.

Key features:

No API key. Uses the ddgs library, with auto mode querying multiple engines and automatically skipping blocked ones.
Local reranking. A small MobileBERT model runs on pure NumPy and fetches only the best pages.
Clean main text. Browser-simulated fetching removes menus, cookie banners, and footers. Code blocks and tables are preserved; different URLs for the same page are returned only once.
Passage-level returns. search returns passages that answer the question, roughly one-tenth the tokens of a full page. When there is no good match, it asks whether you want the full text.
Reports omissions. Skipped pages state the reason (blocked, PDF, duplicate); truncated pages are marked truncated; fallbacks and engine failures are shown as warnings. Google is often blocked and automatically falls back to auto.
Bundled skills. The server provides skill and research prompts, so clients know how to use it without configuration.

The tools are as follows:

Tool	Purpose
search	Entry point; returns only relevant passages; passing keywords works better
rag_search_ddgs	Full-page text, raw mode
rag_search_google	Goes through Google; falls back to auto when blocked
deep_research	Queries multiple search terms across backends, merges and deduplicates
deep_research_ddgs / deep_research_google	Shortcuts for deep_research

Backends support auto (default), brave, duckduckgo, google, grokipedia, mojeek, startpage, wikipedia, yahoo. auto and brave are the most reliable.

Install with uv:

{
  "mcpServers": {
    "mcpsurfr": {
      "command": "uvx",
      "args": ["--from", "git+https://github.com/nkapila6/mcpsurfr", "mcpsurfr"]
    }
  }
}

Or Docker (supports amd64 and arm64):

{
  "mcpServers": {
    "mcpsurfr": {
      "command": "docker",
      "args": ["run", "--rm", "-i", "--init", "ghcr.io/nkapila6/mcpsurfr:latest"]
    }
  }
}

The full-page tool has a 40k-character budget per call (about 10k tokens). To change the size, add the environment variable MCPSURFR_MAX_CHARS in the server configuration, for example 80000. Tested on Claude Desktop, Claude Code, Cursor, and Goose.

The project was renamed from mcp-local-rag; old configurations still work, and the old commands are retained.

Example: asking in Claude Desktop about something that happened after its training data.

example prompt

example result

Someone who saw it said local reranking can save many useless requests and is 'a sensible design.' Others asked whether pages are cached between runs. The project does not mention caching.

The project is MIT open source, has passed the MseeP security audit, and welcomes issues and PRs. GitHub: https://github.com/nkapila6/mcpsurfr . The author is Dan Kornas; he also writes an AI-related newsletter, free to subscribe: https://dankornas.substack.com