AgentSearch MCP

Perplexity-style web retrieval MCP server for LLM agents

// overview

An open-source, Perplexity-style web retrieval system for LLM agents. A typed MCP server in TypeScript exposes search, fetch, and grounded summarization tools through a clean, agent-facing interface, backed by an async Python extraction pipeline that turns live web pages into citation-ready markdown.

// architecture

The TypeScript MCP server defines the agent-facing tool surface (search, fetch, grounded summarization) with typed schemas. Behind it, an async Python pipeline built on FastAPI performs parallel fetching and content extraction, using PostgreSQL and Redis caching to deduplicate work and serve repeated retrievals quickly. Extracted pages are normalized into citation-ready markdown for downstream agent workflows.

// challenges

• ·Designing a tool interface clean enough for arbitrary LLM agents to consume while keeping the underlying retrieval and extraction logic flexible
• ·Keeping latency low under parallel fetching by layering Redis caching and connection reuse over the async Python extraction pipeline
• ·Transforming messy, inconsistent web pages into structured, citation-ready markdown that preserves source attribution

// learnings

• ·A typed MCP boundary makes a retrieval system far easier for agents to use correctly — the schema does a lot of the work
• ·Caching strategy matters as much as the fetch logic when the same queries and pages recur across agent runs
• ·Grounded summarization is only as trustworthy as the citation pipeline feeding it — attribution has to be a first-class concern