BetterWebSearch MCP

Quickstart

Mon, 01 Jan 0001 00:00:00 +0000

BetterWebSearch works out of the box with DuckDuckGo — no API keys, no .env, no configuration.

Option 1: npx (recommended)#

npx -y better-web-search-mcp --help # keyless — DuckDuckGo works immediately
npx -y better-web-search-mcp # start MCP stdio server

Or install globally:

npm i -g better-web-search-mcp
better-web-search-mcp --help
better-web-search-mcp --version

Option 2: From source (30s)#

git clone https://github.com/PhantomPixelDev/BetterWebSearch-MCP.git
cd BetterWebSearch-MCP
npm install
npm run build
npx better-web-search-mcp --help
BRAVE_API_KEY= npx better-web-search-mcp # still works, DuckDuckGo fallback

No .env required for the first test. The server boots, prints a provider banner to stderr, and every tool works.

Installation

Mon, 01 Jan 0001 00:00:00 +0000

Requirements#

Node.js 20+ and npm
No API keys required — DuckDuckGo works out of the box

Install via npx (no install)#

npx -y better-web-search-mcp --help # keyless — DuckDuckGo works immediately
npx -y better-web-search-mcp # start MCP stdio server

Install globally#

npm i -g better-web-search-mcp
better-web-search-mcp --help
better-web-search-mcp --version

Install from source#

git clone https://github.com/PhantomPixelDev/BetterWebSearch-MCP.git
cd BetterWebSearch-MCP
npm ci
npm run build
npm test # 189+ tests, keyless suite
npx tsc --noEmit # lint

Native binding note#

better-sqlite3 needs a compiled native addon. If it fails to load (older Node, missing build tools, CI sandbox), the cache automatically falls back to an in-memory Map — no action needed. To restore the SQLite cache, run npm rebuild better-sqlite3 (Node 20+ recommended).

Tools Reference

Mon, 01 Jan 0001 00:00:00 +0000

BetterWebSearch MCP exposes 6 tools (5 unique + 1 alias). All are keyless-first — DuckDuckGo provides search without any API keys.

Overview#

Tool	What it does	Keyless?
`web_search`	Fast search (aggregated, deduped, reranked)	✓
`web_research` / `deep_search`	Deep research: rewrites question → parallel searches → top-10 extraction → citations	✓
`web_extract`	Clean extraction `HTTP → hydration → browser` with `confidence` & `api_endpoints`	✓
`web_find`	`site:` scoped search	✓
`web_news`	Recent news + timeline + diversity	✓

`web_search`#

Fast multi-provider search. Aggregates results from all enabled providers, deduplicates, re-ranks by relevance, and returns the top matches.

Configuration

Mon, 01 Jan 0001 00:00:00 +0000

All environment variables are optional. The server starts and works via DuckDuckGo when no keys are set. Create a .env only if you want additional providers.

cp .env.example .env # then edit

Environment Variables#

Variable	Required	Description
`BRAVE_API_KEY` or `BETTER_WEB_SEARCH_BRAVE_API_KEY`	no	Brave Search API key (primary). Get one
`TAVILY_API_KEY` or `BETTER_WEB_SEARCH_TAVILY_API_KEY`	no	Tavily key. tavily.com
`BETTER_WEB_SEARCH_DISABLE_CACHE`	no	`true` → in-memory fallback (no SQLite)
`BETTER_WEB_SEARCH_CACHE_PATH`	no	Custom SQLite path (default `data/cache.db`)
`BETTER_WEB_SEARCH_DISABLE_BROWSER`	no	`true` → skip Playwright fallback

Key aliases#

The config loader supports both the bare name and the BETTER_WEB_SEARCH_* prefix:

Providers

Mon, 01 Jan 0001 00:00:00 +0000

BetterWebSearch MCP uses a provider abstraction layer. Each provider implements the same SearchProvider interface, so the aggregation layer treats them uniformly. Providers are enabled or disabled based on environment keys.

Provider overview#

Provider	Key needed?	Env var	Status
DuckDuckGo	No	—	Always on (keyless fallback)
Brave	Yes	`BRAVE_API_KEY`	Primary — better ranking & recency
Tavily	Yes	`TAVILY_API_KEY`	Optional — when present
SerpApi	Yes	`SERPAPI_KEY`	Stub — not yet implemented

Keyless-first design#

DuckDuckGo is always enabled and requires no API key. This means:

Extraction

Mon, 01 Jan 0001 00:00:00 +0000

BetterWebSearch MCP extracts page content through a three-tier escalation pipeline managed by the AccessRouter. Each tier is progressively more powerful (and slower); the router only escalates when the current tier doesn’t produce enough content.

Overview#

URL → Level 1: HTTP fetch (< 1s)
 → Level 2: Hydration / structured data (1–3s)
 → Level 3: Playwright browser + API interception (3–10s)

Only escalates when needed. A page-cache hit short-circuits the entire pipeline.

Tier 1: HTTP Fetch#

Speed: < 1 second
Method: Standard fetch with user-agent spoofing, 10s timeout, 2MB cap
Confidence: 0.85

Fetches the raw HTML and runs Mozilla Readability to extract clean markdown. Wins when the page has enough visible text (> 500 characters after stripping tags).

Architecture

Mon, 01 Jan 0001 00:00:00 +0000

System overview#

graph TD
 A[Query / Question] --> B[expandQueries<br/>4–6 variants]
 B --> C[Providers<br/>Brave / Tavily / DuckDuckGo]
 C -->|Promise.allSettled| D[Deduplicate<br/>normalizeUrl]
 D --> E[Rerank<br/>relevance + domainScore + recency]
 E --> F[Top 10 Results]
 F --> G[AccessRouter]
 G --> H[Level 1: HTTP Fetch<br/>&lt; 1s]
 H -->|not enough content| I[Level 2: Hydration Data<br/>JSON-LD, __NEXT_DATA__]
 I -->|not enough content| J[Level 3: Playwright Browser<br/>3–10s]
 H --> K[Content Fusion]
 I --> K
 J --> K
 K --> L{Confidence scores<br/>api 0.99 &gt; jsonld 0.95<br/>&gt; rendered 0.90 &gt; readability 0.85}
 L --> M[Structured Response]
 M --> N[domainProfile + apiPatterns<br/>SQLite Cache]

Components#

1. MCP Server (`src/index.ts`)#

The CLI entry point. Parses --help / --version flags (exiting before MCP handshake), then starts the stdio MCP server with all 6 tools registered.

Token Efficiency

Mon, 01 Jan 0001 00:00:00 +0000

The point of doing web work inside the server is that the model never has to read the pages. This page reports what that is actually worth, measured rather than asserted. The harness ships in the repository under benchmarks/.

Method#

Two workflows, the same questions, the same pages:

Path	What lands in the agent’s context
baseline	The agent drives the tools: one `web_search`, then `web_extract` on each of the top 5 results. Every extracted page enters context in full.
research	One `web_research` call. The response is the cited passages plus their metadata.

Both paths open the same URLs, so this isolates the compression the pipeline performs from any difference in what was retrieved.

API: extraction/alternative

Mon, 01 Jan 0001 00:00:00 +0000

better-web-search-mcp

better-web-search-mcp / extraction/alternative

extraction/alternative#

Interfaces#

AlternativeDeps#

Defined in: src/extraction/alternative.ts:30

Injectable dependencies, defaulting to the real modules.

Properties#

Property	Type	Description	Defined in
`aggregateSearch?`	(`query`, `opts`) => `Promise`<`SearchResult`[]>	Search aggregator used for snippet fallback.	src/extraction/alternative.ts:34
`cache?`	`Cache`	Page cache consulted before fetching each variant.	src/extraction/alternative.ts:36
`fetchPage?`	(`url`, `opts`) => `Promise`<`FetchedPage`>	Page fetcher used for URL variants.	src/extraction/alternative.ts:32

Variables#

MAX_SNIPPET_CHARS#

const MAX_SNIPPET_CHARS: 200 = 200

Defined in: src/extraction/alternative.ts:27

Maximum snippet length for page-variant evidence.

API: extraction/apiIntercept

Mon, 01 Jan 0001 00:00:00 +0000

better-web-search-mcp

better-web-search-mcp / extraction/apiIntercept

extraction/apiIntercept#

Interfaces#

ApiPatternRow#

Defined in: src/extraction/apiIntercept.ts:145

An api_patterns table row (without the auto-increment id/timestamp).

Properties#

Property	Type	Defined in
`content_type`	`string`	src/extraction/apiIntercept.ts:149
`domain`	`string`	src/extraction/apiIntercept.ts:146
`endpoint_pattern`	`string`	src/extraction/apiIntercept.ts:147
`method`	`string`	src/extraction/apiIntercept.ts:148

ApiResponse#

Defined in: src/extraction/apiIntercept.ts:30

The minimal Playwright Response surface this module depends on.

Methods#

headers()#

headers(): Record<string, string>

Defined in: src/extraction/apiIntercept.ts:33

Returns#

Record<string, string>

json()#

json(): Promise<unknown>

Defined in: src/extraction/apiIntercept.ts:34

Returns#

Promise<unknown>

request()#

request(): { method: string; }

API: extraction/browser

Mon, 01 Jan 0001 00:00:00 +0000

better-web-search-mcp

better-web-search-mcp / extraction/browser

extraction/browser#

Classes#

BrowserPool#

Defined in: src/extraction/browser.ts:151

A pool of reusable browser contexts backed by a single chromium instance.

The browser is launched lazily on first use and reused for every render. Contexts are created on demand up to DEFAULT_POOL_SIZE and then reused round-robin, so no render launches a browser per request.

Constructors#

Constructor#

new BrowserPool(size?): BrowserPool

Defined in: src/extraction/browser.ts:158

Parameters#

Parameter	Type	Default value
`size`	`number`	`DEFAULT_POOL_SIZE`

Returns#

BrowserPool

API: extraction/evidence

Mon, 01 Jan 0001 00:00:00 +0000

better-web-search-mcp

better-web-search-mcp / extraction/evidence

extraction/evidence#

Interfaces#

Evidence#

Defined in: src/extraction/evidence.ts:18

A single piece of extracted content with provenance and confidence.

Properties#

Property	Type	Description	Defined in
`confidence`	`number`	Confidence in this evidence, 0..1.	src/extraction/evidence.ts:24
`snippet?`	`string`	A short snippet of the evidence, when available.	src/extraction/evidence.ts:28
`source`	`EvidenceSource`	Where the evidence came from (page, api, structured data, snippet).	src/extraction/evidence.ts:20
`type`	`EvidenceSource`	The kind of evidence, mirroring EvidenceSource.	src/extraction/evidence.ts:22
`url`	`string`	The URL the evidence was extracted from.	src/extraction/evidence.ts:26

FusedContent#

Defined in: src/extraction/evidence.ts:54

API: extraction/fetch

Mon, 01 Jan 0001 00:00:00 +0000

better-web-search-mcp

better-web-search-mcp / extraction/fetch

extraction/fetch#

Classes#

FetchTimeoutError#

Defined in: src/extraction/fetch.ts:55

A fetch that was aborted by the timeout.

Extends#

Error

Constructors#

Constructor#

new FetchTimeoutError(url, timeoutMs): FetchTimeoutError

Defined in: src/extraction/fetch.ts:56

Parameters#

Parameter	Type
`url`	`string`
`timeoutMs`	`number`

Returns#

FetchTimeoutError

Overrides#

Error.constructor

Properties#

Property	Modifier	Type	Description	Inherited from	Defined in
`cause?`	`public`	`unknown`	-	`Error.cause`	node_modules/typescript/lib/lib.es2022.error.d.ts:26
`message`	`public`	`string`	-	`Error.message`	node_modules/typescript/lib/lib.es5.d.ts:1077
`name`	`public`	`string`	-	`Error.name`	node_modules/typescript/lib/lib.es5.d.ts:1076
`stack?`	`public`	`string`	-	`Error.stack`	node_modules/typescript/lib/lib.es5.d.ts:1078
`stackTraceLimit`	`static`	`number`	The `Error.stackTraceLimit` property specifies the number of stack frames collected by a stack trace (whether generated by `new Error().stack` or `Error.captureStackTrace(obj)`). The default value is `10` but may be set to any valid JavaScript number. Changes will affect any stack trace captured after the value has been changed. If set to a non-number value, or set to a negative number, stack traces will not capture any frames.	`Error.stackTraceLimit`	node_modules/@types/node/globals.d.ts:68

Methods#

captureStackTrace()#

static captureStackTrace(targetObject, constructorOpt?): void

API: extraction/fusion

Mon, 01 Jan 0001 00:00:00 +0000

better-web-search-mcp

better-web-search-mcp / extraction/fusion

extraction/fusion#

Functions#

fuseContent()#

fuseContent(url, title, strategiesResults, metadata?): FusedContent

Defined in: src/extraction/fusion.ts:118

Fuse the results of several extraction strategies into one page extraction.

The strategy with the highest effective confidence and non-empty content becomes the primary body. Structured data and API endpoints from every strategy are merged into the returned object regardless of which strategy won the content race.

Parameters#

Parameter	Type	Description
`url`	`string`	The page URL the strategies were run against.
`title`	`string`	The page title, when known.
`strategiesResults`	readonly `StrategyResult`[]	The per-strategy extraction results.
`metadata`	`Record`<`string`, `string`>	Page metadata (title, description, published, author, siteName) to attach to the result.

Returns#

FusedContent

API: extraction/learn

Mon, 01 Jan 0001 00:00:00 +0000

better-web-search-mcp

better-web-search-mcp / extraction/learn

extraction/learn#

Functions#

recordApiPatterns()#

recordApiPatterns(url, captured, cache): void

Defined in: src/extraction/learn.ts:59

Record discovered API patterns for a domain after a browser render.

Parameters#

Parameter	Type
`url`	`string`
`captured`	readonly `CapturedApiResponse`[]
`cache`	`Cache` \| `undefined`

Returns#

void

toBestMethod()#

toBestMethod(method): BestMethod

Defined in: src/extraction/learn.ts:21

Map a router method to the domain-profile best_method value.

Parameters#

Parameter	Type
`method`	`RouterMethod`

Returns#

BestMethod

updateDomainProfile()#

updateDomainProfile(url, html, method, cache): void

Defined in: src/extraction/learn.ts:34

Update the domain profile after a successful extraction.

API: extraction/metadata

Mon, 01 Jan 0001 00:00:00 +0000

better-web-search-mcp

better-web-search-mcp / extraction/metadata

extraction/metadata#

Interfaces#

PageMetadata#

Defined in: src/extraction/metadata.ts:12

The result of a metadata extraction.

Properties#

Property	Type	Description	Defined in
`author`	`string`	Author from `article:author` / `meta[name=author]`.	src/extraction/metadata.ts:20
`description`	`string`	Description from `og:description` or `meta[name=description]`.	src/extraction/metadata.ts:16
`published`	`string`	Published date from `article:published_time` / `datePublished`.	src/extraction/metadata.ts:18
`siteName`	`string`	Site name from `og:site_name`.	src/extraction/metadata.ts:22
`title`	`string`	The `<title>` tag text.	src/extraction/metadata.ts:14

Functions#

extractMetadata()#

extractMetadata(html): PageMetadata

Defined in: src/extraction/metadata.ts:36

API: extraction/readability

Mon, 01 Jan 0001 00:00:00 +0000

better-web-search-mcp

better-web-search-mcp / extraction/readability

extraction/readability#

Interfaces#

ReadabilityResult#

Defined in: src/extraction/readability.ts:15

The result of a readability extraction.

Properties#

Property	Type	Description	Defined in
`byline`	`string`	Author metadata, when present.	src/extraction/readability.ts:27
`contentMarkdown`	`string`	The article content converted to Markdown.	src/extraction/readability.ts:19
`excerpt`	`string`	A short excerpt / description of the article.	src/extraction/readability.ts:23
`length`	`number`	Length of the article in characters.	src/extraction/readability.ts:25
`textContent`	`string`	Plain-text content with all HTML tags removed.	src/extraction/readability.ts:21
`title`	`string`	The article title.	src/extraction/readability.ts:17

Functions#

extractWithReadability()#

extractWithReadability(html, url): ReadabilityResult | null

API: extraction/router

Mon, 01 Jan 0001 00:00:00 +0000

better-web-search-mcp

better-web-search-mcp / extraction/router

extraction/router#

Interfaces#

GetPageOptions#

Defined in: src/extraction/router.ts:70

Options controlling the escalation pipeline.

Properties#

Property	Type	Description	Defined in
`browser_fallback?`	`boolean`	Whether to fall back to a browser render when HTTP is insufficient.	src/extraction/router.ts:78
`include_api_data?`	`boolean`	Whether to include API data captured during a browser render.	src/extraction/router.ts:74
`include_structured_data?`	`boolean`	Whether to attempt structured / hydration data extraction.	src/extraction/router.ts:76
`mode?`	`"auto"` \| `"browser"` \| `"fast"`	`auto` escalates; `fast` skips the browser; `browser` forces it.	src/extraction/router.ts:72

RoutedPage#

Defined in: src/extraction/router.ts:43

API: extraction/structured

Mon, 01 Jan 0001 00:00:00 +0000

better-web-search-mcp

better-web-search-mcp / extraction/structured

extraction/structured#

Interfaces#

StructuredData#

Defined in: src/extraction/structured.ts:13

The result of a structured-data extraction.

Properties#

Property	Type	Description	Defined in
`apollo`	`unknown`	Parsed `window.__APOLLO_STATE__` payload, when present.	src/extraction/structured.ts:23
`initialState`	`unknown`	Parsed `window.__INITIAL_STATE__` payload, when present.	src/extraction/structured.ts:25
`jsonLd`	`unknown`[]	Parsed JSON-LD objects (with `@graph` unwrapped into its members).	src/extraction/structured.ts:15
`nextData`	`unknown`	Parsed `#__NEXT_DATA__` payload, when present.	src/extraction/structured.ts:17
`nextFlight`	`string`	Concatenated `self.__next_f.push` flight chunks, when present.	src/extraction/structured.ts:19
`nuxt`	`unknown`	Parsed `window.__NUXT__` payload, when present.	src/extraction/structured.ts:21

Variables#

EMPTY_STRUCTURED#

const EMPTY_STRUCTURED: StructuredData

API: extraction/untrusted

Mon, 01 Jan 0001 00:00:00 +0000

better-web-search-mcp

better-web-search-mcp / extraction/untrusted

extraction/untrusted#

Interfaces#

InjectionFinding#

Defined in: src/extraction/untrusted.ts:18

A suspicious span found in extracted content.

Properties#

Property	Type	Description	Defined in
`excerpt`	`string`	The matched text, truncated for reporting.	src/extraction/untrusted.ts:22
`index`	`number`	Character offset of the match within the content.	src/extraction/untrusted.ts:24
`pattern`	`string`	Short identifier for the pattern that matched.	src/extraction/untrusted.ts:20

SecurityReport#

Defined in: src/extraction/untrusted.ts:28

The security annotation attached to extracted content.

Properties#

Property	Type	Description	Defined in
`findings`	`InjectionFinding`[]	The individual matches, capped to keep responses small.	src/extraction/untrusted.ts:37
`injection_suspected`	`boolean`	Whether any injection pattern matched.	src/extraction/untrusted.ts:35
`untrusted`	`true`	Always true. Page content comes from the open web and must be treated as data, never as instructions to the agent.	src/extraction/untrusted.ts:33

Variables#

INJECTION_NOTICE#

const INJECTION_NOTICE: string

API: index

Mon, 01 Jan 0001 00:00:00 +0000

better-web-search-mcp

better-web-search-mcp / index

index#

API: providers

Mon, 01 Jan 0001 00:00:00 +0000

better-web-search-mcp

better-web-search-mcp / providers

providers#

Variables#

PROVIDER_TIMEOUT_MS#

const PROVIDER_TIMEOUT_MS: 10000 = 10_000

Defined in: src/providers/index.ts:33

Wall-clock budget for a single provider, in milliseconds.

Functions#

aggregateSearch()#

aggregateSearch(query, opts?): Promise<SearchResult[]>

Defined in: src/providers/index.ts:69

Aggregate search across all enabled providers.

Uses Promise.allSettled so one failing provider never aborts the others, and caps each provider at PROVIDER_TIMEOUT_MS so one hung provider cannot gate the whole search — the others’ results are still returned. Rejected and timed-out providers are logged as warnings and skipped; fulfilled results are flattened into a single array.

API: providers/brave

Mon, 01 Jan 0001 00:00:00 +0000

better-web-search-mcp

better-web-search-mcp / providers/brave

providers/brave#

Classes#

BraveProvider#

Defined in: src/providers/brave.ts:71

Brave Search provider (primary).

Requires BRAVE_API_KEY in the environment. When the key is missing, or the API returns an auth/rate-limit error, it returns an empty array and logs a warning rather than throwing.

Implements#

SearchProvider

Constructors#

Constructor#

new BraveProvider(): BraveProvider

Returns#

BraveProvider

Properties#

Property	Modifier	Type	Default value	Description	Defined in
`name`	`readonly`	`"brave"`	`"brave"`	Stable provider identifier (e.g. “brave”, “tavily”, “duckduckgo”).	src/providers/brave.ts:72

Methods#

search()#

search(query, opts): Promise<SearchResult[]>

API: providers/duckduckgo

Mon, 01 Jan 0001 00:00:00 +0000

better-web-search-mcp

better-web-search-mcp / providers/duckduckgo

providers/duckduckgo#

Classes#

DuckDuckGoProvider#

Defined in: src/providers/duckduckgo.ts:16

DuckDuckGo provider (free, keyless fallback).

Scrapes the public HTML search endpoint and parses results with cheerio. Always enabled — no API key required. On any failure (network, timeout, parse) it returns an empty array and logs a warning rather than throwing.

Implements#

SearchProvider

Constructors#

Constructor#

new DuckDuckGoProvider(): DuckDuckGoProvider

Returns#

DuckDuckGoProvider

Properties#

Property	Modifier	Type	Default value	Description	Defined in
`name`	`readonly`	`"duckduckgo"`	`"duckduckgo"`	Stable provider identifier (e.g. “brave”, “tavily”, “duckduckgo”).	src/providers/duckduckgo.ts:17

Methods#

search()#

search(query, opts): Promise<SearchResult[]>

API: providers/serpapi

Mon, 01 Jan 0001 00:00:00 +0000

better-web-search-mcp

better-web-search-mcp / providers/serpapi

providers/serpapi#

Classes#

SerpApiProvider#

Defined in: src/providers/serpapi.ts:10

SerpApi provider stub (deferred).

SerpApi integration is not implemented yet. This stub satisfies the SearchProvider contract so the aggregation layer can include it without special-casing, but always returns no results.

Implements#

SearchProvider

Constructors#

Constructor#

new SerpApiProvider(): SerpApiProvider

Returns#

SerpApiProvider

Properties#

Property	Modifier	Type	Default value	Description	Defined in
`name`	`readonly`	`"serpapi"`	`"serpapi"`	Stable provider identifier (e.g. “brave”, “tavily”, “duckduckgo”).	src/providers/serpapi.ts:11

Methods#

search()#

search(_query, _opts): Promise<SearchResult[]>

API: providers/tavily

Mon, 01 Jan 0001 00:00:00 +0000

better-web-search-mcp

better-web-search-mcp / providers/tavily

providers/tavily#

Classes#

TavilyProvider#

Defined in: src/providers/tavily.ts:26

Tavily Search provider (optional).

Requires TAVILY_API_KEY in the environment. When the key is missing or the request fails, it returns an empty array and logs a warning rather than throwing.

Implements#

SearchProvider

Constructors#

Constructor#

new TavilyProvider(): TavilyProvider

Returns#

TavilyProvider

Properties#

Property	Modifier	Type	Default value	Description	Defined in
`name`	`readonly`	`"tavily"`	`"tavily"`	Stable provider identifier (e.g. “brave”, “tavily”, “duckduckgo”).	src/providers/tavily.ts:27

Methods#

search()#

search(query, opts): Promise<SearchResult[]>

API: providers/types

Mon, 01 Jan 0001 00:00:00 +0000

better-web-search-mcp

better-web-search-mcp / providers/types

providers/types#

Interfaces#

SearchOptions#

Defined in: src/providers/types.ts:27

Options controlling a single provider search.

Properties#

Property	Type	Description	Defined in
`count?`	`number`	Maximum number of results to request.	src/providers/types.ts:29
`extraSnippets?`	`boolean`	Request extra snippets from providers that support it (Brave).	src/providers/types.ts:35
`freshness?`	`string`	Provider-specific freshness filter (e.g. Brave “pd”/“pw”/“pm”/“py”).	src/providers/types.ts:31
`recency_days?`	`number`	Number of days back to restrict results to.	src/providers/types.ts:33

SearchProvider#

Defined in: src/providers/types.ts:39

A search provider that can be queried and returns normalized results.

API: ranking/deduplicate

Mon, 01 Jan 0001 00:00:00 +0000

better-web-search-mcp

better-web-search-mcp / ranking/deduplicate

ranking/deduplicate#

Functions#

deduplicate()#

deduplicate(results): SearchResult[]

Defined in: src/ranking/deduplicate.ts:53

Deduplicate search results by normalized URL, keeping the highest score for each canonical URL. Malformed URLs are skipped, not thrown.

Does not mutate the input array.

Parameters#

Parameter	Type
`results`	readonly `SearchResult`[]

Returns#

SearchResult[]

normalizeUrl()#

normalizeUrl(rawUrl): string | undefined

Defined in: src/ranking/deduplicate.ts:18

Normalize a URL to a canonical form suitable for deduplication.

Lowercases the hostname.
Strips utm_*, gclid, and fbclid query parameters.
Sorts the remaining query parameters for stable ordering.
Removes a trailing slash except for the root path.
Drops the hash/fragment.

Returns undefined for malformed URLs so callers can skip them rather than throw.

API: ranking/domainScore

Mon, 01 Jan 0001 00:00:00 +0000

better-web-search-mcp

better-web-search-mcp / ranking/domainScore

ranking/domainScore#

Functions#

domainScore()#

domainScore(urlOrHost): number

Defined in: src/ranking/domainScore.ts:76

Score a hostname against the curated tiers using suffix matching.

Accepts either a full URL or a bare hostname. Returns a value in [0, 1].

Parameters#

Parameter	Type
`urlOrHost`	`string`

Returns#

number

API: ranking/independence

Mon, 01 Jan 0001 00:00:00 +0000

better-web-search-mcp

better-web-search-mcp / ranking/independence

ranking/independence#

Interfaces#

IndependenceInput#

Defined in: src/ranking/independence.ts:30

A document participating in independence analysis.

Properties#

Property	Type	Description	Defined in
`content`	`string`	The extracted page content.	src/ranking/independence.ts:34
`url`	`string`	The source URL, used to derive the registrable-ish host.	src/ranking/independence.ts:32

IndependenceResult#

Defined in: src/ranking/independence.ts:38

Where a document sits among its near-duplicates.

Properties#

Property	Type	Description	Defined in
`cluster`	`number`	Index of the cluster this document belongs to.	src/ranking/independence.ts:40
`duplicates`	`number`	How many other documents share this cluster.	src/ranking/independence.ts:47
`primary`	`boolean`	Whether this is the representative of its cluster. Exactly one document per cluster is primary; the rest are derivative copies of the same text.	src/ranking/independence.ts:45

Variables#

DUPLICATE_THRESHOLD#

const DUPLICATE_THRESHOLD: 0.4 = 0.4

API: ranking/passages

Mon, 01 Jan 0001 00:00:00 +0000

better-web-search-mcp

better-web-search-mcp / ranking/passages

ranking/passages#

Interfaces#

Passage#

Defined in: src/ranking/passages.ts:20

A scored span of text taken from a page.

Properties#

Property	Type	Description	Defined in
`end`	`number`	Character offset of the passage end within the source content.	src/ranking/passages.ts:26
`score`	`number`	BM25 score against the query. Higher is more relevant.	src/ranking/passages.ts:28
`start`	`number`	Character offset of the passage start within the source content.	src/ranking/passages.ts:24
`text`	`string`	The passage text, trimmed.	src/ranking/passages.ts:22

Variables#

MAX_PASSAGE_CHARS#

const MAX_PASSAGE_CHARS: 1200 = 1_200

API: ranking/rerank

Mon, 01 Jan 0001 00:00:00 +0000

better-web-search-mcp

better-web-search-mcp / ranking/rerank

ranking/rerank#

Interfaces#

RankedResult#

Defined in: src/ranking/rerank.ts:10

A ranked result: the original result plus its computed relevance score.

Extends#

SearchResult

Properties#

Property	Type	Description	Inherited from	Defined in
`published?`	`string`	ISO date string of publication, when the provider exposes it.	`SearchResult`.`published`	src/providers/types.ts:19
`relevance`	`number`	Combined relevance score in [0, 1].	-	src/ranking/rerank.ts:12
`score?`	`number`	A 0-1 relevance/quality score assigned by the provider, when available.	`SearchResult`.`score`	src/providers/types.ts:21
`snippet`	`string`	A short text snippet / description of the result.	`SearchResult`.`snippet`	src/providers/types.ts:17
`source`	`string`	The name of the provider that produced this result.	`SearchResult`.`source`	src/providers/types.ts:23
`title`	`string`	The result title.	`SearchResult`.`title`	src/providers/types.ts:13
`url`	`string`	The canonical URL of the result.	`SearchResult`.`url`	src/providers/types.ts:15

Functions#

rerank()#

rerank(results, query, recencyDays?): RankedResult[]

API: tools

Mon, 01 Jan 0001 00:00:00 +0000

better-web-search-mcp

better-web-search-mcp / tools

tools#

Functions#

cacheFromConfig()#

cacheFromConfig(cfg?): Cache

Defined in: src/tools/index.ts:27

Build the shared cache from configuration.

BETTER_WEB_SEARCH_DISABLE_CACHE and BETTER_WEB_SEARCH_CACHE_PATH were parsed by the config loader and printed in the startup banner, but nothing ever passed them to the Cache: the server opened data/cache.db no matter what either said. Both are honored here.

Parameters#

Parameter	Type
`cfg`	`AppConfig`

Returns#

Cache

registerTools()#

registerTools(server, cache?): void

Defined in: src/tools/index.ts:34

Parameters#

Parameter	Type
`server`	`McpServer`
`cache?`	`Cache`

Returns#

void

API: tools/extract

Mon, 01 Jan 0001 00:00:00 +0000

better-web-search-mcp

better-web-search-mcp / tools/extract

tools/extract#

Interfaces#

ExtractResponse#

Defined in: src/tools/extract.ts:39

The spec-shaped per-URL extraction response.

Properties#

Property	Type	Description	Defined in
`api_endpoints`	`unknown`	-	src/tools/extract.ts:49
`content`	`string`	-	src/tools/extract.ts:42
`extraction`	{ `confidence`: `number`; `method`: `string`; `rendered`: `boolean`; }	-	src/tools/extract.ts:43
`extraction.confidence`	`number`	-	src/tools/extract.ts:45
`extraction.method`	`string`	-	src/tools/extract.ts:44
`extraction.rendered`	`boolean`	-	src/tools/extract.ts:46
`metadata`	`PageMetadata`	-	src/tools/extract.ts:50
`security`	`SecurityReport`	Provenance for the content above. Page text is attacker-controlled, so it is always marked untrusted and screened for text addressing the agent.	src/tools/extract.ts:55
`structured_data`	`unknown`	-	src/tools/extract.ts:48
`title`	`string`	-	src/tools/extract.ts:41
`url`	`string`	-	src/tools/extract.ts:40

Variables#

BROWSER_URL_TIMEOUT_MS#

const BROWSER_URL_TIMEOUT_MS: 35000 = 35_000

API: tools/find

Mon, 01 Jan 0001 00:00:00 +0000

better-web-search-mcp

better-web-search-mcp / tools/find

tools/find#

Interfaces#

FindResponse#

Defined in: src/tools/find.ts:26

The spec-shaped web_find response.

Properties#

Property	Type	Defined in
`answer`	`string`	src/tools/find.ts:27
`queries_used`	`string`[]	src/tools/find.ts:29
`sources`	`SearchSource`[]	src/tools/find.ts:28

Variables#

findInputSchema#

const findInputSchema: { max_results: ZodDefault<ZodNumber>; query: ZodString; site: ZodString; }

Defined in: src/tools/find.ts:19

Input schema for web_find.

Type Declaration#

Name	Type	Defined in
`max_results`	`ZodDefault`<`ZodNumber`>	src/tools/find.ts:22
`query`	`ZodString`	src/tools/find.ts:20
`site`	`ZodString`	src/tools/find.ts:21

Functions#

registerFind()#

registerFind(server, cache?): void

API: tools/news

Mon, 01 Jan 0001 00:00:00 +0000

better-web-search-mcp

better-web-search-mcp / tools/news

tools/news#

Interfaces#

NewsResponse#

Defined in: src/tools/news.ts:27

The spec-shaped web_news response.

Properties#

Property	Type	Defined in
`answer`	`string`	src/tools/news.ts:28
`queries_used`	`string`[]	src/tools/news.ts:31
`sources`	`SearchSource`[]	src/tools/news.ts:29
`timeline`	`Record`<`string`, `SearchSource`[]>	src/tools/news.ts:30

Variables#

newsInputSchema#

const newsInputSchema: { max_results: ZodDefault<ZodNumber>; recency_days: ZodDefault<ZodNumber>; topic: ZodString; }

Defined in: src/tools/news.ts:20

Input schema for web_news.

Type Declaration#

Name	Type	Defined in
`max_results`	`ZodDefault`<`ZodNumber`>	src/tools/news.ts:23
`recency_days`	`ZodDefault`<`ZodNumber`>	src/tools/news.ts:22
`topic`	`ZodString`	src/tools/news.ts:21

Functions#

registerNews()#

registerNews(server, cache?): void

API: tools/research

Mon, 01 Jan 0001 00:00:00 +0000

better-web-search-mcp

better-web-search-mcp / tools/research

tools/research#

Interfaces#

Citation#

Defined in: src/tools/research.ts:50

A span of a source page that supports the answer.

Properties#

Property	Type	Description	Defined in
`citation`	`number`	1-based index into `sources`.	src/tools/research.ts:52
`end`	`number`	Character offset of the quote end.	src/tools/research.ts:62
`quote`	`string`	The supporting text, verbatim from the extracted content.	src/tools/research.ts:58
`relevance`	`number`	Relevance of the span to the question. Higher is stronger.	src/tools/research.ts:64
`start`	`number`	Character offset of the quote within that page’s extracted content.	src/tools/research.ts:60
`title`	`string`	The source title.	src/tools/research.ts:56
`url`	`string`	The source URL the span was taken from.	src/tools/research.ts:54

ResearchResponse#

Defined in: src/tools/research.ts:67

API: tools/search

Mon, 01 Jan 0001 00:00:00 +0000

better-web-search-mcp

better-web-search-mcp / tools/search

tools/search#

Interfaces#

SearchResponse#

Defined in: src/tools/search.ts:35

The spec-shaped web_search response.

Properties#

Property	Type	Defined in
`answer`	`string`	src/tools/search.ts:36
`queries_used`	`string`[]	src/tools/search.ts:38
`sources`	`SearchSource`[]	src/tools/search.ts:37

SearchSource#

Defined in: src/tools/search.ts:26

The spec-shaped source entry returned for each result.

Properties#

Property	Type	Defined in
`published?`	`string`	src/tools/search.ts:30
`relevance`	`number`	src/tools/search.ts:31
`snippet`	`string`	src/tools/search.ts:29
`title`	`string`	src/tools/search.ts:27
`url`	`string`	src/tools/search.ts:28

Variables#

searchInputSchema#

const searchInputSchema: { max_results: ZodDefault<ZodNumber>; query: ZodString; recency_days: ZodOptional<ZodNumber>; }

API: utils/cache

Mon, 01 Jan 0001 00:00:00 +0000

better-web-search-mcp

better-web-search-mcp / utils/cache

utils/cache#

Classes#

Cache#

Defined in: src/utils/cache.ts:164

A cache for search results, pages, API patterns, and domain profiles.

Prefers a SQLite database at data/cache.db (WAL mode, auto-created directory). When better-sqlite3 cannot be loaded, when opening the database fails, or when memory: true is passed, every method transparently operates on an in-memory Map with the same TTL semantics.

Constructors#

Constructor#

new Cache(opts?): Cache

Defined in: src/utils/cache.ts:169

Parameters#

Parameter	Type
`opts`	`CacheOptions`

Returns#

Cache

API: utils/cacheTypes

Mon, 01 Jan 0001 00:00:00 +0000

better-web-search-mcp

better-web-search-mcp / utils/cacheTypes

utils/cacheTypes#

Interfaces#

ApiPattern#

Defined in: src/utils/cacheTypes.ts:25

A discovered API endpoint pattern.

Properties#

Property	Type	Defined in
`content_type`	`string`	src/utils/cacheTypes.ts:30
`discovered_at`	`number`	src/utils/cacheTypes.ts:31
`domain`	`string`	src/utils/cacheTypes.ts:27
`endpoint_pattern`	`string`	src/utils/cacheTypes.ts:28
`id`	`number`	src/utils/cacheTypes.ts:26
`method`	`string`	src/utils/cacheTypes.ts:29

CacheOptions#

Defined in: src/utils/cacheTypes.ts:35

Options controlling the Cache backend.

Properties#

Property	Type	Description	Defined in
`dbPath?`	`string`	Path to the SQLite database file. Defaults to `data/cache.db`.	src/utils/cacheTypes.ts:37
`memory?`	`boolean`	Force the in-memory Map backend (used in CI / tests).	src/utils/cacheTypes.ts:39

PageCacheEntry#

Defined in: src/utils/cacheTypes.ts:16

API: utils/config

Mon, 01 Jan 0001 00:00:00 +0000

better-web-search-mcp

better-web-search-mcp / utils/config

utils/config#

Interfaces#

AppConfig#

Defined in: src/utils/config.ts:31

Properties#

Property	Type	Defined in
`braveApiKey`	`string` \| `undefined`	src/utils/config.ts:32
`browserEnabled`	`boolean`	src/utils/config.ts:37
`cacheEnabled`	`boolean`	src/utils/config.ts:35
`cachePath`	`string`	src/utils/config.ts:36
`serpApiKey`	`string` \| `undefined`	src/utils/config.ts:34
`tavilyApiKey`	`string` \| `undefined`	src/utils/config.ts:33

ProviderStatus#

Defined in: src/utils/config.ts:69

Shape for the provider banner (pure, testable).

Properties#

Property	Type	Defined in
`enabled`	`boolean`	src/utils/config.ts:71
`hint?`	`string`	src/utils/config.ts:73
`keyless`	`boolean`	src/utils/config.ts:72
`name`	`string`	src/utils/config.ts:70

Functions#

formatBanner()#

formatBanner(pkg, cfg): string

API: utils/domainProfile

Mon, 01 Jan 0001 00:00:00 +0000

better-web-search-mcp

better-web-search-mcp / utils/domainProfile

utils/domainProfile#

Interfaces#

DomainProfile#

Defined in: src/utils/domainProfile.ts:20

A per-domain capability profile.

Properties#

Property	Type	Description	Defined in
`api_patterns`	`string`[]	API endpoint patterns discovered for this domain.	src/utils/domainProfile.ts:28
`best_method`	`BestMethod`	The extraction method that worked best on the last visit.	src/utils/domainProfile.ts:30
`framework`	`Framework`	The framework detected from HTML markers.	src/utils/domainProfile.ts:24
`has_json_ld`	`boolean`	Whether the page embeds JSON-LD structured data.	src/utils/domainProfile.ts:26
`requires_js`	`boolean`	Whether the page needs JavaScript to produce content.	src/utils/domainProfile.ts:22

ExtractionResult#

Defined in: src/utils/domainProfile.ts:34

API: utils/memoryCache

Mon, 01 Jan 0001 00:00:00 +0000

better-web-search-mcp

better-web-search-mcp / utils/memoryCache

utils/memoryCache#

Classes#

MemoryCache#

Defined in: src/utils/memoryCache.ts:24

A cache backend backed by a plain Map.

Search entries expire after SEARCH_TTL_MS, page entries after PAGE_TTL_MS; domain profiles and API patterns never expire.

Constructors#

Constructor#

new MemoryCache(): MemoryCache

Returns#

MemoryCache

Methods#

addApiPattern()#

addApiPattern(domain, endpointPattern, method, contentType): void

Defined in: src/utils/memoryCache.ts:76

Record a discovered API endpoint pattern for a domain.

Parameters#

Parameter	Type
`domain`	`string`
`endpointPattern`	`string`
`method`	`string`
`contentType`	`string`

Returns#

void

API: utils/queries

Mon, 01 Jan 0001 00:00:00 +0000

better-web-search-mcp

better-web-search-mcp / utils/queries

utils/queries#

Functions#

expandQueries()#

expandQueries(question): string[]

Defined in: src/utils/queries.ts:52

Expand a question into 4-6 unique search variants.

The original question is always first. Each matching synonym group adds a variant with the term replaced. Variants are deduplicated (case-insensitive) and the list is capped to 6.

Returns [""] for an empty question.

Parameters#

Parameter	Type
`question`	`string`

Returns#

string[]

API: utils/retry

Mon, 01 Jan 0001 00:00:00 +0000

better-web-search-mcp

better-web-search-mcp / utils/retry

utils/retry#

Interfaces#

RetryOptions#

Defined in: src/utils/retry.ts:12

Options controlling withRetry.

Properties#

Property	Type	Description	Defined in
`baseMs?`	`number`	Base delay in ms for the first retry (default 200).	src/utils/retry.ts:16
`maxRetries?`	`number`	Maximum number of retries after the initial attempt (default 2).	src/utils/retry.ts:14
`retryNetworkErrors?`	`boolean`	Also retry thrown errors that carry no HTTP status (network errors). Abort errors are never retried — the caller’s `AbortSignal` is already spent, so a retry would fail instantly without waiting.	src/utils/retry.ts:24
`retryOn?`	`number`[]	HTTP status codes that trigger a retry (default [429, 503]).	src/utils/retry.ts:18

Functions#

parseRetryAfter()#

parseRetryAfter(value): number | undefined

API: utils/ssrf

Mon, 01 Jan 0001 00:00:00 +0000

better-web-search-mcp

better-web-search-mcp / utils/ssrf

utils/ssrf#

Classes#

BlockedUrlError#

Defined in: src/utils/ssrf.ts:21

A URL rejected because it does not point at a public internet host.

Extends#

Error

Constructors#

Constructor#

new BlockedUrlError(url, reason): BlockedUrlError

Defined in: src/utils/ssrf.ts:25

Parameters#

Parameter	Type
`url`	`string`
`reason`	`string`

Returns#

BlockedUrlError

Overrides#

Error.constructor

Properties#

Property	Modifier	Type	Description	Inherited from	Defined in
`cause?`	`public`	`unknown`	-	`Error.cause`	node_modules/typescript/lib/lib.es2022.error.d.ts:26
`message`	`public`	`string`	-	`Error.message`	node_modules/typescript/lib/lib.es5.d.ts:1077
`name`	`public`	`string`	-	`Error.name`	node_modules/typescript/lib/lib.es5.d.ts:1076
`stack?`	`public`	`string`	-	`Error.stack`	node_modules/typescript/lib/lib.es5.d.ts:1078
`url`	`readonly`	`string`	The URL that was rejected.	-	src/utils/ssrf.ts:23
`stackTraceLimit`	`static`	`number`	The `Error.stackTraceLimit` property specifies the number of stack frames collected by a stack trace (whether generated by `new Error().stack` or `Error.captureStackTrace(obj)`). The default value is `10` but may be set to any valid JavaScript number. Changes will affect any stack trace captured after the value has been changed. If set to a non-number value, or set to a negative number, stack traces will not capture any frames.	`Error.stackTraceLimit`	node_modules/@types/node/globals.d.ts:68

Methods#

captureStackTrace()#

static captureStackTrace(targetObject, constructorOpt?): void

Changelog

Mon, 01 Jan 0001 00:00:00 +0000

All notable changes to better-web-search-mcp are documented here. Format follows Keep a Changelog and Semantic Versioning.

0.4.4 - 2026-09-02#

Added#

Second benchmark run with the browser tier enabled, recorded in benchmarks/results/browser.json. It is a negative result and is published as one: enabling Tier 3 leaves the compression unchanged within run-to-run noise (86.2% versus 86.5% overall, 83.9% versus 83.8% median) while research spends 24% longer. Tier 3 earns its place on JavaScript-rendered pages that would otherwise extract nothing, not as a way to return less text

0.4.3 - 2026-09-02#

Fixed#

The intermittent CI abort is finally addressed at the right layer. Earlier attempts blamed the wrong thing: switching to forked processes, then serializing them, then closing handles at process exit each lowered the rate without removing it. The mechanism is that vitest tears down the better-sqlite3 addon’s N-API environment between test files, so every file boundary after the addon has loaded is a chance to hit Assertion failed: (env) != nullptr. npm test now runs the one suite that opens a real database in its own vitest invocation, where that environment is destroyed at process exit instead of at a file boundary

0.4.2 - 2026-09-02#

Fixed#

SQLite handles are closed on process exit. A Database left open at exit is finalized during teardown, and when that happened after the N-API environment was gone the addon aborted the process with Assertion failed: (env) != nullptr. It showed up as intermittent CI failures that neither forked nor serialized test runs fully removed, but the same race can strand a WAL file in a real deployment. Open databases are now tracked and closed synchronously on exit, with the hook installed once rather than per database

0.4.1 - 2026-09-02#

Fixed#

A cited passage could be an entire page. splitPassages emitted an over-long block whole, and plenty of extracted pages contain no blank lines at all, so the whole document became a single passage. web_research then returned more text than reading the pages directly would have — the first benchmark run measured a 267% increase against the baseline it was supposed to beat. Blocks longer than 1,200 characters are now windowed on sentence boundaries, without cutting words and with offsets still addressing the source

Added#

Token-efficiency benchmark under benchmarks/, run with npm run bench. It compares an agent driving the tools itself against a single web_research call over the same pages, and is what caught the bug above. Measured over 12 questions: 820,229 characters down to 110,973, an 86.5% reduction overall and 83.8% median. Payload is counted in characters, which is exact and tokenizer-independent; the harness makes no claim about answer quality, which would need a judge model

0.4.0 - 2026-09-02#

Added#

Source independence detection. URL deduplication only ever caught the same page twice; syndication went straight through. Five outlets running one wire story, a vendor press release quoted verbatim, or an article and its own AMP variant all arrived as distinct URLs and were reported as five sources. Pages are now clustered by content shingles (Jaccard over 5-word shingles, union-find so clustering is transitive), and two pages from the same host are treated as one account
evidence block on web_research responses, reporting only things the server actually measured: sources_opened, independent_sources, derivative_sources, query_term_coverage, and cited_spans. There is no language model in this server, so there is no confidence score — an invented one would be worse than none

Changed#

Citations spread across independent accounts rather than distinct URLs, and derivative sources no longer contribute quotes at all. Previously the pass that topped up remaining citation slots could refill them with the same story under a second byline, presenting one claim as corroborated

0.3.0 - 2026-09-02#

Security#

SSRF protection. web_extract took URLs straight from the calling agent and fetchPage followed them with no validation, so http://localhost:8080/, http://192.168.1.1/, and the cloud metadata endpoint at http://169.254.169.254/ were all fetchable and came back as ordinary page content. Every URL is now checked before the request: non-HTTP schemes are refused, hostnames are resolved, and any private, loopback, link-local, carrier-grade-NAT, multicast or reserved address is rejected. IPv4-mapped IPv6 forms such as ::ffff:127.0.0.1 are judged by their IPv4 rules
Redirects are validated per hop. Fetching now uses manual redirects and re-runs the guard on each Location, so a public host cannot bounce the fetcher onto a private one. Chains are capped at 5 hops
Prompt-injection screening. Page text is attacker-controlled, so every web_extract result now carries a security block marking content untrusted and reporting text that tries to address an agent, with the matched pattern and its offset. Suspicious content gets a warning banner; the page text itself is never rewritten, so extraction stays faithful to the source

Changed#

web_research selects passages that answer the question. It previously took the first 400 characters of each of the top 5 pages and never looked at the question, so answers were largely cookie notices and page intros. Passages are now scored against the question with BM25, and the best ones are returned best-first across distinct sources. This also cuts tokens, since only matching paragraphs are included

Added#

web_research responses include citations: each cited span carries its source, the verbatim quote, character offsets into that page’s extracted content, and a relevance score, so a claim can be attributed to a span rather than to a whole page
fetchPage accepts allowPrivateHosts for local fixture servers and ssrf for injecting DNS resolution in tests

0.2.3 - 2026-09-02#

Fixed#

BETTER_WEB_SEARCH_DISABLE_CACHE and BETTER_WEB_SEARCH_CACHE_PATH are finally honored. Both were parsed by the config loader, printed in the startup banner, and documented in --help, but nothing passed them to the cache — the server opened data/cache.db regardless of what either said

Changed#

Test files now run serially in a single forked process. Forks alone were not enough: parallel files still raced the better-sqlite3 addon’s own teardown, tripping Assertion failed: (env) != nullptr inside the native module
Security policy now tracks the 0.2.x line and describes the cache mitigations accurately
The v0.1.0 launch checklist is marked historical rather than reading as a current process

0.2.2 - 2026-09-02#

Fixed#

A locked, corrupt, or unwritable data/cache.db no longer takes the server down at boot. Opening SQLite is guarded and falls back to the in-memory cache with a warning on stderr; directory creation is inside the guard too, so a read-only or permission-denied cache path degrades instead of throwing
The startup banner no longer advertises SerpApi, which has not been part of the provider set since 0.2.0

Added#

mcp.json now tracks the package version automatically via the version lifecycle hook, with a test asserting the two stay in step — the manifest had silently read 0.1.0 while npm was publishing 0.2.1

Changed#

release:* scripts no longer run npm publish locally. Publishing is the tag-triggered workflow’s job; doing both meant the local publish won and CI then failed with E403 cannot publish over the previously published versions, which is how 0.2.1 shipped without provenance and without a GitHub release

0.2.1 - 2026-09-02#

Changed#

Test runner uses forked processes, reducing intermittent Worker exited unexpectedly CI failures caused by loading the better-sqlite3 native addon inside a worker thread (see 0.2.3 for the follow-up that closed the remaining window)
Compiled tests are excluded from the published package: 252 files down to 148, 126.4 kB down to 85.1 kB

0.2.0 - 2026-09-01#

Fixed#

The Level 3 browser tier never ran in production. No call site passed a browserPool to getPage, so the escalation guard skipped it entirely and BrowserPool was constructed only in tests. The router now creates a shared pool lazily on first escalation, so JS-rendered pages actually get rendered
web_extract’s flat 8s per-URL budget was shorter than a browser render’s 23s worst case (15s navigation + 4s intelligent wait + 4s DOM stability), so every Level 3 extraction was killed before it could return. The budget is now sized to the tiers that can run: 8s without the browser, 35s with it
BETTER_WEB_SEARCH_DISABLE_BROWSER was documented but never read; it now actually disables the browser tier
Aborted requests are no longer retried. Providers share one AbortController across attempts, so a retry after the timeout fired could only fail again instantly, burning the retry budget
Brave and Tavily now set retryNetworkErrors, matching DuckDuckGo

Added#

Domain-profile shortcut: the per-domain profile written after every extraction is finally read back, so a domain already served without JavaScript skips the render. mode: "browser" still overrides it
Per-provider 10s timeout inside aggregateSearch — one hung provider no longer gates the other providers’ results
Clean shutdown on SIGINT/SIGTERM so a lazily launched chromium is not orphaned
16 tests (229 total), including a browser-tier suite covering pool reuse, the disable flag, and the profile shortcut

Removed#

The unimplemented SerpApi stub is no longer added to enabledProviders(); it was a guaranteed no-op call on every search. The class remains as a placeholder

Notes#

web_extract on JavaScript-heavy pages is slower than 0.1.2 (up to 35s) but now returns real content instead of a confidence-0 fallback

0.1.2 - 2026-08-31#

Added#

Project banner in the README and on the site homepage
Hugo docs site on the hugo-book theme with native sidebar navigation

Changed#

Node baseline bumped to 22 LTS
Root directory cleanup and a rewritten end-user README

Fixed#

Release workflow npm authentication
Sidebar navigation, duplicate H1 headings, and mobile menu on the docs site

0.1.1 - 2026-08-29#

Fixed#

Packaging and release-workflow fixes on top of the initial release

0.1.0 - 2026-08-29#

Added#

Initial public release — keyless-first MCP server for web research
Multi-provider search abstraction: Brave (primary), Tavily (optional), DuckDuckGo (keyless fallback), SerpApi stub — aggregated via Promise.allSettled, deduplicated and reranked
Query expansion: 4–6 parallel variants incl. German translations (e.g. unlimited mobile internet Germany → unbegrenztes Datenvolumen)
Ranking pipeline: normalizeUrl (utm/gclid/fbclid strip, trailing slash, hash), domainScore curated tiers, rerank (0.5 overlap +0.2 domain +0.2 recency +0.1 score)
3-tier extraction: Level 1 fetchPage (10s, 2MB cap, UA) → Level 2 JSON-LD / __NEXT_DATA__ / self.__next_f / __NUXT__ / __APOLLO_STATE__ → Level 3 Playwright pool (3, image/font/media block, intelligent Race networkidle/article/innerText>1000, page.on('response') JSON capture)
Content Fusion with confidence scores (api 0.99 > jsonld 0.95 > rendered 0.90 > readability 0.85)
Alternative source discovery: AMP variants ?output=1 /amp + quoted-title site: searches, snippet evidence search_snippet 0.6
SQLite cache (better-sqlite3 WAL, data/cache.db) + in-memory fallback — search_cache 15min, page_cache 1h, api_patterns, domain_profiles — domainProfile self-learning per domain
6 MCP tools: web_search, web_research/deep_search (parallel queries, bounded concurrency 3, 8s per page, extractive citations), web_extract (auto/fast/browser), web_find, web_news (recency + timeline + diversity)
Central config loader: BRAVE_API_KEY + BETTER_WEB_SEARCH_BRAVE_API_KEY aliases, cache/browser toggles, keyless hint banner to stderr
CLI: --help / --version (exit before MCP handshake), startup banner listing providers
OpenCode / Claude / Cursor / VS Code configs: mcp.json, .vscode/mcp.json, smithery.yaml
195 tests (vitest), tsc strict, npm run build emits dist/

Notes#

No API keys required for first test — npx better-web-search-mcp works via DuckDuckGo
Add BRAVE_API_KEY for richer ranking & recency filtering
Publish-ready: npm pack --dry-run validated, prepare/prepublishOnly hooks, files whitelist

Contributing

Mon, 01 Jan 0001 00:00:00 +0000

Thank you for considering a contribution! Please see the full Contributing Guide on GitHub for detailed instructions.

Quick start#

git clone https://github.com/PhantomPixelDev/BetterWebSearch-MCP.git
cd BetterWebSearch-MCP
npm ci
npm run build
npm test # 189+ tests — fully keyless
npm run lint # tsc --noEmit — must be zero errors

Key points#

Keyless by default — no API keys needed to develop or test
Node 20+ required
TypeScript strict — no any, no @ts-ignore
Vitest for tests — deterministic mocks, no external keys
Conventional Commits — feat(extraction): ..., fix(providers): ..., docs: ...

PR checklist#

npm run lint — zero errors
npm test — all 189+ tests pass
npm run build — dist/ compiles cleanly
New functionality has tests
Types are explicit — no any or @ts-ignore
CHANGELOG.md updated
npm pack --dry-run — tarball looks correct
No secrets, API keys, or .env values committed

Security#

If you discover a security vulnerability, do not open a public issue. Follow the process in SECURITY.md.

BetterWebSearch MCP

Quickstart

Option 1: npx (recommended)#

Option 2: From source (30s)#

Installation

Requirements#

Install via npx (no install)#

Install globally#

Install from source#

Native binding note#

Tools Reference

Overview#

web_search#

Configuration

Environment Variables#

Key aliases#

Providers

Provider overview#

Keyless-first design#

Extraction

Overview#

Tier 1: HTTP Fetch#

Architecture

System overview#

Components#

1. MCP Server (src/index.ts)#

Token Efficiency

Method#

API: extraction/alternative

extraction/alternative#

Interfaces#

AlternativeDeps#

Properties#

Variables#

MAX_SNIPPET_CHARS#

API: extraction/apiIntercept

extraction/apiIntercept#

Interfaces#

ApiPatternRow#

Properties#

ApiResponse#

Methods#

headers()#

Returns#

json()#

Returns#

request()#

API: extraction/browser

extraction/browser#

Classes#

BrowserPool#

Constructors#

Constructor#

Parameters#

Returns#

API: extraction/evidence

extraction/evidence#

Interfaces#

Evidence#

Properties#

FusedContent#

API: extraction/fetch

extraction/fetch#

Classes#

FetchTimeoutError#

Extends#

Constructors#

Constructor#

Parameters#

Returns#

Overrides#

Properties#

Methods#

captureStackTrace()#

API: extraction/fusion

extraction/fusion#

Functions#

fuseContent()#

Parameters#

Returns#

`web_search`#

1. MCP Server (`src/index.ts`)#