API: ranking/deduplicate

better-web-search-mcp

better-web-search-mcp / ranking/deduplicate

ranking/deduplicate#

Functions#

deduplicate()#

deduplicate(results): SearchResult[]

Defined in: src/ranking/deduplicate.ts:53

Deduplicate search results by normalized URL, keeping the highest score for each canonical URL. Malformed URLs are skipped, not thrown.

Does not mutate the input array.

Parameters#

Returns#

SearchResult[]

normalizeUrl()#

normalizeUrl(rawUrl): string | undefined

Defined in: src/ranking/deduplicate.ts:18

Normalize a URL to a canonical form suitable for deduplication.

• Lowercases the hostname.
• Strips utm_*, gclid, and fbclid query parameters.
• Sorts the remaining query parameters for stable ordering.
• Removes a trailing slash except for the root path.
• Drops the hash/fragment.

Returns undefined for malformed URLs so callers can skip them rather than throw.

Parameters#

Returns#

string | undefined