API: extraction/fetch

better-web-search-mcp

better-web-search-mcp / extraction/fetch

extraction/fetch#

Classes#

FetchTimeoutError#

Defined in: src/extraction/fetch.ts:55

A fetch that was aborted by the timeout.

Extends#

• Error

Constructors#

Constructor#

new FetchTimeoutError(url, timeoutMs): FetchTimeoutError

Defined in: src/extraction/fetch.ts:56

Parameters#

Returns#

FetchTimeoutError

Overrides#

Error.constructor

Properties#

Methods#

captureStackTrace()#

static captureStackTrace(targetObject, constructorOpt?): void

Defined in: node_modules/@types/node/globals.d.ts:52

Creates a .stack property on targetObject, which when accessed returns a string representing the location in the code at which Error.captureStackTrace() was called.

const myObject = {};
Error.captureStackTrace(myObject);
myObject.stack;  // Similar to `new Error().stack`

The first line of the trace will be prefixed with ${myObject.name}: ${myObject.message}.

The optional constructorOpt argument accepts a function. If given, all frames above constructorOpt, including constructorOpt, will be omitted from the generated stack trace.

The constructorOpt argument is useful for hiding implementation details of error generation from the user. For instance:

function a() {
  b();
}

function b() {
  c();
}

function c() {
  // Create an error without stack trace to avoid calculating the stack trace twice.
  const { stackTraceLimit } = Error;
  Error.stackTraceLimit = 0;
  const error = new Error();
  Error.stackTraceLimit = stackTraceLimit;

  // Capture the stack trace above function b
  Error.captureStackTrace(error, b); // Neither function c, nor b is included in the stack trace
  throw error;
}

a();

Parameters#

Returns#

void

Inherited from#

Error.captureStackTrace

prepareStackTrace()#

static prepareStackTrace(err, stackTraces): any

Defined in: node_modules/@types/node/globals.d.ts:56

Parameters#

Returns#

any

See#

https://v8.dev/docs/stack-trace-api#customizing-stack-traces

Inherited from#

Error.prepareStackTrace

NonHtmlError#

Defined in: src/extraction/fetch.ts:71

A fetch that was rejected because the response is not HTML.

Extends#

• Error

Constructors#

Constructor#

new NonHtmlError(url, contentType): NonHtmlError

Defined in: src/extraction/fetch.ts:72

Parameters#

Returns#

NonHtmlError

Overrides#

Error.constructor

Properties#

Methods#

captureStackTrace()#

static captureStackTrace(targetObject, constructorOpt?): void

Defined in: node_modules/@types/node/globals.d.ts:52

Creates a .stack property on targetObject, which when accessed returns a string representing the location in the code at which Error.captureStackTrace() was called.

const myObject = {};
Error.captureStackTrace(myObject);
myObject.stack;  // Similar to `new Error().stack`

The first line of the trace will be prefixed with ${myObject.name}: ${myObject.message}.

The optional constructorOpt argument accepts a function. If given, all frames above constructorOpt, including constructorOpt, will be omitted from the generated stack trace.

The constructorOpt argument is useful for hiding implementation details of error generation from the user. For instance:

function a() {
  b();
}

function b() {
  c();
}

function c() {
  // Create an error without stack trace to avoid calculating the stack trace twice.
  const { stackTraceLimit } = Error;
  Error.stackTraceLimit = 0;
  const error = new Error();
  Error.stackTraceLimit = stackTraceLimit;

  // Capture the stack trace above function b
  Error.captureStackTrace(error, b); // Neither function c, nor b is included in the stack trace
  throw error;
}

a();

Parameters#

Returns#

void

Inherited from#

Error.captureStackTrace

prepareStackTrace()#

static prepareStackTrace(err, stackTraces): any

Defined in: node_modules/@types/node/globals.d.ts:56

Parameters#

Returns#

any

See#

https://v8.dev/docs/stack-trace-api#customizing-stack-traces

Inherited from#

Error.prepareStackTrace

TooManyRedirectsError#

Defined in: src/extraction/fetch.ts:63

A redirect chain that exceeded MAX_REDIRECTS.

Extends#

• Error

Constructors#

Constructor#

new TooManyRedirectsError(url): TooManyRedirectsError

Defined in: src/extraction/fetch.ts:64

Parameters#

Returns#

TooManyRedirectsError

Overrides#

Error.constructor

Properties#

Methods#

captureStackTrace()#

static captureStackTrace(targetObject, constructorOpt?): void

Defined in: node_modules/@types/node/globals.d.ts:52

Creates a .stack property on targetObject, which when accessed returns a string representing the location in the code at which Error.captureStackTrace() was called.

const myObject = {};
Error.captureStackTrace(myObject);
myObject.stack;  // Similar to `new Error().stack`

The first line of the trace will be prefixed with ${myObject.name}: ${myObject.message}.

The optional constructorOpt argument accepts a function. If given, all frames above constructorOpt, including constructorOpt, will be omitted from the generated stack trace.

The constructorOpt argument is useful for hiding implementation details of error generation from the user. For instance:

function a() {
  b();
}

function b() {
  c();
}

function c() {
  // Create an error without stack trace to avoid calculating the stack trace twice.
  const { stackTraceLimit } = Error;
  Error.stackTraceLimit = 0;
  const error = new Error();
  Error.stackTraceLimit = stackTraceLimit;

  // Capture the stack trace above function b
  Error.captureStackTrace(error, b); // Neither function c, nor b is included in the stack trace
  throw error;
}

a();

Parameters#

Returns#

void

Inherited from#

Error.captureStackTrace

prepareStackTrace()#

static prepareStackTrace(err, stackTraces): any

Defined in: node_modules/@types/node/globals.d.ts:56

Parameters#

Returns#

any

See#

https://v8.dev/docs/stack-trace-api#customizing-stack-traces

Inherited from#

Error.prepareStackTrace

Interfaces#

FetchedPage#

Defined in: src/extraction/fetch.ts:43

The result of a successful page fetch.

Properties#

FetchPageOptions#

Defined in: src/extraction/fetch.ts:28

Options controlling a single page fetch.

Properties#

Variables#

DEFAULT_TIMEOUT_MS#

const DEFAULT_TIMEOUT_MS: 10000 = 10_000

Defined in: src/extraction/fetch.ts:16

Default timeout for a single page fetch, in milliseconds.

MAX_BODY_BYTES#

const MAX_BODY_BYTES: number

Defined in: src/extraction/fetch.ts:19

Maximum number of bytes of body we are willing to read.

MAX_REDIRECTS#

const MAX_REDIRECTS: 5 = 5

Defined in: src/extraction/fetch.ts:25

Maximum number of redirects to follow before giving up.

USER_AGENT#

const USER_AGENT: "BetterWebSearch-MCP/1.0" = "BetterWebSearch-MCP/1.0"

Defined in: src/extraction/fetch.ts:22

User agent advertised to servers.

Functions#

fetchPage()#

fetchPage(url, opts?): Promise<FetchedPage>

Defined in: src/extraction/fetch.ts:90

Fetch a page as HTML.

• Uses native fetch with an AbortController timeout.
• Sends the BetterWebSearch-MCP/1.0 user agent and follows redirects.
• Guards on content-type: only text/html is accepted; a missing content-type is allowed through.
• Caps the body at MAX_BODY_BYTES by reading an arrayBuffer slice.

Throws FetchTimeoutError on timeout and NonHtmlError when the content-type is not HTML. Other network errors propagate as-is.

Parameters#

Returns#

Promise<FetchedPage>