Skip to main content
POST
Scrape web page

POST /api/v1/platform/scrapers/web/scrape

Scrape a single web page and return its content.
Scrape a single web page and return readable content. By default the endpoint returns the backward-compatible url, text, and statusCode fields. Use formats when you need markdown, HTML, discovered links, page metadata, or schema-guided JSON extraction.

Example

Formats

formats is optional and defaults to ["text"]. Requested optional formats appear as additional fields under data. If a requested format is unavailable, the field may be null or an empty array.

Schema-guided JSON

Request json only when you provide jsonSchema. The schema defines the returned keys and shape; page content is mapped into that schema and validated before the response is returned. The root schema must be an object; use array properties for repeated data such as comments, products, or plans.
jsonSchema and instructions are only valid when formats includes json. Schema-guided extraction performs additional model work and may use more credits than a plain text or markdown scrape.

Authorizations

X-Access-Key
string
header
required

The public API key from the Credentials page.

X-Secret-Key
string
header
required

The API secret shown when the credential is created.

Body

application/json
url
string
required

URL to scrape

Example:

"https://example.com/page"

renderJs
boolean
default:false

Render JavaScript before scraping

formats
enum<string>[]

Output formats to return. Defaults to ["text"]. Request "json" only with jsonSchema.

Available options:
text,
markdown,
html,
links,
metadata,
json
jsonSchema
object

JSON Schema used for schema-guided extraction when formats includes "json".

instructions
string

Additional extraction instructions used only when formats includes "json".

Maximum string length: 8000
maxAgeMs
number
default:0

Maximum accepted cache age in milliseconds. Use 0 or omit to force a fresh scrape.

Required range: 0 <= x <= 2592000000
maxAgeDays
number
default:0

Maximum accepted cache age in days. Use 0 or omit to force a fresh scrape. Ignored when maxAgeMs is provided.

Required range: 0 <= x <= 30

Response

Scraped web page content.

data
object
required
meta
object
required