Skip to main content
POST
/
api
/
v1
/
platform
/
scrapers
/
web
/
scrape
Scrape web page
curl --request POST \
  --url https://developer.thehog.ai/api/v1/platform/scrapers/web/scrape \
  --header 'Content-Type: application/json' \
  --header 'X-Access-Key: <api-key>' \
  --header 'X-Secret-Key: <api-key>' \
  --data '
{
  "url": "https://example.com/page",
  "renderJs": false,
  "maxAgeMs": 0,
  "maxAgeDays": 0
}
'
{
  "data": {
    "url": "<string>",
    "text": "<string>",
    "statusCode": 123
  },
  "meta": {
    "requestId": "<string>"
  }
}

​
POST /api/v1/platform/scrapers/web/scrape

Scrape a single web page and return its content.
Scrape a single web page and return its content as markdown or HTML.

​
Example

curl -X POST https://developer.thehog.ai/api/v1/platform/scrapers/web/scrape \
  -H "X-Access-Key: ak_xxxxxxxxxxxxxxxx" \
  -H "X-Secret-Key: sk_xxxxxxxxxxxxxxxx" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com/pricing", "renderJs": true}'

Authorizations

​
X-Access-Key
string
header
required

The public API key from the Credentials page.

​
X-Secret-Key
string
header
required

The API secret shown when the credential is created.

Body

application/json
​
url
string
required

URL to scrape

Example:

"https://example.com/page"

​
renderJs
boolean
default:false

Render JavaScript before scraping

​
maxAgeMs
number
default:0

Maximum accepted cache age in milliseconds. Use 0 or omit to force a fresh scrape.

Required range: 0 <= x <= 2592000000
​
maxAgeDays
number
default:0

Maximum accepted cache age in days. Use 0 or omit to force a fresh scrape. Ignored when maxAgeMs is provided.

Required range: 0 <= x <= 30

Response

Scraped web page content.

​
data
object
required
​
meta
object
required