Scout · Endpoints

Extract & inspect

Pull structured fields from the DOM, run an inspect pack, list downloadable files, or read PDF text (no OCR).

Extract

POST/v1/extractAPI key

Provide fields[] and/or template_id. Optional dismiss_cookie_banner for known CMPs. 3 credits.

Name	Type	Required	Description
`url`	`string`	required	Page to open.
`fields`	`array`	optional	Selector fields (max 50). Required unless template_id is set.
`template_id`	`string`	optional	Saved template id or slug.
`dismiss_cookie_banner`	`boolean`	optional	Best-effort known CMP accept (allowlist). No domain verify.
`wait_for / settle_ms`	`string / number`	optional	Legacy wait helpers; prefer wait_for_selector browse option.

Field type values:

Name	Type	Description
`exists`	`boolean`	Whether selector matches at least one node.
`text`	`string\|null`	Trimmed textContent of first match.
`html`	`string\|null`	innerHTML of first match.
`attribute`	`string\|null`	Requires attribute name on the field.
`count`	`number`	Number of matches.
`list`	`string[]`	Trimmed text of each match.

Read fields from a live page

Open the URL in a headless browser, optionally dismiss a known cookie banner, then return named DOM fields.

RequestBody

json

{
  "url": "https://shop.example/product/123",
  "dismiss_cookie_banner": true,
  "wait_for_selector": "[data-price]",
  "fields": [
    { "name": "price", "selector": "[data-price]", "type": "text" },
    { "name": "title", "selector": "h1", "type": "text" }
  ]
}

Response200 OK

json

{
  "status": "success",
  "data": {
    "price": "29,90 €",
    "title": "Example Product"
  },
  "url": "https://shop.example/product/123",
  "captcha_detected": false,
  "captcha_signals": [],
  "cookie_banner_detected": true,
  "cookie_banner_dismissed": true,
  "cookie_banner_vendor": "onetrust",
  "timestamp": "2026-07-21T15:00:00.000Z",
  "meta": {
    "response_time_ms": 1240,
    "final_url": "https://shop.example/product/123"
  }
}

Prefer browse options like wait_for_selector and device. Use dismiss_cookie_banner on third-party shops — it does not require domain verify.

curl

curl -X POST https://scout.macrocontent.dev/v1/extract \
  -H "Authorization: Bearer sk_scout_…" \
  -H "Content-Type: application/json" \
  -d @extract.json

Inspect

POST/v1/inspectAPI key

Composable page pack (meta, headings, links, …). 3 credits.

Name	Type	Required	Description
`url`	`string`	required	Page to inspect.
`include`	`string[]`	optional	meta, headings, links, images, json_ld, headers, security, tech, files, a11y.
`dismiss_cookie_banner`	`boolean`	optional	Same browse option as extract.

Inspect page structure

Collect raw SEO/structure facts for the URL — no scores, just measurements.

RequestBody

json

{
  "url": "https://example.com",
  "include": ["meta", "headings", "links", "json_ld"]
}

Response200 OK (truncated)

json

{
  "status": "success",
  "url": "https://example.com",
  "final_url": "https://example.com/",
  "http_status": 200,
  "data": {
    "meta": {
      "title": "Example Domain",
      "description": null,
      "canonical": null,
      "og_title": "Example Domain"
    },
    "headings": [{ "level": 1, "text": "Example Domain" }],
    "links": [{ "href": "https://www.iana.org/domains/example", "text": "More information…" }],
    "json_ld": []
  },
  "captcha_detected": false,
  "response_time_ms": 680,
  "timestamp": "…"
}

Files

POST/v1/filesAPI key

Discover downloadable links on a page. 3 credits.

List downloadable files

Scan anchor tags for likely file downloads (PDF, ZIP, …).

RequestBody

json

{
  "url": "https://example.com/downloads"
}

Response200 OK (truncated)

json

{
  "status": "success",
  "url": "https://example.com/downloads",
  "final_url": "https://example.com/downloads",
  "data": {
    "files": [
      { "href": "https://example.com/a.pdf", "text": "Brochure", "kind": "pdf" }
    ]
  },
  "response_time_ms": 540,
  "timestamp": "…"
}

PDF extract

POST/v1/pdf/extractAPI key

Text extraction from a PDF URL — no OCR. 4 credits.

Extract PDF text

Download the PDF and return extracted text (no OCR for scanned images).

RequestBody

json

{
  "url": "https://example.com/file.pdf"
}

Response200 OK

json

{
  "status": "success",
  "url": "https://example.com/file.pdf",
  "text": "Page 1 text excerpt…",
  "pages": 3,
  "response_time_ms": 420,
  "timestamp": "…"
}