Scout · Endpoints

Extract & inspect

Pull structured fields from the DOM, run an inspect pack, list downloadable files, or read PDF text (no OCR).

Extract

POST/v1/extractAPI key

Provide fields[] and/or template_id. Optional dismiss_cookie_banner, capture_xhr, fetch_mode. 3 credits (2 with fetch_mode=http).

Name	Type	Required	Description
`url`	`string`	required	Page to open.
`fields`	`array`	optional	Selector fields (max 50). Required unless template_id is set.
`template_id`	`string`	optional	Saved template id or slug.
`dismiss_cookie_banner`	`boolean`	optional	Best-effort known CMP accept (allowlist). No domain verify.
`capture_xhr`	`string`	optional	Glob/substring for XHR/fetch URLs to capture. +1 credit.
`fetch_mode`	`browser \| http`	optional	http disables JS (static document). 2 credits instead of 3.
`solve_captcha`	`boolean`	optional	Owned sites only — cloud requires domain verify. +2 credits.
`wait_for / settle_ms`	`string / number`	optional	Legacy wait helpers; prefer wait_for_selector browse option.

Field type values:

Name	Type	Description
`exists`	`boolean`	Whether selector matches at least one node.
`text`	`string\|null`	Trimmed textContent of first match.
`html`	`string\|null`	innerHTML of first match.
`attribute`	`string\|null`	Requires attribute name on the field.
`count`	`number`	Number of matches.
`list`	`string[]`	Trimmed text of each match.
`similar`	`string[]`	Text of sibling-like nodes sharing tag + most classes with the first match.
`selector`	`string\|null`	Generated CSS path for the first match.
`adaptive`	`boolean`	Field flag: relocate by class/id/text when the selector misses. Returns selector_relocations[].
`match_text`	`string`	Field flag: hint text used with adaptive relocation.

Read fields from a live page

Open the URL in a headless browser, optionally dismiss a known cookie banner, then return named DOM fields.

RequestBody

{
  "url": "https://shop.example/product/123",
  "dismiss_cookie_banner": true,
  "wait_for_selector": "[data-price]",
  "capture_xhr": "*/api/pricing*",
  "fields": [
    { "name": "price", "selector": "[data-price]", "type": "text", "adaptive": true, "match_text": "€" },
    { "name": "title", "selector": "h1", "type": "text" },
    { "name": "cards", "selector": "article.card", "type": "similar" },
    { "name": "cta_path", "selector": "a.buy", "type": "selector" }
  ]
}

Response200 OK

{
  "status": "success",
  "data": {
    "price": "29,90 €",
    "title": "Example Product",
    "cards": ["Starter", "Pro"],
    "cta_path": "section.hero > a.buy"
  },
  "url": "https://shop.example/product/123",
  "captcha_detected": false,
  "captcha_signals": [],
  "cookie_banner_detected": true,
  "cookie_banner_dismissed": true,
  "cookie_banner_vendor": "onetrust",
  "captured_xhr": [
    { "url": "https://shop.example/api/pricing", "method": "GET", "status": 200, "resource_type": "fetch", "content_type": "application/json", "body": { "amount": 2990 }, "truncated": false }
  ],
  "selector_relocations": [],
  "fetch_mode": "browser",
  "timestamp": "2026-08-17T15:00:00.000Z",
  "meta": {
    "response_time_ms": 1240,
    "final_url": "https://shop.example/product/123"
  }
}

Prefer browse options like wait_for_selector and device. Use dismiss_cookie_banner on third-party shops — it does not require domain verify. Set adaptive: true (and optional match_text) so a renamed class still resolves. type: similar collects sibling-like cards; type: selector returns a generated CSS path. capture_xhr records matching XHR/fetch bodies. fetch_mode: "http" loads without JavaScript.

curlbash

curl -X POST https://api.scout.macrocontent.dev/v1/extract \
  -H "Authorization: Bearer sk_scout_…" \
  -H "Content-Type: application/json" \
  -d @extract.json

Inspect

POST/v1/inspectAPI key

Composable page pack (meta, headings, links, …). 3 credits.

Name	Type	Required	Description
`url`	`string`	required	Page to inspect.
`include`	`string[]`	optional	meta, headings, links, images, json_ld, headers, security, tech, files, a11y.
`dismiss_cookie_banner`	`boolean`	optional	Same browse option as extract.
`capture_xhr / fetch_mode / solve_captcha`	`…`	optional	Same browse options as extract.

Inspect page structure

Collect raw SEO/structure facts for the URL — no scores, just measurements.

RequestBody

{
  "url": "https://example.com",
  "include": ["meta", "headings", "links", "json_ld"]
}

Response200 OK (truncated)

{
  "status": "success",
  "url": "https://example.com",
  "final_url": "https://example.com/",
  "http_status": 200,
  "data": {
    "meta": {
      "title": "Example Domain",
      "description": null,
      "canonical": null,
      "og_title": "Example Domain"
    },
    "headings": [{ "level": 1, "text": "Example Domain" }],
    "links": [{ "href": "https://www.iana.org/domains/example", "text": "More information…" }],
    "json_ld": []
  },
  "captcha_detected": false,
  "response_time_ms": 680,
  "timestamp": "…"
}

Files

POST/v1/filesAPI key

Discover downloadable links on a page. 3 credits.

List downloadable files

Scan anchor tags for likely file downloads (PDF, ZIP, …).

RequestBody

{
  "url": "https://example.com/downloads"
}

Response200 OK (truncated)

{
  "status": "success",
  "url": "https://example.com/downloads",
  "final_url": "https://example.com/downloads",
  "data": {
    "files": [
      { "href": "https://example.com/a.pdf", "text": "Brochure", "kind": "pdf" }
    ]
  },
  "response_time_ms": 540,
  "timestamp": "…"
}

PDF extract

POST/v1/pdf/extractAPI key

Text extraction from a PDF URL — no OCR. 4 credits.

Extract PDF text

Download the PDF and return extracted text (no OCR for scanned images).

RequestBody

{
  "url": "https://example.com/file.pdf"
}

Response200 OK

{
  "status": "success",
  "url": "https://example.com/file.pdf",
  "text": "Page 1 text excerpt…",
  "pages": 3,
  "response_time_ms": 420,
  "timestamp": "…"
}