Scout · Endpoints

Jobs & webhooks

Enqueue many independent Scout operations, poll until done, and optionally receive a signed completion webhook.

Journey vs jobs

Journey = one flow, shared browser state (login → click → extract). Jobs = many independent requests (often many URLs). See the comparison on Journey.

Create a job

POST/v1/jobsAPI key

Name	Type	Required	Description
`requests`	`array`	required	List of { type, body } operations to run sequentially.
`requests[].type`	`string`	required	screenshot \| extract \| visibility \| inspect \| performance \| journey \| crawl.
`requests[].body`	`object`	required	Same body shape as the matching POST /v1/… endpoint.
`requests[].stop_if`	`object`	optional	Optional early exit. Match step result fields (visible, exists, overall_pass, status). On match: skip remaining steps; job completed with stopped_early: true.
`webhook_url`	`string`	optional	HTTPS URL to notify when the job finishes.
`webhook_secret`	`string`	optional	HMAC secret for this job (overrides key/env secret).

bash

curl -X POST https://api.scout.macrocontent.dev/v1/jobs \
  -H "Authorization: Bearer sk_scout_…" \
  -H "Content-Type: application/json" \
  -d @job.json

json

{
  "webhook_url": "https://hooks.example.com/scout",
  "webhook_secret": "whsec_…",
  "requests": [
    { "type": "screenshot", "body": { "url": "https://example.com", "format": "jpeg" } },
    { "type": "inspect", "body": { "url": "https://example.com", "include": ["meta"] } },
    {
      "type": "journey",
      "body": {
        "debug": { "on_error_only": true },
        "steps": [
          { "type": "navigate", "url": "https://example.com" },
          { "type": "extract", "fields": [{ "name": "h1", "selector": "h1", "type": "text" }] }
        ]
      }
    }
  ]
}

Early stop (`stop_if`)

Each request may include stop_if. After that step finishes, Scout compares the step result to the given fields. On a match, remaining steps are skipped. The job still ends as completed (not failed) with stopped_early: true, stopped_at (step index), and stop_reason.

Typical pattern: poll a restock selector with your own cron. If “Add to cart” is missing, stop before price extract and screenshot. When it appears on a later run, the full job finishes and your webhook fires.

json

{
  "webhook_url": "https://hooks.shop.example/restock",
  "requests": [
    {
      "type": "visibility",
      "body": { "url": "https://shop.example/p/trail-runner", "selector": "button.add-to-cart" },
      "stop_if": { "visible": false }
    },
    {
      "type": "extract",
      "body": {
        "url": "https://shop.example/p/trail-runner",
        "fields": [{ "name": "price", "selector": ".price", "type": "text" }]
      }
    },
    {
      "type": "screenshot",
      "body": { "url": "https://shop.example/p/trail-runner", "format": "jpeg", "mode": "viewport" }
    }
  ]
}

Poll

GET/v1/jobs/:idAPI key

Poll status/results until expires_at (default TTL 1800s). 0 credits.

bash

curl https://api.scout.macrocontent.dev/v1/jobs/job_… \
  -H "Authorization: Bearer sk_scout_…"

Statuses: queued → running → completed | failed. Results live until expires_at (default ~1800s) — Scout is not a long-term archive. Large jobs (many screenshots / debug journeys) can produce multi‑MB JSON; keep batches modest or omit journey debug in production.

Journey inside a job

Nested journey bodies support the same fields as POST /v1/journey, including debug. Prefer debug only while developing — batch responses get large.

Webhook HMAC

When a secret is available, Scout POSTs the job result and signs it with:

X-Scout-Timestamp — unix seconds
X-Scout-Signature — sha256=<hex> of HMAC-SHA256(secret, "${timestamp}.${rawBody}")

Secret resolution order: job webhook_secret → key webhook secret → env SCOUT_WEBHOOK_SECRET.

javascript

const crypto = require('crypto');
function verify(secret, timestamp, signatureHeader, rawBody, maxAge = 300) {
  if (Math.abs(Date.now() / 1000 - Number(timestamp)) > maxAge) return false;
  const expected = 'sha256=' + crypto.createHmac('sha256', secret)
    .update(`${timestamp}.${rawBody}`).digest('hex');
  return crypto.timingSafeEqual(Buffer.from(expected), Buffer.from(signatureHeader));
}