Scraping and extraction

Selectors may be a CSS string or an object selecting text, inner_html, outer_html, or an attribute. Lists select a root and evaluate fields relative to each item.

{
  "extract": {
    "products": {
      "selector": ".product",
      "type": "list",
      "fields": {
        "name": ".title",
        "url": { "selector": "a", "attribute": "href" }
      }
    }
  }
}

Required misses are reported as extraction issues while optional fields become null. Invalid selectors and page evaluation failures use separate codes. Data assertions such as min_items operate on the extracted result.