{
  "id": "qlOiBu6nltvCzOYf",
  "name": "AWS WAF Scraping \u2014 Price & Product Details \u2014 CapSolver + Schedule + Webhook",
  "tags": [],
  "nodes": [
    {
      "id": "721002ab-5623-44fd-bcd7-44de185ec24f",
      "name": "Sticky Note",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        -1008,
        -368
      ],
      "parameters": {
        "width": 480,
        "height": 896,
        "content": "## AWS WAF Scraping \u2014 Price & Product Details \u2014 CapSolver + Schedule + Webhook\n\n### How it works\n\n1. Triggers at a regular interval or via a webhook request.\n2. Solves AWS WAF challenge then makes a request to fetch the product page.\n3. Extracts product data from the retrieved HTML page.\n4. Compares the current and previously stored data to detect any changes.\n5. Sends an alert if data has changed; else logs no change.\n6. Returns results if triggered via webhook.\n\n### Setup steps\n\n- [ ] Configure schedule settings in 'Every 6 Hours' node.\n- [ ] Set up AWS WAF credentials in 'Solve AWS WAF' nodes.\n- [ ] Input target URL in 'Fetch Product Page' nodes.\n- [ ] Configure webhook URL in 'Receive Monitor Request' node.\n\n### Customization\n\nAdjust the target site URL in 'Fetch Product Page' and related nodes to match different sites or specific pages."
      },
      "typeVersion": 1
    },
    {
      "id": "565d31e9-26a6-412d-92a4-c7da5c7b6985",
      "name": "Sticky Note1",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        -432,
        -192
      ],
      "parameters": {
        "color": 7,
        "width": 496,
        "height": 272,
        "content": "## Scheduled data scraping\n\nTriggers scraping every 6 hours and solves AWS WAF."
      },
      "typeVersion": 1
    },
    {
      "id": "58d3a139-a733-46d6-9f17-574ba6ad7929",
      "name": "Sticky Note2",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        176,
        -208
      ],
      "parameters": {
        "color": 7,
        "width": 480,
        "height": 272,
        "content": "## Scheduled fetch and extract\n\nFetches and processes product page data on schedule."
      },
      "typeVersion": 1
    },
    {
      "id": "fb4b9115-2b2f-49f1-b9be-4b44de4b37bf",
      "name": "Sticky Note3",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        736,
        -240
      ],
      "parameters": {
        "color": 7,
        "width": 432,
        "height": 320,
        "content": "## Scheduled data comparison\n\nCompares scraped data to previous entries and checks for changes."
      },
      "typeVersion": 1
    },
    {
      "id": "ec59cfa5-fe07-4b4a-a060-c3570e42a8a8",
      "name": "Sticky Note4",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        1232,
        -368
      ],
      "parameters": {
        "color": 7,
        "height": 512,
        "content": "## Scheduled alert or log\n\nGenerates alert or logs no change if applicable."
      },
      "typeVersion": 1
    },
    {
      "id": "1324edf4-b4f8-4cab-aa18-401d71fb486b",
      "name": "Sticky Note5",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        -448,
        320
      ],
      "parameters": {
        "color": 7,
        "width": 496,
        "height": 272,
        "content": "## Webhook triggered scraping\n\nHandles requests from webhook to initiate scraping."
      },
      "typeVersion": 1
    },
    {
      "id": "e86f4191-e81c-417e-93e8-ab163ac3024d",
      "name": "Sticky Note6",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        160,
        320
      ],
      "parameters": {
        "color": 7,
        "width": 496,
        "height": 272,
        "content": "## Webhook fetch and extract\n\nFetches and processes product page data via webhook."
      },
      "typeVersion": 1
    },
    {
      "id": "95c5ab10-e493-475a-9fe6-c39d0374a120",
      "name": "Sticky Note7",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        704,
        304
      ],
      "parameters": {
        "color": 7,
        "width": 400,
        "height": 304,
        "content": "## Webhook data comparison\n\nCompares and evaluates changes in data from webhook requests."
      },
      "typeVersion": 1
    },
    {
      "id": "fc36e038-4f18-43c7-84de-c42a32c3177c",
      "name": "Sticky Note8",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        1136,
        192
      ],
      "parameters": {
        "color": 7,
        "width": 384,
        "height": 464,
        "content": "## Webhook alert/log and return\n\nSends an alert or logs and returns data via webhook."
      },
      "typeVersion": 1
    },
    {
      "id": "aw333333-3333-3333-3333-aw3333333301",
      "name": "Every 6 Hours",
      "type": "n8n-nodes-base.scheduleTrigger",
      "position": [
        -384,
        -80
      ],
      "parameters": {
        "rule": {
          "interval": [
            {
              "field": "hours",
              "hoursInterval": 6
            }
          ]
        }
      },
      "typeVersion": 1.3
    },
    {
      "id": "aw333333-3333-3333-3333-aw3333333302",
      "name": "Solve AWS WAF",
      "type": "n8n-nodes-capsolver.capSolver",
      "position": [
        -80,
        -80
      ],
      "parameters": {
        "optional": {},
        "operation": "AWS WAF",
        "websiteURL": "https://YOUR-TARGET-SITE.com/product-page"
      },
      "credentials": {
        "capSolverApi": {
          "name": "<your credential>"
        }
      },
      "typeVersion": 1
    },
    {
      "id": "aw333333-3333-3333-3333-aw3333333303",
      "name": "Fetch Product Page",
      "type": "n8n-nodes-base.httpRequest",
      "position": [
        224,
        -96
      ],
      "parameters": {
        "url": "https://YOUR-TARGET-SITE.com/product-page",
        "options": {
          "response": {
            "response": {}
          }
        },
        "sendHeaders": true,
        "headerParameters": {
          "parameters": [
            {
              "name": "user-agent",
              "value": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.0.0 Safari/537.36"
            },
            {
              "name": "Cookie",
              "value": "={{ $json.data.solution.cookie }}"
            }
          ]
        }
      },
      "typeVersion": 4.3
    },
    {
      "id": "aw333333-3333-3333-3333-aw3333333304",
      "name": "Extract Data",
      "type": "n8n-nodes-base.html",
      "position": [
        512,
        -96
      ],
      "parameters": {
        "options": {},
        "operation": "extractHtmlContent",
        "extractionValues": {
          "values": [
            {
              "key": "price",
              "cssSelector": ".product-price, [data-price], .price"
            },
            {
              "key": "productName",
              "cssSelector": "h1, .product-title"
            }
          ]
        }
      },
      "typeVersion": 1.2
    },
    {
      "id": "aw333333-3333-3333-3333-aw3333333305",
      "name": "Compare Data",
      "type": "n8n-nodes-base.code",
      "position": [
        784,
        -96
      ],
      "parameters": {
        "jsCode": "// Get current and previous price from workflow static data\nconst staticData = $workflow.staticData;\nconst currentPrice = $input.first().json.price;\nconst previousPrice = staticData.lastPrice;\nconst productName = $input.first().json.productName || 'Product';\n\n// Parse numeric values for comparison\nconst parsePrice = (str) => {\n  if (!str) return null;\n  const match = str.match(/[\\d]+\\.?\\d*/);\n  return match ? parseFloat(match[0].replace(',', '')) : null;\n};\n\nconst currentNum = parsePrice(currentPrice);\nconst previousNum = parsePrice(previousPrice);\n\n// Update stored price\nstaticData.lastPrice = currentPrice;\nstaticData.lastChecked = new Date().toISOString();\n\nconst changed = previousNum !== null && currentNum !== null && currentNum !== previousNum;\nconst direction = changed ? (currentNum < previousNum ? 'dropped' : 'increased') : 'unchanged';\nconst diff = changed ? Math.abs(currentNum - previousNum).toFixed(2) : '0';\n\nreturn [{\n  json: {\n    productName,\n    currentPrice,\n    previousPrice: previousPrice || 'first check',\n    changed,\n    direction,\n    diff: changed ? `$${diff}` : null,\n    checkedAt: new Date().toISOString()\n  }\n}];"
      },
      "typeVersion": 2
    },
    {
      "id": "aw333333-3333-3333-3333-aw3333333306",
      "name": "Data Changed?",
      "type": "n8n-nodes-base.if",
      "position": [
        1024,
        -112
      ],
      "parameters": {
        "options": {},
        "conditions": {
          "options": {
            "version": 2,
            "leftValue": "",
            "caseSensitive": true,
            "typeValidation": "strict"
          },
          "combinator": "and",
          "conditions": [
            {
              "id": "price-if-001",
              "operator": {
                "type": "boolean",
                "operation": "true",
                "singleValue": true
              },
              "leftValue": "={{ $json.changed }}"
            }
          ]
        }
      },
      "typeVersion": 2.3
    },
    {
      "id": "aw333333-3333-3333-3333-aw3333333307",
      "name": "Build Alert",
      "type": "n8n-nodes-base.set",
      "position": [
        1296,
        -240
      ],
      "parameters": {
        "options": {},
        "assignments": {
          "assignments": [
            {
              "id": "alert-001",
              "name": "alert",
              "type": "string",
              "value": "=Price {{ $json.direction }} for {{ $json.productName }}: {{ $json.previousPrice }} \u2192 {{ $json.currentPrice }} ({{ $json.direction === 'dropped' ? '-' : '+' }}{{ $json.diff }})"
            },
            {
              "id": "alert-002",
              "name": "severity",
              "type": "string",
              "value": "={{ $json.direction === 'dropped' ? 'deal' : 'info' }}"
            },
            {
              "id": "alert-003",
              "name": "checkedAt",
              "type": "string",
              "value": "={{ $json.checkedAt }}"
            }
          ]
        }
      },
      "typeVersion": 3.4
    },
    {
      "id": "aw333333-3333-3333-3333-aw3333333308",
      "name": "No Change",
      "type": "n8n-nodes-base.set",
      "position": [
        1280,
        -32
      ],
      "parameters": {
        "options": {},
        "assignments": {
          "assignments": [
            {
              "id": "nc-001",
              "name": "status",
              "type": "string",
              "value": "no_change"
            },
            {
              "id": "nc-002",
              "name": "currentPrice",
              "type": "string",
              "value": "={{ $json.currentPrice }}"
            },
            {
              "id": "nc-003",
              "name": "checkedAt",
              "type": "string",
              "value": "={{ $json.checkedAt }}"
            }
          ]
        }
      },
      "typeVersion": 3.4
    },
    {
      "id": "aw333333-3333-3333-3333-aw3333333309",
      "name": "Receive Monitor Request",
      "type": "n8n-nodes-base.webhook",
      "onError": "continueRegularOutput",
      "position": [
        -400,
        432
      ],
      "parameters": {
        "path": "price-monitor-aws-waf",
        "options": {},
        "httpMethod": "POST",
        "responseMode": "responseNode"
      },
      "typeVersion": 2.1
    },
    {
      "id": "aw333333-3333-3333-3333-aw3333333310",
      "name": "Solve AWS WAF [Webhook]",
      "type": "n8n-nodes-capsolver.capSolver",
      "position": [
        -96,
        432
      ],
      "parameters": {
        "optional": {},
        "operation": "AWS WAF",
        "websiteURL": "https://YOUR-TARGET-SITE.com/product-page"
      },
      "credentials": {
        "capSolverApi": {
          "name": "<your credential>"
        }
      },
      "typeVersion": 1
    },
    {
      "id": "aw333333-3333-3333-3333-aw3333333311",
      "name": "Fetch Product Page [Webhook]",
      "type": "n8n-nodes-base.httpRequest",
      "position": [
        208,
        432
      ],
      "parameters": {
        "url": "https://YOUR-TARGET-SITE.com/product-page",
        "options": {
          "response": {
            "response": {}
          }
        },
        "sendHeaders": true,
        "headerParameters": {
          "parameters": [
            {
              "name": "user-agent",
              "value": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.0.0 Safari/537.36"
            },
            {
              "name": "Cookie",
              "value": "={{ $json.data.solution.cookie }}"
            }
          ]
        }
      },
      "typeVersion": 4.3
    },
    {
      "id": "aw333333-3333-3333-3333-aw3333333312",
      "name": "Extract Data [Webhook]",
      "type": "n8n-nodes-base.html",
      "position": [
        512,
        432
      ],
      "parameters": {
        "options": {},
        "operation": "extractHtmlContent",
        "extractionValues": {
          "values": [
            {
              "key": "price",
              "cssSelector": ".product-price, [data-price], .price"
            },
            {
              "key": "productName",
              "cssSelector": "h1, .product-title"
            }
          ]
        }
      },
      "typeVersion": 1.2
    },
    {
      "id": "aw333333-3333-3333-3333-aw3333333313",
      "name": "Compare Data [Webhook]",
      "type": "n8n-nodes-base.code",
      "position": [
        752,
        432
      ],
      "parameters": {
        "jsCode": "// Get current and previous price from workflow static data\nconst staticData = $workflow.staticData;\nconst currentPrice = $input.first().json.price;\nconst previousPrice = staticData.lastPrice;\nconst productName = $input.first().json.productName || 'Product';\n\n// Parse numeric values for comparison\nconst parsePrice = (str) => {\n  if (!str) return null;\n  const match = str.match(/[\\d]+\\.?\\d*/);\n  return match ? parseFloat(match[0].replace(',', '')) : null;\n};\n\nconst currentNum = parsePrice(currentPrice);\nconst previousNum = parsePrice(previousPrice);\n\n// Update stored price\nstaticData.lastPrice = currentPrice;\nstaticData.lastChecked = new Date().toISOString();\n\nconst changed = previousNum !== null && currentNum !== null && currentNum !== previousNum;\nconst direction = changed ? (currentNum < previousNum ? 'dropped' : 'increased') : 'unchanged';\nconst diff = changed ? Math.abs(currentNum - previousNum).toFixed(2) : '0';\n\nreturn [{\n  json: {\n    productName,\n    currentPrice,\n    previousPrice: previousPrice || 'first check',\n    changed,\n    direction,\n    diff: changed ? `$${diff}` : null,\n    checkedAt: new Date().toISOString()\n  }\n}];"
      },
      "typeVersion": 2
    },
    {
      "id": "aw333333-3333-3333-3333-aw3333333314",
      "name": "Data Changed? [Webhook]",
      "type": "n8n-nodes-base.if",
      "position": [
        960,
        432
      ],
      "parameters": {
        "options": {},
        "conditions": {
          "options": {
            "version": 2,
            "leftValue": "",
            "caseSensitive": true,
            "typeValidation": "strict"
          },
          "combinator": "and",
          "conditions": [
            {
              "id": "price-if-002",
              "operator": {
                "type": "boolean",
                "operation": "true",
                "singleValue": true
              },
              "leftValue": "={{ $json.changed }}"
            }
          ]
        }
      },
      "typeVersion": 2.3
    },
    {
      "id": "aw333333-3333-3333-3333-aw3333333315",
      "name": "Build Alert [Webhook]",
      "type": "n8n-nodes-base.set",
      "position": [
        1184,
        320
      ],
      "parameters": {
        "options": {},
        "assignments": {
          "assignments": [
            {
              "id": "alert-004",
              "name": "alert",
              "type": "string",
              "value": "=Price {{ $json.direction }} for {{ $json.productName }}: {{ $json.previousPrice }} \u2192 {{ $json.currentPrice }} ({{ $json.direction === 'dropped' ? '-' : '+' }}{{ $json.diff }})"
            },
            {
              "id": "alert-005",
              "name": "severity",
              "type": "string",
              "value": "={{ $json.direction === 'dropped' ? 'deal' : 'info' }}"
            },
            {
              "id": "alert-006",
              "name": "checkedAt",
              "type": "string",
              "value": "={{ $json.checkedAt }}"
            }
          ]
        }
      },
      "typeVersion": 3.4
    },
    {
      "id": "aw333333-3333-3333-3333-aw3333333316",
      "name": "No Change [Webhook]",
      "type": "n8n-nodes-base.set",
      "position": [
        1184,
        480
      ],
      "parameters": {
        "options": {},
        "assignments": {
          "assignments": [
            {
              "id": "nc-004",
              "name": "status",
              "type": "string",
              "value": "no_change"
            },
            {
              "id": "nc-005",
              "name": "currentPrice",
              "type": "string",
              "value": "={{ $json.currentPrice }}"
            },
            {
              "id": "nc-006",
              "name": "checkedAt",
              "type": "string",
              "value": "={{ $json.checkedAt }}"
            }
          ]
        }
      },
      "typeVersion": 3.4
    },
    {
      "id": "aw333333-3333-3333-3333-aw3333333317",
      "name": "Return Scraped Data",
      "type": "n8n-nodes-base.respondToWebhook",
      "position": [
        1376,
        400
      ],
      "parameters": {
        "options": {},
        "respondWith": "json",
        "responseBody": "={{ JSON.stringify($json) }}"
      },
      "typeVersion": 1.5
    }
  ],
  "active": false,
  "settings": {
    "binaryMode": "separate",
    "callerPolicy": "workflowsFromSameOwner",
    "availableInMCP": false,
    "executionOrder": "v1"
  },
  "versionId": "771e5e00-d1d9-4593-99b1-f33e10eee692",
  "connections": {
    "Compare Data": {
      "main": [
        [
          {
            "node": "Data Changed?",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Extract Data": {
      "main": [
        [
          {
            "node": "Compare Data",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Data Changed?": {
      "main": [
        [
          {
            "node": "Build Alert",
            "type": "main",
            "index": 0
          }
        ],
        [
          {
            "node": "No Change",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Every 6 Hours": {
      "main": [
        [
          {
            "node": "Solve AWS WAF",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Solve AWS WAF": {
      "main": [
        [
          {
            "node": "Fetch Product Page",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Fetch Product Page": {
      "main": [
        [
          {
            "node": "Extract Data",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "No Change [Webhook]": {
      "main": [
        [
          {
            "node": "Return Scraped Data",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Build Alert [Webhook]": {
      "main": [
        [
          {
            "node": "Return Scraped Data",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Compare Data [Webhook]": {
      "main": [
        [
          {
            "node": "Data Changed? [Webhook]",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Extract Data [Webhook]": {
      "main": [
        [
          {
            "node": "Compare Data [Webhook]",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Data Changed? [Webhook]": {
      "main": [
        [
          {
            "node": "Build Alert [Webhook]",
            "type": "main",
            "index": 0
          }
        ],
        [
          {
            "node": "No Change [Webhook]",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Receive Monitor Request": {
      "main": [
        [
          {
            "node": "Solve AWS WAF [Webhook]",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Solve AWS WAF [Webhook]": {
      "main": [
        [
          {
            "node": "Fetch Product Page [Webhook]",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Fetch Product Page [Webhook]": {
      "main": [
        [
          {
            "node": "Extract Data [Webhook]",
            "type": "main",
            "index": 0
          }
        ]
      ]
    }
  }
}