Extract Product Details From Search Result Urls with Mrscraper and Google Sheets

The workflow JSON

Copy or download the full n8n JSON below. Paste it into a new n8n workflow, add your credentials, activate. Full import guide →

{
  "id": "cXh2nQGvFd24V3nD",
  "meta": {
    "templateCredsSetupCompleted": true
  },
  "name": "Search Results to Product Details Extractor",
  "tags": [],
  "nodes": [
    {
      "id": "2c20b891-738d-4f97-b047-3ec77c92393d",
      "name": "Extract All Url ",
      "type": "n8n-nodes-base.code",
      "position": [
        512,
        464
      ],
      "parameters": {
        "language": "python",
        "pythonCode": "items = []\nurls = set()\n\n# Loop through ALL input items, not just one\nfor input_item in _input.all():\n    payload = input_item.json\n    \n    # Extract URLs from response data\n    response = payload.get(\"data\", {}).get(\"response\") or []\n    for page in response:\n        listings = page.get(\"data\", {}).get(\"data\") or []\n        for listing in listings:\n            url = listing.get(\"url\")\n            if isinstance(url, str) and url.strip():\n                urls.add(url)\n    \n    # Extract the search link\n    search_link = payload.get(\"data\", {}).get(\"link\")\n    if isinstance(search_link, str) and search_link.strip():\n        urls.add(search_link)\n\n# Convert set to list of items\nfor url in urls:\n    items.append({\n        \"json\": {\n            \"url\": url\n        }\n    })\n\nreturn items"
      },
      "typeVersion": 2
    },
    {
      "id": "1236a0af-6596-46ed-a79b-fae29e2acea7",
      "name": "When clicking \u2018Execute workflow\u2019",
      "type": "n8n-nodes-base.manualTrigger",
      "position": [
        -288,
        480
      ],
      "parameters": {},
      "typeVersion": 1
    },
    {
      "id": "f15d8061-1638-4aad-ab5d-c0be2c86fcaf",
      "name": "Flatten Object",
      "type": "n8n-nodes-base.code",
      "position": [
        1168,
        464
      ],
      "parameters": {
        "jsCode": "// Recursive function to flatten nested objects\nfunction flattenObject(obj, prefix = '', result = {}) {\n  for (const key in obj) {\n    if (!Object.prototype.hasOwnProperty.call(obj, key)) continue;\n\n    const newKey = prefix ? `${prefix}_${key}` : key;\n    const value = obj[key];\n\n    if (value === null || value === undefined) {\n      result[newKey] = null;\n    } else if (Array.isArray(value)) {\n      // Join array values\n      result[newKey] = value.length ? value.join(', ') : null;\n    } else if (typeof value === 'object' && !(value instanceof Date)) {\n      flattenObject(value, newKey, result);\n    } else {\n      result[newKey] = value;\n    }\n  }\n  return result;\n}\n\n// 1. Get ALL input items (important)\nconst items = $input.all();\n\n// 2. Flatten EACH item\nconst output = items.map(item => {\n  const flattened = flattenObject(item.json);\n  return { json: flattened };\n});\n\nreturn output;\n"
      },
      "executeOnce": false,
      "typeVersion": 2
    },
    {
      "id": "a8477940-b5ed-4627-8468-0ff25bd641b9",
      "name": "Looping Listing Page url",
      "type": "n8n-nodes-base.splitInBatches",
      "position": [
        320,
        480
      ],
      "parameters": {
        "options": {
          "reset": false
        }
      },
      "typeVersion": 3
    },
    {
      "id": "61437fa8-a7dd-481a-bef7-fdd2bac81e42",
      "name": "Looping Detail Page url",
      "type": "n8n-nodes-base.splitInBatches",
      "position": [
        976,
        496
      ],
      "parameters": {
        "options": {
          "reset": false
        }
      },
      "retryOnFail": false,
      "typeVersion": 3
    },
    {
      "id": "e66f7c5e-2a09-4907-97d8-a47b8b9a2668",
      "name": "Sticky Note",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        -1136,
        416
      ],
      "parameters": {
        "width": 720,
        "height": 672,
        "content": "## Phase 0: Setup and Configuration\n### Goal\nPrepare the MrScraper agents, Google Sheets, and n8n credentials so the workflow can run reliably. Also, make sure to create a scraper in MrScraper beforehand so you have a scraperId that can be used in this workflow for reruns.\n\n\n### What you need before running\n1. **Create the scrapers in your MrScraper account first** (one per agent):\n\n   * **Listing Agent Scraper** \u2192 for scraping listing/search pages\n   * **General Agent Scraper** \u2192 for scraping detail pages\n\n   Each scraper will have its own **`scraperId`**. You\u2019ll use these IDs in n8n to **Rerun** each agent and fetch results.\n2. **Enable AI Scraper API access** in your MrScraper account (so n8n can run scrapers via API).\n3. **Create credentials for MrScraper, Gmail, and Google.**\n\n### What To Do\n1. Add your **Google Sheets API token** into your n8n credentials / \"input sheets\" node.\n2. Choose Document and Sheets that contain List Search page that you want to scrape.\n3. Add your **MrScraper API token** into your n8n credentials / \u201cInput re-run API\u201d node.\n4. Fill in all two **scraper IDs** to each node rerun:\n   * `LISTING_SCRAPER_ID`\n   * `GENERAL_SCRAPER_ID`\n5. Connect **Google Sheets** (spreadsheet + sheet tab, append/upsert strategy).\n6. Connect **Gmail** (recipient, subject format, send rules: always / only new rows / only errors).\n\n\n"
      },
      "typeVersion": 1
    },
    {
      "id": "29395412-7f02-4cc7-b40f-36dbbf287cf3",
      "name": "Sticky Note8",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        -400,
        656
      ],
      "parameters": {
        "color": 2,
        "width": 624,
        "height": 352,
        "content": "### Goal\nRetrieve all target search/listing page URLs stored in Google Sheets for use in the n8n workflow.\n\n### What To Do\n1. Use the **Google Sheets** node and select the **Read Rows** operation.\n2. Connect to the correct spreadsheet and choose the sheet that contains your search/listing page URLs.\n3. Retrieve all rows from the sheet and extract the column that stores the target URLs.\n4. Clean the data by removing empty rows, duplicates, and optionally limit the total number of URLs if needed.\n\n### Output\nA structured list of search/listing page URLs fetched directly from Google Sheets, ready to be processed in the next step of the workflow."
      },
      "typeVersion": 1
    },
    {
      "id": "0be7617a-9a3a-40d2-ad91-05cfa9e3078a",
      "name": "Run listing agent scraper",
      "type": "n8n-nodes-mrscraper.mrscraper",
      "position": [
        688,
        512
      ],
      "parameters": {
        "url": "=// Input Your url (required)",
        "timeout": 720,
        "maxPages": 2,
        "operation": "listingAgent",
        "scraperId": "=// Input Your scraperId from mrscraper (required)",
        "requestOptions": {}
      },
      "credentials": {
        "mrscraperApi": {
          "name": "<your credential>"
        }
      },
      "typeVersion": 1
    },
    {
      "id": "4cc501d6-9527-4258-89b5-0d8cd9db9a16",
      "name": "Run general agent scraper",
      "type": "n8n-nodes-mrscraper.mrscraper",
      "position": [
        1312,
        496
      ],
      "parameters": {
        "url": "={{ $json.url }}",
        "operation": "generalAgent",
        "scraperId": "={{ $('Configuration re-run API').item.json.generalScraperId }}",
        "requestOptions": {}
      },
      "credentials": {
        "mrscraperApi": {
          "name": "<your credential>"
        }
      },
      "typeVersion": 1
    },
    {
      "id": "f2b62ecb-76ca-4d28-b143-405d56293ff1",
      "name": "Send a message",
      "type": "n8n-nodes-base.gmail",
      "position": [
        1936,
        480
      ],
      "parameters": {
        "sendTo": "// Input where you want to send",
        "message": "// Input The message",
        "options": {},
        "subject": "// Input The subject of the message",
        "emailType": "text"
      },
      "credentials": {
        "gmailOAuth2": {
          "name": "<your credential>"
        }
      },
      "typeVersion": 2.2
    },
    {
      "id": "87bd29a3-f121-44da-a34d-27b25e47a0bf",
      "name": "Sticky Note9",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        -400,
        416
      ],
      "parameters": {
        "color": 2,
        "width": 624,
        "height": 224,
        "content": "## Phase 1: Load List search Page Url"
      },
      "typeVersion": 1
    },
    {
      "id": "05a40a40-ec5c-45e0-8f8f-8250f8473c1b",
      "name": "Sticky Note10",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        240,
        416
      ],
      "parameters": {
        "color": 5,
        "width": 624,
        "height": 304,
        "content": "## Phase 2: Scrape Listing Page\n"
      },
      "typeVersion": 1
    },
    {
      "id": "15f52a25-622a-4749-a7d8-f2ed139c76f5",
      "name": "Sticky Note11",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        240,
        736
      ],
      "parameters": {
        "color": 5,
        "width": 624,
        "height": 288,
        "content": "### Goal\nFrom each listing/search page, extract **detail page URLs**.\n\n### What To Do\n1. Loop through each listing page URL from Phase 1.\n2. For each URL, call the **Listing Agent (Rerun)** using the scraperId from the mrscraper platform that was previously created.\n3. Extract all detail URLs from the Listing results.\n4. Normalize URLs (convert to absolute), then deduplicate them.\n\n### Output\nA deduped list of detail page URLs."
      },
      "typeVersion": 1
    },
    {
      "id": "6c65be49-d48c-4c40-b03c-9a8389399ed1",
      "name": "Sticky Note12",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        880,
        416
      ],
      "parameters": {
        "color": 6,
        "width": 624,
        "height": 304,
        "content": "## Phase 3: Scrape Detail Data\n"
      },
      "typeVersion": 1
    },
    {
      "id": "fb6302dd-4925-46e6-a7e3-ffb2cf4a86f8",
      "name": "Sticky Note13",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        880,
        736
      ],
      "parameters": {
        "color": 6,
        "width": 624,
        "height": 400,
        "content": "### Goal\nExtract structured fields from each detail page.\n\n### What To Do\n1. Loop through each detail URL from Phase 2.\n2. For each URL, call the **General Agent (Rerun)** using the scraperId from the mrscraper platform that was previously created.\n3. Select the fields you want to keep (examples: title, price, location, attributes, description).\n4. Normalize the output:\n   * flatten nested JSON\n   * format arrays into readable text\n   * add metadata (source URL, scrape timestamp)\n\n### Output\nOne structured record per detail page (ready to export)."
      },
      "typeVersion": 1
    },
    {
      "id": "6acb60f0-6dba-42de-bd4f-becc916c4b95",
      "name": "Sticky Note14",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        1520,
        656
      ],
      "parameters": {
        "color": 3,
        "width": 592,
        "height": 432,
        "content": "### Goal\nSave results into Google Sheets and send a Gmail summary/alert.\n\n### What To Do\n1. Write rows into **Google Sheets**:\n   * Append new rows, or upsert using a unique key (recommended: `source_url`).\n2. Build a run summary:\n   * total listing pages processed\n   * total detail pages scraped\n   * total rows inserted/updated\n   * errors (if any)\n3. Send a Gmail notification:\n   * success digest (optional)\n   * or alert only for new items / errors / threshold triggers\n\n### Output\nUpdated spreadsheet + email notification."
      },
      "typeVersion": 1
    },
    {
      "id": "81c0127b-ae9d-4728-9ba0-daaca7f756ba",
      "name": "Sticky Note15",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        1520,
        416
      ],
      "parameters": {
        "color": 3,
        "width": 592,
        "height": 224,
        "content": "## Phase 4: Export to Spreadsheet + Notify via Gmail"
      },
      "typeVersion": 1
    },
    {
      "id": "179d0db2-b7a0-43c8-b417-a0af14cc7044",
      "name": "Get List Search Page",
      "type": "n8n-nodes-base.googleSheets",
      "position": [
        0,
        480
      ],
      "parameters": {
        "options": {},
        "sheetName": {
          "__rl": true,
          "mode": "list",
          "value": 1823271814,
          "cachedResultUrl": "https://docs.google.com/spreadsheets/d/1fmW05fj0Pu8AXxeLt500l2kE1SvR8bKo4BCRNXsKzPQ/edit#gid=1823271814",
          "cachedResultName": "auxipress_time_diff_analysys"
        },
        "documentId": {
          "__rl": true,
          "mode": "list",
          "value": "1fmW05fj0Pu8AXxeLt500l2kE1SvR8bKo4BCRNXsKzPQ",
          "cachedResultUrl": "https://docs.google.com/spreadsheets/d/1fmW05fj0Pu8AXxeLt500l2kE1SvR8bKo4BCRNXsKzPQ/edit?usp=drivesdk",
          "cachedResultName": "clean auxipress_time_diff_analysys"
        }
      },
      "credentials": {
        "googleSheetsOAuth2Api": {
          "name": "<your credential>"
        }
      },
      "typeVersion": 4.7
    },
    {
      "id": "2fbb77f0-813d-432e-a127-8e2bd911950c",
      "name": "Append or update row in sheet",
      "type": "n8n-nodes-base.googleSheets",
      "position": [
        1648,
        480
      ],
      "parameters": {
        "columns": {
          "value": {},
          "schema": [],
          "mappingMode": "autoMapInputData",
          "matchingColumns": [],
          "attemptToConvertTypes": false,
          "convertFieldsToString": false
        },
        "options": {},
        "operation": "appendOrUpdate",
        "sheetName": {
          "__rl": true,
          "mode": "url",
          "value": "// Input Your sheets Url (required)"
        },
        "documentId": {
          "__rl": true,
          "mode": "url",
          "value": "// Input Your Document Url (required)"
        }
      },
      "credentials": {
        "googleSheetsOAuth2Api": {
          "name": "<your credential>"
        }
      },
      "typeVersion": 4.7
    }
  ],
  "active": false,
  "settings": {
    "availableInMCP": false,
    "executionOrder": "v1"
  },
  "versionId": "11e4d92a-5def-42a1-893a-7b4928c53791",
  "connections": {
    "Flatten Object": {
      "main": [
        [
          {
            "node": "Append or update row in sheet",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Extract All Url ": {
      "main": [
        [
          {
            "node": "Looping Detail Page url",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Get List Search Page": {
      "main": [
        [
          {
            "node": "Looping Listing Page url",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Looping Detail Page url": {
      "main": [
        [
          {
            "node": "Flatten Object",
            "type": "main",
            "index": 0
          }
        ],
        [
          {
            "node": "Run general agent scraper",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Looping Listing Page url": {
      "main": [
        [
          {
            "node": "Extract All Url ",
            "type": "main",
            "index": 0
          }
        ],
        [
          {
            "node": "Run listing agent scraper",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Run general agent scraper": {
      "main": [
        [
          {
            "node": "Looping Detail Page url",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Run listing agent scraper": {
      "main": [
        [
          {
            "node": "Looping Listing Page url",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Append or update row in sheet": {
      "main": [
        [
          {
            "node": "Send a message",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "When clicking \u2018Execute workflow\u2019": {
      "main": [
        [
          {
            "node": "Get List Search Page",
            "type": "main",
            "index": 0
          }
        ]
      ]
    }
  }
}

Credentials you'll need

Each integration node will prompt for credentials when you import. We strip credential IDs before publishing — you'll add your own.

• gmailOAuth2
• googleSheetsOAuth2Api
• mrscraperApi

About this workflow

This n8n template automates the extraction of structured product data from search results pages — using a pre-built list of search/listing URLs stored in Google Sheets as the starting point. Instead of crawling a domain from scratch, this workflow picks up exactly where your…

Source: https://n8n.io/workflows/13800/ — original creator credit. Request a take-down →

More Email & Gmail workflows → · Browse all categories →

Related workflows

Workflows that share integrations, category, or trigger type with this one. All free to copy and import.