AutomationFlowsWeb Scraping › [fc Sub] Map a Website

[fc Sub] Map a Website

[FC Sub] Map a Website. Uses executeWorkflowTrigger, httpRequest, postgres. Event-driven trigger; 7 nodes.

Event trigger★★★★☆ complexity7 nodesExecute Workflow TriggerHTTP RequestPostgres
Web Scraping Trigger: Event Nodes: 7 Complexity: ★★★★☆ Added:

This workflow follows the Execute Workflow Trigger → HTTP Request recipe pattern — see all workflows that pair these two integrations.

The workflow JSON

Copy or download the full n8n JSON below. Paste it into a new n8n workflow, add your credentials, activate. Full import guide →

Download .json
{
  "name": "[FC Sub] Map a Website",
  "settings": {
    "executionOrder": "v1"
  },
  "nodes": [
    {
      "parameters": {
        "content": "## [FC Sub] Map a Website\n**Purpose:** Lightweight URL discovery via Firecrawl `/v1/map`. Returns every URL it can find on the site without fetching page content \u2014 the cheapest way to scope a crawl.\n\n**Called by:** main agent's `map_site` tool.\n\n**Inputs:** `url` (required, site root), `search?` (optional keyword filter), `limit?` (default 500, max 5000), `session_id?`.\n\n**Why this matters:** The winning pattern for any large-site research is **map \u2192 filter via `think` \u2192 batch_scrape selected URLs**. Never call `crawl_site` blind \u2014 it burns credits on pages you don't need. This sub-workflow enables that pattern.\n\n**Credit cost:** 1 credit per call (regardless of URLs returned).\n\n**Flow:** Trigger \u2192 Prep \u2192 Firecrawl `/map` \u2192 Shape \u2192 Ledger \u2192 Return `{ urls[], count, credits_used }`.\n\n**Credentials:** Firecrawl API, Postgres RW.",
        "height": 520,
        "width": 600,
        "color": 6
      },
      "type": "n8n-nodes-base.stickyNote",
      "typeVersion": 1,
      "position": [
        -40,
        -560
      ],
      "id": "sticky-fc-map",
      "name": "README"
    },
    {
      "parameters": {
        "inputSource": "passthrough"
      },
      "type": "n8n-nodes-base.executeWorkflowTrigger",
      "typeVersion": 1.1,
      "position": [
        0,
        0
      ],
      "id": "fc-map-trigger",
      "name": "When Executed by Another Workflow"
    },
    {
      "parameters": {
        "jsCode": "const input = $input.first().json;\nconst q = (input.query && typeof input.query === 'object') ? input.query : {};\nconst pick = (k, def) => {\n  if (input[k] !== undefined) return input[k];\n  if (q[k] !== undefined) return q[k];\n  return def;\n};\n\nlet url = (pick('url') || '').toString().trim();\nif (!url) throw new Error('map_site requires a `url` parameter (the site root).');\nif (!/^https?:\\/\\//i.test(url)) url = 'https://' + url;\ntry { new URL(url); } catch { throw new Error(`Invalid URL: ${url}`); }\n\nconst search = (pick('search') || '').toString().trim();\n\nlet limit = Number(pick('limit', 500));\nif (!Number.isFinite(limit) || limit < 1) limit = 500;\nif (limit > 5000) limit = 5000; // hard cap\n\nconst session_id = (pick('session_id') || $execution.id || 'no-session').toString();\n\nreturn [{ json: {\n  url,\n  search,\n  limit,\n  session_id,\n  execution_id: $execution.id\n} }];"
      },
      "type": "n8n-nodes-base.code",
      "typeVersion": 2,
      "position": [
        220,
        0
      ],
      "id": "fc-map-prep",
      "name": "Prep Input"
    },
    {
      "parameters": {
        "method": "POST",
        "url": "https://api.firecrawl.dev/v1/map",
        "authentication": "genericCredentialType",
        "genericAuthType": "httpHeaderAuth",
        "sendBody": true,
        "specifyBody": "json",
        "jsonBody": "={\n  \"url\": {{ JSON.stringify($json.url) }},\n  \"limit\": {{ $json.limit }}\n  {{ $json.search ? (', \"search\": ' + JSON.stringify($json.search)) : '' }}\n}",
        "options": {
          "response": {
            "response": {
              "responseFormat": "json"
            }
          },
          "timeout": 120000
        }
      },
      "type": "n8n-nodes-base.httpRequest",
      "typeVersion": 4.2,
      "position": [
        440,
        0
      ],
      "id": "fc-map-http",
      "name": "Call Firecrawl Map"
    },
    {
      "parameters": {
        "jsCode": "const prep = $('Prep Input').first().json;\nconst resp = $input.first().json;\nconst urls = Array.isArray(resp.links) ? resp.links\n           : Array.isArray(resp.data)  ? resp.data\n           : Array.isArray(resp.urls)  ? resp.urls\n           : [];\nconst MAX_RETURN = 1000;\nconst returned = urls.slice(0, MAX_RETURN);\nconst truncated = urls.length > MAX_RETURN;\n\nreturn [{ json: {\n  site: prep.url,\n  search: prep.search || null,\n  urls: returned,\n  count: returned.length,\n  total_found: urls.length,\n  truncated,\n  credits_used: 1,\n  session_id: prep.session_id,\n  execution_id: prep.execution_id\n} }];"
      },
      "type": "n8n-nodes-base.code",
      "typeVersion": 2,
      "position": [
        660,
        0
      ],
      "id": "fc-map-shape",
      "name": "Shape Response"
    },
    {
      "parameters": {
        "operation": "executeQuery",
        "query": "INSERT INTO public.firecrawl_credit_ledger (session_id, execution_id, operation, credits_used, url, status, metadata)\nVALUES ($1, $2, 'map', $3, $4, 'ok', $5::jsonb);",
        "options": {
          "queryReplacement": "={{ $json.session_id }}, {{ $json.execution_id }}, {{ $json.credits_used }}, {{ $json.site }}, {{ JSON.stringify({ search: $json.search, count: $json.count, total_found: $json.total_found }) }}"
        }
      },
      "type": "n8n-nodes-base.postgres",
      "typeVersion": 2.6,
      "position": [
        880,
        0
      ],
      "id": "fc-map-ledger",
      "name": "Log Credit Ledger"
    },
    {
      "parameters": {
        "assignments": {
          "assignments": [
            {
              "id": "m1",
              "name": "site",
              "value": "={{ $('Shape Response').first().json.site }}",
              "type": "string"
            },
            {
              "id": "m2",
              "name": "urls",
              "value": "={{ $('Shape Response').first().json.urls }}",
              "type": "array"
            },
            {
              "id": "m3",
              "name": "count",
              "value": "={{ $('Shape Response').first().json.count }}",
              "type": "number"
            },
            {
              "id": "m4",
              "name": "total_found",
              "value": "={{ $('Shape Response').first().json.total_found }}",
              "type": "number"
            },
            {
              "id": "m5",
              "name": "truncated",
              "value": "={{ $('Shape Response').first().json.truncated }}",
              "type": "boolean"
            },
            {
              "id": "m6",
              "name": "credits_used",
              "value": "={{ $('Shape Response').first().json.credits_used }}",
              "type": "number"
            }
          ]
        },
        "options": {}
      },
      "type": "n8n-nodes-base.set",
      "typeVersion": 3.4,
      "position": [
        1100,
        0
      ],
      "id": "fc-map-return",
      "name": "Return URLs"
    }
  ],
  "connections": {
    "When Executed by Another Workflow": {
      "main": [
        [
          {
            "node": "Prep Input",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Prep Input": {
      "main": [
        [
          {
            "node": "Call Firecrawl Map",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Call Firecrawl Map": {
      "main": [
        [
          {
            "node": "Shape Response",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Shape Response": {
      "main": [
        [
          {
            "node": "Log Credit Ledger",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Log Credit Ledger": {
      "main": [
        [
          {
            "node": "Return URLs",
            "type": "main",
            "index": 0
          }
        ]
      ]
    }
  }
}
Pro

For the full experience including quality scoring and batch install features for each workflow upgrade to Pro

About this workflow

[FC Sub] Map a Website. Uses executeWorkflowTrigger, httpRequest, postgres. Event-driven trigger; 7 nodes.

Source: https://github.com/MinaSaad1/n8n-firecrawl-web-crawler-agent/blob/main/workflows/05-sub-map-site.json — original creator credit. Request a take-down →

More Web Scraping workflows → · Browse all categories →

Related workflows

Workflows that share integrations, category, or trigger type with this one. All free to copy and import.

Web Scraping

[FC Sub] Scrape URL with 24h Cache. Uses executeWorkflowTrigger, postgres, httpRequest. Event-driven trigger; 14 nodes.

Execute Workflow Trigger, Postgres, HTTP Request
Web Scraping

[FC Sub] Crawl Site (Bounded). Uses executeWorkflowTrigger, httpRequest, postgres. Event-driven trigger; 12 nodes.

Execute Workflow Trigger, HTTP Request, Postgres
Web Scraping

[FC Sub] Batch Scrape URLs. Uses executeWorkflowTrigger, httpRequest, postgres. Event-driven trigger; 12 nodes.

Execute Workflow Trigger, HTTP Request, Postgres
Web Scraping

[FC Sub] Extract Structured Data. Uses executeWorkflowTrigger, httpRequest, postgres. Event-driven trigger; 12 nodes.

Execute Workflow Trigger, HTTP Request, Postgres
Web Scraping

[FC Sub] Search the Web. Uses executeWorkflowTrigger, httpRequest, postgres. Event-driven trigger; 7 nodes.

Execute Workflow Trigger, HTTP Request, Postgres