Firecrawl Single URL to Resilient Analyzer

The workflow JSON

Copy or download the full n8n JSON below. Paste it into a new n8n workflow, add your credentials, activate. Full import guide →

{
  "name": "03 - Firecrawl Single URL to Resilient Analyzer",
  "nodes": [
    {
      "id": "fc000000-0000-0000-0000-0000000000n1",
      "name": "Overview Note RU",
      "type": "n8n-nodes-base.stickyNote",
      "typeVersion": 1,
      "position": [
        -360,
        -160
      ],
      "parameters": {
        "content": "## 03 \u2014 Firecrawl: \u043e\u0434\u0438\u043d URL \u2192 \u0443\u0441\u0442\u043e\u0439\u0447\u0438\u0432\u044b\u0439 \u0430\u043d\u0430\u043b\u0438\u0437\u0430\u0442\u043e\u0440\n\n\u042d\u0442\u043e \u041f\u0415\u0420\u0412\u042b\u0419 \u0440\u0430\u0431\u043e\u0447\u0438\u0439 \u043f\u0440\u043e\u0446\u0435\u0441\u0441 \u0441 \u0440\u0435\u0430\u043b\u044c\u043d\u044b\u043c \u0438\u0441\u0442\u043e\u0447\u043d\u0438\u043a\u043e\u043c.\n\n\u0427\u0442\u043e \u043e\u043d \u0434\u0435\u043b\u0430\u0435\u0442:\n1. \u0421\u043a\u0440\u0430\u043f\u0438\u0442 \u041e\u0414\u041d\u0423 \u043f\u0443\u0431\u043b\u0438\u0447\u043d\u0443\u044e \u0441\u0442\u0440\u0430\u043d\u0438\u0446\u0443 \u043a\u043e\u043d\u043a\u0443\u0440\u0435\u043d\u0442\u0430 \u0447\u0435\u0440\u0435\u0437 Firecrawl (POST /v2/scrape, \u0442\u043e\u043b\u044c\u043a\u043e markdown).\n2. \u041f\u0440\u0435\u0432\u0440\u0430\u0449\u0430\u0435\u0442 markdown \u0432 \u0437\u0430\u043f\u0438\u0441\u044c-\u0438\u0441\u0442\u043e\u0447\u043d\u0438\u043a (text_context \u043e\u0431\u0440\u0435\u0437\u0430\u0435\u0442\u0441\u044f \u0434\u043e 6000 \u0441\u0438\u043c\u0432\u043e\u043b\u043e\u0432 \u0434\u043b\u044f \u044d\u043a\u043e\u043d\u043e\u043c\u0438\u0438).\n3. \u041e\u0442\u043f\u0440\u0430\u0432\u043b\u044f\u0435\u0442 \u0437\u0430\u043f\u0438\u0441\u044c \u0432 \u0442\u043e\u0442 \u0436\u0435 \u0443\u0441\u0442\u043e\u0439\u0447\u0438\u0432\u044b\u0439 \u0430\u043d\u0430\u043b\u0438\u0437\u0430\u0442\u043e\u0440, \u0447\u0442\u043e \u0438 Workflow 02 (Claude \u2192 \u0440\u0430\u0437\u0431\u043e\u0440 JSON \u2192 \u043f\u0440\u0438 \u043e\u0448\u0438\u0431\u043a\u0435 \u0440\u0435\u043c\u043e\u043d\u0442-\u043f\u0440\u043e\u0445\u043e\u0434 \u2192 \u043d\u043e\u0440\u043c\u0430\u043b\u0438\u0437\u0430\u0446\u0438\u044f \u2192 \u043c\u0430\u0440\u0448\u0440\u0443\u0442).\n4. \u041f\u0438\u0448\u0435\u0442 \u0440\u0435\u0437\u0443\u043b\u044c\u0442\u0430\u0442 \u0432 \u043d\u0443\u0436\u043d\u0443\u044e \u0432\u043a\u043b\u0430\u0434\u043a\u0443 Google Sheets (Sheet Name = route).\n\n\u0415\u0441\u043b\u0438 Firecrawl \u0443\u043f\u0430\u043b \u0438\u043b\u0438 \u0432\u0435\u0440\u043d\u0443\u043b \u043f\u0443\u0441\u0442\u043e\u0439/\u043d\u0435\u043f\u0440\u0438\u0433\u043e\u0434\u043d\u044b\u0439 markdown \u2014 \u0437\u0430\u043f\u0438\u0441\u044c \u0438\u0434\u0451\u0442 \u043d\u0430\u043f\u0440\u044f\u043c\u0443\u044e \u0432 technical_errors, \u0411\u0415\u0417 \u0432\u044b\u0437\u043e\u0432\u0430 Claude (\u044d\u043a\u043e\u043d\u043e\u043c\u0438\u044f).\n\n\u041d\u0435 \u0430\u043a\u0442\u0438\u0432\u0438\u0440\u043e\u0432\u0430\u0442\u044c. \u0417\u0430\u043f\u0443\u0441\u043a \u0442\u043e\u043b\u044c\u043a\u043e \u0432\u0440\u0443\u0447\u043d\u0443\u044e. \u0422\u043e\u043b\u044c\u043a\u043e \u041e\u0414\u0418\u041d URL \u0437\u0430 \u0440\u0430\u0437 \u2014 \u043d\u0435 crawl, \u043d\u0435 batch, \u043d\u0435 \u0440\u0430\u0441\u043f\u0438\u0441\u0430\u043d\u0438\u0435.\n\u041e\u0436\u0438\u0434\u0430\u0435\u043c\u043e: \u0441\u0442\u0440\u0430\u043d\u0438\u0446\u0430 \u043a\u043e\u043d\u043a\u0443\u0440\u0435\u043d\u0442\u0430 \u2192 monitor_queue.",
        "height": 360,
        "width": 480,
        "color": 4
      }
    },
    {
      "id": "rr000000-0000-0000-0000-000000000002",
      "name": "Manual Start",
      "type": "n8n-nodes-base.manualTrigger",
      "typeVersion": 1,
      "position": [
        120,
        300
      ],
      "parameters": {}
    },
    {
      "id": "fc000000-0000-0000-0000-00000000000s",
      "name": "Set Firecrawl URL",
      "type": "n8n-nodes-base.set",
      "typeVersion": 3.4,
      "position": [
        320,
        80
      ],
      "parameters": {
        "mode": "manual",
        "assignments": {
          "assignments": [
            {
              "id": "fc-01",
              "name": "target_url",
              "value": "https://example.com",
              "type": "string"
            },
            {
              "id": "fc-02",
              "name": "source_type",
              "value": "scraped_web",
              "type": "string"
            },
            {
              "id": "fc-03",
              "name": "platform",
              "value": "website",
              "type": "string"
            },
            {
              "id": "fc-04",
              "name": "parsed_at",
              "value": "={{ $now.toISO() }}",
              "type": "string"
            },
            {
              "id": "fc-05",
              "name": "source_note",
              "value": "single_url_firecrawl_test",
              "type": "string"
            }
          ]
        }
      }
    },
    {
      "id": "fc000000-0000-0000-0000-000000000002",
      "name": "Build Firecrawl Request",
      "type": "n8n-nodes-base.code",
      "typeVersion": 2,
      "position": [
        520,
        80
      ],
      "parameters": {
        "jsCode": "const url = $json.target_url || $('Set Firecrawl URL').first().json.target_url || '';\nreturn [{ json: {\n  url: url,\n  formats: ['markdown'],\n  onlyMainContent: true,\n  onlyCleanContent: false,\n  removeBase64Images: true,\n  blockAds: true,\n  timeout: 60000,\n  storeInCache: true\n}}];"
      }
    },
    {
      "id": "fc000000-0000-0000-0000-000000000003",
      "name": "Firecrawl Scrape API",
      "type": "n8n-nodes-base.httpRequest",
      "typeVersion": 4.2,
      "position": [
        740,
        80
      ],
      "onError": "continueRegularOutput",
      "parameters": {
        "method": "POST",
        "url": "https://api.firecrawl.dev/v2/scrape",
        "authentication": "predefinedCredentialType",
        "nodeCredentialType": "httpHeaderAuth",
        "sendHeaders": true,
        "headerParameters": {
          "parameters": [
            {
              "name": "Content-Type",
              "value": "application/json"
            }
          ]
        },
        "sendBody": true,
        "contentType": "raw",
        "rawContentType": "application/json",
        "body": "={{ JSON.stringify($json) }}",
        "options": {}
      },
      "credentials": {
        "httpHeaderAuth": {
          "name": "<your credential>"
        }
      }
    },
    {
      "id": "fc000000-0000-0000-0000-000000000004",
      "name": "Normalize Firecrawl Output",
      "type": "n8n-nodes-base.code",
      "typeVersion": 2,
      "position": [
        960,
        80
      ],
      "parameters": {
        "jsCode": "const resp = $json;\nconst cfg = $('Set Firecrawl URL').first().json;\nconst targetUrl = cfg.target_url || '';\nconst now = new Date().toISOString();\nfunction cap(s, n) { return (s == null ? '' : String(s)).substring(0, n); }\n\nfunction technicalErrorRow(errSummary, preview) {\n  return [{ json: {\n    created_at: now, source_type: 'scraped_web', platform: 'website', source_url: targetUrl, parsed_at: now,\n    published_at: '', freshness_status: 'unknown', entity_type: 'irrelevant', company_name: '', profile_name: '',\n    profile_url: '', region: '', service_type: 'unknown', offer_text: '', terms: '', contact_public: '',\n    text_context: '', detected_need: '', competitor_strength: 1, lead_signal_score: 1, content_idea_score: 1,\n    quality_score: 1, reason: '', recommended_action: 'ignore', status: 'skipped',\n    processing_status: 'technical_error', parse_method: 'firecrawl_error',\n    parse_error: cap('Firecrawl scrape failed: ' + errSummary, 800),\n    raw_response_preview: cap(preview, 500), route: 'technical_errors', needs_manual_review: true,\n    repair_used: false, repair_status: ''\n  }}];\n}\n\nconst apiError = (resp == null) || resp.error || resp.success === false || (resp.code && resp.code >= 400) || (resp.statusCode && resp.statusCode >= 400);\nif (apiError) {\n  const summary = cap(JSON.stringify((resp && (resp.error || resp.message || resp.code || resp.statusCode)) || 'unknown error'), 300);\n  return technicalErrorRow(summary, cap(JSON.stringify(resp), 500));\n}\n\nconst data = resp.data || resp;\nlet markdown = (resp.data && resp.data.markdown) || resp.markdown || (data && data.markdown) || (resp.data && resp.data.data && resp.data.data.markdown) || '';\nconst metadata = (resp.data && resp.data.metadata) || resp.metadata || (data && data.metadata) || {};\nmarkdown = String(markdown).replace(/\\r/g, '').replace(/\\n{3,}/g, '\\n\\n').trim();\nconst meaningful = markdown.replace(/[#>*_|]/g, '').replace(/\\s+/g, ' ').trim();\nif (!markdown || meaningful.length < 80) {\n  return technicalErrorRow('scrape succeeded but markdown empty/unusable (' + meaningful.length + ' meaningful chars)', cap(markdown || JSON.stringify(metadata), 500));\n}\n\nconst sourceUrl = metadata.sourceURL || metadata.url || targetUrl;\nconst title = metadata.title || '';\nconst description = metadata.description || '';\n\nreturn [{ json: {\n  route: '', source_type: 'scraped_web', platform: 'website', source_url: sourceUrl, profile_url: '',\n  published_at: '', parsed_at: now, text_context: cap(markdown, 6000),\n  page_title: cap(title, 300), page_description: cap(description, 500)\n}}];"
      }
    },
    {
      "id": "fc000000-0000-0000-0000-000000000005",
      "name": "IF Firecrawl Normalized OK?",
      "type": "n8n-nodes-base.if",
      "typeVersion": 2,
      "position": [
        1180,
        80
      ],
      "parameters": {
        "conditions": {
          "options": {
            "caseSensitive": true,
            "leftValue": "",
            "typeValidation": "loose"
          },
          "conditions": [
            {
              "id": "fc-if-01",
              "leftValue": "={{ $json.route }}",
              "rightValue": "",
              "operator": {
                "type": "string",
                "operation": "empty",
                "singleValue": true
              }
            }
          ],
          "combinator": "and"
        }
      }
    },
    {
      "id": "rr000000-0000-0000-0000-000000000005",
      "name": "Build Primary Claude Request",
      "type": "n8n-nodes-base.code",
      "typeVersion": 2,
      "position": [
        560,
        300
      ],
      "parameters": {
        "jsCode": "const record = $json;\n\nconst systemPrompt = `You are Marketing Scout Agent v2 -- a market intelligence analyst for a secured lending business in Moscow and Moscow Oblast, Russia.\n\nFor every record ask: What does this mean for the operator's business and what should they do? Reason like a business owner.\n\nANALYSIS PRIORITY ORDER:\n1. LEAD SIGNAL first -- potential client needing secured loan? Moscow/MO? PTS/auto/real estate collateral? Urgency? Contactable?\n2. COMPETITOR second -- active secured lending business Moscow/MO? Threat level?\n3. CONTENT IDEA third -- client fear/objection/knowledge gap for secured lending?\n4. IRRELEVANT if none apply.\n\nIDEAL CLIENT: Car owner (PTS clean title), Moscow/MO, needs cash urgently, bank-rejected, 50k-500k RUB.\n\nHigh-urgency signals (raise lead_signal_score): \u0441\u0440\u043e\u0447\u043d\u043e, \u0441\u0435\u0433\u043e\u0434\u043d\u044f, \u0431\u0430\u043d\u043a\u0438 \u043e\u0442\u043a\u0430\u0437\u0430\u043b\u0438, \u043d\u0435 \u0434\u0430\u044e\u0442 \u043a\u0440\u0435\u0434\u0438\u0442, \u0438\u0441\u043f\u043e\u0440\u0447\u0435\u043d\u0430 \u043a\u0440\u0435\u0434\u0438\u0442\u043d\u0430\u044f \u0438\u0441\u0442\u043e\u0440\u0438\u044f + specific amount + collateral type.\n\nREGION RULES:\n- Moscow/MO explicit: lead_signal_score 60-100\n- Region ambiguous: eligible up to 55\n- Another city/region: lead_signal_score capped at 40\nCompetitors: Moscow/MO or national coverage -> score normally; other region only -> cap competitor_strength at 50.\n\nlead_signal_score calibration:\n- 85-100: fit + urgency + readiness + Moscow/MO confirmed\n- 70-84: strong fit+urgency, region confirmed, readiness partial\n- 55-69: clear product fit, one signal confirmed, region present\n- 35-54: intent apparent, fit unclear or region outside MO\n- 1-34: no real lead signal\n\nrecommended_action=contact requires lead_signal_score>=70.\n\ncompetitor_strength calibration:\n- 85-100: fresh (<=30d), Moscow/MO, stated rate, same-day, bad-credit accepted, contactable\n- 65-84: active professional, Moscow/MO confirmed, rate absent or one signal missing\n- 45-64: present but older content or uncertain coverage\n- 25-44: weak - stale or different region\n- 1: not a competitor\n\nSKIP RULES -- return status=skipped, quality_score=1, all scores=1 when:\n- Fewer than 40 meaningful chars\n- Pure navigation boilerplate\n- No connection to financial services\n- published_at >180 days before parsed_at with no fresh signals\n\nREASON FIELD (3 sentences required):\n1. WHAT: what is this record, key evidence from text\n2. WHY: why scores are what they are, cite specific signals\n3. NEXT: what operator should do and why\n\nOUTPUT FORMAT -- CRITICAL:\nRespond with ONLY a valid JSON object.\nNo markdown, no code fences, no preamble.\nFirst character must be {. Last must be }.\nAll 25 fields required. Empty string for unknown strings. 1 for unknown integers.\n\nREQUIRED JSON SCHEMA:\n{\n  \"created_at\": \"<parsed_at value ISO 8601>\",\n  \"source_type\": \"<from input>\",\n  \"platform\": \"<from input>\",\n  \"source_url\": \"<from input>\",\n  \"parsed_at\": \"<from input>\",\n  \"published_at\": \"<from input or empty>\",\n  \"freshness_status\": \"<fresh|recent|old|unknown>\",\n  \"entity_type\": \"<competitor|lead_signal|market_signal|content_idea|irrelevant>\",\n  \"company_name\": \"<explicitly in text only or empty>\",\n  \"profile_name\": \"<explicitly in text only or empty>\",\n  \"profile_url\": \"<from input or empty>\",\n  \"region\": \"<explicitly mentioned or empty>\",\n  \"service_type\": \"<secured_auto_loan|secured_real_estate_loan|pts_loan|refinancing|mortgage_adjacent|generic_lending|unknown>\",\n  \"offer_text\": \"<1 sentence: what offered/sought or content angle title>\",\n  \"terms\": \"<explicit rate/conditions only or empty>\",\n  \"contact_public\": \"<phone/email/Telegram from text only or empty>\",\n  \"text_context\": \"<cleaned summary max 300 chars>\",\n  \"detected_need\": \"<lead_signal only: need+amount+urgency+bank rejection+region or empty>\",\n  \"competitor_strength\": <integer 1-100; 1 if not competitor>,\n  \"lead_signal_score\": <integer 1-100>,\n  \"content_idea_score\": <integer 1-100>,\n  \"quality_score\": <integer 1-100>,\n  \"reason\": \"<3 sentences: what+evidence; why scores; next action>\",\n  \"recommended_action\": \"<monitor|contact|create_content|ignore|investigate>\",\n  \"status\": \"<analyzed|skipped>\"\n}\n\nREMINDER: Return JSON only. No markdown. No analysis outside JSON. For competitor website records, classify entity_type=competitor if the text offers secured lending services, rates, speed, contact, or Moscow/MO coverage.`;\n\nreturn [{ json: {\n  model: 'claude-sonnet-4-6',\n  max_tokens: 1400,\n  temperature: 0.2,\n  system: systemPrompt,\n  messages: [{\n    role: 'user',\n    content: JSON.stringify({\n      source_type: record.source_type || '',\n      platform: record.platform || '',\n      source_url: record.source_url || '',\n      profile_url: record.profile_url || '',\n      parsed_at: record.parsed_at || '',\n      published_at: record.published_at || '',\n      text_context: record.text_context || ''\n    })\n  }]\n}}];"
      }
    },
    {
      "id": "rr000000-0000-0000-0000-000000000007",
      "name": "Claude Primary API Request",
      "type": "n8n-nodes-base.httpRequest",
      "typeVersion": 4.2,
      "position": [
        820,
        300
      ],
      "onError": "continueRegularOutput",
      "parameters": {
        "method": "POST",
        "url": "https://aiprimetech.io/v1/messages",
        "authentication": "predefinedCredentialType",
        "nodeCredentialType": "httpHeaderAuth",
        "sendHeaders": true,
        "headerParameters": {
          "parameters": [
            {
              "name": "anthropic-version",
              "value": "2023-06-01"
            }
          ]
        },
        "sendBody": true,
        "contentType": "raw",
        "rawContentType": "application/json",
        "body": "={{ JSON.stringify({ model: $json.model, max_tokens: $json.max_tokens, temperature: $json.temperature, system: $json.system, messages: $json.messages }) }}",
        "options": {}
      },
      "credentials": {
        "httpHeaderAuth": {
          "name": "<your credential>"
        }
      }
    },
    {
      "id": "rr000000-0000-0000-0000-000000000008",
      "name": "Parse Primary JSON",
      "type": "n8n-nodes-base.code",
      "typeVersion": 2,
      "position": [
        1080,
        300
      ],
      "parameters": {
        "jsCode": "const response = $json;\nconst srcRecord = $('Normalize Firecrawl Output').first().json;\nfunction cap(s, n) { return (s == null ? '' : String(s)).substring(0, n); }\n\n// Handle HTTP error responses \u2014 preserve everything\nif (response.error || response.status >= 400) {\n  const err = 'Primary HTTP error: ' + cap(JSON.stringify(response.error || response.status), 300);\n  const prev = cap(JSON.stringify(response), 500);\n  return [{ json: {\n    parse_ok: false, parse_method: 'primary_json', repair_used: false,\n    processing_status: 'technical_error_candidate',\n    primary_parse_error: err,\n    primary_raw_response_preview: prev,\n    content_summary: 'http_error',\n    parse_error: err,\n    raw_response_preview: prev,\n    original_record: srcRecord\n  }}];\n}\n\n// Find text item\nlet textItem;\nlet contentSummary = '';\ntry {\n  if (!response.content || !Array.isArray(response.content)) throw new Error('no content array');\n  contentSummary = response.content.map(c => c.type).join(',');\n  textItem = response.content.find(c => c.type === 'text');\n} catch(e) {\n  const err = 'Primary parse failed: no content array: ' + e.message;\n  const prev = cap(JSON.stringify(response), 500);\n  return [{ json: {\n    parse_ok: false, parse_method: 'primary_json', repair_used: false,\n    processing_status: 'technical_error_candidate',\n    primary_parse_error: err,\n    primary_raw_response_preview: prev,\n    content_summary: 'no_content_array',\n    parse_error: err,\n    raw_response_preview: prev,\n    original_record: srcRecord\n  }}];\n}\n\nif (!textItem || !textItem.text) {\n  const err = 'Primary parse failed: no text item';\n  const prev = cap(JSON.stringify(response.content || response), 500);\n  return [{ json: {\n    parse_ok: false, parse_method: 'primary_json', repair_used: false,\n    processing_status: 'technical_error_candidate',\n    primary_parse_error: err,\n    primary_raw_response_preview: prev,\n    content_summary: contentSummary || 'no_text_item',\n    parse_error: err,\n    raw_response_preview: prev,\n    original_record: srcRecord\n  }}];\n}\n\nconst rawPreview = cap(textItem.text, 500);\n\n// Parse with defensive cleanup\nlet parsed;\nlet candidate = '';\ntry {\n  let txt = textItem.text.trim();\n  txt = txt.replace(/^```json\\s*/i, '').replace(/\\s*```$/,'').trim();\n  txt = txt.replace(/^```\\s*/,'').replace(/\\s*```$/,'').trim();\n  const bStart = txt.indexOf('{');\n  const bEnd = txt.lastIndexOf('}');\n  if (bStart !== -1 && bEnd !== -1) txt = txt.substring(bStart, bEnd + 1);\n  candidate = txt;\n  txt = txt.replace(/[\u2018\u2019]/g, \"'\").replace(/[\u201c\u201d\u00ab\u00bb]/g, '\"');\n  parsed = JSON.parse(txt);\n} catch(e) {\n  const err = 'Primary JSON parse failed: ' + e.message;\n  const prev = cap(candidate || rawPreview, 500);\n  return [{ json: {\n    parse_ok: false, parse_method: 'primary_json', repair_used: false,\n    processing_status: 'technical_error_candidate',\n    primary_parse_error: err,\n    primary_raw_response_preview: prev,\n    content_summary: 'text_present_json_invalid',\n    parse_error: err,\n    raw_response_preview: prev,\n    original_record: srcRecord\n  }}];\n}\n\nreturn [{ json: {\n  parse_ok: true,\n  parse_method: 'primary_json',\n  repair_used: false,\n  repair_status: '',\n  parse_error: '',\n  raw_response_preview: rawPreview,\n  ...parsed\n}}];"
      }
    },
    {
      "id": "rr000000-0000-0000-0000-000000000009",
      "name": "IF Primary Parse OK?",
      "type": "n8n-nodes-base.if",
      "typeVersion": 2,
      "position": [
        1340,
        300
      ],
      "parameters": {
        "conditions": {
          "options": {
            "caseSensitive": true,
            "leftValue": "",
            "typeValidation": "loose"
          },
          "conditions": [
            {
              "id": "rr000000-0000-0000-0000-000000000091",
              "leftValue": "={{ $json.parse_ok }}",
              "rightValue": true,
              "operator": {
                "type": "boolean",
                "operation": "true"
              }
            }
          ],
          "combinator": "and"
        }
      }
    },
    {
      "id": "rr000000-0000-0000-0000-000000000010",
      "name": "Build Repair Request",
      "type": "n8n-nodes-base.code",
      "typeVersion": 2,
      "position": [
        1340,
        560
      ],
      "parameters": {
        "jsCode": "const input = $json;\nconst srcRecord = $('Normalize Firecrawl Output').first().json;\nfunction cap(s, n) { return (s == null ? '' : String(s)).substring(0, n); }\n\nconst rawPreview = cap(input.primary_raw_response_preview || input.raw_response_preview || 'empty', 500);\nconst primaryErr = cap(input.primary_parse_error || input.parse_error || 'Primary parse failed', 300);\n\nconst repairSystem = `You are a JSON repair formatter, not a market analyst. Convert the raw response into strict JSON. Do not invent facts. If raw response is unusable, return an object that can be routed to technical_errors.\nReturn JSON only. No markdown. First char {, last char }.\nFields: created_at, source_type, platform, source_url, parsed_at, published_at, freshness_status, entity_type, company_name, profile_name, profile_url, region, service_type, offer_text, terms, contact_public, text_context, detected_need, competitor_strength, lead_signal_score, content_idea_score, quality_score, reason, recommended_action, status.\nUnknown text -> \"\". Unknown score -> 1. Unusable -> status=skipped, entity_type=irrelevant, recommended_action=ignore, all scores=1.\nEnums: entity_type[competitor,lead_signal,market_signal,content_idea,irrelevant]; recommended_action[monitor,contact,create_content,ignore,investigate]; status[analyzed,skipped]; freshness_status[fresh,recent,old,unknown].`;\n\nconst userMsg = JSON.stringify({\n  original_record: {\n    source_type: srcRecord.source_type || '',\n    platform: srcRecord.platform || '',\n    source_url: srcRecord.source_url || '',\n    profile_url: srcRecord.profile_url || '',\n    parsed_at: srcRecord.parsed_at || '',\n    published_at: srcRecord.published_at || '',\n    text_context: cap(srcRecord.text_context || '', 500)\n  },\n  raw_primary_response: rawPreview,\n  primary_parse_error: primaryErr\n});\n\nreturn [{ json: {\n  model: 'claude-sonnet-4-6',\n  max_tokens: 700,\n  temperature: 0,\n  system: repairSystem,\n  messages: [{ role: 'user', content: userMsg }]\n}}];"
      }
    },
    {
      "id": "rr000000-0000-0000-0000-000000000011",
      "name": "Claude Repair API Request",
      "type": "n8n-nodes-base.httpRequest",
      "typeVersion": 4.2,
      "position": [
        1620,
        560
      ],
      "onError": "continueRegularOutput",
      "parameters": {
        "method": "POST",
        "url": "https://aiprimetech.io/v1/messages",
        "authentication": "predefinedCredentialType",
        "nodeCredentialType": "httpHeaderAuth",
        "sendHeaders": true,
        "headerParameters": {
          "parameters": [
            {
              "name": "anthropic-version",
              "value": "2023-06-01"
            }
          ]
        },
        "sendBody": true,
        "contentType": "raw",
        "rawContentType": "application/json",
        "body": "={{ JSON.stringify({ model: $json.model, max_tokens: $json.max_tokens, temperature: $json.temperature, system: $json.system, messages: $json.messages }) }}",
        "options": {}
      },
      "credentials": {
        "httpHeaderAuth": {
          "name": "<your credential>"
        }
      }
    },
    {
      "id": "rr000000-0000-0000-0000-000000000012",
      "name": "Parse Repaired JSON",
      "type": "n8n-nodes-base.code",
      "typeVersion": 2,
      "position": [
        1900,
        560
      ],
      "parameters": {
        "jsCode": "const response = $json;\nconst primary = $('Parse Primary JSON').first().json;\nfunction cap(s, n) { return (s == null ? '' : String(s)).substring(0, n); }\n\nconst primaryErr = cap(primary.primary_parse_error || primary.parse_error || 'unknown', 300);\nconst primaryPreview = cap(primary.primary_raw_response_preview || primary.raw_response_preview || '', 500);\n\nfunction technicalError(repairErr) {\n  const combined = cap('Primary: ' + primaryErr + ' | Repair: ' + repairErr, 800);\n  // preserve primary raw preview first; append repair error only if space allows\n  let preview = primaryPreview;\n  if (preview.length < 440) {\n    preview = cap(preview + ' || REPAIR_ERR: ' + repairErr, 500);\n  }\n  return [{ json: {\n    parse_ok: false,\n    parse_method: 'technical_error',\n    repair_used: true,\n    repair_status: 'failed',\n    processing_status: 'technical_error',\n    route: 'technical_errors',\n    needs_manual_review: true,\n    parse_error: combined,\n    raw_response_preview: cap(preview, 500),\n    original_record: primary.original_record || {}\n  }}];\n}\n\n// Handle HTTP error from repair call\nif (response.error || response.status >= 400) {\n  return technicalError('Repair HTTP error: ' + cap(JSON.stringify(response.error || response.status), 300));\n}\n\nlet textItem;\ntry {\n  textItem = (response.content || []).find(c => c.type === 'text');\n} catch(e) { textItem = null; }\n\nif (!textItem || !textItem.text) {\n  return technicalError('No text item in repair response');\n}\n\nconst rawPreview = cap(textItem.text, 500);\n\nlet parsed;\ntry {\n  let txt = textItem.text.trim();\n  txt = txt.replace(/^```json\\s*/i,'').replace(/\\s*```$/,'').trim();\n  txt = txt.replace(/^```\\s*/,'').replace(/\\s*```$/,'').trim();\n  const bStart = txt.indexOf('{'); const bEnd = txt.lastIndexOf('}');\n  if (bStart !== -1 && bEnd !== -1) txt = txt.substring(bStart, bEnd + 1);\n  txt = txt.replace(/[\u2018\u2019]/g,\"'\").replace(/[\u201c\u201d\u00ab\u00bb]/g,'\"');\n  parsed = JSON.parse(txt);\n} catch(e) {\n  return technicalError('Repair JSON parse failed: ' + e.message);\n}\n\nreturn [{ json: {\n  parse_ok: true,\n  parse_method: 'repaired_json',\n  repair_used: true,\n  repair_status: 'success',\n  parse_error: '',\n  raw_response_preview: rawPreview,\n  ...parsed\n}}];"
      }
    },
    {
      "id": "rr000000-0000-0000-0000-000000000013",
      "name": "Normalize + Route",
      "type": "n8n-nodes-base.code",
      "typeVersion": 2,
      "position": [
        2160,
        300
      ],
      "parameters": {
        "jsCode": "// embedded n8n/lib/error_sanitizer.js (drift-proof; test asserts equality)\n// error_sanitizer.js \u2014 what may be persisted about a FAILED record (technical_errors / skipped_log routes).\n//\n// WF04-ROUTE-002. The resilient routers persist `raw_response_preview` + `parse_error` for triage. Those strings are\n// built from a provider response, so without a gate they can carry an Authorization header, an api key, a cookie, a\n// Claude thinking block, or a customer's phone/email straight into a durable Google Sheets tab that many people can\n// open. Diagnosis needs a bounded, scrubbed EXCERPT \u2014 never the raw body.\n//\n// Keep: provider, source URL, safe error category, bounded sanitized excerpt, request/run lineage, timestamp.\n// Drop: secrets, credentials, cookies, hidden reasoning, private PII, anything past the cap.\n//\n// Embeddable: unique es*-prefixed names, no cross-lib require.\n\nfunction esStr(v) { return v == null ? '' : String(v); }\n\nvar ES_MAX_PREVIEW = 300;      // enough to recognise a failure shape; far too short to be a \"raw body\"\nvar ES_REDACTED = '[\u0441\u043a\u0440\u044b\u0442\u043e]';\n\n// Ordered: the most specific secret shapes first, then generic key/value pairs, then PII.\nvar ES_RULES = [\n  // Authorization / bearer / api-key headers (with or without a header name)\n  { re: /(authorization|proxy-authorization)\\s*[:=]\\s*\\S+/gi, to: '$1: ' + ES_REDACTED },\n  { re: /\\bbearer\\s+[A-Za-z0-9._\\-~+/]{8,}=*/gi, to: 'bearer ' + ES_REDACTED },\n  { re: /\\bbasic\\s+[A-Za-z0-9+/]{8,}=*/gi, to: 'basic ' + ES_REDACTED },\n  // cookies / set-cookie\n  { re: /(set-cookie|cookie)\\s*[:=]\\s*[^\\n;]+/gi, to: '$1: ' + ES_REDACTED },\n  // provider key formats (Anthropic sk-ant-\u2026, generic sk-\u2026, Google AIza\u2026, GitHub gh[pousr]_\u2026)\n  { re: /\\bsk-ant-[A-Za-z0-9._\\-]{8,}/g, to: ES_REDACTED },\n  { re: /\\bsk-[A-Za-z0-9._\\-]{16,}/g, to: ES_REDACTED },\n  { re: /\\bAIza[0-9A-Za-z._\\-]{20,}/g, to: ES_REDACTED },\n  { re: /\\bgh[pousr]_[A-Za-z0-9]{20,}/g, to: ES_REDACTED },\n  // JWTs\n  { re: /\\beyJ[A-Za-z0-9._\\-]{10,}\\.[A-Za-z0-9._\\-]{10,}\\.[A-Za-z0-9._\\-]{4,}/g, to: ES_REDACTED },\n  // generic \"<something>key|token|secret|password\" : \"<value>\" (JSON or header style)\n  { re: /(\"?\\b[\\w.\\-]*(?:api[_-]?key|access[_-]?token|refresh[_-]?token|token|secret|password|passwd|pwd|credential)\\b\"?)\\s*[:=]\\s*\"?[^\"\\s,}{&]+\"?/gi, to: '$1: ' + ES_REDACTED },\n  // url query secrets: ?key=\u2026 &token=\u2026\n  { re: /([?&](?:key|token|api_key|apikey|access_token|secret|password)=)[^&\\s]+/gi, to: '$1' + ES_REDACTED },\n  // private PII \u2014 we analyze public positioning, never harvest contacts\n  { re: /[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}/g, to: '[email]' },\n  { re: /\\+?\\d[\\d\\s().-]{9,}\\d/g, to: '[\u0442\u0435\u043b]' }\n];\n\n// Strip a model's hidden reasoning: it is never persisted, in any shape.\nfunction esStripThinking(s) {\n  return esStr(s)\n    .replace(/<thinking>[\\s\\S]*?<\\/thinking>/gi, ' ')\n    .replace(/<thinking>[\\s\\S]*?<\\/antml:thinking>/gi, ' ')\n    .replace(/\"type\"\\s*:\\s*\"thinking\"[\\s\\S]*?(?=[,}]\\s*\"type\"|$)/gi, ' ')\n    .replace(/\\bthinking\\s*[:=]\\s*\"[^\"]*\"/gi, ' ');\n}\n\n// sanitizeErrorPreview(raw, opts) -> a bounded, secret-free, PII-free excerpt safe to persist.\nfunction sanitizeErrorPreview(raw, opts) {\n  opts = opts || {};\n  var max = Number(opts.max_chars);\n  if (!isFinite(max) || max <= 0) max = ES_MAX_PREVIEW;\n  var s = esStripThinking(raw);\n  for (var i = 0; i < ES_RULES.length; i++) s = s.replace(ES_RULES[i].re, ES_RULES[i].to);\n  s = s.replace(/\\s+/g, ' ').trim();\n  if (s.length > max) s = s.slice(0, max - 1) + '\u2026';\n  return s;\n}\n\n// Does a string still look like it carries a secret? Used as a fail-closed assertion before persisting.\nfunction esLooksSecret(s) {\n  s = esStr(s);\n  return /\\b(sk-ant-|sk-[A-Za-z0-9]{16,}|AIza[0-9A-Za-z]{20,}|gh[pousr]_[A-Za-z0-9]{20,}|eyJ[A-Za-z0-9._-]{10,}\\.)/.test(s) ||\n    /(authorization|set-cookie)\\s*[:=]\\s*(?!\\[\u0441\u043a\u0440\u044b\u0442\u043e\\])\\S+/i.test(s);\n}\n\n// The ONLY diagnostic fields a failed record may persist. Anything not listed here is dropped by construction.\nfunction sanitizeErrorRecord(rec, opts) {\n  rec = rec || {};\n  return {\n    provider: esStr(rec.provider),\n    source_url: esStr(rec.source_url),\n    error_category: esStr(rec.error_category || rec.processing_status),\n    parse_error: sanitizeErrorPreview(rec.parse_error, { max_chars: (opts && opts.max_chars) || ES_MAX_PREVIEW }),\n    raw_response_preview: sanitizeErrorPreview(rec.raw_response_preview, { max_chars: (opts && opts.max_chars) || ES_MAX_PREVIEW }),\n    agent_request_id: esStr(rec.agent_request_id),\n    source_run_id: esStr(rec.source_run_id),\n    run_id: esStr(rec.run_id),\n    created_at: esStr(rec.created_at)\n  };\n}\n// --- end embedded error_sanitizer ---\n\nconst data = $json;\nconst srcRecord = $('Normalize Firecrawl Output').first().json;\n\nfunction clamp(v) { const n = parseInt(v); return isNaN(n) ? 1 : Math.min(100, Math.max(1, n)); }\nfunction truncate(s, n) { return (s == null ? '' : String(s)).substring(0, n); }\n\n// Normalize free-text service_type into allowed enum values\nfunction normalizeServiceType(raw, hay) {\n  const allowed = ['secured_auto_loan','secured_real_estate_loan','pts_loan','refinancing','mortgage_adjacent','generic_lending'];\n  const r = (raw || '').toLowerCase().trim();\n  if (allowed.includes(r)) return r;\n  const s = r + ' ' + (hay || '');\n  if (s.includes('\u043f\u0442\u0441') || s.includes('pts')) return 'pts_loan';\n  if ((s.includes('\u0430\u0432\u0442\u043e') || s.includes('\u043c\u0430\u0448\u0438\u043d')) && (s.includes('\u0437\u0430\u043b\u043e\u0433') || s.includes('collateral') || s.includes('\u043b\u043e\u043c\u0431\u0430\u0440\u0434'))) return 'secured_auto_loan';\n  if (s.includes('\u043d\u0435\u0434\u0432\u0438\u0436') || s.includes('\u043a\u0432\u0430\u0440\u0442\u0438\u0440') || s.includes('\u0434\u043e\u043c') || s.includes('\u0437\u0435\u043c\u043b')) return 'secured_real_estate_loan';\n  if (s.includes('\u0440\u0435\u0444\u0438\u043d\u0430\u043d\u0441') || s.includes('refinanc')) return 'refinancing';\n  if (s.includes('\u0438\u043f\u043e\u0442\u0435\u043a') || s.includes('mortgage')) return 'mortgage_adjacent';\n  if (s.includes('\u0431\u0438\u0437\u043d\u0435\u0441') || s.includes('business')) {\n    if (s.includes('\u043d\u0435\u0434\u0432\u0438\u0436') || s.includes('\u043a\u0432\u0430\u0440\u0442\u0438\u0440')) return 'secured_real_estate_loan';\n    return 'generic_lending';\n  }\n  return 'unknown';\n}\n\n// Descriptive company_name fallback for competitors (never invent a brand)\nfunction companyNameFallback(existing, entity, hay) {\n  if (existing && String(existing).trim() !== '') return existing;\n  if (entity === 'competitor' && (hay.includes('\u043c\u0444\u043e') || hay.includes('\u043c\u0438\u043a\u0440\u043e\u0444\u0438\u043d\u0430\u043d\u0441') || hay.includes('mfo'))) return '\u041c\u0424\u041e / \u0447\u0430\u0441\u0442\u043d\u044b\u0439 \u043a\u0440\u0435\u0434\u0438\u0442\u043e\u0440';\n  if (hay.includes('\u0447\u0430\u0441\u0442\u043d\u044b\u0439 \u0438\u043d\u0432\u0435\u0441\u0442\u043e\u0440')) return '\u0427\u0430\u0441\u0442\u043d\u044b\u0439 \u0438\u043d\u0432\u0435\u0441\u0442\u043e\u0440';\n  if (hay.includes('\u0430\u0432\u0442\u043e\u043b\u043e\u043c\u0431\u0430\u0440\u0434')) return '\u0410\u0432\u0442\u043e\u043b\u043e\u043c\u0431\u0430\u0440\u0434';\n  if (hay.includes('\u0431\u0440\u043e\u043a\u0435\u0440')) return '\u0411\u0440\u043e\u043a\u0435\u0440';\n  if (entity === 'competitor') return '\u041a\u043e\u043d\u043a\u0443\u0440\u0435\u043d\u0442 \u0431\u0435\u0437 \u0431\u0440\u0435\u043d\u0434\u0430';\n  return '';\n}\n\n// recommended_action normalization driven by final route\nfunction normalizeAction(route, entity, action, leadScore) {\n  if (route === 'technical_errors') return 'ignore';\n  if (route === 'skipped_log') return 'ignore';\n  if (route === 'results' && entity === 'lead_signal') return 'contact';\n  if (route === 'review_queue') {\n    if (entity === 'lead_signal' && action === 'contact' && leadScore >= 70) return 'contact';\n    return 'investigate';\n  }\n  if (route === 'monitor_queue' && entity === 'competitor') return 'monitor';\n  if (route === 'content_queue') {\n    if (action === 'contact' || action === 'investigate') return action;\n    return 'create_content';\n  }\n  return action;\n}\n\n// Pass-through for confirmed technical_error (from Parse Repaired JSON)\nif (data.processing_status === 'technical_error' && data.route === 'technical_errors') {\n  return [{ json: {\n    created_at: data.created_at || srcRecord.parsed_at || '',\n    source_type: data.source_type || srcRecord.source_type || '',\n    platform: data.platform || srcRecord.platform || '',\n    source_url: data.source_url || srcRecord.source_url || '',\n    parsed_at: data.parsed_at || srcRecord.parsed_at || '',\n    published_at: data.published_at || srcRecord.published_at || '',\n    freshness_status: data.freshness_status || 'unknown',\n    entity_type: data.entity_type || 'irrelevant',\n    company_name: data.company_name || '',\n    profile_name: data.profile_name || '',\n    profile_url: data.profile_url || srcRecord.profile_url || '',\n    region: data.region || '',\n    service_type: data.service_type || 'unknown',\n    offer_text: data.offer_text || '',\n    terms: data.terms || '',\n    contact_public: data.contact_public || '',\n    text_context: data.text_context || srcRecord.text_context || '',\n    detected_need: data.detected_need || '',\n    competitor_strength: clamp(data.competitor_strength),\n    lead_signal_score: clamp(data.lead_signal_score),\n    content_idea_score: clamp(data.content_idea_score),\n    quality_score: clamp(data.quality_score),\n    reason: data.reason || '',\n    recommended_action: 'ignore',\n    status: data.status || 'analyzed',\n    processing_status: 'technical_error',\n    parse_method: data.parse_method || 'technical_error',\n    parse_error: sanitizeErrorPreview(data.parse_error || ''),\n    raw_response_preview: sanitizeErrorPreview(data.raw_response_preview),\n    route: 'technical_errors',\n    needs_manual_review: true,\n    repair_used: data.repair_used || false,\n    repair_status: data.repair_status || ''\n  }}];\n}\n\n// Normalize schema fields\nconst entity = data.entity_type || 'irrelevant';\nconst status = data.status || 'analyzed';\nconst recAction = data.recommended_action || 'ignore';\nconst leadScore = clamp(data.lead_signal_score);\nconst compScore = clamp(data.competitor_strength);\nconst contentScore = clamp(data.content_idea_score);\nconst qualScore = clamp(data.quality_score);\n\nconst validEntities = ['competitor','lead_signal','market_signal','content_idea','irrelevant'];\nconst validActions = ['monitor','contact','create_content','ignore','investigate'];\nconst safeEntity = validEntities.includes(entity) ? entity : 'irrelevant';\nconst safeAction = validActions.includes(recAction) ? recAction : 'ignore';\nconst safeStatus = (status === 'skipped') ? 'skipped' : 'analyzed';\n\n// Derived text haystacks for normalization and routing keyword checks\nconst hay = ((data.text_context||'') + ' ' + (data.offer_text||'') + ' ' + (data.detected_need||'') + ' ' + (data.reason||'') + ' ' + (data.service_type||'') + ' ' + (srcRecord.text_context||'')).toLowerCase();\nconst companyHay = ((data.text_context||'') + ' ' + (data.reason||'') + ' ' + (data.offer_text||'') + ' ' + (data.source_url||srcRecord.source_url||'') + ' ' + (data.detected_need||'')).toLowerCase();\n\nconst safeServiceType = normalizeServiceType(data.service_type, hay);\nconst safeCompanyName = companyNameFallback(data.company_name, safeEntity, companyHay);\n\n// Determine processing_status\nlet procStatus = 'parsed_success';\nif (safeStatus === 'skipped' || safeEntity === 'irrelevant') procStatus = 'business_skip';\n\n// === Post-repair business-consistency hardening (DEC-043 / DEC-044) ===\n// Repaired JSON is structurally valid but not trusted for business scores/language.\nfunction _hasCyrillic(s){ return /[\\u0400-\\u04FF]/.test(s || ''); }\nfunction _hasCJK(s){ return /[\\u3400-\\u9FFF\\uF900-\\uFAFF]/.test(s || ''); }\n\nconst srcTypeLc = (data.source_type || srcRecord.source_type || '').toLowerCase();\nconst platformLc = (data.platform || srcRecord.platform || '').toLowerCase();\nconst isWebsiteScrape = (srcTypeLc === 'scraped_web') && (platformLc === 'website');\nconst evidence = ((data.text_context||'') + ' ' + (data.offer_text||'') + ' ' + (data.terms||'') + ' ' + (data.reason||'') + ' ' + (srcRecord.text_context||'')).toLowerCase();\nconst hasParseError = !!(data.parse_error && String(data.parse_error).trim() !== '');\nconst textUsable = ((srcRecord.text_context || data.text_context || '').trim().length >= 80);\n\n// Competitor signal counting (B)\nconst signalTerms = ['\u043a\u0440\u0435\u0434\u0438\u0442','\u0437\u0430\u0439\u043c','\u0437\u0430\u043b\u043e\u0433','\u043f\u0442\u0441','\u0430\u0432\u0442\u043e','\u043d\u0435\u0434\u0432\u0438\u0436','\u0440\u0435\u0444\u0438\u043d\u0430\u043d\u0441','\u0438\u043f\u043e\u0442\u0435\u043a','\u0441\u0442\u0430\u0432\u043a\u0430','\u0441\u0443\u043c\u043c\u0430','\u043e\u0434\u043e\u0431\u0440\u0435\u043d','\u043f\u043b\u043e\u0445\u0430\u044f \u043a\u0440\u0435\u0434\u0438\u0442\u043d','\u043f\u0440\u043e\u0441\u0440\u043e\u0447\u043a','\u0442\u0435\u043b\u0435\u0444\u043e\u043d','\u043c\u043e\u0441\u043a\u0432\u0430','\u043c\u043e\u0441\u043a\u043e\u0432\u0441\u043a'];\nlet compSignals = 0;\nfor (const t of signalTerms) { if (evidence.includes(t)) compSignals++; }\n\n// Effective (hardened) values default to the normalized values\nlet effAction = safeAction;\nlet effComp = compScore;\nlet effQual = qualScore;\nlet effServiceType = safeServiceType;\n\nconst isCompetitorWebsite = (safeEntity === 'competitor' && isWebsiteScrape);\nconst richCompetitor = isCompetitorWebsite && compSignals >= 3;\n\n// B + C: competitor consistency rule (applies even when repaired output under-scored)\nif (isCompetitorWebsite && compSignals >= 3) {\n  effComp = Math.max(effComp, 65);\n  effQual = Math.max(effQual, 65);\n  effAction = 'monitor';\n}\nif (isCompetitorWebsite && compSignals >= 5) {\n  effComp = Math.max(effComp, 75);\n  effQual = Math.max(effQual, 75);\n}\n// C: repaired competitor scores are unreliable \u2014 never below 45 unless text unusable\nif (data.repair_used === true && safeEntity === 'competitor' && textUsable) {\n  effComp = Math.max(effComp, 45);\n}\n\n// D: multi-product website \u2192 generic_lending unless one product dominates\nconst prodCats = [];\nif (evidence.includes('\u0437\u0430\u043b\u043e\u0433') && (evidence.includes('\u043d\u0435\u0434\u0432\u0438\u0436')||evidence.includes('\u043a\u0432\u0430\u0440\u0442\u0438\u0440')||evidence.includes('\u0434\u043e\u043c ')||evidence.includes('\u0437\u0435\u043c\u043b')||evidence.includes('\u043a\u043e\u043c\u043c\u0435\u0440\u0447\u0435\u0441\u043a'))) prodCats.push('real_estate');\nif (evidence.includes('\u0440\u0435\u0444\u0438\u043d\u0430\u043d\u0441')) prodCats.push('refinancing');\nif (evidence.includes('\u0438\u043f\u043e\u0442\u0435\u043a')) prodCats.push('mortgage');\nif ((evidence.includes('\u0430\u0432\u0442\u043e')||evidence.includes('\u043c\u0430\u0448\u0438\u043d')) && (evidence.includes('\u0437\u0430\u043b\u043e\u0433')||evidence.includes('\u043f\u0442\u0441')||evidence.includes('pts'))) prodCats.push('auto');\nif (isWebsiteScrape && prodCats.length >= 2) {\n  effServiceType = 'generic_lending';\n}\n\n// A: language guard \u2014 Russian source but reason in Chinese / mostly foreign script\nfunction _russianFallbackReason(entity, company, service, terms, region, lead, comp, qual) {\n  if (entity === 'competitor') {\n    return '\u0421\u0442\u0440\u0430\u043d\u0438\u0446\u0430 \u0441\u043e\u0434\u0435\u0440\u0436\u0438\u0442 \u043f\u0440\u0438\u0437\u043d\u0430\u043a\u0438 \u043a\u043e\u043d\u043a\u0443\u0440\u0435\u043d\u0442\u0430 \u0432 \u043d\u0438\u0448\u0435 \u0437\u0430\u043b\u043e\u0433\u043e\u0432\u043e\u0433\u043e \u043a\u0440\u0435\u0434\u0438\u0442\u043e\u0432\u0430\u043d\u0438\u044f: \u0443\u043a\u0430\u0437\u0430\u043d\u044b \u043f\u0440\u043e\u0434\u0443\u043a\u0442\u044b, \u0443\u0441\u043b\u043e\u0432\u0438\u044f/\u0441\u0442\u0430\u0432\u043a\u0438, \u0440\u0435\u0433\u0438\u043e\u043d \u0438\u043b\u0438 \u043a\u043e\u043d\u0442\u0430\u043a\u0442. \u0417\u0430\u043f\u0438\u0441\u044c \u043e\u0442\u043f\u0440\u0430\u0432\u043b\u0435\u043d\u0430 \u0432 \u043c\u043e\u043d\u0438\u0442\u043e\u0440\u0438\u043d\u0433 \u043a\u043e\u043d\u043a\u0443\u0440\u0435\u043d\u0442\u043e\u0432 \u0434\u043b\u044f \u043f\u0440\u043e\u0432\u0435\u0440\u043a\u0438 \u043e\u0444\u0444\u0435\u0440\u043e\u0432 \u0438 \u0443\u0441\u043b\u043e\u0432\u0438\u0439.';\n  }\n  const parts = ['\u0410\u0432\u0442\u043e\u043c\u0430\u0442\u0438\u0447\u0435\u0441\u043a\u0438 \u0441\u0444\u043e\u0440\u043c\u0438\u0440\u043e\u0432\u0430\u043d\u043d\u043e\u0435 \u043e\u043f\u0438\u0441\u0430\u043d\u0438\u0435 (\u0438\u0441\u0445\u043e\u0434\u043d\u044b\u0439 reason \u0431\u044b\u043b \u043d\u0435 \u043d\u0430 \u0440\u0443\u0441\u0441\u043a\u043e\u043c).'];\n  parts.push('\u0422\u0438\u043f \u0437\u0430\u043f\u0438\u0441\u0438: ' + entity + (company ? ', ' + company : '') + (service && service !== 'unknown' ? ', \u0443\u0441\u043b\u0443\u0433\u0430: ' + service : '') + '.');\n  if (region) parts.push('\u0420\u0435\u0433\u0438\u043e\u043d: ' + region + '.');\n  if (terms) parts.push('\u0423\u0441\u043b\u043e\u0432\u0438\u044f: ' + truncate(terms, 120) + '.');\n  parts.push('\u041e\u0446\u0435\u043d\u043a\u0438 \u2014 \u043b\u0438\u0434: ' + lead + ', \u043a\u043e\u043d\u043a\u0443\u0440\u0435\u043d\u0442: ' + comp + ', \u043a\u0430\u0447\u0435\u0441\u0442\u0432\u043e: ' + qual + '.');\n  return parts.join(' ');\n}\nlet finalReason = data.reason || '';\nconst srcHasCyrillic = _hasCyrillic(srcRecord.text_context || data.text_context || evidence);\nconst reasonLetters = (finalReason.match(/[A-Za-z\\u0400-\\u04FF]/g) || []).length;\nconst reasonNonSpace = (finalReason.match(/\\S/g) || []).length;\nconst reasonForeign = reasonNonSpace >= 8 && (reasonLetters / reasonNonSpace) < 0.4;\nif (srcHasCyrillic && finalReason && (_hasCJK(finalReason) || reasonForeign || (!_hasCyrillic(finalReason) && reasonLetters === 0))) {\n  finalReason = _russianFallbackReason(safeEntity, safeCompanyName, effServiceType, data.terms || '', data.region || '', leadScore, effComp, effQual);\n}\n\n// Weak / potential lead detection (must run before content_queue routing)\nconst srcType = (data.source_type || srcRecord.source_type || '').toLowerCase();\nconst isSocialOrClassified = (srcType === 'social' || srcType === 'classified');\nconst productTerms = ['loan','collateral','pts','auto','real estate','refinanc','\u0437\u0430\u0439\u043c','\u043a\u0440\u0435\u0434\u0438\u0442','\u0437\u0430\u043b\u043e\u0433','\u043f\u0442\u0441','\u0430\u0432\u0442\u043e','\u043c\u0430\u0448\u0438\u043d','\u043d\u0435\u0434\u0432\u0438\u0436','\u043a\u0432\u0430\u0440\u0442\u0438\u0440','\u0437\u0435\u043c\u043b','\u0440\u0435\u0444\u0438\u043d\u0430\u043d\u0441','\u0438\u043f\u043e\u0442\u0435\u043a'];\nconst mentionsProduct = productTerms.some(t => hay.includes(t));\n\nconst weakLead = (\n  (safeEntity === 'lead_signal' && leadScore >= 30 && leadScore <= 69) ||\n  (effAction === 'investigate') ||\n  (leadScore >= 30 && isSocialOrClassified && mentionsProduct) ||\n  (safeEntity === 'content_idea' && leadScore >= 30 && isSocialOrClassified && safeServiceType !== 'unknown')\n);\n\n// Routing priority (F, after consistency hardening \u2014 DEC-043):\n// 1 technical_errors (handled above) 2 skipped/irrelevant 3 hot lead\n// 4 rich competitor website (never review_queue, rule C) 5 weak lead\n// 6 competitor>=45 7 pure content idea 8 fallback review_queue\nlet route;\nlet needsReview;\n\nif (safeStatus === 'skipped' || safeEntity === 'irrelevant') {\n  route = 'skipped_log'; needsReview = false;\n} else if (safeEntity === 'lead_signal' && leadScore >= 70 && effAction === 'contact') {\n  route = 'results'; needsReview = false;\n} else if (richCompetitor) {\n  route = 'monitor_queue'; needsReview = hasParseError;\n} else if (weakLead) {\n  route = 'review_queue'; needsReview = true;\n} else if (safeEntity === 'competitor' && effComp >= 45) {\n  route = 'monitor_queue'; needsReview = false;\n} else if (safeEntity === 'content_idea' && contentScore >= 50) {\n  route = 'content_queue'; needsReview = true;\n} else {\n  route = 'review_queue'; needsReview = true;\n}\n\n// Safety: enforce route is exactly one of the six valid sheet tabs\nconst validRoutes = ['results','review_queue','monitor_queue','content_queue','skipped_log','technical_errors'];\nlet parseError = data.parse_error || '';\nif (!validRoutes.includes(route)) {\n  route = 'technical_errors';\n  procStatus = 'technical_error';\n  needsReview = true;\n  parseError = (parseError ? parseError + '; ' : '') + 'invalid_route';\n}\n\n// Final recommended_action consistent with the route\nconst finalAction = normalizeAction(route, safeEntity, effAction, leadScore);\n\n// v0.1 dedup hint: source_url is the first dedup key. Real scraper workflow\n// should check existing source_url in the target tab before append. No dedup\n// column is emitted yet (see DEC-037 / TABLE_SCHEMA notes).\n\nreturn [{ json: {\n  created_at: data.created_at || srcRecord.parsed_at || '',\n  source_type: data.source_type || srcRecord.source_type || '',\n  platform: data.platform || srcRecord.platform || '',\n  source_url: data.source_url || srcRecord.source_url || '',\n  parsed_at: data.parsed_at || srcRecord.parsed_at || '',\n  published_at: data.published_at || srcRecord.published_at || '',\n  freshness_status: data.freshness_status || 'unknown',\n  entity_type: safeEntity,\n  company_name: safeCompanyName,\n  profile_name: data.profile_name || '',\n  profile_url: data.profile_url || srcRecord.profile_url || '',\n  region: data.region || '',\n  service_type: effServiceType,\n  offer_text: data.offer_text || '',\n  terms: data.terms || '',\n  contact_public: data.contact_public || '',\n  text_context: data.text_context || '',\n  detected_need: data.detected_need || '',\n  competitor_strength: effComp,\n  lead_signal_score: leadScore,\n  content_idea_score: contentScore,\n  quality_score: effQual,\n  reason: finalReason,\n  recommended_action: finalAction,\n  status: safeStatus,\n  processing_status: procStatus,\n  parse_method: data.parse_method || 'primary_json',\n  parse_error: sanitizeErrorPreview(parseError),\n  raw_response_preview: sanitizeErrorPreview(data.raw_response_preview),\n  route: route,\n  needs_manual_review: needsReview,\n  repair_used: data.repair_used || false,\n  repair_status: data.repair_status || ''\n}}];"
      }
    },
    {
      "id": "rr000000-0000-0000-0000-000000000099",
      "name": "Append to Dynamic Route Sheet",
      "type": "n8n-nodes-base.googleSheets",
      "typeVersion": 4,
      "position": [
        2440,
        300
      ],
      "parameters": {
        "authentication": "serviceAccount",
        "operation": "append",
        "documentId": {
          "__rl": true,
          "value": "PASTE_SPREADSHEET_ID_HERE",
          "mode": "id"
        },
        "sheetName": {
          "__rl": true,
          "value": "={{ $json.route }}",
          "mode": "name"
        },
        "columns": {
          "mappingMode": "autoMapInputData",
          "value": {},
          "matchingColumns": [],
          "schema": []
        },
        "options": {}
      },
      "credentials": {
        "googleApi": {
          "name": "<your credential>"
        }
      },
      "retryOnFail": true,
      "maxTries": 3,
      "waitBetweenTries": 5000,
      "onError": "continueRegularOutput",
      "alwaysOutputData": true
    },
    {
      "id": "fc000000-0000-0000-0000-0000000000n2",
      "name": "Test Instructions RU",
      "type": "n8n-nodes-base.stickyNote",
      "typeVersion": 1,
      "position": [
        2640,
        -160
      ],
      "parameters": {
        "content": "## \u0422\u0435\u0441\u0442 (\u0432\u0440\u0443\u0447\u043d\u0443\u044e, \u043e\u0434\u0438\u043d \u0440\u0430\u0437)\n\n1. \u041d\u0415 \u0430\u043a\u0442\u0438\u0432\u0438\u0440\u043e\u0432\u0430\u0442\u044c workflow.\n2. \u0421\u043e\u0437\u0434\u0430\u0442\u044c \u043a\u0440\u0435\u0434\u0435\u043d\u0448\u043b Firecrawl: Header Auth \u2014 Authorization = Bearer <FIRECRAWL_API_KEY>, \u0434\u043e\u043c\u0435\u043d api.firecrawl.dev. \u041f\u0440\u0438\u0432\u044f\u0437\u0430\u0442\u044c \u043a 'Firecrawl Scrape API'.\n3. \u041f\u0440\u0438\u0432\u044f\u0437\u0430\u0442\u044c \u043a\u0440\u0435\u0434\u0435\u043d\u0448\u043b Claude \u043a \u043e\u0431\u043e\u0438\u043c HTTP-\u043d\u043e\u0434\u0430\u043c Claude \u0438 Google Sheets \u043a 'Append to Dynamic Route Sheet' + \u0432\u0441\u0442\u0430\u0432\u0438\u0442\u044c \u0440\u0435\u0430\u043b\u044c\u043d\u044b\u0439 Spreadsheet ID.\n4. \u0423\u0431\u0435\u0434\u0438\u0442\u044c\u0441\u044f, \u0447\u0442\u043e \u0435\u0441\u0442\u044c 6 \u0432\u043a\u043b\u0430\u0434\u043e\u043a \u0441 33-\u043a\u043e\u043b\u043e\u043d\u043e\u0447\u043d\u044b\u043c \u0437\u0430\u0433\u043e\u043b\u043e\u0432\u043a\u043e\u043c (docs/TABLE_SCHEMA.md).\n5. \u0412\u0441\u0442\u0430\u0432\u0438\u0442\u044c \u041e\u0414\u0418\u041d \u043f\u0443\u0431\u043b\u0438\u0447\u043d\u044b\u0439 URL \u043a\u043e\u043d\u043a\u0443\u0440\u0435\u043d\u0442\u0430 \u0432 'Set Firecrawl URL' \u2192 target_url.\n6. \u0417\u0430\u043f\u0438\u0441\u0430\u0442\u044c \u0431\u0430\u043b\u0430\u043d\u0441 Firecrawl \u0438 Claude \u0414\u041e.\n7. \u0417\u0430\u043f\u0443\u0441\u0442\u0438\u0442\u044c \u0432\u0440\u0443\u0447\u043d\u0443\u044e \u043e\u0434\u0438\u043d \u0440\u0430\u0437.\n8. \u0417\u0430\u043f\u0438\u0441\u0430\u0442\u044c \u0431\u0430\u043b\u0430\u043d\u0441 \u041f\u041e\u0421\u041b\u0415.\n\n\u041e\u0436\u0438\u0434\u0430\u0435\u043c\u043e:\n- \u0421\u0442\u0440\u0430\u043d\u0438\u0446\u0430 \u043a\u043e\u043d\u043a\u0443\u0440\u0435\u043d\u0442\u0430 (\u0437\u0430\u043b\u043e\u0433/\u041f\u0422\u0421, \u0441\u0442\u0430\u0432\u043a\u0438, \u0441\u043a\u043e\u0440\u043e\u0441\u0442\u044c, \u041c\u043e\u0441\u043a\u0432\u0430/\u041c\u041e, \u043a\u043e\u043d\u0442\u0430\u043a\u0442) \u2192 monitor_queue.\n- Firecrawl \u0443\u043f\u0430\u043b/\u043f\u0443\u0441\u0442\u043e \u2192 technical_errors (Claude \u043d\u0435 \u0432\u044b\u0437\u044b\u0432\u0430\u0435\u0442\u0441\u044f).\n- Primary \u0440\u0430\u0437\u0431\u043e\u0440 \u043d\u0435 \u0443\u0434\u0430\u043b\u0441\u044f, \u0440\u0435\u043c\u043e\u043d\u0442 \u0443\u0434\u0430\u043b\u0441\u044f \u2192 repair_used=true.\n- \u041e\u0431\u0430 \u043d\u0435 \u0443\u0434\u0430\u043b\u0438\u0441\u044c \u2192 technical_errors \u0441 \u0441\u043e\u0445\u0440\u0430\u043d\u0451\u043d\u043d\u044b\u043c\u0438 Primary+Repair \u0434\u0438\u0430\u0433\u043d\u043e\u0441\u0442\u0438\u043a\u0430\u043c\u0438.",
        "height": 420,
        "width": 480,
        "color": 4
      }
    }
  ],
  "connections": {
    "Manual Start": {
      "main": [
        [
          {
            "node": "Set Firecrawl URL",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Set Firecrawl URL": {
      "main": [
        [
          {
            "node": "Build Firecrawl Request",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Build Firecrawl Request": {
      "main": [
        [
          {
            "node": "Firecrawl Scrape API",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Firecrawl Scrape API": {
      "main": [
        [
          {
            "node": "Normalize Firecrawl Output",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Normalize Firecrawl Output": {
      "main": [
        [
          {
            "node": "IF Firecrawl Normalized OK?",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "IF Firecrawl Normalized OK?": {
      "main": [
        [
          {
            "node": "Build Primary Claude Request",
            "type": "main",
            "index": 0
          }
        ],
        [
          {
            "node": "Append to Dynamic Route Sheet",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Build Primary Claude Request": {
      "main": [
        [
          {
            "node": "Claude Primary API Request",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Claude Primary API Request": {
      "main": [
        [
          {
            "node": "Parse Primary JSON",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Parse Primary JSON": {
      "main": [
        [
          {
            "node": "IF Primary Parse OK?",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "IF Primary Parse OK?": {
      "main": [
        [
          {
            "node": "Normalize + Route",
            "type": "main",
            "index": 0
          }
        ],
        [
          {
            "node": "Build Repair Request",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Build Repair Request": {
      "main": [
        [
          {
            "node": "Claude Repair API Request",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Claude Repair API Request": {
      "main": [
        [
          {
            "node": "Parse Repaired JSON",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Parse Repaired JSON": {
      "main": [
        [
          {
            "node": "Normalize + Route",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Normalize + Route": {
      "main": [
        [
          {
            "node": "Append to Dynamic Route Sheet",
            "type": "main",
            "index": 0
          }
        ]
      ]
    }
  },
  "active": false,
  "settings": {
    "executionOrder": "v1"
  },
  "versionId": "fc000000-firecrawl-single-url-resilient-v001-20260607"
}

Credentials you'll need

Each integration node will prompt for credentials when you import. We strip credential IDs before publishing — you'll add your own.

• googleApi
• httpHeaderAuth

About this workflow

03 - Firecrawl Single URL to Resilient Analyzer. Uses httpRequest, googleSheets. Event-driven trigger; 17 nodes.

Source: https://github.com/CodeVinci8/vinci-ai-pilot/blob/main/n8n/workflows/03_firecrawl_single_url_resilient.json — original creator credit. Request a take-down →

More Web Scraping workflows → · Browse all categories →

Related workflows

Workflows that share integrations, category, or trigger type with this one. All free to copy and import.