{
  "id": "0rv9SHoVxoahDbcv",
  "meta": {
    "templateCredsSetupCompleted": true
  },
  "name": "n8n Template Catalog Vector Sync",
  "tags": [],
  "nodes": [
    {
      "id": "139cd54f-25c6-4a65-8ddb-894c68a95d91",
      "name": "Sticky Note - Intro",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        960,
        -192
      ],
      "parameters": {
        "color": 6,
        "width": 1852,
        "height": 508,
        "content": "## \ud83d\udee0\ufe0f n8n Template Catalog Vector Sync\n\n**Purpose:** Daily incremental ingestion and security auditing of public community workflow templates into Qdrant for local RAG retrieval.\n\n**\ud83d\udc64 Who It's For:**\nAutomation architects wanting a searchable, locally hosted vector database of ~749k n8n community workflow patterns.\n\n**\ud83d\udd17 Quick Links & Seed Database:**\n\u2022 **GitHub Repository**: [jdm6457/n8n-rag-suite](https://github.com/jdm6457/n8n-rag-suite)\n\u2022 **Instant Seed Download**: `wget https://huggingface.co/datasets/jdm6457/n8n-rag-suite-snapshot/resolve/main/n8n_templates_749k.snapshot.gz`\n\n**\u2699\ufe0f Prerequisites:**\n\u2022 Host Ollama running `nomic-embed-text:latest`\n\u2022 Local Qdrant instance on port 6333 (`n8n_templates` collection)\n\n**\ud83d\ude80 How to Use:**\n1. Run manually once to fetch recent templates, or let the Schedule Trigger execute daily at 2:00 AM.\n2. Alternatively, use the Hugging Face snapshot link above to seed the Qdrant database instantly without scraping."
      },
      "typeVersion": 1
    },
    {
      "id": "bc89b7cb-3f19-4b86-881a-ca4b4b206a1f",
      "name": "Sticky Note - Ingestion",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        960,
        336
      ],
      "parameters": {
        "color": 4,
        "width": 592,
        "height": 348,
        "content": "### 1. Ingestion & Incremental Fetch\nFetches top community templates from n8n public catalog API.\n* Pages 1\u20132 only (~200 items) for daily updates.\n* Splits collection array into individual workflow items."
      },
      "typeVersion": 1
    },
    {
      "id": "44f285fd-0a2e-43c5-ac04-b1a29c5471b3",
      "name": "Sticky Note - Audit",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        1568,
        336
      ],
      "parameters": {
        "color": 3,
        "width": 440,
        "height": 348,
        "content": "### 2. Full Extraction & Security Audit\n* Pulls full workflow metadata, sticky note text, and system prompts.\n* Strips non-printable ASCII/hex control characters.\n* Scans node configuration strings for exposed API keys."
      },
      "typeVersion": 1
    },
    {
      "id": "9c46e271-f68f-4dc0-bac4-f219c5c71dea",
      "name": "Sticky Note - Qdrant",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        2016,
        336
      ],
      "parameters": {
        "color": 5,
        "width": 804,
        "height": 524,
        "content": "### 3. Purge Dupes, Embed & Vector Indexing\n* Check for duplicate templates and delete them\n* Embeds text content via Ollama (`nomic-embed-text`).\n* Upserts vectors and metadata into Qdrant `n8n_templates` collection."
      },
      "typeVersion": 1
    },
    {
      "id": "0c4dad43-ae9d-4324-830c-548e6c3c388a",
      "name": "Fetch Template Catalog",
      "type": "n8n-nodes-base.httpRequest",
      "position": [
        1232,
        528
      ],
      "parameters": {
        "url": "https://api.n8n.io/templates/search",
        "options": {
          "pagination": {
            "pagination": {
              "parameters": {
                "parameters": [
                  {
                    "name": "page",
                    "value": "={{ $pageCount + 1 }}"
                  }
                ]
              },
              "completeExpression": "={{ $pageCount >= 2 }}",
              "paginationCompleteWhen": "other"
            }
          }
        },
        "sendQuery": true,
        "queryParameters": {
          "parameters": [
            {
              "name": "rows",
              "value": "100"
            },
            {
              "name": "page",
              "value": "1"
            }
          ]
        }
      },
      "typeVersion": 4.2
    },
    {
      "id": "620452a5-ba13-48ad-8680-2b872cf5315a",
      "name": "Split Templates",
      "type": "n8n-nodes-base.itemLists",
      "position": [
        1424,
        528
      ],
      "parameters": {
        "options": {},
        "fieldToSplitOut": "workflows"
      },
      "typeVersion": 3
    },
    {
      "id": "e6ea8c75-8949-434b-a49b-f3d3d7588c2d",
      "name": "Fetch Full Workflow JSON",
      "type": "n8n-nodes-base.httpRequest",
      "onError": "continueRegularOutput",
      "position": [
        1648,
        528
      ],
      "parameters": {
        "url": "=https://api.n8n.io/workflows/templates/{{ $json.id }}",
        "options": {}
      },
      "typeVersion": 4.2
    },
    {
      "id": "2bc2cffd-76b1-4148-93dd-b8b4d6653a03",
      "name": "Security Audit & Text Sanitizer",
      "type": "n8n-nodes-base.code",
      "position": [
        1856,
        528
      ],
      "parameters": {
        "jsCode": "const splitItems = $('Split Templates').all();\n\nconst secretRegexes = [\n  /sk-[a-zA-Z0-9]{20,}/i,\n  /bearer\\s+[a-zA-Z0-9\\-\\._~\\+\\/]+=*/i,\n  /(A3T[A-Z0-9]|AKIA|AGPA|AIDA|AROA|AIPA|ANPA|ANVA|ASIA)[A-Z0-9]{16}/,\n  /ghp_[a-zA-Z0-9]{36}|github_pat_[a-zA-Z0-9]{22}_[a-zA-Z0-9]{59}/,\n  /AIza[0-9A-Za-z-_]{35}/,\n  /xox[baprs]-[0-9a-zA-Z]{10,48}/,\n  /(postgres|mysql|mongodb|redis):\\/\\/[^:]+:[^@]+@/i\n];\n\n// Strips surrogate pairs (emojis), lone surrogates, null bytes, and control chars\nfunction sanitizeString(str) {\n  if (typeof str !== 'string') return str;\n\n  let s = str;\n\n  // 1. Remove 4-byte surrogate pairs (emojis, symbols) to prevent Text Splitter truncation errors\n  s = s.replace(/[\\uD800-\\uDBFF][\\uDC00-\\uDFFF]/g, ' ');\n\n  // 2. Remove any remaining orphan surrogates\n  s = s.replace(/[\\uD800-\\uDFFF]/g, '');\n\n  // 3. Strip null bytes and non-printable control characters (preserving newlines/tabs)\n  s = s.replace(/[\\x00-\\x08\\x0B\\x0C\\x0E-\\x1F]/g, '');\n\n  return s;\n}\n\nfunction sanitize(val) {\n  if (typeof val === 'string') {\n    return sanitizeString(val);\n  }\n  if (Array.isArray(val)) {\n    return val.map(sanitize);\n  }\n  if (val && typeof val === 'object') {\n    const res = {};\n    for (const [k, v] of Object.entries(val)) {\n      res[k] = sanitize(v);\n    }\n    return res;\n  }\n  return val;\n}\n\nreturn $input.all().map((item, index) => {\n  const fullData = item.json || {};\n  const splitData = splitItems[index]?.json || {};\n  \n  const workflowObj = fullData.workflow || {};\n  const nodes = fullData.nodes || workflowObj.nodes || [];\n\n  const rawId = fullData.id ?? fullData.templateId ?? splitData.id;\n  const templateId = rawId ? Number(rawId) : null;\n\n  const name = fullData.name || splitData.name || 'Untitled';\n  const description = fullData.description || splitData.description || '';\n  \n  const rawCategories = fullData.categories || splitData.categories || [];\n  const categories = rawCategories.map(c => typeof c === 'object' ? (c.name || c.label || '') : c).filter(Boolean);\n\n  const nodeTypes = [...new Set(nodes.map(n => n.type))];\n  const nodeCount = nodes.length;\n\n  const triggerTypes = [...new Set(\n    nodes.filter(n => (n.type || '').toLowerCase().includes('trigger')).map(n => n.type)\n  )];\n\n  const hasAiNodes = nodes.some(n => \n    (n.type || '').includes('langchain') || \n    (n.type || '').includes('openai') || \n    (n.type || '').includes('anthropic') || \n    (n.type || '').includes('ollama') || \n    (n.type || '').includes('qdrant') ||\n    (n.type || '').includes('vectorStore')\n  );\n\n  const views = Number(fullData.views ?? splitData.views ?? fullData.totalViews ?? 0);\n  const createdAt = fullData.createdAt || splitData.createdAt || fullData.created_at || null;\n  const slug = fullData.slug || splitData.slug || '';\n  const templateUrl = slug ? `https://n8n.io/workflows/${slug}` : (templateId ? `https://n8n.io/workflows/${templateId}` : '');\n\n  const stickyNotes = nodes\n    .filter(n => n.type === 'n8n-nodes-base.stickyNote')\n    .map(n => n.parameters?.content || '');\n    \n  const systemPrompts = nodes\n    .filter(n => n.parameters?.options?.systemMessage || n.parameters?.systemMessage)\n    .map(n => n.parameters?.options?.systemMessage || n.parameters?.systemMessage || '');\n\n  const nodeDetails = nodes.map(n => {\n    return `Node: ${n.name} (${n.type})\\nConfig: ${JSON.stringify(n.parameters || {})}`;\n  }).join('\\n');\n\n  const pageContent = [\n    `Title: ${name}`,\n    `Description: ${description || 'No description provided'}`,\n    `Categories: ${categories.join(', ')}`,\n    `Node Types Used: ${nodeTypes.join(', ')}`,\n    stickyNotes.length > 0 ? `Sticky Notes:\\n${stickyNotes.join('\\n')}` : '',\n    systemPrompts.length > 0 ? `System Prompts:\\n${systemPrompts.join('\\n')}` : '',\n    nodeDetails ? `Workflow Configurations:\\n${nodeDetails}` : ''\n  ].filter(Boolean).join('\\n\\n');\n\n  const paramString = JSON.stringify(nodes.map(n => n.parameters || {}));\n  const combinedTextToScan = [\n    paramString,\n    stickyNotes.join(' '),\n    systemPrompts.join(' '),\n    description\n  ].join(' ');\n\n  const hasSecurityWarning = secretRegexes.some(regex => regex.test(combinedTextToScan));\n\n  const payload = {\n    metadata: {\n      template_id: templateId,\n      name: name,\n      description: description,\n      categories: categories,\n      nodeTypes: nodeTypes,\n      node_count: nodeCount,\n      trigger_types: triggerTypes,\n      has_ai_nodes: hasAiNodes,\n      has_security_warning: hasSecurityWarning,\n      views: views,\n      created_at: createdAt,\n      template_url: templateUrl\n    },\n    pageContent: pageContent\n  };\n\n  return {\n    json: sanitize(payload)\n  };\n});"
      },
      "typeVersion": 2
    },
    {
      "id": "52de46c7-b2ba-4337-909b-c4ac9a0fc052",
      "name": "Qdrant Vector Store",
      "type": "@n8n/n8n-nodes-langchain.vectorStoreQdrant",
      "position": [
        2464,
        528
      ],
      "parameters": {
        "mode": "insert",
        "options": {},
        "qdrantCollection": {
          "__rl": true,
          "mode": "list",
          "value": "n8n_templates",
          "cachedResultName": "n8n_templates"
        },
        "embeddingBatchSize": 10
      },
      "credentials": {
        "qdrantApi": {
          "name": "<your credential>"
        }
      },
      "typeVersion": 1.3
    },
    {
      "id": "93d316fa-79ae-4809-bf28-5977fcbb00b5",
      "name": "Embeddings Ollama",
      "type": "@n8n/n8n-nodes-langchain.embeddingsOllama",
      "position": [
        2464,
        720
      ],
      "parameters": {
        "model": "nomic-embed-text:latest"
      },
      "credentials": {
        "ollamaApi": {
          "name": "<your credential>"
        }
      },
      "typeVersion": 1
    },
    {
      "id": "118c5326-4d0d-448b-9c75-8199d84c1aa0",
      "name": "Default Data Loader",
      "type": "@n8n/n8n-nodes-langchain.documentDefaultDataLoader",
      "position": [
        2624,
        720
      ],
      "parameters": {
        "options": {
          "metadata": {
            "metadataValues": [
              {
                "name": "template_id",
                "value": "={{ $json.metadata.template_id }}"
              },
              {
                "name": "name",
                "value": "={{ $json.metadata.name }}"
              },
              {
                "name": "has_security_warning",
                "value": "={{ $json.metadata.has_security_warning }}"
              },
              {
                "name": "categories",
                "value": "={{ $json.metadata.categories }}"
              },
              {
                "name": "nodeTypes",
                "value": "={{ $json.metadata.nodeTypes }}"
              },
              {
                "name": "description",
                "value": "={{ $json.metadata.description }}"
              },
              {
                "name": "node_count",
                "value": "={{ $json.metadata.node_count }}"
              },
              {
                "name": "trigger_types",
                "value": "={{ $json.metadata.trigger_types }}"
              },
              {
                "name": "has_ai_nodes",
                "value": "={{ $json.metadata.has_ai_nodes }}"
              },
              {
                "name": "views",
                "value": "={{ $json.metadata.views }}"
              },
              {
                "name": "created_at",
                "value": "={{ $json.metadata.created_at }}"
              },
              {
                "name": "template_url",
                "value": "={{ $json.metadata.template_url }}"
              }
            ]
          }
        }
      },
      "typeVersion": 1.1
    },
    {
      "id": "5225300a-844b-47df-a711-f8139b9273e7",
      "name": "Schedule Trigger",
      "type": "n8n-nodes-base.scheduleTrigger",
      "position": [
        1024,
        528
      ],
      "parameters": {
        "rule": {
          "interval": [
            {
              "triggerAtHour": 2
            }
          ]
        }
      },
      "typeVersion": 1.3
    },
    {
      "id": "e0b2974b-f1d4-445f-8edd-e20fdf9d91d9",
      "name": "Pass-Through Workflow Items",
      "type": "n8n-nodes-base.code",
      "position": [
        2288,
        528
      ],
      "parameters": {
        "jsCode": "return $(\"Security Audit & Text Sanitizer\").all();;"
      },
      "typeVersion": 2
    },
    {
      "id": "237009a5-4a5a-4dbc-91b2-c9dcf5301781",
      "name": "Purge Existing Template Vectors",
      "type": "n8n-nodes-base.httpRequest",
      "position": [
        2080,
        528
      ],
      "parameters": {
        "url": "http://qdrant:6333/collections/n8n_templates/points/delete",
        "method": "POST",
        "options": {},
        "jsonBody": "={{\n  {\n    \"filter\": {\n      \"must\": [\n        {\n          \"key\": \"metadata.template_id\",\n          \"match\": {\n            \"any\": $(\"Security Audit & Text Sanitizer\").all().map(item => item.json.metadata?.template_id || item.json.template_id)\n          }\n        }\n      ]\n    }\n  }\n}}",
        "sendBody": true,
        "specifyBody": "json"
      },
      "executeOnce": true,
      "typeVersion": 4.4
    }
  ],
  "active": true,
  "settings": {
    "binaryMode": "separate",
    "availableInMCP": false,
    "executionOrder": "v1"
  },
  "versionId": "b2392a57-c293-4e5b-a5d6-d5f202767047",
  "nodeGroups": [],
  "connections": {
    "Split Templates": {
      "main": [
        [
          {
            "node": "Fetch Full Workflow JSON",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Schedule Trigger": {
      "main": [
        [
          {
            "node": "Fetch Template Catalog",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Embeddings Ollama": {
      "ai_embedding": [
        [
          {
            "node": "Qdrant Vector Store",
            "type": "ai_embedding",
            "index": 0
          }
        ]
      ]
    },
    "Default Data Loader": {
      "ai_document": [
        [
          {
            "node": "Qdrant Vector Store",
            "type": "ai_document",
            "index": 0
          }
        ]
      ]
    },
    "Fetch Template Catalog": {
      "main": [
        [
          {
            "node": "Split Templates",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Fetch Full Workflow JSON": {
      "main": [
        [
          {
            "node": "Security Audit & Text Sanitizer",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Pass-Through Workflow Items": {
      "main": [
        [
          {
            "node": "Qdrant Vector Store",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Purge Existing Template Vectors": {
      "main": [
        [
          {
            "node": "Pass-Through Workflow Items",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Security Audit & Text Sanitizer": {
      "main": [
        [
          {
            "node": "Purge Existing Template Vectors",
            "type": "main",
            "index": 0
          }
        ]
      ]
    }
  }
}