{
  "id": "DtuDy7JtfybUocZU",
  "meta": {
    "templateCredsSetupCompleted": true
  },
  "name": "Template - MySQL Schema Embeddings in Pinecone",
  "tags": [
    {
      "id": "pH0FjEImo9WfHqWC",
      "name": "Template",
      "createdAt": "2025-12-19T17:46:15.670Z",
      "updatedAt": "2025-12-19T17:46:15.670Z"
    }
  ],
  "nodes": [
    {
      "id": "cda78dc2-e6da-4039-87e2-829d0295a726",
      "name": "Sticky Note",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        -1696,
        -1232
      ],
      "parameters": {
        "width": 736,
        "height": 960,
        "content": "## Workflow Overview - MySQL Schema Embeddings in Pinecone\n\nThis workflow synchronizes MySQL database table schemas with a vector database in a controlled, idempotent manner.\nEach database table is indexed as a single vector to preserve complete schema context for AI-based retrieval and reasoning.\nThe workflow prevents duplicate vectors and automatically handles schema changes by detecting differences and re-indexing only when required.\n\n### How it works\n- The workflow starts with a manual trigger and loads global configuration values.\n- All database tables are discovered and processed one-by-one inside a loop.\n- For each table, a normalized schema representation is generated and a deterministic hash is calculated.\n- A metadata table is checked to determine whether a vector already exists for the table.\n- If a vector exists, the stored schema hash is compared with the current hash to detect schema changes.\n- When a schema change is detected, the existing vector and metadata are deleted.\n- The updated table schema is embedded as a single vector (without chunking) and upserted into the vector database.\n- Vector identifiers and schema hashes are persisted for future executions.\n\n### Setup steps\n- Set the MySQL database name using mysql_database_name.\n- Configure the Pinecone index name using pinecone_index.\n- Set the vector namespace using vector_namespace.\n- Configure the Pinecone index host using vector_index_host.\n- Add your Pinecone API key using pinecone_apikey.\n- Select the embedding model using embedding_model.\n- Configure text processing options:\n    - chunk_size\n    - chunk_overlap\n- Set the metadata table identifier using dataTable_Id.\n- Save and run the workflow manually to perform the initial schema synchronization.\n\n### Limitations\n- This workflow indexes database table schemas only. Table data (rows) are not embedded or indexed.\n- Each table is stored as a single vector. Very large or highly complex schemas may approach model token limits depending on the selected embedding model.\n- Schema changes are detected using a hash-based comparison. Non-structural changes that do not affect the schema representation will not trigger re-indexing."
      },
      "typeVersion": 1
    },
    {
      "id": "5c8558c4-b7a8-4dae-8f6e-839f4867e57d",
      "name": "Sync DB Schema to Vector Store",
      "type": "n8n-nodes-base.manualTrigger",
      "position": [
        -800,
        -752
      ],
      "parameters": {},
      "typeVersion": 1
    },
    {
      "id": "5d1589fb-28dd-45a7-8002-1b0a0ab8cf18",
      "name": "Load Global Configuration",
      "type": "n8n-nodes-base.set",
      "position": [
        -624,
        -752
      ],
      "parameters": {
        "mode": "raw",
        "options": {},
        "jsonOutput": "{\n  \"vector_namespace\": \"\",\n  \"embedding_model\": \"\",\n  \"chunk_size\": \"\",\n  \"chunk_overlap\": \"\",\n  \"pinecone_index\": \"\",\n  \"mysql_database_name\" : \"\",\n  \"vector_index_host\" : \"\",\n  \"pinecone_apikey\" : \"\",\n  \"dataTable_Id\" : \"\"\n}"
      },
      "typeVersion": 3.4
    },
    {
      "id": "dd9a808b-0be9-4af6-9b9f-3832db3b8306",
      "name": "Fetch All Database Tables",
      "type": "n8n-nodes-base.mySql",
      "position": [
        -464,
        -752
      ],
      "parameters": {
        "query": "SELECT CONCAT('SHOW CREATE TABLE `', TABLE_NAME, '`;') AS queries, TABLE_NAME as tbl, DATABASE() as db\nFROM information_schema.tables\nWHERE table_schema = DATABASE()\nAND TABLE_TYPE = 'BASE TABLE';",
        "options": {},
        "operation": "executeQuery"
      },
      "credentials": {
        "mySql": {
          "name": "<your credential>"
        }
      },
      "typeVersion": 2.4
    },
    {
      "id": "b5924c88-248d-4e26-a82d-ebbe8ad3c52e",
      "name": "Set Table Schema Context",
      "type": "n8n-nodes-base.set",
      "position": [
        -48,
        -736
      ],
      "parameters": {
        "mode": "raw",
        "options": {},
        "jsonOutput": "={\n  \"vector_id\" : \"mysql_schema::{{ $node['Loop Over executable queries'].json.db }}::{{ $node['Loop Over executable queries'].json.tbl }}\",\n\"table_name\" : \"{{ $node['Loop Over executable queries'].json.tbl }}\",\n\"source_type\" : \"mysql_schema\"\n}\n"
      },
      "typeVersion": 3.4
    },
    {
      "id": "695c1804-f072-4272-9a14-80d1103946ec",
      "name": "Fetch Table Schema Definition",
      "type": "n8n-nodes-base.mySql",
      "position": [
        112,
        -736
      ],
      "parameters": {
        "query": "{{ $node['Loop Over executable queries'].json.queries }}",
        "options": {},
        "operation": "executeQuery"
      },
      "credentials": {
        "mySql": {
          "name": "<your credential>"
        }
      },
      "typeVersion": 2.4
    },
    {
      "id": "9fc19d70-5435-46ce-9ede-1f511a4e52ed",
      "name": "Generate Schema Hash",
      "type": "n8n-nodes-base.code",
      "position": [
        288,
        -736
      ],
      "parameters": {
        "jsCode": "// Only hash what defines the schema\nconst schemaText = $input.first().json['Create Table']\n\nlet hash = 5381;\nfor (let i = 0; i < schemaText.length; i++) {\n  hash = ((hash << 5) + hash) + schemaText.charCodeAt(i);\n}\n\nreturn {\n  ...$input.first().json,\n  vector_id: $('Set Table Schema Context').first().json.vector_id,\n  source_type: $('Set Table Schema Context').first().json.source_type,\n  schema_hash: hash.toString()\n};\n"
      },
      "typeVersion": 2
    },
    {
      "id": "5a8761b1-dc3a-4b6a-9bf7-8cdba5c185eb",
      "name": "Check Existing Vector Metadata",
      "type": "n8n-nodes-base.dataTable",
      "position": [
        544,
        -736
      ],
      "parameters": {
        "filters": {
          "conditions": [
            {
              "keyName": "vector_id",
              "keyValue": "={{ $node['Generate Schema Hash'].json.vector_id }}"
            }
          ]
        },
        "operation": "get",
        "returnAll": true,
        "dataTableId": {
          "__rl": true,
          "mode": "id",
          "value": "={{ $('Load Global Configuration').item.json.dataTable_Id }}"
        }
      },
      "typeVersion": 1,
      "alwaysOutputData": true
    },
    {
      "id": "d5302713-4ea7-4448-9c7a-fc4ae75ad56f",
      "name": "Vector Exists?",
      "type": "n8n-nodes-base.if",
      "position": [
        736,
        -736
      ],
      "parameters": {
        "options": {},
        "conditions": {
          "options": {
            "version": 2,
            "leftValue": "",
            "caseSensitive": true,
            "typeValidation": "loose"
          },
          "combinator": "and",
          "conditions": [
            {
              "id": "db0cb7c0-f0b9-44b6-8427-fc0787c9e3d5",
              "operator": {
                "type": "object",
                "operation": "empty",
                "singleValue": true
              },
              "leftValue": "={{ $node[\"Check Existing Vector Metadata\"].json }}",
              "rightValue": 0
            }
          ]
        },
        "looseTypeValidation": true
      },
      "typeVersion": 2.2,
      "alwaysOutputData": false
    },
    {
      "id": "2de84f88-0e49-4a87-8e58-4fbb7b1aca4b",
      "name": "Schema Changed?",
      "type": "n8n-nodes-base.if",
      "position": [
        880,
        -640
      ],
      "parameters": {
        "options": {},
        "conditions": {
          "options": {
            "version": 2,
            "leftValue": "",
            "caseSensitive": true,
            "typeValidation": "strict"
          },
          "combinator": "and",
          "conditions": [
            {
              "id": "ff61f3da-c1b0-49da-b7a5-6103932fd4ac",
              "operator": {
                "type": "string",
                "operation": "notEquals"
              },
              "leftValue": "={{ $node['Vector Exists?'].json.schema_hash }}",
              "rightValue": "={{ $node['Generate Schema Hash'].json.schema_hash }}"
            }
          ]
        }
      },
      "typeVersion": 2.2
    },
    {
      "id": "5bc39c82-e002-4d23-a702-b5f2b5a11e25",
      "name": "Delete Existing Vector (Pinecone)",
      "type": "n8n-nodes-base.httpRequest",
      "position": [
        1136,
        -656
      ],
      "parameters": {
        "url": "={{ $('Load Global Configuration').item.json.vector_index_host }}/vectors/delete",
        "method": "POST",
        "options": {
          "response": {
            "response": {}
          },
          "allowUnauthorizedCerts": true
        },
        "jsonBody": "={\n  \"filter\": {\n    \"vector_id\": {\n      \"$eq\": \"{{ $('Generate Schema Hash').item.json.vector_id }}\"\n    }\n  },\n  \"namespace\": \"{{ $('Load Global Configuration').item.json.vector_namespace }}\"\n}",
        "sendBody": true,
        "sendHeaders": true,
        "specifyBody": "json",
        "headerParameters": {
          "parameters": [
            {
              "name": "Api-Key",
              "value": "={{ $('Load Global Configuration').item.json.pinecone_apikey }}"
            }
          ]
        }
      },
      "typeVersion": 4.3
    },
    {
      "id": "d8048466-aca3-4f57-b1b4-413c13b01bfe",
      "name": "Delete Vector Metadata Record",
      "type": "n8n-nodes-base.dataTable",
      "position": [
        1312,
        -656
      ],
      "parameters": {
        "filters": {
          "conditions": [
            {
              "keyName": "vector_id",
              "keyValue": "={{ $('Generate Schema Hash').item.json.vector_id }}"
            }
          ]
        },
        "options": {},
        "operation": "deleteRows",
        "dataTableId": {
          "__rl": true,
          "mode": "list",
          "value": "2IZryiCLAMg3BCTg",
          "cachedResultUrl": "/projects/Jz2ZhOCR6j1Rldr1/datatables/2IZryiCLAMg3BCTg",
          "cachedResultName": "rag_embedding_log"
        }
      },
      "typeVersion": 1
    },
    {
      "id": "5f8e1571-e4a1-460d-a5f1-05b887e66944",
      "name": "Split Schema Text (No Chunking)",
      "type": "@n8n/n8n-nodes-langchain.textSplitterRecursiveCharacterTextSplitter",
      "position": [
        1600,
        -512
      ],
      "parameters": {
        "options": {
          "splitCode": "markdown"
        },
        "chunkSize": "={{ $('Load Global Configuration').item.json.chunk_size }}",
        "chunkOverlap": "={{ $('Load Global Configuration').item.json.chunk_overlap }}"
      },
      "typeVersion": 1
    },
    {
      "id": "2f9bc643-4faa-47aa-9daa-3f200cb1d8ae",
      "name": "Generate Schema Embeddings",
      "type": "@n8n/n8n-nodes-langchain.embeddingsOpenAi",
      "position": [
        1488,
        -640
      ],
      "parameters": {
        "model": "={{ $('Load Global Configuration').item.json.embedding_model }}",
        "options": {}
      },
      "credentials": {
        "openAiApi": {
          "name": "<your credential>"
        }
      },
      "typeVersion": 1.2
    },
    {
      "id": "20ca023e-76bb-46dd-81ed-a67c06da80aa",
      "name": "Upsert Vector Metadata Record",
      "type": "n8n-nodes-base.dataTable",
      "position": [
        1888,
        -544
      ],
      "parameters": {
        "columns": {
          "value": {
            "vector_id": "={{ $node['Generate Schema Hash'].json.vector_id }}",
            "table_name": "={{ $node['Generate Schema Hash'].json.Table }}",
            "schema_hash": "={{ $node['Generate Schema Hash'].json.schema_hash }}"
          },
          "schema": [
            {
              "id": "vector_id",
              "type": "string",
              "display": true,
              "removed": false,
              "readOnly": false,
              "required": false,
              "displayName": "vector_id",
              "defaultMatch": false
            },
            {
              "id": "table_name",
              "type": "string",
              "display": true,
              "removed": false,
              "readOnly": false,
              "required": false,
              "displayName": "table_name",
              "defaultMatch": false
            },
            {
              "id": "schema_hash",
              "type": "string",
              "display": true,
              "removed": false,
              "readOnly": false,
              "required": false,
              "displayName": "schema_hash",
              "defaultMatch": false
            }
          ],
          "mappingMode": "defineBelow",
          "matchingColumns": [],
          "attemptToConvertTypes": false,
          "convertFieldsToString": false
        },
        "filters": {
          "conditions": [
            {
              "keyName": "vector_id",
              "keyValue": "={{ $node['Generate Schema Hash'].json.vector_id }}"
            }
          ]
        },
        "options": {},
        "operation": "upsert",
        "dataTableId": {
          "__rl": true,
          "mode": "list",
          "value": "2IZryiCLAMg3BCTg",
          "cachedResultUrl": "/projects/Jz2ZhOCR6j1Rldr1/datatables/2IZryiCLAMg3BCTg",
          "cachedResultName": "rag_embedding_log"
        }
      },
      "typeVersion": 1
    },
    {
      "id": "11105b1b-70e2-4115-ab1f-06f99bcdac21",
      "name": "Prepare Schema Document",
      "type": "@n8n/n8n-nodes-langchain.documentDefaultDataLoader",
      "position": [
        1600,
        -640
      ],
      "parameters": {
        "options": {
          "metadata": {
            "metadataValues": [
              {
                "name": "table_name",
                "value": "={{ $('Generate Schema Hash').item.json.Table }}"
              },
              {
                "name": "vector_id",
                "value": "={{ $('Generate Schema Hash').item.json.vector_id }}"
              },
              {
                "name": "source_type",
                "value": "={{ $('Generate Schema Hash').item.json.source_type }}"
              },
              {
                "name": "schema_hash",
                "value": "={{ $('Generate Schema Hash').item.json.schema_hash }}"
              }
            ]
          }
        },
        "jsonData": "={{ $('Generate Schema Hash').item.json['Create Table'] }}",
        "jsonMode": "expressionData"
      },
      "typeVersion": 1
    },
    {
      "id": "e1c786bf-e9f5-4ddd-b347-edecef79fa9d",
      "name": "Insert Schema Vector to Pinecone",
      "type": "@n8n/n8n-nodes-langchain.vectorStorePinecone",
      "position": [
        1536,
        -800
      ],
      "parameters": {
        "mode": "insert",
        "options": {
          "pineconeNamespace": "={{ $node['Load Global Configuration'].json.vector_namespace }}"
        },
        "pineconeIndex": {
          "__rl": true,
          "mode": "list",
          "value": "dbrag",
          "cachedResultName": "dbrag"
        }
      },
      "credentials": {
        "pineconeApi": {
          "name": "<your credential>"
        }
      },
      "typeVersion": 1
    },
    {
      "id": "a0bbece4-1c0a-488d-8ec0-b6b84ddfebd4",
      "name": "Sticky Note1",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        -848,
        -880
      ],
      "parameters": {
        "width": 1280,
        "height": 560,
        "content": "## Setup Workflow & Schema Discovery\n     Initialize configuration, discover database tables, and generate schema fingerprints"
      },
      "typeVersion": 1
    },
    {
      "id": "79fbc689-c786-41b0-978a-63e866f4cef8",
      "name": "Sticky Note2",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        480,
        -880
      ],
      "parameters": {
        "width": 560,
        "height": 560,
        "content": "## Vector Existence & Change Detection\n     Duplicate prevention and schema drift logic"
      },
      "typeVersion": 1
    },
    {
      "id": "16f257cc-f4ba-44cb-bc12-f766228b4248",
      "name": "Sticky Note3",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        1088,
        -880
      ],
      "parameters": {
        "width": 960,
        "height": 560,
        "content": "## Vector Cleanup & Indexing \n     Safe deletion and re-indexing pipeline"
      },
      "typeVersion": 1
    },
    {
      "id": "94b17575-2d49-41b4-b8d6-0a4be701f049",
      "name": "Loop Over executable queries",
      "type": "n8n-nodes-base.splitInBatches",
      "position": [
        -272,
        -752
      ],
      "parameters": {
        "options": {
          "reset": false
        }
      },
      "executeOnce": false,
      "typeVersion": 3
    },
    {
      "id": "bfd300b1-8c5d-49b4-b662-75cc796f9bc9",
      "name": "Sticky Note4",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        -592,
        -1232
      ],
      "parameters": {
        "width": 464,
        "height": 320,
        "content": "## How to Use This Workflow\n@[youtube](elykvVRjVL0)"
      },
      "typeVersion": 1
    }
  ],
  "active": false,
  "settings": {
    "executionOrder": "v1"
  },
  "versionId": "dad112b4-574c-4cc0-b54b-125fffd6922f",
  "connections": {
    "Vector Exists?": {
      "main": [
        [
          {
            "node": "Insert Schema Vector to Pinecone",
            "type": "main",
            "index": 0
          }
        ],
        [
          {
            "node": "Schema Changed?",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Schema Changed?": {
      "main": [
        [
          {
            "node": "Delete Existing Vector (Pinecone)",
            "type": "main",
            "index": 0
          }
        ],
        [
          {
            "node": "Loop Over executable queries",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Generate Schema Hash": {
      "main": [
        [
          {
            "node": "Check Existing Vector Metadata",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Prepare Schema Document": {
      "ai_document": [
        [
          {
            "node": "Insert Schema Vector to Pinecone",
            "type": "ai_document",
            "index": 0
          }
        ]
      ]
    },
    "Set Table Schema Context": {
      "main": [
        [
          {
            "node": "Fetch Table Schema Definition",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Fetch All Database Tables": {
      "main": [
        [
          {
            "node": "Loop Over executable queries",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Load Global Configuration": {
      "main": [
        [
          {
            "node": "Fetch All Database Tables",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Generate Schema Embeddings": {
      "ai_embedding": [
        [
          {
            "node": "Insert Schema Vector to Pinecone",
            "type": "ai_embedding",
            "index": 0
          }
        ]
      ]
    },
    "Loop Over executable queries": {
      "main": [
        [],
        [
          {
            "node": "Set Table Schema Context",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Delete Vector Metadata Record": {
      "main": [
        [
          {
            "node": "Insert Schema Vector to Pinecone",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Fetch Table Schema Definition": {
      "main": [
        [
          {
            "node": "Generate Schema Hash",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Upsert Vector Metadata Record": {
      "main": [
        [
          {
            "node": "Loop Over executable queries",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Check Existing Vector Metadata": {
      "main": [
        [
          {
            "node": "Vector Exists?",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Sync DB Schema to Vector Store": {
      "main": [
        [
          {
            "node": "Load Global Configuration",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Split Schema Text (No Chunking)": {
      "ai_textSplitter": [
        [
          {
            "node": "Prepare Schema Document",
            "type": "ai_textSplitter",
            "index": 0
          }
        ]
      ]
    },
    "Insert Schema Vector to Pinecone": {
      "main": [
        [
          {
            "node": "Upsert Vector Metadata Record",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Delete Existing Vector (Pinecone)": {
      "main": [
        [
          {
            "node": "Delete Vector Metadata Record",
            "type": "main",
            "index": 0
          }
        ]
      ]
    }
  }
}