The workflow JSON
Copy or download the full n8n JSON below. Paste it into a new n8n workflow, add your credentials, activate. Full import guide →
{
"id": "pPtCy6qPfEv1qNRn",
"name": "[1/3 - anomaly detection] [1/2 - KNN classification] Batch upload dataset to Qdrant (crops dataset)",
"tags": [
{
"id": "n3zAUYFhdqtjhcLf",
"name": "qdrant",
"createdAt": "2024-12-10T11:56:59.987Z",
"updatedAt": "2024-12-10T11:56:59.987Z"
}
],
"nodes": [
{
"id": "53831410-b4f3-4374-8bdd-c2a33cd873cb",
"name": "When clicking \u2018Test workflow\u2019",
"type": "n8n-nodes-base.manualTrigger",
"position": [
-640,
0
],
"parameters": {},
"typeVersion": 1
},
{
"id": "e303ccea-c0e0-4fe5-bd31-48380a0e438f",
"name": "Google Cloud Storage",
"type": "n8n-nodes-base.googleCloudStorage",
"position": [
820,
160
],
"parameters": {
"resource": "object",
"returnAll": true,
"bucketName": "n8n-qdrant-demo",
"listFilters": {
"prefix": "agricultural-crops"
},
"requestOptions": {}
},
"credentials": {
"googleCloudStorageOAuth2Api": {
"name": "<your credential>"
}
},
"typeVersion": 1
},
{
"id": "737bdb15-61cf-48eb-96af-569eb5986ee8",
"name": "Get fields for Qdrant",
"type": "n8n-nodes-base.set",
"position": [
1080,
160
],
"parameters": {
"options": {},
"assignments": {
"assignments": [
{
"id": "10d9147f-1c0c-4357-8413-3130829c2e24",
"name": "=publicLink",
"type": "string",
"value": "=https://storage.googleapis.com/{{ $json.bucket }}/{{ $json.selfLink.split('/').splice(-1) }}"
},
{
"id": "ff9e6a0b-e47a-4550-a13b-465507c75f8f",
"name": "cropName",
"type": "string",
"value": "={{ $json.id.split('/').slice(-3, -2)[0].toLowerCase()}}"
}
]
}
},
"typeVersion": 3.4
},
{
"id": "2b18ed0c-38d3-49e9-be3d-4f7b35f4d9e5",
"name": "Qdrant cluster variables",
"type": "n8n-nodes-base.set",
"position": [
-360,
0
],
"parameters": {
"options": {},
"assignments": {
"assignments": [
{
"id": "58b7384d-fd0c-44aa-9f8e-0306a99be431",
"name": "qdrantCloudURL",
"type": "string",
"value": "=https://152bc6e2-832a-415c-a1aa-fb529f8baf8d.eu-central-1-0.aws.cloud.qdrant.io"
},
{
"id": "e34c4d88-b102-43cc-a09e-e0553f2da23a",
"name": "collectionName",
"type": "string",
"value": "=agricultural-crops"
},
{
"id": "33581e0a-307f-4380-9533-615791096de7",
"name": "VoyageEmbeddingsDim",
"type": "number",
"value": 1024
},
{
"id": "6e390343-2cd2-4559-aba9-82b13acb7f52",
"name": "batchSize",
"type": "number",
"value": 4
}
]
}
},
"typeVersion": 3.4
},
{
"id": "f88d290e-3311-4322-b2a5-1350fc1f8768",
"name": "Embed crop image",
"type": "n8n-nodes-base.httpRequest",
"position": [
2120,
160
],
"parameters": {
"url": "https://api.voyageai.com/v1/multimodalembeddings",
"method": "POST",
"options": {},
"jsonBody": "={{\n{\n \"inputs\": $json.batchVoyage,\n \"model\": \"voyage-multimodal-3\",\n \"input_type\": \"document\"\n}\n}}",
"sendBody": true,
"specifyBody": "json",
"authentication": "genericCredentialType",
"genericAuthType": "httpHeaderAuth"
},
"credentials": {
"httpHeaderAuth": {
"name": "<your credential>"
}
},
"typeVersion": 4.2
},
{
"id": "250c6a8d-f545-4037-8069-c834437bbe15",
"name": "Create Qdrant Collection",
"type": "n8n-nodes-base.httpRequest",
"position": [
320,
160
],
"parameters": {
"url": "={{ $('Qdrant cluster variables').first().json.qdrantCloudURL }}/collections/{{ $('Qdrant cluster variables').first().json.collectionName }}",
"method": "PUT",
"options": {},
"jsonBody": "={{\n{\n \"vectors\": {\n \"voyage\": { \n \"size\": $('Qdrant cluster variables').first().json.VoyageEmbeddingsDim, \n \"distance\": \"Cosine\" \n } \n }\n}\n}}",
"sendBody": true,
"specifyBody": "json",
"authentication": "predefinedCredentialType",
"nodeCredentialType": "qdrantApi"
},
"credentials": {
"qdrantApi": {
"name": "<your credential>"
}
},
"typeVersion": 4.2
},
{
"id": "20b612ff-4794-43ef-bf45-008a16a2f30f",
"name": "Check Qdrant Collection Existence",
"type": "n8n-nodes-base.httpRequest",
"position": [
-100,
0
],
"parameters": {
"url": "={{ $json.qdrantCloudURL }}/collections/{{ $json.collectionName }}/exists",
"options": {},
"authentication": "predefinedCredentialType",
"nodeCredentialType": "qdrantApi"
},
"credentials": {
"qdrantApi": {
"name": "<your credential>"
}
},
"typeVersion": 4.2
},
{
"id": "c067740b-5de3-452e-a614-bf14985a73a0",
"name": "Batches in the API's format",
"type": "n8n-nodes-base.set",
"position": [
1860,
160
],
"parameters": {
"options": {},
"assignments": {
"assignments": [
{
"id": "f14db112-6f15-4405-aa47-8cb56bb8ae7a",
"name": "=batchVoyage",
"type": "array",
"value": "={{ $json.batch.map(item => ({ \"content\": ([{\"type\": \"image_url\", \"image_url\": item[\"publicLink\"]}])}))}}"
},
{
"id": "3885fd69-66f5-4435-86a4-b80eaa568ac1",
"name": "=batchPayloadQdrant",
"type": "array",
"value": "={{ $json.batch.map(item => ({\"crop_name\":item[\"cropName\"], \"image_path\":item[\"publicLink\"]})) }}"
},
{
"id": "8ea7a91e-af27-49cb-9a29-41dae15c4e33",
"name": "uuids",
"type": "array",
"value": "={{ $json.uuids }}"
}
]
}
},
"typeVersion": 3.4
},
{
"id": "bf9a9532-db64-4c02-b91d-47e708ded4d3",
"name": "Batch Upload to Qdrant",
"type": "n8n-nodes-base.httpRequest",
"position": [
2320,
160
],
"parameters": {
"url": "={{ $('Qdrant cluster variables').first().json.qdrantCloudURL }}/collections/{{ $('Qdrant cluster variables').first().json.collectionName }}/points",
"method": "PUT",
"options": {},
"jsonBody": "={{\n{\n \"batch\": {\n \"ids\" : $('Batches in the API\\'s format').item.json.uuids,\n \"vectors\": {\"voyage\": $json.data.map(item => item[\"embedding\"]) },\n \"payloads\": $('Batches in the API\\'s format').item.json.batchPayloadQdrant\n }\n}\n}}",
"sendBody": true,
"specifyBody": "json",
"authentication": "predefinedCredentialType",
"nodeCredentialType": "qdrantApi"
},
"credentials": {
"qdrantApi": {
"name": "<your credential>"
}
},
"typeVersion": 4.2
},
{
"id": "3c30373f-c84c-405f-bb84-ec8b4c7419f4",
"name": "Split in batches, generate uuids for Qdrant points",
"type": "n8n-nodes-base.code",
"position": [
1600,
160
],
"parameters": {
"language": "python",
"pythonCode": "import uuid\n\ncrops = [item.json for item in _input.all()]\nbatch_size = int(_('Qdrant cluster variables').first()['json']['batchSize'])\n\ndef split_into_batches_add_uuids(array, batch_size):\n return [\n {\n \"batch\": array[i:i + batch_size],\n \"uuids\": [str(uuid.uuid4()) for j in range(len(array[i:i + batch_size]))]\n }\n for i in range(0, len(array), batch_size)\n ]\n\n# Split crops into batches\nbatched_crops = split_into_batches_add_uuids(crops, batch_size)\n\nreturn batched_crops"
},
"typeVersion": 2
},
{
"id": "2b028f8c-0a4c-4a3a-9e2b-14b1c2401c6d",
"name": "If collection exists",
"type": "n8n-nodes-base.if",
"position": [
120,
0
],
"parameters": {
"options": {},
"conditions": {
"options": {
"version": 2,
"leftValue": "",
"caseSensitive": true,
"typeValidation": "strict"
},
"combinator": "and",
"conditions": [
{
"id": "2104b862-667c-4a34-8888-9cb81a2e10f8",
"operator": {
"type": "boolean",
"operation": "true",
"singleValue": true
},
"leftValue": "={{ $json.result.exists }}",
"rightValue": "true"
}
]
}
},
"typeVersion": 2.2
},
{
"id": "768793f6-391e-4cc9-b637-f32ee2f77156",
"name": "Sticky Note",
"type": "n8n-nodes-base.stickyNote",
"position": [
500,
340
],
"parameters": {
"width": 280,
"height": 200,
"content": "In the next workflow, we're going to use Qdrant to get the number of images belonging to each crop type defined by `crop_name` (for example, *\"cucumber\"*). \nTo get this information about counts in payload fields, we need to create an index on that field to optimise the resources (it needs to be done once). That's what is happening here"
},
"typeVersion": 1
},
{
"id": "0c8896f7-8c57-4add-bc4d-03c4a774bdf1",
"name": "Payload index on crop_name",
"type": "n8n-nodes-base.httpRequest",
"position": [
500,
160
],
"parameters": {
"url": "={{ $('Qdrant cluster variables').first().json.qdrantCloudURL }}/collections/{{ $('Qdrant cluster variables').first().json.collectionName }}/index",
"method": "PUT",
"options": {},
"jsonBody": "={\n \"field_name\": \"crop_name\",\n \"field_schema\": \"keyword\"\n}",
"sendBody": true,
"specifyBody": "json",
"authentication": "predefinedCredentialType",
"nodeCredentialType": "qdrantApi"
},
"credentials": {
"qdrantApi": {
"name": "<your credential>"
}
},
"typeVersion": 4.2
},
{
"id": "342186f6-41bf-46be-9be8-a9b1ca290d55",
"name": "Sticky Note1",
"type": "n8n-nodes-base.stickyNote",
"position": [
-360,
-360
],
"parameters": {
"height": 300,
"content": "Setting up variables\n1) Cloud URL - to connect to Qdrant Cloud (your personal cluster URL)\n2) Collection name in Qdrant\n3) Size of Voyage embeddings (needed for collection creation in Qdrant) <this one should not be changed unless the embedding model is changed>\n4) Batch size for batch embedding/batch uploading to Qdrant "
},
"typeVersion": 1
},
{
"id": "fae9248c-dbcc-4b6d-b977-0047f120a587",
"name": "Sticky Note2",
"type": "n8n-nodes-base.stickyNote",
"position": [
-100,
-220
],
"parameters": {
"content": "In Qdrant, you can create a collection once; if you try to create it two times with the same name, you'll get an error, so I am adding here a check if a collection with this name exists already"
},
"typeVersion": 1
},
{
"id": "f7aea242-3d98-4a1c-a98a-986ac2b4928b",
"name": "Sticky Note3",
"type": "n8n-nodes-base.stickyNote",
"position": [
180,
340
],
"parameters": {
"height": 280,
"content": "If a collection with the name set up in variables doesn't exist yet, I create an empty one; \n\nCollection will contain [named vectors](https://qdrant.tech/documentation/concepts/vectors/#named-vectors), with a name *\"voyage\"*\nFor these named vectors, I define two parameters:\n1) Vectors size (in our case, Voyage embeddings size)\n2) Similarity metric to compare embeddings: in our case, **\"Cosine\"**.\n"
},
"typeVersion": 1
},
{
"id": "b84045c1-f66a-4543-8d42-1e76de0b6e91",
"name": "Sticky Note4",
"type": "n8n-nodes-base.stickyNote",
"position": [
800,
-280
],
"parameters": {
"height": 400,
"content": "Now it's time to embed & upload to Qdrant our image datasets;\nBoth of them, [crops](https://www.kaggle.com/datasets/mdwaquarazam/agricultural-crops-image-classification) and [lands](https://www.kaggle.com/datasets/apollo2506/landuse-scene-classification) were uploaded to our Google Cloud Storage bucket, and in this workflow we're fetching **the crops dataset** (for lands it will be a nearly identical workflow, up to variable names)\n(you should replace it with your image datasets)\n\nDatasets consist of **image URLs**; images are grouped by folders based on their class. For example, we have a system of subfolders like *\"tomato\"* and *\"cucumber\"* for the crops dataset with image URLs of the respective class.\n"
},
"typeVersion": 1
},
{
"id": "255dfad8-c545-4d75-bc9c-529aa50447a9",
"name": "Sticky Note5",
"type": "n8n-nodes-base.stickyNote",
"position": [
1080,
-140
],
"parameters": {
"height": 240,
"content": "Google Storage node returns **mediaLink**, which can be used directly for downloading images; however, we just need a public image URL so that Voyage API can process it; so here we construct this public link and extract a crop name from the folder in which image was stored (for example, *\"cucumber\"*)\n"
},
"typeVersion": 1
},
{
"id": "a6acce75-cce0-4de3-bc64-37592c97359b",
"name": "Sticky Note6",
"type": "n8n-nodes-base.stickyNote",
"position": [
1600,
-80
],
"parameters": {
"height": 180,
"content": "I regroup images into batches of `batchSize` size and, to make batch upload to Qdrant possible, generate UUIDs to use them as batch [point IDs](https://qdrant.tech/documentation/concepts/points/#point-ids) (Qdrant doesn't set up id's for the user; users have to choose them themselves)"
},
"typeVersion": 1
},
{
"id": "cab3cc83-b50c-41f4-8d51-59e04bba5556",
"name": "Sticky Note7",
"type": "n8n-nodes-base.stickyNote",
"position": [
1340,
-60
],
"parameters": {
"content": "Since we build anomaly detection based on the crops dataset, to test it properly, I didn't upload to Qdrant pictures of tomatoes at all; I filter them out here"
},
"typeVersion": 1
},
{
"id": "e5cdcce5-efdc-41f2-9796-656bd345f783",
"name": "Sticky Note9",
"type": "n8n-nodes-base.stickyNote",
"position": [
1860,
-100
],
"parameters": {
"height": 200,
"content": "Since Voyage API requires a [specific json structure](https://docs.voyageai.com/reference/multimodal-embeddings-api) for batch embeddings, as does [Qdrant's API for uploading points in batches](https://api.qdrant.tech/api-reference/points/upsert-points), I am adapting the structure of jsons\n\n[NB] - [payload = meta data in Qdrant]"
},
"typeVersion": 1
},
{
"id": "a7f15c44-3d5c-4b43-bfb2-94fe27a32071",
"name": "Sticky Note11",
"type": "n8n-nodes-base.stickyNote",
"position": [
2120,
20
],
"parameters": {
"width": 180,
"height": 80,
"content": "Embedding images with Voyage model (mind `input_type`)"
},
"typeVersion": 1
},
{
"id": "01b92e7e-d954-4d58-85b1-109c336546c4",
"name": "Filtering out tomato to test anomalies",
"type": "n8n-nodes-base.filter",
"position": [
1340,
160
],
"parameters": {
"options": {},
"conditions": {
"options": {
"version": 2,
"leftValue": "",
"caseSensitive": true,
"typeValidation": "strict"
},
"combinator": "and",
"conditions": [
{
"id": "f7953ae2-5333-4805-abe5-abf6da645c5e",
"operator": {
"type": "string",
"operation": "notEquals"
},
"leftValue": "={{ $json.cropName }}",
"rightValue": "tomato"
}
]
}
},
"typeVersion": 2.2
},
{
"id": "8d564817-885e-453a-a087-900b34b84d9c",
"name": "Sticky Note8",
"type": "n8n-nodes-base.stickyNote",
"position": [
-1160,
-280
],
"parameters": {
"width": 440,
"height": 460,
"content": "## Batch Uploading Dataset to Qdrant \n### This template imports dataset images from storage, creates embeddings for them in batches, and uploads them to Qdrant in batches. In this particular template, we work with [crops dataset](https://www.kaggle.com/datasets/mdwaquarazam/agricultural-crops-image-classification). However, it's analogous to [lands dataset](https://www.kaggle.com/datasets/apollo2506/landuse-scene-classification), and in general, it's adaptable to any dataset consisting of image URLs (as the following pipelines are).\n\n* First, check for an existing Qdrant collection to use; otherwise, create it here. Additionally, when creating the collection, we'll create a [payload index](https://qdrant.tech/documentation/concepts/indexing/#payload-index), which is required for a particular type of Qdrant requests we will use later.\n* Next, import all (dataset) images from Google Storage but keep only non-tomato-related ones (for anomaly detection testing).\n* Create (per batch) embeddings for all imported images using the Voyage AI multimodal embeddings API.\n* Finally, upload the resulting embeddings and image descriptors to Qdrant via batch uploading."
},
"typeVersion": 1
},
{
"id": "0233d3d0-bbdf-4d5b-a366-53cbfa4b6f9c",
"name": "Sticky Note10",
"type": "n8n-nodes-base.stickyNote",
"position": [
-860,
360
],
"parameters": {
"color": 4,
"width": 540,
"height": 420,
"content": "### For anomaly detection\n**1. This is the first pipeline to upload (crops) dataset to Qdrant's collection.**\n2. The second pipeline is to set up cluster (class) centres in this Qdrant collection & cluster (class) threshold scores.\n3. The third is the anomaly detection tool, which takes any image as input and uses all preparatory work done with Qdrant (crops) collection.\n\n### For KNN (k nearest neighbours) classification\n**1. This is the first pipeline to upload (lands) dataset to Qdrant's collection.**\n2. The second is the KNN classifier tool, which takes any image as input and classifies it based on queries to the Qdrant (lands) collection.\n\n### To recreate both\nYou'll have to upload [crops](https://www.kaggle.com/datasets/mdwaquarazam/agricultural-crops-image-classification) and [lands](https://www.kaggle.com/datasets/apollo2506/landuse-scene-classification) datasets from Kaggle to your own Google Storage bucket, and re-create APIs/connections to [Qdrant Cloud](https://qdrant.tech/documentation/quickstart-cloud/) (you can use **Free Tier** cluster), Voyage AI API & Google Cloud Storage\n\n**In general, pipelines are adaptable to any dataset of images**\n"
},
"typeVersion": 1
}
],
"active": false,
"settings": {
"executionOrder": "v1"
},
"versionId": "27776c4a-3bf9-4704-9c13-345b75ffacc0",
"connections": {
"Embed crop image": {
"main": [
[
{
"node": "Batch Upload to Qdrant",
"type": "main",
"index": 0
}
]
]
},
"Google Cloud Storage": {
"main": [
[
{
"node": "Get fields for Qdrant",
"type": "main",
"index": 0
}
]
]
},
"If collection exists": {
"main": [
[
{
"node": "Google Cloud Storage",
"type": "main",
"index": 0
}
],
[
{
"node": "Create Qdrant Collection",
"type": "main",
"index": 0
}
]
]
},
"Get fields for Qdrant": {
"main": [
[
{
"node": "Filtering out tomato to test anomalies",
"type": "main",
"index": 0
}
]
]
},
"Batch Upload to Qdrant": {
"main": [
[]
]
},
"Create Qdrant Collection": {
"main": [
[
{
"node": "Payload index on crop_name",
"type": "main",
"index": 0
}
]
]
},
"Qdrant cluster variables": {
"main": [
[
{
"node": "Check Qdrant Collection Existence",
"type": "main",
"index": 0
}
]
]
},
"Payload index on crop_name": {
"main": [
[
{
"node": "Google Cloud Storage",
"type": "main",
"index": 0
}
]
]
},
"Batches in the API's format": {
"main": [
[
{
"node": "Embed crop image",
"type": "main",
"index": 0
}
]
]
},
"Check Qdrant Collection Existence": {
"main": [
[
{
"node": "If collection exists",
"type": "main",
"index": 0
}
]
]
},
"When clicking \u2018Test workflow\u2019": {
"main": [
[
{
"node": "Qdrant cluster variables",
"type": "main",
"index": 0
}
]
]
},
"Filtering out tomato to test anomalies": {
"main": [
[
{
"node": "Split in batches, generate uuids for Qdrant points",
"type": "main",
"index": 0
}
]
]
},
"Split in batches, generate uuids for Qdrant points": {
"main": [
[
{
"node": "Batches in the API's format",
"type": "main",
"index": 0
}
]
]
}
}
}
Credentials you'll need
Each integration node will prompt for credentials when you import. We strip credential IDs before publishing — you'll add your own.
googleCloudStorageOAuth2ApihttpHeaderAuthqdrantApi
Pro
For the full experience including quality scoring and batch install features for each workflow upgrade to Pro
How this works
This workflow enables farmers and agronomists to swiftly upload and organise crop image datasets into Qdrant for anomaly detection and KNN classification, transforming raw data from Google Cloud Storage into a searchable vector database without manual effort. It saves hours of preprocessing by automating embedding generation for each image and batching uploads to handle large volumes efficiently. The key step involves using HTTP requests to create and populate a Qdrant collection, ensuring your data is ready for machine learning analysis on crop health anomalies.
Use this workflow when you have batches of crop images stored in Google Cloud Storage and need to build a foundation for AI-driven anomaly detection in agriculture, such as identifying diseased plants. Avoid it for real-time processing or non-image datasets, as it's optimised for offline batch uploads. Common variations include adapting the embedding endpoint for different models or scaling batches for massive datasets exceeding API limits.
About this workflow
[1/3 - anomaly detection] [1/2 - KNN classification] Batch upload dataset to Qdrant (crops dataset). Uses manualTrigger, googleCloudStorage, httpRequest, stickyNote. Event-driven trigger; 25 nodes.
Source: https://github.com/Zie619/n8n-workflows — original creator credit. Request a take-down →
Related workflows
Workflows that share integrations, category, or trigger type with this one. All free to copy and import.
[1/3 - anomaly detection] [1/2 - KNN classification] Batch upload dataset to Qdrant (crops dataset). Uses manualTrigger, googleCloudStorage, httpRequest, stickyNote. Event-driven trigger; 25 nodes.
Google Cloud Storage, HTTP Request
Workflows from the webinar "Build production-ready AI Agents with Qdrant and n8n".
Google Cloud Storage, HTTP Request
This pipeline is the first part of "Hybrid Search with Qdrant & n8n, Legal AI"*. The second part, "Hybrid Search with Qdrant & n8n, Legal AI: Retrieval", covers retrieval and simple evaluation.*
N8N Nodes Qdrant, HTTP Request
This is the second part of "Hybrid Search with Qdrant & n8n, Legal AI."* The first part, "Indexing", covers preparing and uploading the dataset to Qdrant.*
HTTP Request, N8N Nodes Qdrant
Api Schema Extractor. Uses manualTrigger, httpRequest, splitOut, textSplitterRecursiveCharacterTextSplitter. Event-driven trigger; 88 nodes.
HTTP Request, Text Splitter Recursive Character Text Splitter, Document Default Data Loader +9