Redowan Ahmed Farhan This workflow corresponds to n8n.io template #16633 — we link there as the canonical source.
This workflow follows the Agent → OpenAI Chat recipe pattern — see all workflows that pair these two integrations.
The workflow JSON
Copy or download the full n8n JSON below. Paste it into a new n8n workflow, add your credentials, activate. Full import guide →
{
"id": "J5NwzcdqWOHJbH5y",
"meta": {
"templateCredsSetupCompleted": true
},
"name": "Autonomous AI Crawler",
"tags": [],
"nodes": [
{
"id": "e2ec4c3c-940e-4993-908d-9d515e6e5816",
"name": "Text",
"type": "@n8n/n8n-nodes-langchain.toolWorkflow",
"position": [
-512,
1072
],
"parameters": {
"name": "text_retrieval_tool",
"source": "parameter",
"description": "Call this tool to return all text from the given website. Query should be full website URL.",
"workflowJson": "{\n \"nodes\": [\n {\n \"parameters\": {},\n \"id\": \"05107436-c9cb-419b-ae8a-b74d309a130d\",\n \"name\": \"Execute workflow\",\n \"type\": \"n8n-nodes-base.manualTrigger\",\n \"typeVersion\": 1,\n \"position\": [\n 2220,\n 620\n ]\n },\n {\n \"parameters\": {\n \"assignments\": {\n \"assignments\": [\n {\n \"id\": \"253c2b17-c749-4f0a-93e8-5ff74f1ce49b\",\n \"name\": \"domain\",\n \"value\": \"={{ $json.query }}\",\n \"type\": \"string\"\n }\n ]\n },\n \"options\": {}\n },\n \"id\": \"bb8be616-3227-4705-8520-1827069faacd\",\n \"name\": \"Set domain\",\n \"type\": \"n8n-nodes-base.set\",\n \"typeVersion\": 3.3,\n \"position\": [\n 2440,\n 620\n ]\n },\n {\n \"parameters\": {\n \"assignments\": {\n \"assignments\": [\n {\n \"id\": \"ed0f1505-82b6-4393-a0d8-088055137ec9\",\n \"name\": \"domain\",\n \"value\": \"={{ $json.domain.startsWith(\\\"http\\\") ? $json.domain : \\\"http://\\\" + $json.domain }}\",\n \"type\": \"string\"\n }\n ]\n },\n \"options\": {}\n },\n \"id\": \"bdf29340-f135-489f-848e-1c7fa43a01df\",\n \"name\": \"Add protocool to domain\",\n \"type\": \"n8n-nodes-base.set\",\n \"typeVersion\": 3.3,\n \"position\": [\n 2640,\n 620\n ]\n },\n {\n \"parameters\": {\n \"assignments\": {\n \"assignments\": [\n {\n \"id\": \"2b1c7ff8-06a7-448b-99b7-5ede4b2e0bf0\",\n \"name\": \"response\",\n \"value\": \"={{ $json.data }}\",\n \"type\": \"string\"\n }\n ]\n },\n \"options\": {}\n },\n \"id\": \"9f0aa264-08c1-459a-bb99-e28599fe8f76\",\n \"name\": \"Set response\",\n \"type\": \"n8n-nodes-base.set\",\n \"typeVersion\": 3.3,\n \"position\": [\n 3300,\n 620\n ]\n },\n {\n \"parameters\": {\n \"url\": \"={{ $json.domain }}\",\n \"options\": {}\n },\n \"id\": \"cec7c8e8-bf5e-43d5-aa41-876293dbec78\",\n \"name\": \"Get website\",\n \"type\": \"n8n-nodes-base.httpRequest\",\n \"typeVersion\": 4.2,\n \"position\": [\n 2860,\n 620\n ]\n },\n {\n \"parameters\": {\n \"html\": \"={{ $json.data }}\",\n \"options\": {\n \"ignore\": \"a,img\"\n }\n },\n \"id\": \"1af94fcb-bca3-45c4-9277-18878c75d417\",\n \"name\": \"Convert HTML to Markdown\",\n \"type\": \"n8n-nodes-base.markdown\",\n \"typeVersion\": 1,\n \"position\": [\n 3080,\n 620\n ]\n }\n ],\n \"connections\": {\n \"Execute workflow\": {\n \"main\": [\n [\n {\n \"node\": \"Set domain\",\n \"type\": \"main\",\n \"index\": 0\n }\n ]\n ]\n },\n \"Set domain\": {\n \"main\": [\n [\n {\n \"node\": \"Add protocool to domain\",\n \"type\": \"main\",\n \"index\": 0\n }\n ]\n ]\n },\n \"Add protocool to domain\": {\n \"main\": [\n [\n {\n \"node\": \"Get website\",\n \"type\": \"main\",\n \"index\": 0\n }\n ]\n ]\n },\n \"Get website\": {\n \"main\": [\n [\n {\n \"node\": \"Convert HTML to Markdown\",\n \"type\": \"main\",\n \"index\": 0\n }\n ]\n ]\n },\n \"Convert HTML to Markdown\": {\n \"main\": [\n [\n {\n \"node\": \"Set response\",\n \"type\": \"main\",\n \"index\": 0\n }\n ]\n ]\n }\n },\n \"pinData\": {}\n}"
},
"typeVersion": 1.1
},
{
"id": "e47a5b79-c589-4350-90a7-6a3bff079ec6",
"name": "URLs",
"type": "@n8n/n8n-nodes-langchain.toolWorkflow",
"position": [
-336,
1072
],
"parameters": {
"name": "url_retrieval_tool",
"source": "parameter",
"description": "Call this tool to return all URLs from the given website. Query should be full website URL.",
"workflowJson": "{\n \"nodes\": [\n {\n \"parameters\": {},\n \"id\": \"05107436-c9cb-419b-ae8a-b74d309a130d\",\n \"name\": \"Execute workflow\",\n \"type\": \"n8n-nodes-base.manualTrigger\",\n \"typeVersion\": 1,\n \"position\": [\n 2200,\n 740\n ]\n },\n {\n \"parameters\": {\n \"operation\": \"extractHtmlContent\",\n \"extractionValues\": {\n \"values\": [\n {\n \"key\": \"output\",\n \"cssSelector\": \"a\",\n \"returnValue\": \"attribute\",\n \"returnArray\": true\n }\n ]\n },\n \"options\": {}\n },\n \"id\": \"1972e13e-d923-45e8-9752-e4bf45faaccf\",\n \"name\": \"Retrieve URLs\",\n \"type\": \"n8n-nodes-base.html\",\n \"typeVersion\": 1.2,\n \"position\": [\n 3060,\n 740\n ]\n },\n {\n \"parameters\": {\n \"fieldToSplitOut\": \"output\",\n \"options\": {}\n },\n \"id\": \"19703fbc-05ff-4d80-ab53-85ba6d39fc3f\",\n \"name\": \"Split out URLs\",\n \"type\": \"n8n-nodes-base.splitOut\",\n \"typeVersion\": 1,\n \"position\": [\n 3280,\n 740\n ]\n },\n {\n \"parameters\": {\n \"compare\": \"selectedFields\",\n \"fieldsToCompare\": \"href\",\n \"options\": {}\n },\n \"id\": \"5cc988e7-de9b-4177-b5e7-edb3842202c8\",\n \"name\": \"Remove duplicated\",\n \"type\": \"n8n-nodes-base.removeDuplicates\",\n \"typeVersion\": 1,\n \"position\": [\n 3720,\n 740\n ]\n },\n {\n \"parameters\": {\n \"assignments\": {\n \"assignments\": [\n {\n \"id\": \"04ced063-09f0-496c-9b28-b8095f9e2297\",\n \"name\": \"href\",\n \"value\": \"={{ $json.href.startsWith(\\\"/\\\") ? $('Add protocool to domain (URL)').item.json[\\\"domain\\\"] + $json.href : $json.href }}\",\n \"type\": \"string\"\n }\n ]\n },\n \"includeOtherFields\": true,\n \"include\": \"selected\",\n \"includeFields\": \"title\",\n \"options\": {}\n },\n \"id\": \"4715a25d-93a7-4056-8768-e3f886a1a0c9\",\n \"name\": \"Set domain to path\",\n \"type\": \"n8n-nodes-base.set\",\n \"typeVersion\": 3.3,\n \"position\": [\n 3940,\n 740\n ]\n },\n {\n \"parameters\": {\n \"conditions\": {\n \"options\": {\n \"caseSensitive\": true,\n \"leftValue\": \"\",\n \"typeValidation\": \"strict\"\n },\n \"conditions\": [\n {\n \"id\": \"d01ea6a8-7e75-40d4-98f2-25d42b245f36\",\n \"leftValue\": \"={{ $json.href.isUrl() }}\",\n \"rightValue\": \"\",\n \"operator\": {\n \"type\": \"boolean\",\n \"operation\": \"true\",\n \"singleValue\": true\n }\n }\n ],\n \"combinator\": \"and\"\n },\n \"options\": {}\n },\n \"id\": \"353deefb-ae69-440c-95b6-fdadacf4bf91\",\n \"name\": \"Filter out invalid URLs\",\n \"type\": \"n8n-nodes-base.filter\",\n \"typeVersion\": 2,\n \"position\": [\n 4160,\n 740\n ]\n },\n {\n \"parameters\": {\n \"aggregate\": \"aggregateAllItemData\",\n \"include\": \"specifiedFields\",\n \"fieldsToInclude\": \"title,href\",\n \"options\": {}\n },\n \"id\": \"9f87be8c-72d7-4ab1-b297-dc7069b2dd11\",\n \"name\": \"Aggregate URLs\",\n \"type\": \"n8n-nodes-base.aggregate\",\n \"typeVersion\": 1,\n \"position\": [\n 4380,\n 740\n ]\n },\n {\n \"parameters\": {\n \"conditions\": {\n \"options\": {\n \"caseSensitive\": true,\n \"leftValue\": \"\",\n \"typeValidation\": \"strict\"\n },\n \"conditions\": [\n {\n \"id\": \"5b9b7353-bd04-4af2-9480-8de135ff4223\",\n \"leftValue\": \"={{ $json.href }}\",\n \"rightValue\": \"\",\n \"operator\": {\n \"type\": \"string\",\n \"operation\": \"exists\",\n \"singleValue\": true\n }\n }\n ],\n \"combinator\": \"and\"\n },\n \"options\": {}\n },\n \"id\": \"35c8323a-5350-403a-9c2d-114b0527e395\",\n \"name\": \"Filter out empty hrefs\",\n \"type\": \"n8n-nodes-base.filter\",\n \"typeVersion\": 2,\n \"position\": [\n 3500,\n 740\n ]\n },\n {\n \"parameters\": {\n \"assignments\": {\n \"assignments\": [\n {\n \"id\": \"253c2b17-c749-4f0a-93e8-5ff74f1ce49b\",\n \"name\": \"domain\",\n \"value\": \"={{ $json.query }}\",\n \"type\": \"string\"\n }\n ]\n },\n \"options\": {}\n },\n \"id\": \"d9f6a148-6c8c-4a58-89f5-4e9cfcd8d910\",\n \"name\": \"Set domain (URL)\",\n \"type\": \"n8n-nodes-base.set\",\n \"typeVersion\": 3.3,\n \"position\": [\n 2400,\n 740\n ]\n },\n {\n \"parameters\": {\n \"assignments\": {\n \"assignments\": [\n {\n \"id\": \"ed0f1505-82b6-4393-a0d8-088055137ec9\",\n \"name\": \"domain\",\n \"value\": \"={{ $json.domain.startsWith(\\\"http\\\") ? $json.domain : \\\"http://\\\" + $json.domain }}\",\n \"type\": \"string\"\n }\n ]\n },\n \"options\": {}\n },\n \"id\": \"1f974444-da58-4a47-a9c3-ba3091fc1e96\",\n \"name\": \"Add protocool to domain (URL)\",\n \"type\": \"n8n-nodes-base.set\",\n \"typeVersion\": 3.3,\n \"position\": [\n 2620,\n 740\n ]\n },\n {\n \"parameters\": {\n \"url\": \"={{ $json.domain }}\",\n \"options\": {}\n },\n \"id\": \"31d7c7d4-8f61-402b-858d-63dd68ac69ee\",\n \"name\": \"Get website (URL)\",\n \"type\": \"n8n-nodes-base.httpRequest\",\n \"typeVersion\": 4.2,\n \"position\": [\n 2840,\n 740\n ]\n },\n {\n \"parameters\": {\n \"assignments\": {\n \"assignments\": [\n {\n \"id\": \"53c1c016-7983-4eba-a91d-da2a0523d805\",\n \"name\": \"response\",\n \"value\": \"={{ JSON.stringify($json.data) }}\",\n \"type\": \"string\"\n }\n ]\n },\n \"options\": {}\n },\n \"id\": \"f4b6df77-96be-4b12-9a8b-ae9b7009f13d\",\n \"name\": \"Set response (URL)\",\n \"type\": \"n8n-nodes-base.set\",\n \"typeVersion\": 3.3,\n \"position\": [\n 4600,\n 740\n ]\n }\n ],\n \"connections\": {\n \"Execute workflow\": {\n \"main\": [\n [\n {\n \"node\": \"Set domain (URL)\",\n \"type\": \"main\",\n \"index\": 0\n }\n ]\n ]\n },\n \"Retrieve URLs\": {\n \"main\": [\n [\n {\n \"node\": \"Split out URLs\",\n \"type\": \"main\",\n \"index\": 0\n }\n ]\n ]\n },\n \"Split out URLs\": {\n \"main\": [\n [\n {\n \"node\": \"Filter out empty hrefs\",\n \"type\": \"main\",\n \"index\": 0\n }\n ]\n ]\n },\n \"Remove duplicated\": {\n \"main\": [\n [\n {\n \"node\": \"Set domain to path\",\n \"type\": \"main\",\n \"index\": 0\n }\n ]\n ]\n },\n \"Set domain to path\": {\n \"main\": [\n [\n {\n \"node\": \"Filter out invalid URLs\",\n \"type\": \"main\",\n \"index\": 0\n }\n ]\n ]\n },\n \"Filter out invalid URLs\": {\n \"main\": [\n [\n {\n \"node\": \"Aggregate URLs\",\n \"type\": \"main\",\n \"index\": 0\n }\n ]\n ]\n },\n \"Aggregate URLs\": {\n \"main\": [\n [\n {\n \"node\": \"Set response (URL)\",\n \"type\": \"main\",\n \"index\": 0\n }\n ]\n ]\n },\n \"Filter out empty hrefs\": {\n \"main\": [\n [\n {\n \"node\": \"Remove duplicated\",\n \"type\": \"main\",\n \"index\": 0\n }\n ]\n ]\n },\n \"Set domain (URL)\": {\n \"main\": [\n [\n {\n \"node\": \"Add protocool to domain (URL)\",\n \"type\": \"main\",\n \"index\": 0\n }\n ]\n ]\n },\n \"Add protocool to domain (URL)\": {\n \"main\": [\n [\n {\n \"node\": \"Get website (URL)\",\n \"type\": \"main\",\n \"index\": 0\n }\n ]\n ]\n },\n \"Get website (URL)\": {\n \"main\": [\n [\n {\n \"node\": \"Retrieve URLs\",\n \"type\": \"main\",\n \"index\": 0\n }\n ]\n ]\n }\n },\n \"pinData\": {}\n}"
},
"typeVersion": 1.1
},
{
"id": "8f422426-17c5-478d-ac94-f0a78822d40e",
"name": "OpenAI Chat Model",
"type": "@n8n/n8n-nodes-langchain.lmChatOpenAi",
"position": [
-672,
1072
],
"parameters": {
"model": "gpt-4o",
"options": {
"temperature": 0,
"responseFormat": "json_object"
}
},
"typeVersion": 1
},
{
"id": "09db3556-ca91-4c26-8659-ac81f4f038c3",
"name": "JSON Parser",
"type": "@n8n/n8n-nodes-langchain.outputParserStructured",
"position": [
-176,
1072
],
"parameters": {
"schemaType": "manual",
"inputSchema": "{\n \"type\": \"object\",\n \"properties\": {\n \"social_media\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"object\",\n \"properties\": {\n \"platform\": {\n \"type\": \"string\",\n \"description\": \"The name of the social media platform (e.g., LinkedIn, Instagram)\"\n },\n \"urls\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\",\n \"format\": \"uri\",\n \"description\": \"A URL for the social media platform\"\n }\n }\n },\n \"required\": [\"platform\", \"urls\"],\n \"additionalProperties\": false\n }\n }\n },\n \"required\": [\"platforms\"],\n \"additionalProperties\": false\n}\n"
},
"typeVersion": 1.2
},
{
"id": "40b3d61d-c058-4afc-ac6f-c6c9205e5351",
"name": "Map company name and website",
"type": "n8n-nodes-base.set",
"position": [
-464,
528
],
"parameters": {
"options": {},
"assignments": {
"assignments": [
{
"id": "ae484e44-36bc-4d88-9772-545e579a261c",
"name": "company_name",
"type": "string",
"value": "={{ $json.name }}"
},
{
"id": "c426ab19-649c-4443-aabb-eb0826680452",
"name": "company_website",
"type": "string",
"value": "={{ $json.website }}"
}
]
}
},
"typeVersion": 3.3
},
{
"id": "a561cf9b-3cf8-4776-a6f7-6e32263574ca",
"name": "Execute workflow",
"type": "n8n-nodes-base.manualTrigger",
"position": [
-1536,
752
],
"parameters": {},
"typeVersion": 1
},
{
"id": "f817872b-b5b4-4392-b630-c9e42d1bc90e",
"name": "Get companies",
"type": "n8n-nodes-base.supabase",
"position": [
-1296,
752
],
"parameters": {
"tableId": "companies_input",
"operation": "getAll"
},
"typeVersion": 1
},
{
"id": "8ca87503-35d3-4016-911a-77d25287171c",
"name": "Select company name and website",
"type": "n8n-nodes-base.set",
"position": [
-1040,
752
],
"parameters": {
"include": "selected",
"options": {},
"assignments": {
"assignments": []
},
"includeFields": "name,website",
"includeOtherFields": true
},
"typeVersion": 3.3
},
{
"id": "1c59b72e-f9a7-41fc-8bae-d15a51175e4c",
"name": "Set social media array",
"type": "n8n-nodes-base.set",
"position": [
-176,
816
],
"parameters": {
"options": {},
"assignments": {
"assignments": [
{
"id": "a6e109b7-9333-44e8-aa13-590aeb91a56b",
"name": "social_media",
"type": "array",
"value": "={{ $json.output.social_media }}"
}
]
}
},
"typeVersion": 3.3
},
{
"id": "4a3bf11e-ffae-4b3a-bd91-34374d9af5e9",
"name": "Merge all data",
"type": "n8n-nodes-base.merge",
"position": [
80,
720
],
"parameters": {
"mode": "combine",
"options": {},
"combinationMode": "mergeByPosition"
},
"typeVersion": 2.1
},
{
"id": "79d96472-bca6-4b17-8040-729d8ea7f411",
"name": "Insert new row",
"type": "n8n-nodes-base.supabase",
"position": [
320,
720
],
"parameters": {
"tableId": "companies_output",
"dataToSend": "autoMapInputData"
},
"typeVersion": 1
},
{
"id": "5929dc22-4ed0-4a08-bc3f-5c918c10dda0",
"name": "Crawl website",
"type": "@n8n/n8n-nodes-langchain.agent",
"position": [
-576,
816
],
"parameters": {
"text": "=Retrieve social media profile URLs from this website: {{ $json.website }}",
"options": {
"systemMessage": "You are an automated web crawler tasked with extracting social media URLs from a webpage provided by the user. You have access to a text retrieval tool to gather all text content from the page and a URL retrieval tool to identify and navigate through links on the page. Utilize the URLs retrieved to crawl additional pages. Your objective is to provide a unified JSON output containing the extracted data (links to all possible social media profiles from the website)."
},
"promptType": "define",
"hasOutputParser": true
},
"retryOnFail": true,
"typeVersion": 1.6
},
{
"id": "8063a285-0f0e-4f76-bb1a-dbbc5d8beee4",
"name": "Main Overview",
"type": "n8n-nodes-base.stickyNote",
"position": [
-2256,
720
],
"parameters": {
"color": 7,
"width": 580,
"height": 1688,
"content": "## \ud83d\udd77\ufe0f Autonomous AI Crawler\n\nAn AI agent autonomously visits each company website, follows internal links, and extracts social media profile URLs into a clean JSON structure, then writes the results back to your database.\n\n**Perfect for:** Growth, sales-ops, and lead-enrichment teams who need to enrich a company list with social profiles at scale.\n\n---\n\n## How it works\n\n1. **Execute workflow** \u2014 Manual trigger that starts the run.\n2. **Get companies** \u2014 Pulls company name and website rows from a Supabase table.\n3. **Select company name and website** \u2014 Keeps only the `name` and `website` fields to pass downstream.\n4. **Crawl website** \u2014 The AI agent that drives the crawl, calling its tools until it has the social links.\n5. **OpenAI Chat Model** \u2014 GPT-4o brain for the agent, set to temperature 0 and JSON response mode.\n6. **Text** \u2014 `text_retrieval_tool`; returns all readable text from a given page.\n7. **URLs** \u2014 `url_retrieval_tool`; returns all links found on a given page so the agent can navigate.\n8. **JSON Parser** \u2014 Structured output parser that forces the agent's answer into a fixed social-links schema.\n9. **Map company name and website** \u2014 Carries the original company name and website forward in parallel.\n10. **Set social media array** \u2014 Shapes the agent's extracted profiles into an array field.\n11. **Merge all data** \u2014 Combines the mapped company info with the extracted social media array.\n12. **Insert new row** \u2014 Writes the enriched record back into Supabase.\n\n**Text scraper tool** *(called by the agent)*\n13. **Set domain (text)** \u2014 Reads the requested URL from the agent's query.\n14. **Add protocool to domain (text)** \u2014 Prepends `http://` if no protocol is present.\n15. **Get website (text)** \u2014 HTTP request that fetches the raw page HTML.\n16. **Convert HTML to Markdown** \u2014 Strips links and images, returns clean readable text.\n17. **Set response (text)** \u2014 Returns the text payload to the agent.\n\n**URL scraper tool** *(called by the agent)*\n18. **Set domain (URL)** \u2014 Reads the requested URL from the agent's query.\n19. **Add protocool to domain (URL)** \u2014 Prepends `http://` if missing.\n20. **Get website (URL)** \u2014 HTTP request that fetches the raw page HTML.\n21. **Retrieve URLs** \u2014 Extracts every `<a>` href from the HTML.\n22. **Split out URLs** \u2014 Splits the href array into individual items.\n23. **Filter out empty hrefs** \u2014 Drops items with no link.\n24. **Remove duplicated** \u2014 Removes duplicate hrefs.\n25. **Set domain to path** \u2014 Resolves relative `/path` links into absolute URLs.\n26. **Filter out invalid URLs** \u2014 Keeps only well-formed URLs.\n27. **Aggregate URLs** \u2014 Collects all valid links back into one list.\n28. **Set response (URL)** \u2014 Returns the link list to the agent.\n\n---\n\n## Setup (~10 minutes)\n\n1. **Supabase** \u2014 Add your credential in the *Get companies* and *Insert new row* nodes, and point them at your company table.\n2. **OpenAI** \u2014 Add your API key in the *OpenAI Chat Model* node (uses GPT-4o).\n3. **Field names** \u2014 If your table does not use `name` and `website`, update the *Select company name and website* node to match.\n> Crawling unfamiliar sites can hit rate limits or blocks. Consider adding a proxy to the *Get website (text)* and *Get website (URL)* HTTP nodes for reliable crawling. OpenAI usage is billed per token.\n"
},
"typeVersion": 1
},
{
"id": "3a9bad9d-e3fd-4242-be93-13888dc24ded",
"name": "Section - Trigger & Intake",
"type": "n8n-nodes-base.stickyNote",
"position": [
-1648,
592
],
"parameters": {
"color": 5,
"width": 760,
"height": 360,
"content": "## 1\ufe0f\u20e3 Trigger & Company Intake\n\nThe **Execute workflow** trigger starts the run and **Get companies** pulls the list of company names and websites from your Supabase table. **Select company name and website** trims each record down to just the `name` and `website` fields before handing them to the crawling agent.\n"
},
"typeVersion": 1
},
{
"id": "7dc5f0d9-ad2e-4bfa-aeef-a9657a775ff1",
"name": "Section - AI Agent",
"type": "n8n-nodes-base.stickyNote",
"position": [
-832,
176
],
"parameters": {
"color": 3,
"width": 820,
"height": 1072,
"content": "## 2\ufe0f\u20e3 AI Crawling Agent\n\n**Crawl website** is the autonomous agent that navigates each site to find social media profile links. It thinks with the **OpenAI Chat Model** (GPT-4o), gathers page content through the **Text** and **URLs** tools, and is forced into a clean JSON structure by the **JSON Parser**. In parallel, **Map company name and website** preserves the original company details, and **Set social media array** shapes the agent's findings into an array.\n"
},
"typeVersion": 1
},
{
"id": "2e04f241-ceea-46ab-9b22-0e8c5347f1d5",
"name": "Section - Output & Logging",
"type": "n8n-nodes-base.stickyNote",
"position": [
32,
496
],
"parameters": {
"color": 4,
"width": 520,
"height": 424,
"content": "## 3\ufe0f\u20e3 Output & Logging\n\n**Merge all data** combines the preserved company name and website with the social media array extracted by the agent into a single record. **Insert new row** then writes that enriched record back into your Supabase table.\n"
},
"typeVersion": 1
}
],
"active": false,
"settings": {
"binaryMode": "separate",
"executionOrder": "v1"
},
"versionId": "1b4f0bd9-306a-4fc1-a95c-e708e5517203",
"nodeGroups": [],
"connections": {
"Text": {
"ai_tool": [
[
{
"node": "Crawl website",
"type": "ai_tool",
"index": 0
}
]
]
},
"URLs": {
"ai_tool": [
[
{
"node": "Crawl website",
"type": "ai_tool",
"index": 0
}
]
]
},
"JSON Parser": {
"ai_outputParser": [
[
{
"node": "Crawl website",
"type": "ai_outputParser",
"index": 0
}
]
]
},
"Crawl website": {
"main": [
[
{
"node": "Set social media array",
"type": "main",
"index": 0
}
]
]
},
"Get companies": {
"main": [
[
{
"node": "Select company name and website",
"type": "main",
"index": 0
}
]
]
},
"Merge all data": {
"main": [
[
{
"node": "Insert new row",
"type": "main",
"index": 0
}
]
]
},
"Execute workflow": {
"main": [
[
{
"node": "Get companies",
"type": "main",
"index": 0
}
]
]
},
"OpenAI Chat Model": {
"ai_languageModel": [
[
{
"node": "Crawl website",
"type": "ai_languageModel",
"index": 0
}
]
]
},
"Set social media array": {
"main": [
[
{
"node": "Merge all data",
"type": "main",
"index": 1
}
]
]
},
"Map company name and website": {
"main": [
[
{
"node": "Merge all data",
"type": "main",
"index": 0
}
]
]
},
"Select company name and website": {
"main": [
[
{
"node": "Crawl website",
"type": "main",
"index": 0
},
{
"node": "Map company name and website",
"type": "main",
"index": 0
}
]
]
}
}
}
Pro
For the full experience including quality scoring and batch install features for each workflow upgrade to Pro
About this workflow
This workflow pulls companies from Supabase, uses an OpenAI (GPT-4o) agent to crawl each company website and collect social media profile URLs via HTTP requests and HTML parsing, and then writes the enriched company record (name, website, and social links) back to Supabase. Runs…
Source: https://n8n.io/workflows/16633/ — original creator credit. Request a take-down →
Related workflows
Workflows that share integrations, category, or trigger type with this one. All free to copy and import.
Autonomous Ai Crawler. Uses toolWorkflow, lmChatOpenAi, outputParserStructured, manualTrigger. Event-driven trigger; 38 nodes.
Tool Workflow, OpenAI Chat, Output Parser Structured +3
This workflow with AI agent is designed to navigate through the page to retrieve specific type of information (in this example: social media profile links).
Tool Workflow, OpenAI Chat, Output Parser Structured +3
Financial Agent with SQL (Supabase). Uses lmChatOpenAi, agent, telegram, textClassifier. Event-driven trigger; 16 nodes.
OpenAI Chat, Agent, Telegram +5
This workflow is manually triggered to pull company names and websites from Supabase, crawl each site with an OpenAI-powered agent to extract social media profile URLs, and write the enriched results
Tool Workflow, OpenAI Chat, Output Parser Structured +2
RAG CHATBOT Main. Uses telegram, telegramTrigger, lmChatOpenAi, n8n-nodes-mcp. Event-driven trigger; 87 nodes.
Telegram, Telegram Trigger, OpenAI Chat +8