{ "id": "t0UZclB5tuoTbCyS", "meta": { "templateCredsSetupCompleted": true }, "name": "RSS Feed Latest Article Fetcher", "tags": [], "nodes": [ { "id": "4c654b62-69b7-463f-861b-90a78ae8344a", "name": "When clicking \u2018Execute workflow\u2019", "type": "n8n-nodes-base.manualTrigger", "position": [ -1168, -32 ], "parameters": {}, "typeVersion": 1 }, { "id": "f52bba62-6bae-4905-8dc3-f0b00b5da0da", "name": "RSS Read", "type": "n8n-nodes-base.rssFeedRead", "position": [ 160, -48 ], "parameters": { "url": "={{ $json.RssUrl }}", "options": {} }, "typeVersion": 1.2 }, { "id": "1fe09cfa-97db-4d5a-bc83-3e7f5d7f3170", "name": "Sticky Note", "type": "n8n-nodes-base.stickyNote", "position": [ -960, -96 ], "parameters": { "color": 7, "width": 208, "height": 224, "content": "List of Urls to check for RSS feed\n\n" }, "typeVersion": 1 }, { "id": "f279a2e9-fd75-4d7d-aab6-a6e6cd1ca161", "name": "Sticky Note2", "type": "n8n-nodes-base.stickyNote", "position": [ -480, -192 ], "parameters": { "color": 7, "width": 896, "height": 464, "content": "Fetch data from a specific URL, most likely to retrieve the RSS feed,check which urls have RSS feed and returns true then pass only those urls whose RSS flag is true. Then check for the latest articles published if not found then stop the workflow.\n\n" }, "typeVersion": 1 }, { "id": "3feef12a-8a93-4f5a-8558-f81afecfe4ed", "name": "Sticky Note1", "type": "n8n-nodes-base.stickyNote", "position": [ -704, -128 ], "parameters": { "color": 7, "width": 176, "height": 240, "content": "Get Each one using this code\n\n" }, "typeVersion": 1 }, { "id": "f61e39ed-c2f3-44e8-b99f-635bd41af765", "name": "Fetch Each URL", "type": "n8n-nodes-base.code", "position": [ -672, -64 ], "parameters": { "mode": "runOnceForEachItem", "jsCode": "const websites = $json.URLs; // Get the websites array from the Set Node\n\n// Prepare the list of websites for processing in the next node\nreturn {\n json: {\n websiteUrl: $json.URLs\n }\n};" }, "typeVersion": 2 }, { "id": "2e1dfe6e-c394-4f66-9b46-509a0e5d2b61", "name": "Fetched Article", "type": "n8n-nodes-base.set", "position": [ 480, -128 ], "parameters": { "options": {}, "assignments": { "assignments": [ { "id": "300d34b8-b078-4099-a772-6b42b131596d", "name": "link", "type": "string", "value": "={{ $json.link }}" }, { "id": "8547123a-0e39-4224-8c7d-c6fdd19fdc12", "name": "title", "type": "string", "value": "={{ $json.title }}" }, { "id": "21bf4bda-2906-4b4c-bf91-e8fa7974a0dc", "name": "article", "type": "string", "value": "={{ $json.contentSnippet }}" } ] } }, "typeVersion": 3.4 }, { "id": "4687a009-113e-47ad-99e4-3e501b1a7ba4", "name": "Stop if no Article found", "type": "n8n-nodes-base.stopAndError", "position": [ 480, 80 ], "parameters": { "errorMessage": "No latest Article is found" }, "typeVersion": 1 }, { "id": "45a65885-0755-44d8-ac3e-bfbdb9b2de8b", "name": "Sticky Note3", "type": "n8n-nodes-base.stickyNote", "position": [ -1840, -368 ], "parameters": { "width": 560, "height": 672, "content": "## How it works\n\nThis workflow is designed to find RSS feed URLs from a given list of URLs and fetch the latest articles from those that have a valid RSS feed. It works by:\n\n1.Checking each URL from a list.\n2.Identifying which URLs contain a valid RSS feed.\n3.Fetching the latest articles from those identified URLs.\n4.Stopping the workflow if no valid article is found.\n\n## Setup Steps\n\n1.Trigger: Workflow starts when you click \"Execute Workflow.\"\n\n2.Fetch RSS URLs: Retrieves URLs from Google Sheets.\n\n3.Check for RSS Feed: Validates whether each URL contains a valid RSS feed.\n\n4.Fetch Latest Articles: For valid feeds, the workflow fetches the latest articles.\n\n5.Fetch Existing Titles: Pulls existing titles from Google Sheets to compare against new titles.\n\n6.Check for Duplicates: Compares new titles with existing ones. If duplicate, skips; if not, proceeds.\n\n7.Store Non-Duplicate Titles: Adds non-duplicate articles to Google Sheets.\n\n8.Message Node: Optionally logs duplicate data (e.g., \"Data already exists\").\n\n9.Stop if No Articles: Stops the workflow if no new articles are found." }, "typeVersion": 1 }, { "id": "fae7faa7-80d8-4482-bd8f-e84e5a619e0c", "name": "RSS URLs", "type": "n8n-nodes-base.googleSheets", "position": [ -912, -32 ], "parameters": { "options": {}, "sheetName": { "__rl": true, "mode": "list", "value": "gid=0", "cachedResultUrl": "https://docs.google.com/spreadsheets/d/1ndtTAw9_UstPpMWrEUP4GFESac_5139BqZOrkuROFzg/edit#gid=0", "cachedResultName": "Sheet1" }, "documentId": { "__rl": true, "mode": "list", "value": "1ndtTAw9_UstPpMWrEUP4GFESac_5139BqZOrkuROFzg", "cachedResultUrl": "https://docs.google.com/spreadsheets/d/1ndtTAw9_UstPpMWrEUP4GFESac_5139BqZOrkuROFzg/edit?usp=drivesdk", "cachedResultName": "RSS URLs" } }, "credentials": { "googleSheetsOAuth2Api": { "name": "" } }, "typeVersion": 4.7 }, { "id": "3d551427-981b-4b1d-9513-eb2084a7bcaa", "name": "Get Website Data", "type": "n8n-nodes-base.httpRequest", "position": [ -448, -32 ], "parameters": { "url": "={{ $json.websiteUrl }}", "options": {} }, "typeVersion": 4.3 }, { "id": "dd64a948-47ab-40e3-8fa0-5c880cbf16b5", "name": "Check RSS URLs", "type": "n8n-nodes-base.code", "position": [ -304, -32 ], "parameters": { "mode": "runOnceForEachItem", "jsCode": "// Get the content (HTML) from the previous node (HTTP Request Node)\nconst content = $json[\"data\"]; // This contains the raw HTML data\n\n// Log the content for debugging purposes\nconsole.log('Content:', content);\n\n// Regular expression to search for both RSS and Atom feed links in the HTML content\nconst rssFeedRegex = /]+rel=[\"']alternate[\"'][^>]+type=[\"'](application\\/rss\\+xml|application\\/atom\\+xml)[\"'][^>]+href=[\"']([^\"']+)[\"']/i;\n\n// Extract the website URL dynamically from the input node (previous node's data)\nconst websiteUrl = $('Fetch Each URL').item.json.websiteUrl;\n\n// Function to check if the content is valid HTML and contains RSS feeds\nif (content.trim().startsWith('') || content.trim().startsWith('" } }, "typeVersion": 4.7 }, { "id": "81e92f75-af73-4ecc-85fb-484642ea76db", "name": "Sticky Note4", "type": "n8n-nodes-base.stickyNote", "position": [ 448, -176 ], "parameters": { "color": 7, "width": 160, "height": 208, "content": "Format Data" }, "typeVersion": 1 }, { "id": "e7e931ae-d5a5-45bb-86d4-a71a8444cdf2", "name": "Fetch title", "type": "n8n-nodes-base.googleSheets", "position": [ 624, -304 ], "parameters": { "options": {}, "sheetName": { "__rl": true, "mode": "list", "value": "gid=0", "cachedResultUrl": "https://docs.google.com/spreadsheets/d/1ME5tNqq4YpxEoS7v3Ve4ckxBLH-9K3NjbcOeeO1pB1k/edit#gid=0", "cachedResultName": "Sheet1" }, "documentId": { "__rl": true, "mode": "list", "value": "1ME5tNqq4YpxEoS7v3Ve4ckxBLH-9K3NjbcOeeO1pB1k", "cachedResultUrl": "https://docs.google.com/spreadsheets/d/1ME5tNqq4YpxEoS7v3Ve4ckxBLH-9K3NjbcOeeO1pB1k/edit?usp=drivesdk", "cachedResultName": "RSS Fetched Feed" }, "combineFilters": "=AND" }, "credentials": { "googleSheetsOAuth2Api": { "name": "" } }, "typeVersion": 4.7, "alwaysOutputData": true }, { "id": "2c2bbdca-947e-480a-aff1-49680d104655", "name": "Get Titles List", "type": "n8n-nodes-base.code", "position": [ 1024, -208 ], "parameters": { "jsCode": "// Initialize an empty array to store titles\nlet titlesList = [];\n\n// Loop through each item in the previous result (fetched titles)\nfor (let i = 0; i < items.length; i++) {\n // Extract title from each item (you may need to adjust this part based on the structure)\n let title = items[i].json.title;\n\n // Add the title to the list if it's not already in the list\n if (!titlesList.includes(title)) {\n titlesList.push(title);\n }\n}\n\n// Return the list as a new output\nreturn [\n {\n json: {\n titlesList: titlesList // The final array of titles without duplicates\n }\n }\n];" }, "typeVersion": 2 }, { "id": "232091eb-047f-433c-855e-30f106d9226c", "name": "Check if data is empty", "type": "n8n-nodes-base.if", "position": [ 784, -304 ], "parameters": { "options": {}, "conditions": { "options": { "version": 2, "leftValue": "", "caseSensitive": true, "typeValidation": "strict" }, "combinator": "and", "conditions": [ { "id": "a2c8f7f5-b156-4239-987c-40a1e0726562", "operator": { "type": "object", "operation": "empty", "singleValue": true }, "leftValue": "={{ $json }}", "rightValue": "" } ] } }, "typeVersion": 2.2 }, { "id": "c314c121-f0c0-4d32-9dc3-5b34b77c0b8a", "name": "Get Each item", "type": "n8n-nodes-base.code", "position": [ 1056, -512 ], "parameters": { "jsCode": "// Get the fetched articles from the previous node\nconst fetchedArticles = $('Fetched Article').all();\n\n// Initialize an array to store the rows for Google Sheets\nlet rows = [];\n\n// Loop through each item and format the data for Google Sheets\nfetchedArticles.forEach(item => {\n const title = item.json.title;\n const link = item.json.link;\n const article = item.json.article;\n\n // Create a row for Google Sheets (you can add more fields as needed)\n const row = [title, link, article];\n\n // Push the row to the rows array\n rows.push(row);\n});\n\n// Return the rows array to pass data to the next node (Google Sheets)\nreturn rows.map(row => ({\n json: {\n title: row[0],\n link: row[1],\n article: row[2],\n },\n}));" }, "typeVersion": 2 }, { "id": "69b1fbed-3a7d-4334-8202-8c24f1055610", "name": "Append row in sheet", "type": "n8n-nodes-base.googleSheets", "position": [ 1680, -96 ], "parameters": { "columns": { "value": { "link": "={{ $json.link }}", "title": "={{ $json.title }}", "article": "={{ $json.article }}" }, "schema": [ { "id": "title", "type": "string", "display": true, "required": false, "displayName": "title", "defaultMatch": false, "canBeUsedToMatch": true }, { "id": "link", "type": "string", "display": true, "required": false, "displayName": "link", "defaultMatch": false, "canBeUsedToMatch": true }, { "id": "article", "type": "string", "display": true, "required": false, "displayName": "article", "defaultMatch": false, "canBeUsedToMatch": true } ], "mappingMode": "defineBelow", "matchingColumns": [], "attemptToConvertTypes": false, "convertFieldsToString": false }, "options": {}, "operation": "append", "sheetName": { "__rl": true, "mode": "list", "value": "gid=0", "cachedResultUrl": "https://docs.google.com/spreadsheets/d/1ME5tNqq4YpxEoS7v3Ve4ckxBLH-9K3NjbcOeeO1pB1k/edit#gid=0", "cachedResultName": "Sheet1" }, "documentId": { "__rl": true, "mode": "list", "value": "1ME5tNqq4YpxEoS7v3Ve4ckxBLH-9K3NjbcOeeO1pB1k", "cachedResultUrl": "https://docs.google.com/spreadsheets/d/1ME5tNqq4YpxEoS7v3Ve4ckxBLH-9K3NjbcOeeO1pB1k/edit?usp=drivesdk", "cachedResultName": "RSS Fetched Feed" } }, "credentials": { "googleSheetsOAuth2Api": { "name": "" } }, "typeVersion": 4.7 }, { "id": "5840fdc8-f143-43d8-9b74-34671170a659", "name": "Check if title exits in sheet", "type": "n8n-nodes-base.code", "position": [ 1168, -208 ], "parameters": { "jsCode": "// Get the list of new titles from Fetched Article\nconst newTitles = $('Fetched Article').all().map(item => item.json.title);\n\n// Get the list of existing titles from Google Sheets (or wherever you're storing them)\nconst titlesList = $json[\"titlesList\"];\n\n// Flag to indicate if all titles are found\nlet allTitlesFound = true;\n\n// Array to store titles that were not found\nlet notFoundTitles = [];\n\n// Iterate over each new title and check if it exists in titlesList\nfor (let newTitle of newTitles) {\n if (!titlesList.includes(newTitle)) {\n // If the title is not found, mark it and add to the notFoundTitles array\n allTitlesFound = false;\n notFoundTitles.push(newTitle);\n }\n}\n\n// Return the result (true/false) and the list of not found titles\nreturn [\n {\n json: {\n allTitlesFound: allTitlesFound, // This will be true if all titles are found\n notFoundTitles: notFoundTitles // This will contain the titles that are not found\n }\n }\n];" }, "typeVersion": 2 }, { "id": "880cc11d-0181-496a-88d2-d62ab8aa858a", "name": "Stop if no Url found", "type": "n8n-nodes-base.stopAndError", "position": [ 48, 112 ], "parameters": { "errorMessage": "RSS field not found for this URL" }, "typeVersion": 1 }, { "id": "a682204c-4b42-4d01-b89b-02ec54489773", "name": "Each RSS URL", "type": "n8n-nodes-base.code", "position": [ 16, -48 ], "parameters": { "mode": "runOnceForEachItem", "jsCode": "const websites = $json.rssFeedUrl; // Get the websites array from the Set Node\n\n// Prepare the list of websites for processing in the next node\nreturn {\n json: {\n RssUrl: $json.rssFeedUrl\n }\n};" }, "typeVersion": 2 }, { "id": "f92d02b0-5492-4110-a8a4-d04e506337a7", "name": "Get matching titles", "type": "n8n-nodes-base.code", "position": [ 1520, -96 ], "parameters": { "jsCode": "// Get the list of not found titles (those we need to match)\nconst notFoundTitles = $json[\"notFoundTitles\"];\n\n// Get the list of new titles from Fetched Article\nconst fetchedArticles = $('Fetched Article').all(); // This contains the articles with title, link and article\n\n// Initialize an array to store the matched articles\nlet matchingArticles = [];\n\n// Loop through each item in the fetched articles\nfor (let i = 0; i < fetchedArticles.length; i++) {\n const article = fetchedArticles[i].json;\n\n // Check if the current article's title exists in the notFoundTitles array\n if (notFoundTitles.includes(article.title)) {\n // If found, push the link, title and article content in the desired format\n matchingArticles.push({\n link: article.link,\n title: article.title,\n article: article.article\n });\n }\n}\n\n// If you just want to return the matching articles in the format you mentioned:\n// Return as single objects, no need for a list of \"matchingArticles\"\nreturn matchingArticles.map(article => ({\n json: {\n link: article.link,\n title: article.title,\n article: article.article\n }\n}));" }, "typeVersion": 2 }, { "id": "ee10fdee-47e6-415d-820c-0a4f898e99c9", "name": "Message", "type": "n8n-nodes-base.set", "position": [ 1504, -304 ], "parameters": { "options": {}, "assignments": { "assignments": [ { "id": "bc68727f-c8c0-45a1-aa5c-722e585f5347", "name": "message", "type": "string", "value": "Data already exists" } ] } }, "typeVersion": 3.4 }, { "id": "264b0b94-c3d7-488c-9aa1-0c70dde13a88", "name": "Check if matches", "type": "n8n-nodes-base.if", "position": [ 1328, -208 ], "parameters": { "options": {}, "conditions": { "options": { "version": 2, "leftValue": "", "caseSensitive": true, "typeValidation": "strict" }, "combinator": "and", "conditions": [ { "id": "b7310eaf-6316-4ec8-8236-a9d364fb7066", "operator": { "type": "boolean", "operation": "true", "singleValue": true }, "leftValue": "={{ $json.allTitlesFound }}", "rightValue": "={{ $json.titlesList }}" } ] } }, "typeVersion": 2.2 }, { "id": "244caa88-ab6b-4310-86de-3f542f9433d8", "name": "Sticky Note5", "type": "n8n-nodes-base.stickyNote", "position": [ 608, -384 ], "parameters": { "color": 7, "width": 320, "height": 224, "content": "Fetch existing title from sheets and check that it exists\n" }, "typeVersion": 1 }, { "id": "f39d2140-6456-42bb-b203-3ac82eace4a4", "name": "Sticky Note6", "type": "n8n-nodes-base.stickyNote", "position": [ 1024, -560 ], "parameters": { "color": 7, "width": 352, "height": 192, "content": "Get each item and store in sheet.\n\n" }, "typeVersion": 1 }, { "id": "5f37ac61-0cbb-4387-aab9-5119eda14d19", "name": "Sticky Note7", "type": "n8n-nodes-base.stickyNote", "position": [ 992, -288 ], "parameters": { "color": 7, "width": 432, "height": 240, "content": "Store existing title in a list and check that if any new titles are exiting if not then store new title in a seperate list.\n" }, "typeVersion": 1 }, { "id": "dae98fd9-03bc-40d1-8c94-26118d17c2f0", "name": "Sticky Note8", "type": "n8n-nodes-base.stickyNote", "position": [ 1488, -144 ], "parameters": { "color": 7, "width": 384, "height": 208, "content": "New titles matching with fetched title and store in sheet.\n" }, "typeVersion": 1 }, { "id": "1fe04ed5-7dd1-4ab2-a6ff-81e687801d60", "name": "Sticky Note9", "type": "n8n-nodes-base.stickyNote", "position": [ 1440, -352 ], "parameters": { "color": 7, "width": 256, "height": 144, "content": "Add message that data already exits." }, "typeVersion": 1 } ], "active": false, "settings": { "executionOrder": "v1" }, "versionId": "5df42ae9-c615-4e76-a029-15e5647622a4", "connections": { "RSS Read": { "main": [ [ { "node": "Latest Uploaded Articles", "type": "main", "index": 0 } ] ] }, "RSS URLs": { "main": [ [ { "node": "Fetch Each URL", "type": "main", "index": 0 } ] ] }, "Fetch title": { "main": [ [ { "node": "Check if data is empty", "type": "main", "index": 0 } ] ] }, "Each RSS URL": { "main": [ [ { "node": "RSS Read", "type": "main", "index": 0 } ] ] }, "Get Each item": { "main": [ [ { "node": "Add data in sheets", "type": "main", "index": 0 } ] ] }, "Check RSS URLs": { "main": [ [ { "node": "Fetch RSS URLs", "type": "main", "index": 0 } ] ] }, "Fetch Each URL": { "main": [ [ { "node": "Get Website Data", "type": "main", "index": 0 } ] ] }, "Fetch RSS URLs": { "main": [ [ { "node": "Each RSS URL", "type": "main", "index": 0 } ], [ { "node": "Stop if no Url found", "type": "main", "index": 0 } ] ] }, "Fetched Article": { "main": [ [ { "node": "Fetch title", "type": "main", "index": 0 } ] ] }, "Get Titles List": { "main": [ [ { "node": "Check if title exits in sheet", "type": "main", "index": 0 } ] ] }, "Check if matches": { "main": [ [ { "node": "Message", "type": "main", "index": 0 } ], [ { "node": "Get matching titles", "type": "main", "index": 0 } ] ] }, "Get Website Data": { "main": [ [ { "node": "Check RSS URLs", "type": "main", "index": 0 } ] ] }, "Get matching titles": { "main": [ [ { "node": "Append row in sheet", "type": "main", "index": 0 } ] ] }, "Check if data is empty": { "main": [ [ { "node": "Get Each item", "type": "main", "index": 0 } ], [ { "node": "Get Titles List", "type": "main", "index": 0 } ] ] }, "Latest Uploaded Articles": { "main": [ [ { "node": "Fetched Article", "type": "main", "index": 0 } ], [ { "node": "Stop if no Article found", "type": "main", "index": 0 } ] ] }, "Check if title exits in sheet": { "main": [ [ { "node": "Check if matches", "type": "main", "index": 0 } ] ] }, "When clicking \u2018Execute workflow\u2019": { "main": [ [ { "node": "RSS URLs", "type": "main", "index": 0 } ] ] } } }