{
  "id": "uFO3zp2bl5cqZqMS",
  "meta": {
    "templateCredsSetupCompleted": true
  },
  "name": "YouTube Crawler",
  "tags": [],
  "nodes": [
    {
      "id": "230d2269-6b2e-4915-82e8-b42c2a36746a",
      "name": "When clicking \u2018Execute workflow\u2019",
      "type": "n8n-nodes-base.manualTrigger",
      "position": [
        20,
        -40
      ],
      "parameters": {},
      "typeVersion": 1
    },
    {
      "id": "2ffb8769-8769-4636-adff-f70a65df8fa5",
      "name": "Sticky Note",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        860,
        140
      ],
      "parameters": {
        "color": 4,
        "width": 340,
        "height": 440,
        "content": "This step is going to call YouTube's API and ask for it to return videos that match the query, looking only in the category of education, sorted by relevance, and  no more than 50.\n\n**What you need to do**:\n\n- Enable the YouTube Data API v3: YouTube Data API v3 page in Google Cloud Console. Instructions can be found here: https://console.cloud.google.com/apis/library/youtube.googleapis.com\n\n- Add your youtube credentials to the workspace before starting. Instructions can be found here: https://developers.google.com/youtube/registering_an_application\n\n- Change the credentials in this \"Search YouTube\" node to use your new credentials you just added"
      },
      "typeVersion": 1
    },
    {
      "id": "b0becf14-1e2b-47a4-83b0-2db297518479",
      "name": "Sticky Note1",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        220,
        140
      ],
      "parameters": {
        "width": 340,
        "height": 240,
        "content": "This step is going to set the value of what you're going to be searching for in YouTube. It's optional, but easier to update and you can attach a chat to this if you want to update it outside of the node.\n\n**What you need to do**\n\n- Change the query to whatever you want to search for videos on"
      },
      "typeVersion": 1
    },
    {
      "id": "d437a506-e2b8-48fa-98cf-940657d7a879",
      "name": "Set Query",
      "type": "n8n-nodes-base.set",
      "position": [
        340,
        -40
      ],
      "parameters": {
        "options": {},
        "assignments": {
          "assignments": [
            {
              "id": "58609f1a-03c7-4e5a-8199-2fcdaf495325",
              "name": "query",
              "type": "array",
              "value": "={{[\n\"AI agents tutorial step by step\",\n\"AI tools tutorial for beginners\",\n\"AI agent building tutorial\",\n\"AI prompt engineering techniques\",\n\"best AI tools training\",\n\"prompt engineering best practices\"]}}"
            }
          ]
        }
      },
      "typeVersion": 3.4
    },
    {
      "id": "ea7e2dad-4f8f-40d8-9e77-25fbfe222f6f",
      "name": "Search YouTube",
      "type": "n8n-nodes-base.httpRequest",
      "position": [
        980,
        -40
      ],
      "parameters": {
        "url": "https://www.googleapis.com/youtube/v3/search",
        "options": {},
        "sendQuery": true,
        "authentication": "predefinedCredentialType",
        "queryParameters": {
          "parameters": [
            {
              "name": "part",
              "value": "id,snippet"
            },
            {
              "name": "q",
              "value": "={{ $json.query }}"
            },
            {
              "name": "type",
              "value": "video"
            },
            {
              "name": "maxResults",
              "value": "50"
            },
            {
              "name": "order",
              "value": "relevance"
            },
            {
              "name": "videoCategoryId",
              "value": "27"
            }
          ]
        },
        "nodeCredentialType": "youTubeOAuth2Api"
      },
      "credentials": {
        "youTubeOAuth2Api": {
          "name": "<your credential>"
        }
      },
      "typeVersion": 4.2
    },
    {
      "id": "f68554af-5f38-4296-a38e-e917dba605f4",
      "name": "Sticky Note2",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        1980,
        220
      ],
      "parameters": {
        "color": 3,
        "width": 440,
        "height": 340,
        "content": "This loop is going to collect all the metadata that we need from the videos, such as view and like count, and then filter based on those values as well as recency of publishing. Finally it will remove unnecessary fields before sending on to the final steps in the worfklow.\n\n**What you need to do**\n\n- Change the YouTube credentials in the \"Get Video Metadata\" node\n\n\n**Try Modifying**\n\n- The filter criteria to broaden, narrow, or change your aperture.\n\n"
      },
      "typeVersion": 1
    },
    {
      "id": "62956d67-ee59-4131-8f4f-de263dafa7bc",
      "name": "Sticky Note3",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        1380,
        -320
      ],
      "parameters": {
        "color": 5,
        "width": 340,
        "height": 220,
        "content": "This step is going to be the first quality and relevance filter, excluding anything that doesn't seem purely educational or that seems promotional or scammy.\n\n**Try Modifying**\n\n- The filter criteria to better reflect what you want to include or exclude.\n"
      },
      "typeVersion": 1
    },
    {
      "id": "c0f112d7-99d3-487a-9def-71bf8a879fdd",
      "name": "Wait",
      "type": "n8n-nodes-base.wait",
      "position": [
        2920,
        -340
      ],
      "parameters": {
        "amount": 3
      },
      "typeVersion": 1.1
    },
    {
      "id": "43bb3d2b-0022-4cb0-a13c-ba7df4d4d9f4",
      "name": "Filter for Quality",
      "type": "n8n-nodes-base.filter",
      "position": [
        2280,
        -40
      ],
      "parameters": {
        "options": {},
        "conditions": {
          "options": {
            "version": 2,
            "leftValue": "",
            "caseSensitive": true,
            "typeValidation": "loose"
          },
          "combinator": "and",
          "conditions": [
            {
              "id": "0fe9a82c-6c5c-44c1-8da7-24e2f8783a0e",
              "operator": {
                "type": "number",
                "operation": "gt"
              },
              "leftValue": "={{ $json.items[0].statistics.viewCount }}",
              "rightValue": 10000
            },
            {
              "id": "7eae8c9b-c85d-4f0f-92bd-01b047e05d0e",
              "operator": {
                "type": "number",
                "operation": "gt"
              },
              "leftValue": "={{ $json.items[0].statistics.likeCount }}",
              "rightValue": 100
            },
            {
              "id": "51d609d3-0b04-476e-9abd-a68a1f9c095c",
              "operator": {
                "type": "dateTime",
                "operation": "after"
              },
              "leftValue": "={{ $json.items[0].snippet.publishedAt }}",
              "rightValue": "2023-12-01T00:00:00"
            }
          ]
        },
        "looseTypeValidation": true
      },
      "typeVersion": 2.2
    },
    {
      "id": "f844dd50-6a38-4e9f-874b-885bddf52b4e",
      "name": "Sticky Note5",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        2260,
        -820
      ],
      "parameters": {
        "color": 3,
        "width": 440,
        "height": 440,
        "content": "This final set of steps is going to remove duplicate videos, assign a relevance score based on keywords and quality metrics, and then send only the top one's to a google sheet. \n\n**What you need to do**\n\n- Create a new google sheet in your drive with the columns Title, Channel, Published At, Views, Likes, Description, URL. This is the sheet your workflow will write into.\n\n- Add your google sheet credentials to your workspace\n\n- Change out the credential values in the \"Send to Google Sheets\" node\n\n\n**Try Modifying**\n\n- The formula for the relevance score to reflect your criteria for quality\n"
      },
      "typeVersion": 1
    },
    {
      "id": "40ec5dda-3c74-4203-b769-896cdb17c703",
      "name": "Split Query",
      "type": "n8n-nodes-base.splitOut",
      "position": [
        660,
        -40
      ],
      "parameters": {
        "options": {},
        "fieldToSplitOut": "query"
      },
      "typeVersion": 1
    },
    {
      "id": "0f3f6e07-b0ad-4558-a268-97b61094671a",
      "name": "Split Results",
      "type": "n8n-nodes-base.splitOut",
      "position": [
        1240,
        -40
      ],
      "parameters": {
        "options": {},
        "fieldToSplitOut": "items"
      },
      "typeVersion": 1
    },
    {
      "id": "c44d4550-3bcc-46e5-b294-65c5455b112f",
      "name": "Filter for Relevance",
      "type": "n8n-nodes-base.filter",
      "position": [
        1480,
        -40
      ],
      "parameters": {
        "options": {},
        "conditions": {
          "options": {
            "version": 2,
            "leftValue": "",
            "caseSensitive": true,
            "typeValidation": "strict"
          },
          "combinator": "and",
          "conditions": [
            {
              "id": "051c8268-ace1-4d87-a57e-798ed9992331",
              "operator": {
                "type": "string",
                "operation": "regex"
              },
              "leftValue": "={{ $json.snippet.title.toLowerCase() }}",
              "rightValue": "tutorial|course|guide|training|learn|education|explained|masterclass|complete.*guide|step.*by.*step|beginners.*guide|how.*to.*build|fundamentals"
            },
            {
              "id": "3bb11116-7ac6-4aa1-b1ac-3ebd1a6ceb82",
              "operator": {
                "type": "string",
                "operation": "notRegex"
              },
              "leftValue": "={{ $json.snippet.title.toLowerCase() }}",
              "rightValue": "money|rich|profit|earn|cash|income|\\$|\ud83d\udcb0|sell|agency|business|client|freelance|side.*hustle|make.*money"
            },
            {
              "id": "5edc6bdd-514c-4cbd-bbbc-51da2f87c41e",
              "operator": {
                "type": "string",
                "operation": "notRegex"
              },
              "leftValue": "={{ $json.snippet.title.toLowerCase() }}",
              "rightValue": "autopilot|passive.*income|[\\d]+k.*subs|faceless.*videos|youtube.*automation.*money|100%.*automated.*money"
            },
            {
              "id": "320ed472-8b8d-49bc-b830-e95c39017397",
              "operator": {
                "type": "string",
                "operation": "notRegex"
              },
              "leftValue": "={{ $json.snippet.title.toLowerCase() }}",
              "rightValue": "insane|crazy|ultimate.*secret|hack|trick|must.*do|steal.*these|best.*way.*to.*make|ways.*to.*make|you.*need.*to|[\\d]+.*ways|[\\d]+.*insane"
            }
          ]
        }
      },
      "typeVersion": 2.2
    },
    {
      "id": "74f3670e-e61a-4dd1-bde7-5fbf845e14ca",
      "name": "Loop over Items",
      "type": "n8n-nodes-base.splitInBatches",
      "position": [
        1820,
        -80
      ],
      "parameters": {
        "options": {},
        "batchSize": 10
      },
      "executeOnce": false,
      "typeVersion": 3,
      "alwaysOutputData": false
    },
    {
      "id": "8ad9d1d8-83f7-4edc-bac3-3d7bffbc0c83",
      "name": "Get Video Metadata",
      "type": "n8n-nodes-base.httpRequest",
      "position": [
        2060,
        -40
      ],
      "parameters": {
        "url": "https://www.googleapis.com/youtube/v3/videos",
        "options": {},
        "sendQuery": true,
        "authentication": "predefinedCredentialType",
        "queryParameters": {
          "parameters": [
            {
              "name": "part",
              "value": "snippet,statistics"
            },
            {
              "name": "id",
              "value": "={{ $json.id.videoId }}"
            }
          ]
        },
        "nodeCredentialType": "youTubeOAuth2Api"
      },
      "credentials": {
        "youTubeOAuth2Api": {
          "name": "<your credential>"
        }
      },
      "typeVersion": 4.2
    },
    {
      "id": "040a4ed8-ec93-4461-acc1-5403154d3d61",
      "name": "Remove Unnecessary Fields",
      "type": "n8n-nodes-base.set",
      "position": [
        2480,
        -40
      ],
      "parameters": {
        "options": {},
        "assignments": {
          "assignments": [
            {
              "id": "bd1dfa28-997f-45e7-983f-06b4afdf2224",
              "name": "Title",
              "type": "string",
              "value": "={{ $json.items[0].snippet.title }}"
            },
            {
              "id": "9921ef02-a86d-42ae-a4df-9c2dd674505c",
              "name": "Channel",
              "type": "string",
              "value": "={{ $json.items[0].snippet.channelTitle }}"
            },
            {
              "id": "ec38edef-66e2-480f-9617-66123decfcef",
              "name": "Published At",
              "type": "string",
              "value": "={{ $json.items[0].snippet.publishedAt }}"
            },
            {
              "id": "9bb69372-553a-4c8d-8286-00f41c90a78c",
              "name": "Views",
              "type": "string",
              "value": "={{ $json.items[0].statistics.viewCount }}"
            },
            {
              "id": "80edec74-08c1-4d68-82f7-ba061cb2173e",
              "name": "Likes",
              "type": "string",
              "value": "={{ $json.items[0].statistics.likeCount }}"
            },
            {
              "id": "9700bbee-7f25-43fe-a653-42a7c8fade77",
              "name": "Description",
              "type": "string",
              "value": "={{ $json.items[0].snippet.description }}"
            },
            {
              "id": "9afaaa69-1afd-4f45-bfe6-24671651ac82",
              "name": "URL",
              "type": "string",
              "value": "=https://www.youtube.com/watch?v={{ $json.items[0].id }}"
            }
          ]
        }
      },
      "typeVersion": 3.4
    },
    {
      "id": "79b34b24-290f-43b7-8c45-990a5c30cdb0",
      "name": "Remove Duplicate Videos",
      "type": "n8n-nodes-base.removeDuplicates",
      "position": [
        2060,
        -340
      ],
      "parameters": {
        "compare": "={{ $json.URL }}",
        "options": {}
      },
      "typeVersion": 2
    },
    {
      "id": "756fa598-ab05-46ff-aaea-be6c9533568d",
      "name": "Generate Relevance Score",
      "type": "n8n-nodes-base.set",
      "position": [
        2280,
        -340
      ],
      "parameters": {
        "options": {},
        "assignments": {
          "assignments": [
            {
              "id": "d691e3bf-9fc4-406f-b6e7-d62487820ea1",
              "name": "RelevanceScore",
              "type": "number",
              "value": "={{\n  // Educational keywords (higher scores)\n  (($json.Title.toLowerCase().includes('tutorial')) ? 15 : 0) +\n  (($json.Title.toLowerCase().includes('course')) ? 15 : 0) +\n  (($json.Title.toLowerCase().includes('guide')) ? 12 : 0) +\n  (($json.Title.toLowerCase().includes('explained')) ? 12 : 0) +\n  (($json.Title.toLowerCase().includes('beginner')) ? 10 : 0) +\n  (($json.Title.toLowerCase().includes('step by step')) ? 15 : 0) +\n  (($json.Title.toLowerCase().includes('complete')) ? 10 : 0) +\n  \n  // AI-specific terms\n  (($json.Title.toLowerCase().includes('prompt engineering')) ? 20 : 0) +\n  (($json.Title.toLowerCase().includes('ai agent')) ? 18 : 0) +\n  (($json.Title.toLowerCase().includes('ai tools')) ? 15 : 0) +\n  (($json.Title.toLowerCase().includes('prompting')) ? 15 : 0) +\n  \n  // Quality indicators\n  ((parseInt($json.Views) > 100000) ? 10 : 0) +\n  ((parseInt($json.Views) > 50000) ? 5 : 0) +\n  ((parseInt($json.Likes) > 1000) ? 8 : 0) +\n  \n  // Penalty for monetization indicators\n  (($json.Title.toLowerCase().includes('money') || \n    $json.Title.toLowerCase().includes('rich') || \n    $json.Title.toLowerCase().includes('earn')) ? -20 : 0)\n}}"
            },
            {
              "id": "a9e446de-dc9c-4dba-ab0b-d5647a24ba39",
              "name": "Title",
              "type": "string",
              "value": "={{ $json.Title }}"
            },
            {
              "id": "1669b129-9e9a-45e6-925f-c5e41e6de692",
              "name": "Channel",
              "type": "string",
              "value": "={{ $json.Channel }}"
            },
            {
              "id": "3d45d0ce-3068-424c-80a3-06c11785846d",
              "name": "Published at",
              "type": "string",
              "value": "={{ $json['Published At'] }}"
            },
            {
              "id": "16e45f2b-1d0b-48e4-b86c-7e8ba82c5aef",
              "name": "Views",
              "type": "string",
              "value": "={{ $json.Views }}"
            },
            {
              "id": "86f2af64-27f9-4e3c-9883-c279edc938c6",
              "name": "Likes",
              "type": "string",
              "value": "={{ $json.Likes }}"
            },
            {
              "id": "d967b6c4-ab40-4f87-a84e-e0cb1bbb51d6",
              "name": "Descriptoin",
              "type": "string",
              "value": "={{ $json.Description }}"
            },
            {
              "id": "e0af7129-2215-4583-b236-63587a4974b6",
              "name": "URL",
              "type": "string",
              "value": "={{ $json.URL }}"
            }
          ]
        }
      },
      "typeVersion": 3.4
    },
    {
      "id": "452016d9-50ce-49bb-bed8-d3213bd5bcec",
      "name": "Sort by Relevance",
      "type": "n8n-nodes-base.sort",
      "position": [
        2480,
        -340
      ],
      "parameters": {
        "options": {
          "disableDotNotation": false
        },
        "sortFieldsUi": {
          "sortField": [
            {
              "order": "descending",
              "fieldName": "RelevanceScore"
            }
          ]
        }
      },
      "typeVersion": 1
    },
    {
      "id": "916ff54f-b7a3-472f-af8e-df2e501aef65",
      "name": "Force Limit",
      "type": "n8n-nodes-base.limit",
      "position": [
        2700,
        -340
      ],
      "parameters": {
        "maxItems": 50
      },
      "typeVersion": 1
    },
    {
      "id": "b4007c9b-cf09-4292-b22d-2778bf7c03a6",
      "name": "Send to Google Sheets",
      "type": "n8n-nodes-base.googleSheets",
      "position": [
        3120,
        -340
      ],
      "parameters": {
        "columns": {
          "value": {
            "URL": "={{ $json.URL }}",
            "Likes": "={{ $json.Likes }}",
            "Title": "={{ $json.Title }}",
            "Views": "={{ $json.Views }}",
            "Channel": "={{ $json.Channel }}",
            "Description": "={{ $json.Descriptoin }}",
            "Published At": "={{ $json['Published at'] }}"
          },
          "schema": [
            {
              "id": "Title",
              "type": "string",
              "display": true,
              "required": false,
              "displayName": "Title",
              "defaultMatch": false,
              "canBeUsedToMatch": true
            },
            {
              "id": "Channel",
              "type": "string",
              "display": true,
              "required": false,
              "displayName": "Channel",
              "defaultMatch": false,
              "canBeUsedToMatch": true
            },
            {
              "id": "Published At",
              "type": "string",
              "display": true,
              "required": false,
              "displayName": "Published At",
              "defaultMatch": false,
              "canBeUsedToMatch": true
            },
            {
              "id": "Views",
              "type": "string",
              "display": true,
              "required": false,
              "displayName": "Views",
              "defaultMatch": false,
              "canBeUsedToMatch": true
            },
            {
              "id": "Likes",
              "type": "string",
              "display": true,
              "required": false,
              "displayName": "Likes",
              "defaultMatch": false,
              "canBeUsedToMatch": true
            },
            {
              "id": "Description",
              "type": "string",
              "display": true,
              "required": false,
              "displayName": "Description",
              "defaultMatch": false,
              "canBeUsedToMatch": true
            },
            {
              "id": "URL",
              "type": "string",
              "display": true,
              "required": false,
              "displayName": "URL",
              "defaultMatch": false,
              "canBeUsedToMatch": true
            }
          ],
          "mappingMode": "defineBelow",
          "matchingColumns": [],
          "attemptToConvertTypes": false,
          "convertFieldsToString": false
        },
        "options": {},
        "operation": "append",
        "sheetName": {
          "__rl": true,
          "mode": "list",
          "value": "gid=0",
          "cachedResultUrl": "https://docs.google.com/spreadsheets/d/1LbV3_SHHrGiTm7h-4WjBlbaNXDcv7fzzvHhmrU_GY3c/edit#gid=0",
          "cachedResultName": "Sheet1"
        },
        "documentId": {
          "__rl": true,
          "mode": "list",
          "value": "1LbV3_SHHrGiTm7h-4WjBlbaNXDcv7fzzvHhmrU_GY3c",
          "cachedResultUrl": "https://docs.google.com/spreadsheets/d/1LbV3_SHHrGiTm7h-4WjBlbaNXDcv7fzzvHhmrU_GY3c/edit?usp=drivesdk",
          "cachedResultName": "YouTube AI Resources"
        }
      },
      "credentials": {
        "googleSheetsOAuth2Api": {
          "name": "<your credential>"
        }
      },
      "typeVersion": 4.6
    }
  ],
  "active": false,
  "settings": {
    "executionOrder": "v1"
  },
  "versionId": "cfed5f6c-f15f-4dfe-bf4a-11541a75fc71",
  "connections": {
    "Wait": {
      "main": [
        [
          {
            "node": "Send to Google Sheets",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Set Query": {
      "main": [
        [
          {
            "node": "Split Query",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Force Limit": {
      "main": [
        [
          {
            "node": "Wait",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Split Query": {
      "main": [
        [
          {
            "node": "Search YouTube",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Split Results": {
      "main": [
        [
          {
            "node": "Filter for Relevance",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Search YouTube": {
      "main": [
        [
          {
            "node": "Split Results",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Loop over Items": {
      "main": [
        [
          {
            "node": "Remove Duplicate Videos",
            "type": "main",
            "index": 0
          }
        ],
        [
          {
            "node": "Get Video Metadata",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Sort by Relevance": {
      "main": [
        [
          {
            "node": "Force Limit",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Filter for Quality": {
      "main": [
        [
          {
            "node": "Remove Unnecessary Fields",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Get Video Metadata": {
      "main": [
        [
          {
            "node": "Filter for Quality",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Filter for Relevance": {
      "main": [
        [
          {
            "node": "Loop over Items",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Send to Google Sheets": {
      "main": [
        []
      ]
    },
    "Remove Duplicate Videos": {
      "main": [
        [
          {
            "node": "Generate Relevance Score",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Generate Relevance Score": {
      "main": [
        [
          {
            "node": "Sort by Relevance",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Remove Unnecessary Fields": {
      "main": [
        [
          {
            "node": "Loop over Items",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "When clicking \u2018Execute workflow\u2019": {
      "main": [
        [
          {
            "node": "Set Query",
            "type": "main",
            "index": 0
          }
        ]
      ]
    }
  }
}