AutomationFlowsWeb Scraping › Weekly Shodan Query - Report Accidents

Weekly Shodan Query - Report Accidents

Byn8n Team @n8n-team on n8n.io

This n8n workflow, which runs every Monday at 5:00 AM, initiates a comprehensive process to monitor and analyze network security by scrutinizing IP addresses and their associated ports. It begins by fetching a list of watched IP addresses and expected ports through an HTTP…

Cron / scheduled trigger★★★★☆ complexity15 nodesHTTP RequestItem ListsThe Hive
Web Scraping Trigger: Cron / scheduled Nodes: 15 Complexity: ★★★★☆ Added:

This workflow corresponds to n8n.io template #1977 — we link there as the canonical source.

This workflow follows the HTTP Request → Itemlists recipe pattern — see all workflows that pair these two integrations.

The workflow JSON

Copy or download the full n8n JSON below. Paste it into a new n8n workflow, add your credentials, activate. Full import guide →

Download .json
{
  "id": "VoLT6Omw9KMQgPum",
  "name": "Weekly_Shodan_Query___Report_Accidents__no_function_node_",
  "tags": [
    {
      "id": "GCHVocImoXoEVnzP",
      "name": "\ud83d\udee0\ufe0f In progress",
      "createdAt": "2023-10-31T02:17:21.618Z",
      "updatedAt": "2023-10-31T02:17:21.618Z"
    },
    {
      "id": "QPJKatvLSxxtrE8U",
      "name": "Secops",
      "createdAt": "2023-10-31T02:15:11.396Z",
      "updatedAt": "2023-10-31T02:15:11.396Z"
    }
  ],
  "nodes": [
    {
      "id": "54b2b2bd-9101-402c-b7cb-3d5e1070fcd2",
      "name": "Scan each IP",
      "type": "n8n-nodes-base.httpRequest",
      "position": [
        2123,
        202
      ],
      "parameters": {
        "url": "=https://api.shodan.io/shodan/host/{{ $json.ip }}",
        "options": {},
        "authentication": "genericCredentialType",
        "genericAuthType": "httpQueryAuth"
      },
      "credentials": {
        "httpQueryAuth": {
          "name": "<your credential>"
        }
      },
      "typeVersion": 4.1
    },
    {
      "id": "f6b194a7-a38d-46b4-899f-a9cb71de247e",
      "name": "Get watched IPs & Ports",
      "type": "n8n-nodes-base.httpRequest",
      "position": [
        1448.635348143835,
        200
      ],
      "parameters": {
        "url": "https://internal.users.n8n.cloud/webhook/mock-shodan-ips",
        "options": {}
      },
      "typeVersion": 4.1
    },
    {
      "id": "a6754adf-610b-46f0-9019-7ea21ac22690",
      "name": "Split out services",
      "type": "n8n-nodes-base.itemLists",
      "position": [
        2323,
        202
      ],
      "parameters": {
        "options": {},
        "fieldToSplitOut": "data"
      },
      "typeVersion": 3
    },
    {
      "id": "fa9dd77c-32e9-48c5-a1bf-8b95720aad43",
      "name": "Unexpected port?",
      "type": "n8n-nodes-base.filter",
      "position": [
        2543,
        202
      ],
      "parameters": {
        "conditions": {
          "boolean": [
            {
              "value1": "={{ $('For each IP').item.json.ports.includes($json.port) }}"
            }
          ]
        }
      },
      "typeVersion": 1
    },
    {
      "id": "addfeaf8-0c5d-4e4a-924e-53b3e28a23de",
      "name": "Set data to post for each port",
      "type": "n8n-nodes-base.set",
      "position": [
        2763,
        202
      ],
      "parameters": {
        "values": {
          "string": [
            {
              "name": "ip",
              "value": "={{ $('Get watched IPs & Ports').item.json.ip }}"
            },
            {
              "name": "hostnames",
              "value": "={{ $json.hostnames.join(', ') }}"
            },
            {
              "name": "port",
              "value": "={{ $json.port }}"
            },
            {
              "name": "description",
              "value": "={{ $json.description }}"
            },
            {
              "name": "data",
              "value": "={{ $json.data }}"
            }
          ]
        },
        "options": {},
        "keepOnlySet": true
      },
      "typeVersion": 2
    },
    {
      "id": "aaef71c0-927c-4297-9fa1-331e7009bf7e",
      "name": "Convert to table",
      "type": "n8n-nodes-base.html",
      "position": [
        2983,
        202
      ],
      "parameters": {
        "options": {},
        "operation": "convertToHtmlTable"
      },
      "typeVersion": 1
    },
    {
      "id": "2f257556-cf1b-4a80-8f40-7989ea077f48",
      "name": "Convert to Markdown",
      "type": "n8n-nodes-base.markdown",
      "position": [
        3203,
        202
      ],
      "parameters": {
        "html": "={{ $json.table }}",
        "options": {},
        "destinationKey": "markdown"
      },
      "typeVersion": 1
    },
    {
      "id": "9fdd40ba-1ab4-43a2-9e9d-53af5fc32f9f",
      "name": "For each IP",
      "type": "n8n-nodes-base.splitInBatches",
      "position": [
        1740,
        200
      ],
      "parameters": {
        "options": {},
        "batchSize": 1
      },
      "typeVersion": 2
    },
    {
      "id": "01823f1f-9612-4e56-a8f2-62aa2a0d5d5e",
      "name": "Sticky Note",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        2722,
        -137.71236116790237
      ],
      "parameters": {
        "width": 607.8070576425011,
        "height": 564.6974012965735,
        "content": "![Shodan](https://i.imgur.com/tK0RXSK.png)\n## Format port service data as a Markdown table\nAfter identifying the open ports, the next step is to organize this information neatly. This node converts the data gathered from the previous steps into a `Markdown table format`. \n\nIt's crucial for readability and makes it easier to parse through the port and service information. This formatted data can then be seamlessly integrated into documentation or reports, ensuring that the information is accessible and understandable for further analysis or sharing with team members."
      },
      "typeVersion": 1
    },
    {
      "id": "7ef1232b-8386-4b47-8617-a65749357ede",
      "name": "Sticky Note1",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        2083.3970861885364,
        -134.67908072298724
      ],
      "parameters": {
        "width": 606.297701841459,
        "height": 562.5474916374191,
        "content": "![Shodan](https://i.imgur.com/q4G3kQf.png)\n## Query Shodan for unexpected open ports\nThis stage of the workflow leverages `Shodan`, a search engine for internet-connected devices, to identify running services on each IP port.\n\nOnce the services and ports are returned, the `split out services` node extracts all the services to be filtered at once. \n\nIf an unexpected port is found, it allows the service to pass the service through the filter node."
      },
      "typeVersion": 1
    },
    {
      "id": "afd6651a-bcf1-4d66-ae1c-0f5616d3f29a",
      "name": "Sticky Note2",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        928,
        -465
      ],
      "parameters": {
        "width": 650.8045775723033,
        "height": 890.9735108226744,
        "content": "![Scheduled](https://i.imgur.com/PcOuvAL.png)\n# Workflow Overview\n\nThe n8n workflow initiates with a node that fetches a list of IP addresses and their specified ports from a security system, which is essential for ongoing surveillance of network integrity. \n\nThe data is expected to be in JSON format, detailing each IP with an array of associated ports to be monitored. While the example provided showcases a basic API call, in practice, this should be replaced with a call to the organization's own security system. \n\n`It's important to note that error handling is not included in this example and should be implemented according to the specific data formatting and error response protocols of the user's system.` The expectation for successful execution is that the incoming data conforms to the predefined JSON structure.\n\n## Get list of IPs from your IPS or database\n\nThis section retrieves a current list of IP addresses and their associated ports that require monitoring from your Intrusion Prevention System (IPS). It is essential to maintain an updated list to monitor for any unauthorized changes or traffic. The expected format for each entry is a JSON object containing the IP address and an array of ports.\n\nOur sample api call below can be replaced with api access to your IPS. To ensure it works, this workflow expects data to be in the following format:\n```\n[\n  {\n    \"ip\": \"116.202.106.35\",\n    \"ports\": [\n      5678,\n      80\n    ]\n  },\n  {\n    \"ip\": \"188.114.96.9\",\n    \"ports\": [\n      8080,\n      80\n    ]\n  }\n]\n```"
      },
      "typeVersion": 1
    },
    {
      "id": "14730a2f-42cc-4d96-b401-8a6a2c41aa27",
      "name": "Sticky Note4",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        3362,
        -167.68304244703404
      ],
      "parameters": {
        "width": 438.8550109331452,
        "height": 594.7981050471616,
        "content": "![thehive](https://i.imgur.com/y2Yw1ZP.png)\n## Post to TheHive\nThe final step in the process involves posting the findings to TheHive - a scalable, open-source and free Security Incident Response Platform. \n\nIf the workflow has identified an unexpected open port, it creates an alert in TheHive. This integration ensures that any potential security issues are escalated appropriately, and the relevant teams can begin the incident response process immediately, leveraging TheHive's powerful case management capabilities."
      },
      "typeVersion": 1
    },
    {
      "id": "f8cd7e3f-b55e-400d-ba45-486d4c736a16",
      "name": "Every Monday",
      "type": "n8n-nodes-base.scheduleTrigger",
      "position": [
        1228.635348143835,
        200
      ],
      "parameters": {
        "rule": {
          "interval": [
            {
              "field": "weeks",
              "triggerAtDay": [
                1
              ],
              "triggerAtHour": 5
            }
          ]
        }
      },
      "typeVersion": 1.1
    },
    {
      "id": "804ef38c-3ecb-41b2-ab38-ef8c231b7425",
      "name": "Create TheHive alert",
      "type": "n8n-nodes-base.theHive",
      "position": [
        3423,
        202
      ],
      "parameters": {
        "date": "={{$now}}",
        "tags": "={{ $('For each IP').last().json.ip }}",
        "type": "Unexpected open port",
        "title": "=Unexpected ports for {{ $('For each IP').last().json.ip }}",
        "source": "n8n",
        "sourceRef": "={{ $('For each IP').last().json.ip }}:{{$now.toUnixInteger()}}",
        "description": "=Unexpected open ports:\n\n{{ $json.markdown }}",
        "additionalFields": {}
      },
      "credentials": {
        "theHiveApi": {
          "name": "<your credential>"
        }
      },
      "typeVersion": 1
    },
    {
      "id": "5f09a27b-c6a4-4f74-a5ff-4c684e5e8917",
      "name": "Sticky Note5",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        1620,
        -261.14340884889145
      ],
      "parameters": {
        "width": 432.3140705656865,
        "height": 690.0398460499007,
        "content": "![n8n](https://i.imgur.com/lKnBNnH.png)\n## Iterate Through IP addresses\nThe \"`Split In Batches`\" node is configured with a batch size of one, ensuring that the array of IP addresses received is processed one at a time. \n\nThis approach allows for a focused analysis of each detection, ensuring no detail is overlooked. \n\nFollowing this, the \"`Split out services`\" node further along dissects each service to extract and separately handle the array of behaviors associated with them. \n\nBy processing these elements one by one, we effectively manage the workflow's load, maintaining optimal performance and adherence to external APIs' rate limits, crucial for the seamless operation of our security protocols.\n\n"
      },
      "typeVersion": 1
    }
  ],
  "active": false,
  "settings": {
    "executionOrder": "v1"
  },
  "versionId": "28b191c5-e23c-42db-9815-6770a2b72260",
  "connections": {
    "For each IP": {
      "main": [
        [
          {
            "node": "Scan each IP",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Every Monday": {
      "main": [
        [
          {
            "node": "Get watched IPs & Ports",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Scan each IP": {
      "main": [
        [
          {
            "node": "Split out services",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Convert to table": {
      "main": [
        [
          {
            "node": "Convert to Markdown",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Unexpected port?": {
      "main": [
        [
          {
            "node": "Set data to post for each port",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Split out services": {
      "main": [
        [
          {
            "node": "Unexpected port?",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Convert to Markdown": {
      "main": [
        [
          {
            "node": "Create TheHive alert",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Create TheHive alert": {
      "main": [
        [
          {
            "node": "For each IP",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Get watched IPs & Ports": {
      "main": [
        [
          {
            "node": "For each IP",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Set data to post for each port": {
      "main": [
        [
          {
            "node": "Convert to table",
            "type": "main",
            "index": 0
          }
        ]
      ]
    }
  }
}

Credentials you'll need

Each integration node will prompt for credentials when you import. We strip credential IDs before publishing — you'll add your own.

Pro

For the full experience including quality scoring and batch install features for each workflow upgrade to Pro

About this workflow

This n8n workflow, which runs every Monday at 5:00 AM, initiates a comprehensive process to monitor and analyze network security by scrutinizing IP addresses and their associated ports. It begins by fetching a list of watched IP addresses and expected ports through an HTTP…

Source: https://n8n.io/workflows/1977/ — original creator credit. Request a take-down →

More Web Scraping workflows → · Browse all categories →

Related workflows

Workflows that share integrations, category, or trigger type with this one. All free to copy and import.

Web Scraping

Weekly_Shodan_Query___Report_Accidents__no_function_node_. Uses httpRequest, itemLists, html, markdown. Scheduled trigger; 15 nodes.

HTTP Request, Item Lists, The Hive
Web Scraping

This workflow pushes Stripe charges to HubSpot contacts. It uses the Stripe API to get all charges and the HubSpot API to update the contacts. The workflow will create a new HubSpot property to store

Stripe, Item Lists, HubSpot +1
Web Scraping

Freelancers, founders, households, and side-hustlers who work with several bank accounts but want one, always-up-to-date budget inside Maybe Finance—no more CSV exports or copy-paste. Schedule Trigger

HTTP Request, N8N Nodes Resend, Item Lists
Web Scraping

This workflow syncs Zendesk tickets to Pipedrive contact owners.

Function Item, HTTP Request, Zendesk +2
Web Scraping

This workflow snoozes any Todoist tasks, by moving them into a Snoozed todoist list and unsnoozes them 3 days before due date. Helps keep inbox clear only of tasks you need to worry about soon. Add yo

Todoist, Crypto, Item Lists +1