AutomationFlowsWeb Scraping › Unstop — Hackathons (direct Upsert)

Unstop — Hackathons (direct Upsert)

Unstop — Hackathons (direct upsert). Uses httpRequest. Scheduled trigger; 8 nodes.

Cron / scheduled trigger★★★★☆ complexity8 nodesHTTP Request
Web Scraping Trigger: Cron / scheduled Nodes: 8 Complexity: ★★★★☆ Added:

The workflow JSON

Copy or download the full n8n JSON below. Paste it into a new n8n workflow, add your credentials, activate. Full import guide →

Download .json
{
  "name": "Unstop \u2014 Hackathons (direct upsert)",
  "nodes": [
    {
      "parameters": {
        "rule": {
          "interval": [
            {
              "field": "hours",
              "hoursInterval": 12
            }
          ]
        }
      },
      "type": "n8n-nodes-base.scheduleTrigger",
      "typeVersion": 1.3,
      "position": [
        0,
        0
      ],
      "id": "50ust001-0000-4000-8000-000000000001",
      "name": "Schedule Trigger"
    },
    {
      "parameters": {
        "jsCode": "// === Unstop Public Search API ===\n// India-centric platform for hackathons, case competitions, internships.\n// The public search endpoint powers their listings UI; returns rich JSON.\n//\n// We hit the hackathons surface here. To add competitions/internships,\n// clone this workflow and change `opportunity=` query param + category mapping.\n\nlet data;\ntry {\n  data = await this.helpers.httpRequest({\n    method: 'GET',\n    url: 'https://unstop.com/api/public/opportunity/search-result?opportunity=hackathons&per_page=20&oppstatus=open',\n    json: true,\n    headers: { Accept: 'application/json' },\n  });\n} catch (e) {\n  throw new Error('Unstop API fetch failed: ' + ((e && e.message) || e));\n}\n\n// Response shape: { data: { current_page, data: [...] } }\nconst items = (data && data.data && Array.isArray(data.data.data)) ? data.data.data : [];\nif (items.length === 0) {\n  throw new Error('Unstop returned 0 hackathons');\n}\n\nfunction extractTags(h) {\n  const tags = new Set();\n  if (Array.isArray(h.required_skills)) {\n    for (const s of h.required_skills) {\n      const name = (s && (s.name || s.title)) || (typeof s === 'string' ? s : null);\n      if (name) tags.add(String(name).toLowerCase());\n    }\n  }\n  if (Array.isArray(h.filters)) {\n    for (const f of h.filters) {\n      const name = (f && (f.name || f.title)) || (typeof f === 'string' ? f : null);\n      if (name) tags.add(String(name).toLowerCase());\n    }\n  }\n  return Array.from(tags).slice(0, 6);\n}\n\nfunction extractLocation(h) {\n  // Unstop nests location info in address_with_country_logo or similar\n  if (h.address_with_country_logo) {\n    const a = h.address_with_country_logo;\n    return [a.city, a.state, a.country].filter(Boolean).join(', ') || 'India';\n  }\n  if (typeof h.location === 'string') return h.location;\n  return 'India';\n}\n\nfunction extractCompensation(h) {\n  // Prize info lives in different fields depending on the listing.\n  // Always strip HTML \u2014 Unstop sometimes returns currency-formatted markup.\n  if (h.prize_money) return stripHtml(String(h.prize_money));\n  if (Array.isArray(h.prizes) && h.prizes.length) {\n    const first = h.prizes[0];\n    if (first && first.amount) return stripHtml(String(first.amount));\n  }\n  // Sometimes embedded in description as \u20b9X,XX,XXX \u2014 let admin fix manually.\n  return null;\n}\n\nfunction extractEligibility(h) {\n  if (h.regnRequirements) {\n    const r = h.regnRequirements;\n    const parts = [];\n    if (r.team_size_min || r.team_size_max) {\n      parts.push(`Team size: ${r.team_size_min || 1}-${r.team_size_max || 'any'}`);\n    }\n    if (r.eligibility) parts.push(String(r.eligibility));\n    return parts.length ? parts.join('. ') : null;\n  }\n  return null;\n}\n\nfunction stripHtml(s) {\n  if (!s) return null;\n  return String(s)\n    .replace(/<[^>]*>/g, ' ')\n    .replace(/&nbsp;/g, ' ')\n    .replace(/&amp;/g, '&')\n    .replace(/&lt;/g, '<')\n    .replace(/&gt;/g, '>')\n    .replace(/\\s+/g, ' ')\n    .trim();\n}\n\nconst out = [];\nfor (const h of items) {\n  const orgName = (h.organisation && h.organisation.name) || 'Unstop';\n  const cleanDesc = stripHtml(h.description);\n  const summary = cleanDesc\n    ? (cleanDesc.length > 280 ? cleanDesc.slice(0, 277) + '...' : cleanDesc)\n    : `Hackathon by ${orgName}`;\n\n  const opportunity = {\n    title: h.title,\n    organization: orgName,\n    category: 'hackathon',\n    description: cleanDesc,\n    summary: summary,\n    tags: extractTags(h),\n    deadline: h.end_date || null,  // already ISO 8601\n    eligibility: extractEligibility(h),\n    location: extractLocation(h),\n    compensation: extractCompensation(h),\n    is_remote: false,  // Unstop hackathons are mostly offline at colleges\n    apply_url: h.seo_url,\n    difficulty: null,\n    estimated_value_score: null,\n    extraction_confidence: 0.9,\n  };\n\n  out.push({ json: { opportunity, source_url: h.seo_url } });\n}\n\nreturn out;"
      },
      "type": "n8n-nodes-base.code",
      "typeVersion": 2,
      "position": [
        208,
        0
      ],
      "id": "50ust001-0000-4000-8000-000000000002",
      "name": "Fetch Unstop Hackathons"
    },
    {
      "parameters": {
        "maxItems": 15
      },
      "type": "n8n-nodes-base.limit",
      "typeVersion": 1,
      "position": [
        416,
        0
      ],
      "id": "50ust001-0000-4000-8000-000000000003",
      "name": "Limit"
    },
    {
      "parameters": {
        "options": {
          "reset": false
        }
      },
      "type": "n8n-nodes-base.splitInBatches",
      "typeVersion": 3,
      "position": [
        624,
        0
      ],
      "id": "50ust001-0000-4000-8000-000000000004",
      "name": "Loop Over Items"
    },
    {
      "parameters": {
        "method": "POST",
        "url": "http://host.docker.internal:3000/api/ingest/check-exists",
        "sendHeaders": true,
        "headerParameters": {
          "parameters": [
            {
              "name": "X-Ingest-Secret",
              "value": "REPLACE_WITH_YOUR_INGEST_SHARED_SECRET"
            }
          ]
        },
        "sendBody": true,
        "bodyParameters": {
          "parameters": [
            {
              "name": "source_url",
              "value": "={{ $json.source_url }}"
            }
          ]
        },
        "options": {}
      },
      "type": "n8n-nodes-base.httpRequest",
      "typeVersion": 4.4,
      "position": [
        832,
        0
      ],
      "id": "50ust001-0000-4000-8000-000000000005",
      "name": "Check Exists"
    },
    {
      "parameters": {
        "conditions": {
          "options": {
            "caseSensitive": true,
            "leftValue": "",
            "typeValidation": "strict",
            "version": 3
          },
          "conditions": [
            {
              "id": "n1",
              "leftValue": "={{ $json.exists }}",
              "rightValue": "",
              "operator": {
                "type": "boolean",
                "operation": "false",
                "singleValue": true
              }
            }
          ],
          "combinator": "and"
        },
        "options": {}
      },
      "type": "n8n-nodes-base.if",
      "typeVersion": 2.3,
      "position": [
        1040,
        0
      ],
      "id": "50ust001-0000-4000-8000-000000000006",
      "name": "If New"
    },
    {
      "parameters": {
        "method": "POST",
        "url": "http://host.docker.internal:3000/api/ingest/upsert",
        "sendHeaders": true,
        "headerParameters": {
          "parameters": [
            {
              "name": "X-Ingest-Secret",
              "value": "REPLACE_WITH_YOUR_INGEST_SHARED_SECRET"
            }
          ]
        },
        "sendBody": true,
        "bodyParameters": {
          "parameters": [
            {
              "name": "opportunity",
              "value": "={{ $('Loop Over Items').item.json.opportunity }}"
            },
            {
              "name": "source_url",
              "value": "={{ $('Loop Over Items').item.json.source_url }}"
            },
            {
              "name": "source_name",
              "value": "Unstop"
            }
          ]
        },
        "options": {
          "timeout": 15000
        }
      },
      "type": "n8n-nodes-base.httpRequest",
      "typeVersion": 4.4,
      "position": [
        1280,
        -64
      ],
      "id": "50ust001-0000-4000-8000-000000000007",
      "name": "Upsert Opportunity"
    },
    {
      "parameters": {
        "method": "POST",
        "url": "http://host.docker.internal:3000/api/log",
        "sendHeaders": true,
        "headerParameters": {
          "parameters": [
            {
              "name": "X-Ingest-Secret",
              "value": "REPLACE_WITH_YOUR_INGEST_SHARED_SECRET"
            }
          ]
        },
        "sendBody": true,
        "bodyParameters": {
          "parameters": [
            {
              "name": "status",
              "value": "skipped_duplicate"
            },
            {
              "name": "source_url",
              "value": "={{ $('Loop Over Items').item.json.source_url }}"
            },
            {
              "name": "source_name",
              "value": "Unstop"
            }
          ]
        },
        "options": {}
      },
      "type": "n8n-nodes-base.httpRequest",
      "typeVersion": 4.4,
      "position": [
        1280,
        272
      ],
      "id": "50ust001-0000-4000-8000-000000000008",
      "name": "Log Duplicate"
    }
  ],
  "connections": {
    "Schedule Trigger": {
      "main": [
        [
          {
            "node": "Fetch Unstop Hackathons",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Fetch Unstop Hackathons": {
      "main": [
        [
          {
            "node": "Limit",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Limit": {
      "main": [
        [
          {
            "node": "Loop Over Items",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Loop Over Items": {
      "main": [
        [],
        [
          {
            "node": "Check Exists",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Check Exists": {
      "main": [
        [
          {
            "node": "If New",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "If New": {
      "main": [
        [
          {
            "node": "Upsert Opportunity",
            "type": "main",
            "index": 0
          }
        ],
        [
          {
            "node": "Log Duplicate",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Upsert Opportunity": {
      "main": [
        [
          {
            "node": "Loop Over Items",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Log Duplicate": {
      "main": [
        [
          {
            "node": "Loop Over Items",
            "type": "main",
            "index": 0
          }
        ]
      ]
    }
  },
  "settings": {
    "executionOrder": "v1",
    "binaryMode": "separate"
  },
  "tags": []
}
Pro

For the full experience including quality scoring and batch install features for each workflow upgrade to Pro

About this workflow

Unstop — Hackathons (direct upsert). Uses httpRequest. Scheduled trigger; 8 nodes.

Source: https://github.com/krishnagahlod/opportunity-os/blob/399dbb38874ec13c22922b64013cbc57a2845dc0/n8n-workflows/05-unstop-hackathons.json — original creator credit. Request a take-down →

More Web Scraping workflows → · Browse all categories →

Related workflows

Workflows that share integrations, category, or trigger type with this one. All free to copy and import.

Web Scraping

As n8n instances scale, teams often lose track of sub-workflows—who uses them, where they are referenced, and whether they can be safely updated. This leads to inefficiencies like unnecessary copies o

HTTP Request, n8n, N8N Trigger +1
Web Scraping

This workflow is an improvement of this workflow by Greg Brzezinka.

HTTP Request, Email Send, XML +1
Web Scraping

N8N-Workflow-Github-Manager. Uses github, httpRequest, n8n. Scheduled trigger; 38 nodes.

GitHub, HTTP Request, n8n
Web Scraping

This workflow uses KlickTipp community nodes, available for self-hosted n8n instances only.

N8N Nodes Klicktipp, Salesforce, Salesforce Trigger +1
Web Scraping

This workflow acts as an automated engagement bot. It sends a Direct Message (DM) with a link or resource to any follower who replies to your post with a specific target keyword.

HTTP Request