Track Job Skill Demand and Pay with Bright Data, Openai, Google Sheets and Slack

The workflow JSON

Copy or download the full n8n JSON below. Paste it into a new n8n workflow, add your credentials, activate. Full import guide →

{
  "name": "Track skill demand and salaries for any role with Bright Data, GPT-5.6, Google Sheets and Slack",
  "nodes": [
    {
      "id": "sticky-main",
      "name": "Sticky Note main",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        -700,
        309
      ],
      "parameters": {
        "color": 1,
        "width": 560,
        "height": 934,
        "content": "## Track what a job market actually demands, and what it pays\n\nPoint this at one role in one location and it comes back every week with the skills employers are asking for, ranked by how many live postings require each one, plus whatever the market is willing to say about pay.\n\n### How it works\nA weekly schedule triggers a Bright Data keyword search of live LinkedIn job postings for your role and location. Because discovery takes about half a minute, the workflow uses Bright Data's asynchronous API properly: it triggers the search, then polls the snapshot on a Wait loop with a seven-minute cap until the rows are ready.\n\nEach posting is trimmed and sent to GPT-5.6, which reads it for required skills, tools, years of experience, working pattern and any stated pay. Skills are counted by posting rather than by mention, so one wordy advert cannot dominate the ranking. The result is diffed against the previous scan held in Google Sheets, which is what turns a snapshot into a trend: rising, falling, new and dropped skills. A short brief goes to Slack with the ranking, and every skill lands in the sheet as its own row so the next run has something to compare against.\n\n### Setup\nAdd a Bright Data API key as a Header Auth credential (`Authorization` / `Bearer YOUR_KEY`), connect OpenAI, Google Sheets and Slack, then set your role, location and channel in Set Market Config. The first run writes the baseline, so movement appears from the second run onwards.\n\n### Customization\nSwap the LinkedIn jobs dataset for the Indeed or Glassdoor equivalent, or point several copies at the same role in different cities to compare markets."
      },
      "typeVersion": 1
    },
    {
      "id": "sticky-sec1",
      "name": "Sticky Note sec1",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        -60,
        310
      ],
      "parameters": {
        "color": 7,
        "width": 696,
        "height": 366,
        "content": "## 1. Pick the market\nOne role in one location, on a weekly schedule. Everything you would tune lives in one config node."
      },
      "typeVersion": 1
    },
    {
      "id": "sticky-sec2",
      "name": "Sticky Note sec2",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        816,
        310
      ],
      "parameters": {
        "color": 7,
        "width": 1416,
        "height": 366,
        "content": "## 2. Discover postings asynchronously\nBright Data searches live postings by keyword. Discovery takes about half a minute, so this polls the snapshot until the rows are ready."
      },
      "typeVersion": 1
    },
    {
      "id": "sticky-sec3",
      "name": "Sticky Note sec3",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        2412,
        310
      ],
      "parameters": {
        "color": 7,
        "width": 456,
        "height": 366,
        "content": "## 3. Clean the postings\nDrops duplicates and postings with no description, then trims each one to the part that carries the requirements."
      },
      "typeVersion": 1
    },
    {
      "id": "sticky-sec4",
      "name": "Sticky Note sec4",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        3048,
        309
      ],
      "parameters": {
        "color": 7,
        "width": 440,
        "height": 591,
        "content": "## 4. Read each posting\nThe extractor runs once per posting and returns skills, years of experience, working pattern and any stated pay."
      },
      "typeVersion": 1
    },
    {
      "id": "sticky-sec5",
      "name": "Sticky Note sec5",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        3572,
        310
      ],
      "parameters": {
        "color": 7,
        "width": 1064,
        "height": 590,
        "content": "## 5. Rank and compare\nSkills are counted by posting, then diffed against the previous scan in the sheet to show what is rising, new or gone."
      },
      "typeVersion": 1
    },
    {
      "id": "sticky-sec6",
      "name": "Sticky Note sec6",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        4816,
        310
      ],
      "parameters": {
        "color": 7,
        "width": 936,
        "height": 366,
        "content": "## 6. Deliver\nOne sheet row per skill, which is what the next run compares against, plus a Slack digest."
      },
      "typeVersion": 1
    },
    {
      "id": "sticky-warn",
      "name": "Sticky Note warn",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        816,
        736
      ],
      "parameters": {
        "color": 3,
        "width": 380,
        "height": 223,
        "content": "## Many postings never state a salary\nPay was disclosed on 4 of 14 postings in testing and the rate swings hard by market, so the range comes from whichever postings said anything. The report always prints how many disclosed. Treat a small number as indicative, not as the market rate."
      },
      "typeVersion": 1
    },
    {
      "id": "t5-trigger",
      "name": "Weekly Market Scan",
      "type": "n8n-nodes-base.scheduleTrigger",
      "position": [
        0,
        460
      ],
      "parameters": {
        "rule": {
          "interval": [
            {
              "field": "days",
              "triggerAtHour": 7,
              "triggerAtMinute": 0
            }
          ]
        }
      },
      "typeVersion": 1.2
    },
    {
      "id": "t5-cfg",
      "name": "Set Market Config",
      "type": "n8n-nodes-base.set",
      "position": [
        240,
        460
      ],
      "parameters": {
        "mode": "manual",
        "options": {},
        "assignments": {
          "assignments": [
            {
              "id": "c0",
              "name": "role_keyword",
              "type": "string",
              "value": "growth marketing manager"
            },
            {
              "id": "c1",
              "name": "location",
              "type": "string",
              "value": "New York, NY"
            },
            {
              "id": "c2",
              "name": "country",
              "type": "string",
              "value": "US"
            },
            {
              "id": "c3",
              "name": "time_range",
              "type": "string",
              "value": "Past month"
            },
            {
              "id": "c4",
              "name": "job_type",
              "type": "string",
              "value": "Full-time"
            },
            {
              "id": "c5",
              "name": "jobs_per_scan",
              "type": "number",
              "value": 25
            },
            {
              "id": "c6",
              "name": "sheet_url",
              "type": "string",
              "value": "https://docs.google.com/spreadsheets/d/YOUR_SHEET_ID"
            },
            {
              "id": "c7",
              "name": "slack_channel",
              "type": "string",
              "value": "#hiring"
            }
          ]
        },
        "includeOtherFields": false
      },
      "typeVersion": 3.4
    },
    {
      "id": "t5-input",
      "name": "Build Discovery Input",
      "type": "n8n-nodes-base.code",
      "position": [
        480,
        460
      ],
      "parameters": {
        "mode": "runOnceForAllItems",
        "jsCode": "// single-item input: the config node emits exactly one row.\nconst cfg = $input.first().json;\n\nconst role = String(cfg.role_keyword || '').trim();\nconst location = String(cfg.location || '').trim();\nif (!role || !location) {\n  throw new Error('Set both role_keyword and location in ' + 'Set Market Config' + '.');\n}\n\n// Bright Data discovers by keyword, so the payload is the search itself rather\n// than a list of URLs. One search per run keeps the ranking about one market.\nconst payload = [{\n  keyword: role,\n  location,\n  country: String(cfg.country || 'US').toUpperCase(),\n  time_range: cfg.time_range || 'Past month',\n  job_type: cfg.job_type || 'Full-time',\n}];\n\nreturn [{ json: { payload, role, location, scanned_at: new Date().toISOString() } }];"
      },
      "typeVersion": 2
    },
    {
      "id": "t5-fetch",
      "name": "Discover Jobs With Bright Data",
      "type": "n8n-nodes-base.httpRequest",
      "onError": "continueRegularOutput",
      "maxTries": 3,
      "position": [
        876,
        460
      ],
      "parameters": {
        "url": "=https://api.brightdata.com/datasets/v3/trigger?dataset_id=gd_lpfll7v5hcqtkxl6l&format=json&include_errors=true&type=discover_new&discover_by=keyword&limit_per_input={{ $('Set Market Config').first().json.jobs_per_scan }}",
        "method": "POST",
        "options": {
          "timeout": 120000,
          "response": {
            "response": {
              "neverError": true
            }
          }
        },
        "jsonBody": "={{ JSON.stringify($json.payload) }}",
        "sendBody": true,
        "specifyBody": "json",
        "authentication": "genericCredentialType",
        "genericAuthType": "httpHeaderAuth"
      },
      "retryOnFail": true,
      "typeVersion": 4.4,
      "waitBetweenTries": 2000
    },
    {
      "id": "t5-accept",
      "name": "Check Discovery Accepted",
      "type": "n8n-nodes-base.code",
      "position": [
        1116,
        460
      ],
      "parameters": {
        "mode": "runOnceForAllItems",
        "jsCode": "// single-item input: the trigger endpoint answers once with one snapshot id.\nconst res = $input.first().json;\n\n// Bright Data answers validation errors with HTTP 200 and an error body, and\n// this node runs with neverError, so an unchecked response looks like a success\n// and fails several nodes later blaming the wrong thing.\nif (res.error || res.errors) {\n  throw new Error('Bright Data rejected the search: ' + JSON.stringify(res.error || res.errors));\n}\nif (!res.snapshot_id) {\n  throw new Error('No snapshot_id in the Bright Data response: ' + JSON.stringify(res).slice(0, 400));\n}\n\nreturn [{ json: { snapshot_id: res.snapshot_id } }];"
      },
      "typeVersion": 2
    },
    {
      "id": "t5-wait",
      "name": "Wait For Snapshot",
      "type": "n8n-nodes-base.wait",
      "position": [
        1356,
        460
      ],
      "parameters": {
        "unit": "seconds",
        "amount": 20
      },
      "typeVersion": 1.1
    },
    {
      "id": "t5-progress",
      "name": "Check Snapshot Progress",
      "type": "n8n-nodes-base.httpRequest",
      "onError": "continueRegularOutput",
      "maxTries": 3,
      "position": [
        1596,
        460
      ],
      "parameters": {
        "url": "=https://api.brightdata.com/datasets/v3/progress/{{ $('Check Discovery Accepted').first().json.snapshot_id }}",
        "method": "GET",
        "options": {
          "timeout": 60000,
          "response": {
            "response": {
              "neverError": true
            }
          }
        },
        "authentication": "genericCredentialType",
        "genericAuthType": "httpHeaderAuth"
      },
      "retryOnFail": true,
      "typeVersion": 4.4,
      "waitBetweenTries": 2000
    },
    {
      "id": "t5-eval",
      "name": "Evaluate Scrape Progress",
      "type": "n8n-nodes-base.code",
      "position": [
        1836,
        460
      ],
      "parameters": {
        "mode": "runOnceForAllItems",
        "jsCode": "const MAX_POLLS = 20; // 20 x 20s = about 7 minutes\n\n// single-item input: the progress endpoint returns one status object.\nconst res = $input.first().json;\nconst snapshotId = $('Check Discovery Accepted').first().json.snapshot_id;\nconst attempt = $runIndex + 1;\n\nif (res.error) {\n  throw new Error('Progress check failed: ' + JSON.stringify(res.error));\n}\nif (res.status === 'failed') {\n  throw new Error('Bright Data reported the snapshot failed: ' + JSON.stringify(res).slice(0, 400));\n}\nif (res.status !== 'ready' && attempt >= MAX_POLLS) {\n  throw new Error(\n    'Snapshot ' + snapshotId + ' still ' + res.status + ' after ' + attempt +\n    ' polls. Raise MAX_POLLS or narrow the search.'\n  );\n}\n\nreturn [{ json: { status: res.status, snapshot_id: snapshotId, attempt, records: res.records || 0 } }];"
      },
      "typeVersion": 2
    },
    {
      "id": "t5-ready",
      "name": "Jobs Ready?",
      "type": "n8n-nodes-base.if",
      "position": [
        2076,
        460
      ],
      "parameters": {
        "options": {},
        "conditions": {
          "options": {
            "version": 2,
            "leftValue": "",
            "caseSensitive": true,
            "typeValidation": "strict"
          },
          "combinator": "and",
          "conditions": [
            {
              "id": "cond1",
              "operator": {
                "type": "string",
                "operation": "equals"
              },
              "leftValue": "={{ $json.status }}",
              "rightValue": "ready"
            }
          ]
        }
      },
      "typeVersion": 2.2
    },
    {
      "id": "t5-download",
      "name": "Download Job Posts",
      "type": "n8n-nodes-base.httpRequest",
      "onError": "continueRegularOutput",
      "maxTries": 3,
      "position": [
        2472,
        460
      ],
      "parameters": {
        "url": "=https://api.brightdata.com/datasets/v3/snapshot/{{ $('Check Discovery Accepted').first().json.snapshot_id }}?format=json",
        "method": "GET",
        "options": {
          "timeout": 180000,
          "response": {
            "response": {
              "neverError": true
            }
          }
        },
        "authentication": "genericCredentialType",
        "genericAuthType": "httpHeaderAuth"
      },
      "retryOnFail": true,
      "typeVersion": 4.4,
      "waitBetweenTries": 2000
    },
    {
      "id": "t5-norm",
      "name": "Normalise Job Posts",
      "type": "n8n-nodes-base.code",
      "position": [
        2712,
        460
      ],
      "parameters": {
        "mode": "runOnceForAllItems",
        "jsCode": "const cfg = $('Set Market Config').first().json;\nconst maxChars = 2500;\n\n// A post with no description gives the extractor nothing to read, and an empty\n// extraction still costs a model call, so drop those here.\nconst seen = new Set();\nconst out = [];\n\nfor (const item of $input.all()) {\n  const j = item.json;\n  if (j.error || j.warning) continue;\n\n  const id = String(j.job_posting_id || j.url || '');\n  if (!id || seen.has(id)) continue;\n  seen.add(id);\n\n  const summary = String(j.job_summary || '').replace(/\\s+/g, ' ').trim();\n  if (summary.length < 200) continue;\n\n  // Descriptions run to 10,000 characters. The requirements live near the top,\n  // so a head slice keeps the model cost roughly flat per post.\n  const text = summary.length > maxChars ? summary.slice(0, maxChars) : summary;\n\n  // Compensation is stated near the bottom - it sat between characters 4,000 and\n  // 9,000 on every post that had it - so the head slice above throws away the\n  // one sentence the pay question depends on. Carry it separately.\n  const payMatch = summary.match(\n    /[^.]{0,130}(?:salary|compensation|base pay|pay range|hourly rate)[^.]{0,170}/i\n  );\n\n  const bs = j.base_salary || null;\n  const statedPay = bs && bs.min_amount\n    ? `${bs.currency || '$'}${bs.min_amount} - ${bs.currency || '$'}${bs.max_amount} per ${bs.payment_period || 'yr'}`\n    : '';\n\n  out.push({\n    json: {\n      job_id: id,\n      title: String(j.job_title || '').trim(),\n      company: String(j.company_name || '').trim(),\n      location: String(j.job_location || '').trim(),\n      seniority: String(j.job_seniority_level || '').trim(),\n      employment_type: String(j.job_employment_type || '').trim(),\n      industry: String(j.job_industries || '').trim(),\n      applicants: Number(j.job_num_applicants || 0),\n      posted: String(j.job_posted_date || '').slice(0, 10),\n      url: String(j.url || '').split('?')[0],\n      stated_pay: statedPay,\n      pay_text: payMatch ? payMatch[0].trim() : '',\n      // Bright Data's own structured pay, kept as numbers so the ranking can\n      // prefer it over the model's reading of the prose.\n      struct_pay_min: bs && bs.min_amount ? Number(bs.min_amount) : 0,\n      struct_pay_max: bs && bs.max_amount ? Number(bs.max_amount) : 0,\n      struct_pay_period: bs && bs.payment_period ? String(bs.payment_period) : '',\n      description: text,\n      role: cfg.role_keyword,\n      market: cfg.location,\n    },\n  });\n}\n\nif (!out.length) {\n  throw new Error(\n    'No usable job posts came back. Widen time_range, or check that the ' +\n    'keyword and location match a real LinkedIn search.'\n  );\n}\n\nreturn out;"
      },
      "typeVersion": 2
    },
    {
      "id": "t5-extract",
      "name": "Extract Role Requirements",
      "type": "@n8n/n8n-nodes-langchain.informationExtractor",
      "onError": "continueRegularOutput",
      "position": [
        3108,
        460
      ],
      "parameters": {
        "text": "=Extract the requirements from this job posting.\n\nFor `skills`, name at most 10 concrete, checkable skills and tools: named\nsoftware (hubspot, salesforce, ga4, figma, snowflake), named techniques (seo,\npaid search, sql, a/b testing, lifecycle marketing) and named languages. Use the\nshort common name in lower case. Do not return responsibilities, seniority\nwords, soft skills or whole phrases from the advert: \"relationship building\",\n\"cross-functional collaboration\", \"strategy\" and \"ownership\" are all wrong.\n\nFor pay, read the description text as well as the structured field. Set\n`pay_disclosed` false when the posting states no figure, and never guess a range.\nReport the pay exactly as written, and set `pay_period` to match it.\n\nJob title: {{ $json.title }}\nCompany: {{ $json.company }}\nSeniority label: {{ $json.seniority }}\nPay stated in structured data: {{ $json.stated_pay || 'none' }}\nPay sentence found in the description: {{ $json.pay_text || 'none' }}\n\nDescription:\n{{ $json.description }}",
        "options": {},
        "schemaType": "fromJson",
        "jsonSchemaExample": "{\n  \"skills\": [\n    \"seo\",\n    \"hubspot\",\n    \"google analytics\",\n    \"paid search\",\n    \"sql\"\n  ],\n  \"must_have_years\": 0,\n  \"pay_min\": 0,\n  \"pay_max\": 0,\n  \"pay_period\": \"year | month | hour | unknown\",\n  \"pay_currency\": \"USD\",\n  \"pay_disclosed\": true,\n  \"remote_policy\": \"remote | hybrid | onsite | unclear\"\n}"
      },
      "typeVersion": 1.2
    },
    {
      "id": "t5-llm1",
      "name": "OpenAI Requirements Reader",
      "type": "@n8n/n8n-nodes-langchain.lmChatOpenAi",
      "onError": "continueRegularOutput",
      "position": [
        3116,
        700
      ],
      "parameters": {
        "model": {
          "__rl": true,
          "mode": "list",
          "value": "gpt-5.6-terra"
        },
        "options": {
          "temperature": 0.2
        }
      },
      "typeVersion": 1.3
    },
    {
      "id": "t5-rank",
      "name": "Rank Skills And Pay",
      "type": "n8n-nodes-base.code",
      "position": [
        3632,
        460
      ],
      "parameters": {
        "mode": "runOnceForAllItems",
        "jsCode": "const cfg = $('Set Market Config').first().json;\nconst posts = $('Normalise Job Posts').all().map((i) => i.json);\n// The extractor nests its result under `output` on some versions. Reading the\n// item directly gives undefined skills on every post, which produces an empty\n// ranking on a run where every node is green.\nconst extracted = $input.all().map((i) => (i.json && i.json.output) || i.json || {});\n\n// Merge each extraction back onto the post it came from. The extractor emits one\n// item per input item in the same order, so index alignment is the join key.\nconst rows = posts.map((p, i) => ({ ...p, ...(extracted[i] || {}) }));\n\n// --- skills -------------------------------------------------------------\n// Post counts, not mention counts. A post that says \"SQL\" six times is still one\n// employer asking for SQL, and counting mentions would let one verbose ad\n// dominate the ranking.\nconst ALIAS = {\n  'js': 'javascript', 'ts': 'typescript', 'py': 'python',\n  'postgres': 'postgresql', 'k8s': 'kubernetes', 'gcp': 'google cloud',\n  'aws cloud': 'aws', 'ga4': 'google analytics', 'node': 'node.js',\n};\nconst clean = (s) => {\n  let t = String(s || '').toLowerCase().trim()\n    .replace(/[.,;:()]+$/g, '')\n    .replace(/\\s+/g, ' ');\n  return ALIAS[t] || t;\n};\n\nconst perPost = rows.map(\n  (r) => new Set((r.skills || []).map(clean).filter((s) => s.length > 1 && s.length < 40))\n);\n\n// \"automation\" from one posting and \"automation workflows\" from another are one\n// demand signal, not two. Collapse the longer name onto the shorter one, but\n// only when that shorter name is itself something an employer wrote, so the\n// ranking never shows a label nobody used. Counting has to happen after this,\n// or a posting naming both variants would be counted twice.\nconst allNames = [...new Set(perPost.flatMap((s) => [...s]))].sort((a, b) => a.length - b.length);\nconst canon = new Map();\nfor (const name of allNames) {\n  const shorter = allNames.find((s) => s.length < name.length && name.startsWith(s + ' '));\n  canon.set(name, shorter ? canon.get(shorter) || shorter : name);\n}\n\nconst skillPosts = new Map();\nfor (const set of perPost) {\n  const uniq = new Set([...set].map((s) => canon.get(s) || s));\n  for (const s of uniq) skillPosts.set(s, (skillPosts.get(s) || 0) + 1);\n}\n\nconst total = rows.length;\n\n// On a real scan of 25 postings a skill named by one employer is noise, not\n// demand, and those one-offs otherwise fill the whole ranking. On a small sample\n// there is nothing to spare, so the floor only applies once the sample is big\n// enough for it to mean something.\nconst floor = total >= 10 ? 2 : 1;\n\nconst skills = [...skillPosts.entries()]\n  .filter(([, posts_requiring]) => posts_requiring >= floor)\n  .map(([skill, posts_requiring]) => ({\n    skill,\n    posts_requiring,\n    share_pct: Math.round((posts_requiring / total) * 1000) / 10,\n  }))\n  .sort((a, b) => b.posts_requiring - a.posts_requiring || a.skill.localeCompare(b.skill));\n\n// --- pay ----------------------------------------------------------------\n// Only 1 of 6 posts carried structured pay in testing, so most of this comes\n// from the model reading the text. Normalise everything to an annual figure and\n// throw out anything that cannot be an annual base, which is what stops a \"$600\"\n// day rate or a stray bonus number from moving the median.\nconst ANNUALISE = { year: 1, month: 12, hour: 2080 };\n\n// The period arrives as \"yr\" from Bright Data, \"per yr\" from the string handed to\n// the model, and \"year\" from the schema, depending on which source answered.\n// Matching the bare word only means every salary annualises to zero and the\n// report then states that nobody disclosed pay - wrong, and wrong in a way that\n// reads like a real finding.\nconst period = (p) => {\n  const t = String(p || '').toLowerCase();\n  if (/(yr|year|annual|annum|\\bpa\\b)/.test(t)) return 'year';\n  if (/(month|\\bmo\\b|mth)/.test(t)) return 'month';\n  if (/(hour|\\bhr\\b|hourly)/.test(t)) return 'hour';\n  return '';\n};\nconst annual = (v, p) => {\n  const n = Number(v) || 0;\n  const mult = ANNUALISE[period(p)] || 0;\n  return mult ? n * mult : 0;\n};\n\nconst paid = [];\nfor (const r of rows) {\n  // Bright Data's structured field is authoritative where it exists; the model\n  // reading the prose is the fallback for the postings that lack it.\n  const useStruct = Boolean(r.struct_pay_min);\n  if (!useStruct && !r.pay_disclosed) continue;\n\n  const rawLo = useStruct ? r.struct_pay_min : r.pay_min;\n  const rawHi = useStruct ? r.struct_pay_max : r.pay_max;\n  const per = useStruct ? r.struct_pay_period : r.pay_period;\n\n  const lo = annual(rawLo, per);\n  const hi = annual(rawHi, per) || lo;\n  // Anything that cannot be an annual base is noise: a day rate, a bonus figure\n  // or an equity number the model mistook for salary.\n  if (!lo || lo < 20000 || lo > 2000000) continue;\n  paid.push({ lo, hi: Math.max(lo, hi), mid: (lo + Math.max(lo, hi)) / 2, title: r.title, company: r.company });\n}\n\nconst median = (arr) => {\n  if (!arr.length) return 0;\n  const s = [...arr].sort((a, b) => a - b);\n  const m = Math.floor(s.length / 2);\n  return s.length % 2 ? s[m] : Math.round((s[m - 1] + s[m]) / 2);\n};\n\nconst pay = {\n  disclosed_posts: paid.length,\n  total_posts: total,\n  disclosure_pct: Math.round((paid.length / total) * 1000) / 10,\n  low: paid.length ? Math.min(...paid.map((p) => p.lo)) : 0,\n  median_mid: median(paid.map((p) => p.mid)),\n  high: paid.length ? Math.max(...paid.map((p) => p.hi)) : 0,\n};\n\n// --- shape of the market ------------------------------------------------\n// The model ignores the enum often enough to matter - one run came back with\n// \"not specified\", \"in-person\" and \"in-office\" for three postings, which reads\n// as three different working patterns when it is really one.\nconst REMOTE = (v) => {\n  const t = String(v || '').toLowerCase();\n  if (/hybrid/.test(t)) return 'hybrid';\n  if (/remote|work from home|wfh|distributed/.test(t)) return 'remote';\n  if (/on-?site|in-?person|in-?office|on premise/.test(t)) return 'onsite';\n  return 'unclear';\n};\n\nconst tally = (key, map) => {\n  const m = {};\n  for (const r of rows) {\n    const v = map ? map(r[key]) : (String(r[key] || 'unclear').trim() || 'unclear');\n    m[v] = (m[v] || 0) + 1;\n  }\n  return Object.entries(m).sort((a, b) => b[1] - a[1]).map(([k, v]) => ({ name: k, posts: v }));\n};\n\nconst years = rows.map((r) => Number(r.must_have_years) || 0).filter((y) => y > 0 && y < 30);\n\nreturn [{\n  json: {\n    role: cfg.role_keyword,\n    market: cfg.location,\n    scanned_at: new Date().toISOString().slice(0, 10),\n    total_posts: total,\n    skills,\n    pay,\n    remote: tally('remote_policy', REMOTE),\n    seniority: tally('seniority'),\n    top_companies: tally('company').slice(0, 5),\n    median_years: median(years),\n    sample_titles: [...new Set(rows.map((r) => r.title))].slice(0, 6),\n  },\n}];"
      },
      "typeVersion": 2
    },
    {
      "id": "t5-history",
      "name": "Read Previous Scan",
      "type": "n8n-nodes-base.googleSheets",
      "onError": "continueRegularOutput",
      "position": [
        3872,
        460
      ],
      "parameters": {
        "options": {},
        "sheetName": {
          "__rl": true,
          "mode": "name",
          "value": "Skill demand"
        },
        "documentId": {
          "__rl": true,
          "mode": "url",
          "value": "={{ $('Set Market Config').first().json.sheet_url }}"
        }
      },
      "typeVersion": 4.7,
      "alwaysOutputData": true
    },
    {
      "id": "t5-compare",
      "name": "Compare With Last Scan",
      "type": "n8n-nodes-base.code",
      "position": [
        4112,
        460
      ],
      "parameters": {
        "mode": "runOnceForAllItems",
        "jsCode": "const current = $('Rank Skills And Pay').first().json;\n\n// The sheet read runs with alwaysOutputData, so on the very first run this is a\n// single empty item rather than no items. Treat that as \"no history\" instead of\n// letting an undefined row through as a real previous scan.\nconst history = $input.all()\n  .map((i) => i.json || {})\n  .filter((r) => r && r.skill && r.scanned_at);\n\nconst sameMarket = history.filter(\n  (r) => String(r.role) === String(current.role) && String(r.market) === String(current.market)\n);\n\nconst dates = [...new Set(sameMarket.map((r) => String(r.scanned_at)))]\n  .filter((d) => d !== current.scanned_at)\n  .sort();\nconst previousDate = dates.length ? dates[dates.length - 1] : null;\n\nconst prev = new Map();\nif (previousDate) {\n  for (const r of sameMarket.filter((r) => String(r.scanned_at) === previousDate)) {\n    prev.set(String(r.skill), Number(r.share_pct) || 0);\n  }\n}\n\nconst TOP = 15;\nconst top = current.skills.slice(0, TOP);\n\nconst movement = top.map((s) => {\n  const before = prev.has(s.skill) ? prev.get(s.skill) : null;\n  const delta = before === null ? null : Math.round((s.share_pct - before) * 10) / 10;\n  let status = 'steady';\n  if (before === null) status = previousDate ? 'new' : 'first scan';\n  else if (delta >= 5) status = 'rising';\n  else if (delta <= -5) status = 'falling';\n  return { ...s, previous_pct: before, delta_pct: delta, status };\n});\n\n// Skills that were in the previous top list and are not in this one at all.\nconst nowHas = new Set(current.skills.map((s) => s.skill));\nconst dropped = [...prev.entries()]\n  .filter(([skill]) => !nowHas.has(skill))\n  .sort((a, b) => b[1] - a[1])\n  .slice(0, 5)\n  .map(([skill, share_pct]) => ({ skill, share_pct }));\n\nreturn [{ json: { ...current, previous_scan: previousDate, movement, dropped } }];"
      },
      "typeVersion": 2
    },
    {
      "id": "t5-brief",
      "name": "Write Market Brief",
      "type": "@n8n/n8n-nodes-langchain.chainLlm",
      "onError": "continueRegularOutput",
      "position": [
        4352,
        460
      ],
      "parameters": {
        "text": "=You are a labour market analyst. Write a short brief for someone deciding what this role needs to know and what it should pay.\n\nRole: {{ $json.role }}\nMarket: {{ $json.market }}\nJob postings analysed: {{ $json.total_posts }}\nPrevious scan: {{ $json.previous_scan || 'none, this is the first scan' }}\nMost-demanded skills with the share of posts requiring each, and movement since the last scan: {{ JSON.stringify($json.movement) }}\nSkills that dropped out since the last scan: {{ JSON.stringify($json.dropped) }}\nPay, in annual terms: {{ JSON.stringify($json.pay) }}\nWorking pattern: {{ JSON.stringify($json.remote) }}\nSeniority mix: {{ JSON.stringify($json.seniority) }}\nTypical years of experience asked for: {{ $json.median_years }}\n\nWrite 4 to 6 sentences covering: what this market clearly expects, anything that has moved since the last scan, and what the pay data does and does not support.\n\nBe accurate about the evidence. `disclosed_posts` is how many of the posts stated pay at all; if that number is small, say the pay figures are indicative rather than reliable, and never present a median drawn from a handful of posts as the market rate. If there is no previous scan, say so instead of inventing a trend. Plain sentences, no bullet points, no headings.",
        "batching": {},
        "promptType": "define"
      },
      "typeVersion": 1.9
    },
    {
      "id": "t5-llm2",
      "name": "OpenAI Brief Writer",
      "type": "@n8n/n8n-nodes-langchain.lmChatOpenAi",
      "onError": "continueRegularOutput",
      "position": [
        4360,
        700
      ],
      "parameters": {
        "model": {
          "__rl": true,
          "mode": "list",
          "value": "gpt-5.6-terra"
        },
        "options": {
          "temperature": 0.3
        }
      },
      "typeVersion": 1.3
    },
    {
      "id": "t5-report",
      "name": "Build Report",
      "type": "n8n-nodes-base.code",
      "position": [
        4876,
        460
      ],
      "parameters": {
        "mode": "runOnceForAllItems",
        "jsCode": "const data = $('Compare With Last Scan').first().json;\n// single-item input: the brief chain is fed one aggregated item and answers with\n// one block of prose, so there is no second item to lose here.\nconst brief = String($input.first().json.text || $input.first().json.output || '').trim();\n\nconst money = (n) => (n ? '$' + Math.round(n / 1000) + 'k' : 'n/a');\nconst plural = (n, word) => n + ' ' + word + (n === 1 ? '' : 's');\n\n// One row per skill, so the sheet accumulates a history the next run can diff.\nconst rows = data.movement.map((s, i) => ({\n  scanned_at: data.scanned_at,\n  role: data.role,\n  market: data.market,\n  rank: i + 1,\n  skill: s.skill,\n  posts_requiring: s.posts_requiring,\n  share_pct: s.share_pct,\n  previous_pct: s.previous_pct === null ? '' : s.previous_pct,\n  delta_pct: s.delta_pct === null ? '' : s.delta_pct,\n  status: s.status,\n  posts_analysed: data.total_posts,\n  pay_disclosed_posts: data.pay.disclosed_posts,\n  pay_low: data.pay.low || '',\n  pay_median: data.pay.median_mid || '',\n  pay_high: data.pay.high || '',\n}));\n\nconst lines = [];\nlines.push(`*${data.role} - ${data.market}*`);\nlines.push(`${plural(data.total_posts, 'live posting')} analysed` +\n  (data.previous_scan ? `, compared with ${data.previous_scan}.` : ', first scan so no comparison yet.'));\n\nlines.push('');\nlines.push('*Most demanded skills*');\ndata.movement.slice(0, 8).forEach((s, i) => {\n  const move = s.delta_pct === null || s.delta_pct === 0\n    ? ''\n    : ` (${s.delta_pct > 0 ? '+' : ''}${s.delta_pct} pts)`;\n  const tag = s.status === 'new' ? ' - new' : '';\n  lines.push(`${i + 1}. ${s.skill} - ${s.share_pct}% of postings${move}${tag}`);\n});\n\nif (data.dropped.length) {\n  lines.push('');\n  lines.push('*Dropped out since last scan*');\n  lines.push(data.dropped.map((d) => `${d.skill} (was ${d.share_pct}%)`).join(', '));\n}\n\nlines.push('');\nif (data.pay.disclosed_posts) {\n  lines.push('*Pay*');\n  lines.push(`${money(data.pay.low)} to ${money(data.pay.high)}, midpoint ${money(data.pay.median_mid)} - ` +\n    `from the ${plural(data.pay.disclosed_posts, 'posting')} that disclosed, ${data.pay.disclosure_pct}% of the sample.`);\n} else {\n  lines.push('*Pay*');\n  lines.push('No posting in this sample stated a salary.');\n}\n\nconst remote = data.remote.map((r) => `${r.name} ${r.posts}`).join(', ');\nlines.push('');\nlines.push(`*Working pattern* ${remote}`);\nlines.push(`*Typical experience asked* ${data.median_years || 'not stated'} years`);\n\nif (brief) {\n  lines.push('');\n  lines.push(brief);\n}\n\n// One item carrying both payloads. Split Out turns `rows` into one item per\n// skill for the sheet; the Slack node runs once and reads `message` from here.\n// Emitting the message as its own item instead would append it to the sheet as\n// a row, and post the digest once per skill.\nreturn [{ json: { message: lines.join('\\n'), rows, row_count: rows.length } }];"
      },
      "typeVersion": 2
    },
    {
      "id": "t5-split",
      "name": "Split Out Skill Rows",
      "type": "n8n-nodes-base.splitOut",
      "position": [
        5116,
        460
      ],
      "parameters": {
        "options": {},
        "fieldToSplitOut": "rows"
      },
      "typeVersion": 1
    },
    {
      "id": "t5-sheet",
      "name": "Log Skills To Sheet",
      "type": "n8n-nodes-base.googleSheets",
      "onError": "continueRegularOutput",
      "position": [
        5356,
        460
      ],
      "parameters": {
        "columns": {
          "value": {},
          "mappingMode": "autoMapInputData",
          "matchingColumns": []
        },
        "options": {},
        "operation": "append",
        "sheetName": {
          "__rl": true,
          "mode": "name",
          "value": "Skill demand"
        },
        "documentId": {
          "__rl": true,
          "mode": "url",
          "value": "={{ $('Set Market Config').first().json.sheet_url }}"
        }
      },
      "typeVersion": 4.7
    },
    {
      "id": "t5-slack",
      "name": "Post Digest To Slack",
      "type": "n8n-nodes-base.slack",
      "onError": "continueRegularOutput",
      "position": [
        5596,
        460
      ],
      "parameters": {
        "text": "={{ $('Build Report').first().json.message }}",
        "select": "channel",
        "resource": "message",
        "channelId": {
          "__rl": true,
          "mode": "name",
          "value": "={{ $('Set Market Config').first().json.slack_channel }}"
        },
        "operation": "post",
        "otherOptions": {}
      },
      "executeOnce": true,
      "typeVersion": 2.3
    }
  ],
  "settings": {
    "executionOrder": "v1"
  },
  "connections": {
    "Jobs Ready?": {
      "main": [
        [
          {
            "node": "Download Job Posts",
            "type": "main",
            "index": 0
          }
        ],
        [
          {
            "node": "Wait For Snapshot",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Build Report": {
      "main": [
        [
          {
            "node": "Split Out Skill Rows",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Set Market Config": {
      "main": [
        [
          {
            "node": "Build Discovery Input",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Wait For Snapshot": {
      "main": [
        [
          {
            "node": "Check Snapshot Progress",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Download Job Posts": {
      "main": [
        [
          {
            "node": "Normalise Job Posts",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Read Previous Scan": {
      "main": [
        [
          {
            "node": "Compare With Last Scan",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Weekly Market Scan": {
      "main": [
        [
          {
            "node": "Set Market Config",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Write Market Brief": {
      "main": [
        [
          {
            "node": "Build Report",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Log Skills To Sheet": {
      "main": [
        [
          {
            "node": "Post Digest To Slack",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Normalise Job Posts": {
      "main": [
        [
          {
            "node": "Extract Role Requirements",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "OpenAI Brief Writer": {
      "ai_languageModel": [
        [
          {
            "node": "Write Market Brief",
            "type": "ai_languageModel",
            "index": 0
          }
        ]
      ]
    },
    "Rank Skills And Pay": {
      "main": [
        [
          {
            "node": "Read Previous Scan",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Split Out Skill Rows": {
      "main": [
        [
          {
            "node": "Log Skills To Sheet",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Build Discovery Input": {
      "main": [
        [
          {
            "node": "Discover Jobs With Bright Data",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Compare With Last Scan": {
      "main": [
        [
          {
            "node": "Write Market Brief",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Check Snapshot Progress": {
      "main": [
        [
          {
            "node": "Evaluate Scrape Progress",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Check Discovery Accepted": {
      "main": [
        [
          {
            "node": "Wait For Snapshot",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Evaluate Scrape Progress": {
      "main": [
        [
          {
            "node": "Jobs Ready?",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Extract Role Requirements": {
      "main": [
        [
          {
            "node": "Rank Skills And Pay",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "OpenAI Requirements Reader": {
      "ai_languageModel": [
        [
          {
            "node": "Extract Role Requirements",
            "type": "ai_languageModel",
            "index": 0
          }
        ]
      ]
    },
    "Discover Jobs With Bright Data": {
      "main": [
        [
          {
            "node": "Check Discovery Accepted",
            "type": "main",
            "index": 0
          }
        ]
      ]
    }
  }
}

About this workflow

This workflow runs weekly to discover live LinkedIn job postings via Bright Data, uses OpenAI (GPT-5.6) to extract required skills and any stated pay, compares results with prior runs stored in Google Sheets, and posts a market digest to Slack. Runs on a weekly schedule and sets…

Source: https://n8n.io/workflows/18234/ — original creator credit. Request a take-down →

More AI & RAG workflows → · Browse all categories →

Related workflows

Workflows that share integrations, category, or trigger type with this one. All free to copy and import.