AutomationFlowsSocial Media › Scrape Twitter Profiles with Bright Data API and Export to Google Sheets

Scrape Twitter Profiles with Bright Data API and Export to Google Sheets

ByIncrementors @incrementors on n8n.io

A comprehensive n8n automation that scrapes Twitter profile data using Bright Data's Twitter dataset and stores comprehensive tweet analytics, user metrics, and engagement data directly into Google Sheets.

Event trigger★★★★☆ complexity15 nodesForm TriggerHTTP RequestGoogle Sheets
Social Media Trigger: Event Nodes: 15 Complexity: ★★★★☆ Added:

This workflow corresponds to n8n.io template #6405 — we link there as the canonical source.

This workflow follows the Form Trigger → Google Sheets recipe pattern — see all workflows that pair these two integrations.

The workflow JSON

Copy or download the full n8n JSON below. Paste it into a new n8n workflow, add your credentials, activate. Full import guide →

Download .json
{
  "id": "aklI9zFDCmfAo9hA",
  "name": "Twitter Profile Scraper via Bright Data API with Google Sheets Output",
  "tags": [],
  "nodes": [
    {
      "id": "ac0410b2-4a26-4b4b-a4fe-b676754337df",
      "name": "\ud83d\udce5 User Input Trigger",
      "type": "n8n-nodes-base.formTrigger",
      "position": [
        -1312,
        16
      ],
      "parameters": {
        "options": {},
        "formTitle": "Twitter profile url",
        "formFields": {
          "values": [
            {
              "fieldLabel": "Twitter Profile url"
            },
            {
              "fieldType": "date",
              "fieldLabel": "start date"
            },
            {
              "fieldType": "date",
              "fieldLabel": "end date"
            }
          ]
        }
      },
      "typeVersion": 2.2
    },
    {
      "id": "4541f9d3-7c19-44e1-935b-56fce3e52317",
      "name": "Sticky Note",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        -1376,
        -48
      ],
      "parameters": {
        "width": 224,
        "height": 208,
        "content": "\u2192 Starts flow with Twitter URL & date range from form\n\n"
      },
      "typeVersion": 1
    },
    {
      "id": "4c03a5bd-3121-4c13-ade9-33c8eaed1043",
      "name": "\ud83d\ude80 Trigger Twitter Scraping",
      "type": "n8n-nodes-base.httpRequest",
      "position": [
        -1040,
        16
      ],
      "parameters": {
        "url": "https://api.brightdata.com/datasets/v3/trigger",
        "method": "POST",
        "options": {},
        "jsonBody": "{\n  \"input\": [\n    {\n      \"url\": \"{{ $json['Twitter Profile url'] }}\",\n      \"start_date\": \"{{ $json['start date'] }}\",\n      \"end_date\": \"{{ $json['end date'] }}\"\n    }\n  ],\n  \"custom_output_fields\": [\n    \"id\",\n    \"user_posted\",\n    \"name\",\n    \"description\",\n    \"date_posted\",\n    \"photos\",\n    \"url\",\n    \"quoted_post\",\n    \"tagged_users\",\n    \"replies\",\n    \"reposts\",\n    \"likes\",\n    \"views\",\n    \"external_url\",\n    \"hashtags\",\n    \"followers\",\n    \"posts_count\",\n    \"profile_image_link\",\n    \"following\",\n    \"is_verified\",\n    \"quotes\",\n    \"parent_post_details\",\n    \"external_image_urls\",\n    \"videos\",\n    \"external_video_urls\",\n    \"user_id\",\n    \"timestamp\"\n  ]\n}",
        "sendBody": true,
        "sendQuery": true,
        "sendHeaders": true,
        "specifyBody": "json",
        "queryParameters": {
          "parameters": [
            {
              "name": "dataset_id",
              "value": "gd_lwxkxvnf1cynvib9co"
            },
            {
              "name": "include_errors",
              "value": "true"
            },
            {
              "name": "type",
              "value": "discover_new"
            },
            {
              "name": "discover_by",
              "value": "profile_url"
            }
          ]
        },
        "headerParameters": {
          "parameters": [
            {
              "name": "Authorization",
              "value": "Bearer YOUR_TOKEN_HERE"
            }
          ]
        }
      },
      "typeVersion": 4.2,
      "alwaysOutputData": true
    },
    {
      "id": "9fa82381-8c38-4704-8cb0-e8aa4e469d5a",
      "name": "Sticky Note1",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        -1120,
        -48
      ],
      "parameters": {
        "color": 2,
        "height": 208,
        "content": "\u2192 Sends scrape request to BrightData with user input\n\n"
      },
      "typeVersion": 1
    },
    {
      "id": "4f544be2-c578-4de3-8cdf-db3f9e443530",
      "name": "\ud83d\udd04 Monitor Scraping Progress",
      "type": "n8n-nodes-base.httpRequest",
      "position": [
        -784,
        16
      ],
      "parameters": {
        "url": "=https://api.brightdata.com/datasets/v3/progress/{{ $json.snapshot_id }}",
        "options": {},
        "sendHeaders": true,
        "headerParameters": {
          "parameters": [
            {
              "name": "Authorization",
              "value": "Bearer YOUR_TOKEN_HERE"
            }
          ]
        }
      },
      "typeVersion": 4.2,
      "alwaysOutputData": true
    },
    {
      "id": "701e0ed1-3658-48cf-aebf-cefac22a94e3",
      "name": "Sticky Note2",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        -848,
        -48
      ],
      "parameters": {
        "color": 3,
        "width": 224,
        "height": 208,
        "content": "\u2192 Checks if scraping is still running or ready\n\n"
      },
      "typeVersion": 1
    },
    {
      "id": "88cfbb92-c12f-45ba-bdab-c48476d33676",
      "name": "\u23f1\ufe0f Delay Before Recheck",
      "type": "n8n-nodes-base.wait",
      "position": [
        -544,
        16
      ],
      "parameters": {
        "unit": "minutes",
        "amount": 1
      },
      "typeVersion": 1.1
    },
    {
      "id": "c976c82b-6e1d-4f51-8224-e314bfc703c7",
      "name": "Sticky Note3",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        -608,
        -48
      ],
      "parameters": {
        "color": 4,
        "width": 224,
        "height": 208,
        "content": "\u2192 Waits 1 minute before checking status again\n\n"
      },
      "typeVersion": 1
    },
    {
      "id": "d107f520-3621-4d2d-9887-07a860461b95",
      "name": "\u2705 Is Scraping Ready?",
      "type": "n8n-nodes-base.if",
      "position": [
        -304,
        16
      ],
      "parameters": {
        "options": {},
        "conditions": {
          "options": {
            "version": 2,
            "leftValue": "",
            "caseSensitive": true,
            "typeValidation": "strict"
          },
          "combinator": "or",
          "conditions": [
            {
              "id": "35ed620d-b5d5-4e97-bcc5-52b283d85616",
              "operator": {
                "name": "filter.operator.equals",
                "type": "string",
                "operation": "equals"
              },
              "leftValue": "={{ $json.status }}",
              "rightValue": "ready"
            }
          ]
        }
      },
      "typeVersion": 2.2
    },
    {
      "id": "8ee62b09-4073-424d-b051-381134548eb9",
      "name": "Sticky Note4",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        -352,
        -48
      ],
      "parameters": {
        "color": 5,
        "width": 224,
        "height": 208,
        "content": "\u2192 If ready, fetch data; if not, repeat check loop\n\n"
      },
      "typeVersion": 1
    },
    {
      "id": "e53c14eb-e7cb-443a-bfeb-1a6842d839a6",
      "name": "\ud83d\udce6 Fetch Twitter Data",
      "type": "n8n-nodes-base.httpRequest",
      "position": [
        -48,
        0
      ],
      "parameters": {
        "url": "=https://api.brightdata.com/datasets/v3/snapshot/{{ $json.snapshot_id }}",
        "options": {},
        "sendQuery": true,
        "sendHeaders": true,
        "queryParameters": {
          "parameters": [
            {
              "name": "format",
              "value": "json"
            }
          ]
        },
        "headerParameters": {
          "parameters": [
            {
              "name": "Authorization",
              "value": "Bearer YOUR_TOKEN_HERE"
            }
          ]
        }
      },
      "typeVersion": 4.2
    },
    {
      "id": "9b713be8-53f8-42e9-8c70-ae158ffee142",
      "name": "Sticky Note5",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        -112,
        -48
      ],
      "parameters": {
        "color": 6,
        "width": 224,
        "height": 208,
        "content": "\u2192 Downloads scraped tweet data using snapshot ID\n\n"
      },
      "typeVersion": 1
    },
    {
      "id": "1182993d-0cf0-4a65-99d0-7cd65bfe56c2",
      "name": "\ud83d\udcca Store Twitter Data in Google Sheet",
      "type": "n8n-nodes-base.googleSheets",
      "position": [
        176,
        0
      ],
      "parameters": {
        "columns": {
          "value": {
            "id": "={{ $json.id }}",
            "name": "={{ $json.name }}",
            "likes": "={{ $json.likes }}",
            "views": "={{ $json.views }}",
            "photos": "={{ $json.photos }}",
            "quotes": "={{ $json.quotes }}",
            "videos": "={{ $json.videos }}",
            "replies": "={{ $json.replies }}",
            "reposts": "={{ $json.reposts }}",
            "user_id": "={{ $json.user_id }}",
            "hashtags": "={{ $json.hashtags }}",
            "followers": "={{ $json.followers }}",
            "following": "={{ $json.following }}",
            "timestamp": "={{ $json.timestamp }}",
            "date_posted": "={{ $json.date_posted }}",
            "description": "={{ $json.description }}",
            "is_verified": "={{ $json.is_verified }}",
            "posts_count": "={{ $json.posts_count }}",
            "quoted_post": "={{ $json.quoted_post }}",
            "user_posted": "={{ $json.user_posted }}",
            "tagged_users": "={{ $json.tagged_users }}",
            "profile_image_link": "={{ $json.profile_image_link }}",
            "external_image_urls": "={{ $json.external_image_urls }}",
            "external_video_urls": "={{ $json.external_video_urls }}"
          },
          "schema": [
            {
              "id": "id",
              "type": "string",
              "display": true,
              "removed": false,
              "required": false,
              "displayName": "id",
              "defaultMatch": true,
              "canBeUsedToMatch": true
            },
            {
              "id": "user_posted",
              "type": "string",
              "display": true,
              "removed": false,
              "required": false,
              "displayName": "user_posted",
              "defaultMatch": false,
              "canBeUsedToMatch": true
            },
            {
              "id": "name",
              "type": "string",
              "display": true,
              "removed": false,
              "required": false,
              "displayName": "name",
              "defaultMatch": false,
              "canBeUsedToMatch": true
            },
            {
              "id": "description",
              "type": "string",
              "display": true,
              "removed": false,
              "required": false,
              "displayName": "description",
              "defaultMatch": false,
              "canBeUsedToMatch": true
            },
            {
              "id": "date_posted",
              "type": "string",
              "display": true,
              "removed": false,
              "required": false,
              "displayName": "date_posted",
              "defaultMatch": false,
              "canBeUsedToMatch": true
            },
            {
              "id": "photos",
              "type": "string",
              "display": true,
              "removed": false,
              "required": false,
              "displayName": "photos",
              "defaultMatch": false,
              "canBeUsedToMatch": true
            },
            {
              "id": "url",
              "type": "string",
              "display": true,
              "removed": true,
              "required": false,
              "displayName": "url",
              "defaultMatch": false,
              "canBeUsedToMatch": true
            },
            {
              "id": "quoted_post",
              "type": "string",
              "display": true,
              "removed": false,
              "required": false,
              "displayName": "quoted_post",
              "defaultMatch": false,
              "canBeUsedToMatch": true
            },
            {
              "id": "tagged_users",
              "type": "string",
              "display": true,
              "removed": false,
              "required": false,
              "displayName": "tagged_users",
              "defaultMatch": false,
              "canBeUsedToMatch": true
            },
            {
              "id": "replies",
              "type": "string",
              "display": true,
              "removed": false,
              "required": false,
              "displayName": "replies",
              "defaultMatch": false,
              "canBeUsedToMatch": true
            },
            {
              "id": "reposts",
              "type": "string",
              "display": true,
              "removed": false,
              "required": false,
              "displayName": "reposts",
              "defaultMatch": false,
              "canBeUsedToMatch": true
            },
            {
              "id": "likes",
              "type": "string",
              "display": true,
              "removed": false,
              "required": false,
              "displayName": "likes",
              "defaultMatch": false,
              "canBeUsedToMatch": true
            },
            {
              "id": "views",
              "type": "string",
              "display": true,
              "removed": false,
              "required": false,
              "displayName": "views",
              "defaultMatch": false,
              "canBeUsedToMatch": true
            },
            {
              "id": "external_url",
              "type": "string",
              "display": true,
              "removed": true,
              "required": false,
              "displayName": "external_url",
              "defaultMatch": false,
              "canBeUsedToMatch": true
            },
            {
              "id": "hashtags",
              "type": "string",
              "display": true,
              "removed": false,
              "required": false,
              "displayName": "hashtags",
              "defaultMatch": false,
              "canBeUsedToMatch": true
            },
            {
              "id": "followers",
              "type": "string",
              "display": true,
              "removed": false,
              "required": false,
              "displayName": "followers",
              "defaultMatch": false,
              "canBeUsedToMatch": true
            },
            {
              "id": "posts_count",
              "type": "string",
              "display": true,
              "removed": false,
              "required": false,
              "displayName": "posts_count",
              "defaultMatch": false,
              "canBeUsedToMatch": true
            },
            {
              "id": "profile_image_link",
              "type": "string",
              "display": true,
              "removed": false,
              "required": false,
              "displayName": "profile_image_link",
              "defaultMatch": false,
              "canBeUsedToMatch": true
            },
            {
              "id": "following",
              "type": "string",
              "display": true,
              "removed": false,
              "required": false,
              "displayName": "following",
              "defaultMatch": false,
              "canBeUsedToMatch": true
            },
            {
              "id": "is_verified",
              "type": "string",
              "display": true,
              "removed": false,
              "required": false,
              "displayName": "is_verified",
              "defaultMatch": false,
              "canBeUsedToMatch": true
            },
            {
              "id": "quotes",
              "type": "string",
              "display": true,
              "removed": false,
              "required": false,
              "displayName": "quotes",
              "defaultMatch": false,
              "canBeUsedToMatch": true
            },
            {
              "id": "parent_post_details",
              "type": "string",
              "display": true,
              "removed": true,
              "required": false,
              "displayName": "parent_post_details",
              "defaultMatch": false,
              "canBeUsedToMatch": true
            },
            {
              "id": "external_image_urls",
              "type": "string",
              "display": true,
              "removed": false,
              "required": false,
              "displayName": "external_image_urls",
              "defaultMatch": false,
              "canBeUsedToMatch": true
            },
            {
              "id": "videos",
              "type": "string",
              "display": true,
              "removed": false,
              "required": false,
              "displayName": "videos",
              "defaultMatch": false,
              "canBeUsedToMatch": true
            },
            {
              "id": "external_video_urls",
              "type": "string",
              "display": true,
              "removed": false,
              "required": false,
              "displayName": "external_video_urls",
              "defaultMatch": false,
              "canBeUsedToMatch": true
            },
            {
              "id": "user_id",
              "type": "string",
              "display": true,
              "removed": false,
              "required": false,
              "displayName": "user_id",
              "defaultMatch": false,
              "canBeUsedToMatch": true
            },
            {
              "id": "timestamp",
              "type": "string",
              "display": true,
              "removed": false,
              "required": false,
              "displayName": "timestamp",
              "defaultMatch": false,
              "canBeUsedToMatch": true
            }
          ],
          "mappingMode": "defineBelow",
          "matchingColumns": [
            "id"
          ],
          "attemptToConvertTypes": false,
          "convertFieldsToString": false
        },
        "options": {},
        "operation": "append",
        "sheetName": {
          "__rl": true,
          "mode": "list",
          "value": "gid=0",
          "cachedResultUrl": "https://docs.google.com/spreadsheets/d/YOUR_GOOGLE_SHEET_ID/edit#gid=0",
          "cachedResultName": "Sheet1"
        },
        "documentId": {
          "__rl": true,
          "mode": "url",
          "value": "YOUR_GOOGLE_SHEET_ID"
        }
      },
      "credentials": {
        "googleSheetsOAuth2Api": {
          "name": "<your credential>"
        }
      },
      "typeVersion": 4.6
    },
    {
      "id": "eda5f904-6e95-4ffa-b894-e13915991745",
      "name": "Sticky Note6",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        128,
        -64
      ],
      "parameters": {
        "color": 7,
        "width": 256,
        "height": 240,
        "content": "\u2192 Saves post data (likes, replies, user, etc.) into Sheet\n\n"
      },
      "typeVersion": 1
    },
    {
      "id": "1681773c-b304-4a73-a4a2-0406ae9090e5",
      "name": "Sticky Note8",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        -1328,
        -272
      ],
      "parameters": {
        "content": "Create a Google Sheet with the following columns:\nid | user_posted | name | description | date_posted | photos | quoted_post | tagged_users | replies | reposts | likes | views | hashtags | followers | posts_count | profile_image_link | following | is_verified | quotes | external_image_urls | videos | external_video_urls | user_id | timestamp"
      },
      "typeVersion": 1
    }
  ],
  "active": false,
  "settings": {
    "executionOrder": "v1"
  },
  "versionId": "0e65c515-c16f-4470-85ee-1a3a64017e80",
  "connections": {
    "\u2705 Is Scraping Ready?": {
      "main": [
        [
          {
            "node": "\ud83d\udce6 Fetch Twitter Data",
            "type": "main",
            "index": 0
          }
        ],
        [
          {
            "node": "\ud83d\udd04 Monitor Scraping Progress",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "\ud83d\udce5 User Input Trigger": {
      "main": [
        [
          {
            "node": "\ud83d\ude80 Trigger Twitter Scraping",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "\ud83d\udce6 Fetch Twitter Data": {
      "main": [
        [
          {
            "node": "\ud83d\udcca Store Twitter Data in Google Sheet",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "\u23f1\ufe0f Delay Before Recheck": {
      "main": [
        [
          {
            "node": "\u2705 Is Scraping Ready?",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "\ud83d\ude80 Trigger Twitter Scraping": {
      "main": [
        [
          {
            "node": "\ud83d\udd04 Monitor Scraping Progress",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "\ud83d\udd04 Monitor Scraping Progress": {
      "main": [
        [
          {
            "node": "\u23f1\ufe0f Delay Before Recheck",
            "type": "main",
            "index": 0
          }
        ]
      ]
    }
  }
}

Credentials you'll need

Each integration node will prompt for credentials when you import. We strip credential IDs before publishing — you'll add your own.

Pro

For the full experience including quality scoring and batch install features for each workflow upgrade to Pro

About this workflow

A comprehensive n8n automation that scrapes Twitter profile data using Bright Data's Twitter dataset and stores comprehensive tweet analytics, user metrics, and engagement data directly into Google Sheets.

Source: https://n8n.io/workflows/6405/ — original creator credit. Request a take-down →

More Social Media workflows → · Browse all categories →

Related workflows

Workflows that share integrations, category, or trigger type with this one. All free to copy and import.

Social Media

Disclaimer: this workflow only works on self-hosted instances due to the file system usage.

Execute Workflow Trigger, HTTP Request, Form Trigger +3
Social Media

This n8n workflow automates the process of scraping job listings from both LinkedIn and Indeed platforms simultaneously, combining results, and exporting data to Google Sheets for comprehensive job ma

Form Trigger, HTTP Request, Google Sheets
Social Media

💼 LinkedIn Job Finder Automation using Bright Data API & Google Sheets

Form Trigger, HTTP Request, Google Sheets
Social Media

This n8n workflow automatically converts LinkedIn video URLs into downloadable MP4 files using the LinkedIn Video Downloader API, uploads them to Google Drive with public access, and logs both the ori

Form Trigger, HTTP Request, Google Drive +1
Social Media

LinkedIn Hiring Signal Scraper — Jobs & Prospecting Using Bright Data

HTTP Request, Form Trigger, Google Sheets