{ "nodes": [ { "parameters": { "content": "```\nstatus retry_count error_type severity is_retryable suggested_action failed_node\n-------------------------------------------------------------------------------------------------------\npending_retry 0 unknown medium FALSE manual_review \npending_retry 0 unknown medium FALSE manual_review \npending_retry 0 unknown medium FALSE manual_review \npending_retry 0 unknown medium FALSE manual_review \npending_retry 0 parse_error medium FALSE check_llm_output \npending_retry 0 parse_error medium FALSE check_llm_output Accountant-concierge-LM\npending_retry 0 parse_error medium FALSE check_llm_output Accountant-concierge-LM\npending_retry 0 parse_error medium FALSE check_llm_output Accountant-concierge-LM\npending_retry 0 parse_error medium FALSE check_llm_output Call 'smart-table-fill'\npending_retry 0 parse_error medium FALSE check_llm_output Call 'smart-table-fill'\n```", "height": 256, "width": 1008 }, "type": "n8n-nodes-base.stickyNote", "typeVersion": 1, "position": [ 224, 64 ], "id": "31d79e15-243e-4416-a1eb-ef083978548f", "name": "Sticky Note" }, { "parameters": { "content": "## Error Handler Workflow\n\n**Purpose:** Catch failed executions from all workflows, classify errors, log to Google Sheets, send Telegram alerts.\n\n**Setup:**\n1. Activate this workflow\n2. In n8n Settings \u2192 Error Workflow \u2192 Select this workflow\n3. Create `FailedItems` sheet tab with headers\n\n**CODE RED Alert:**\nIf the failing workflow is \"8-hour Task Resolver\", sends an immediate CODE RED Telegram alert and then continues into normal logging + alerts (FailedItems, Telegram, email). This catches the watchdog-failure scenario where automatic error recovery is broken.\n\n**Error Types:**\n- `auth_error` (401/403) \u2192 Critical, refresh credentials\n- `rate_limit` (429) \u2192 High, wait and retry\n- `network_error` \u2192 High, retry with backoff\n- `llm_schema_error` \u2192 High, auto-retry after 8h (retryable)\n- `parse_error` \u2192 Medium, check LLM output\n- `validation_error` \u2192 Medium, check input data\n- `unknown` \u2192 Medium, manual review\n\n**Long Message Support:**\nMessages over 4000 chars are split into chunks and sent as multiple Telegram messages with part numbers [1/3], [2/3], etc.\n\n\nuse a code-node like this to test:\n```\n// Test Error Handler - uncomment ONE at a time\n\n// Rate Limit (retryable)\nthrow new Error(\"429 Too Many Requests - rate limit exceeded\"); \n\n// Auth Error (critical)\n// throw new Error(\"401 Unauthorized - invalid credentials\"); \n\n// Network Error (retryable)\n// throw new Error(\"ETIMEDOUT - connection timed out\");\n\n// Parse Error\n// throw new Error(\"Unexpected token in JSON at position 0\"); \n\n// Validation Error\n// throw new Error(\"Required field 'email' is missing\");\n\n// Resource Error (critical)\n// throw new Error(\"Out of memory - cannot allocate buffer\"); \n\n// Unknown Error\n// throw new Error(\"Something weird happened\");\n```", "height": 944, "width": 480 }, "type": "n8n-nodes-base.stickyNote", "typeVersion": 1, "position": [ -1312, 144 ], "id": "97da6e44-5c8f-4f09-b9a8-85dfd7b87899", "name": "Overview" }, { "parameters": {}, "type": "n8n-nodes-base.errorTrigger", "typeVersion": 1, "position": [ -768, 592 ], "id": "ac4bfaba-1429-46fb-a165-c58f6302ab7a", "name": "Error Trigger" }, { "parameters": { "jsCode": "// Prepare & Classify Error\n// 1) Extracts execution context from the error trigger (record fields)\n// 2) Classifies error type/severity/retryability, LLM cost estimate, auto-retry registry\n\nconst execution = $json.execution || {};\nconst workflow = $json.workflow || {};\nconst error = $json.error || {};\n\n// ====== 1) Prepare record ======\n\n// Extract failing node name from multiple possible locations\nconst failedNode = execution.lastNodeExecuted ||\n error.node ||\n error.nodeName ||\n (error.context && error.context.nodeName) ||\n 'unknown';\n\n// Extract error message from multiple possible locations\nlet rawErrorMsg = error.message;\nif (!rawErrorMsg) {\n rawErrorMsg = error.description || error.reason || error.cause ||\n error.details || error.errorMessage;\n}\n// If still no message but error has a name that's descriptive, use it\nif (!rawErrorMsg && error.name && error.name !== 'Error' && error.name !== 'NodeOperationError') {\n rawErrorMsg = error.name;\n}\n// Last resort: stringify, but check if it's actually useful\nif (!rawErrorMsg) {\n const stringified = JSON.stringify(error);\n if (stringified && stringified !== '{}' && stringified !== 'null' && stringified.length > 2) {\n rawErrorMsg = stringified;\n }\n}\n// Final fallback with specific hint\nif (!rawErrorMsg) {\n rawErrorMsg = 'LLM output did not match required schema - check model response format';\n}\n\n// Extract retry_input - data needed to restart workflow from scratch\nlet retryInputData = {};\nconst runData = execution.data?.resultData?.runData || {};\nconst nodesToCheck = ['Set File ID', 'Gmail', 'Gmail Trigger', 'Clean Email', 'Manual Trigger'];\nfor (const nodeName of nodesToCheck) {\n const nodeData = runData[nodeName]?.[0]?.data?.main?.[0]?.[0]?.json;\n if (nodeData) {\n if (nodeData.email_ID) retryInputData.email_ID = nodeData.email_ID;\n if (nodeData.id && !retryInputData.email_ID) retryInputData.email_ID = nodeData.id;\n if (nodeData.messageId) retryInputData.messageId = nodeData.messageId;\n break; // Found data, stop searching\n }\n}\n\n// ====== Extract attachment count for LLM cost estimation ======\nlet attachmentCount = 0;\n\n// Strategy 1: Check Gmail node for binary attachments\nconst gmailNodeData = runData['Gmail']?.[0]?.data?.main?.[0]?.[0];\nif (gmailNodeData?.binary) {\n attachmentCount = Object.keys(gmailNodeData.binary).length;\n}\n// Strategy 2: Check Gmail Trigger node\nif (attachmentCount === 0) {\n const gmailTriggerData = runData['Gmail Trigger']?.[0]?.data?.main?.[0]?.[0];\n if (gmailTriggerData?.binary) {\n attachmentCount = Object.keys(gmailTriggerData.binary).length;\n }\n}\n// Strategy 3: Check \"sp\" (Split Out) node output count\nif (attachmentCount === 0) {\n const spNodeData = runData['sp'];\n if (spNodeData && spNodeData[0]?.data?.main?.[0]) {\n attachmentCount = spNodeData[0].data.main[0].length;\n }\n}\n// Strategy 4: Check Clean Email object for attachmentNames\nif (attachmentCount === 0) {\n const cleanEmailData = runData['Clean Email object']?.[0]?.data?.main?.[0]?.[0]?.json;\n if (cleanEmailData?.data?.[0]?.attachmentNames) {\n attachmentCount = cleanEmailData.data[0].attachmentNames.length;\n }\n}\n// Default to 1 if we couldn't determine (conservative estimate)\nif (attachmentCount === 0) {\n attachmentCount = 1;\n}\n\nconst record = {\n timestamp: new Date().toISOString(),\n workflow_id: workflow.id || 'unknown',\n workflow_name: workflow.name || 'unknown',\n execution_id: execution.retryOf || execution.id || 'unknown',\n execution_mode: execution.mode || 'unknown',\n failed_node: failedNode,\n retry_of: execution.retryOf || '',\n error_message: rawErrorMsg.substring(0, 500),\n error_stack: (error.stack || '').substring(0, 1000),\n error_name: error.name || 'Error',\n started_at: execution.startedAt || '',\n status: 'pending_retry',\n retry_count: 0,\n retry_input: JSON.stringify(retryInputData),\n attachment_count: attachmentCount\n};\n\n// ====== 2) Classify ======\n\nconst errorMsg = record.error_message.toLowerCase();\nconst errorMsgOriginal = record.error_message;\nconst errorName = record.error_name.toLowerCase();\nconst workflowName = record.workflow_name;\n\n// ====== Auto-Retry Registry ======\n// Workflows that support automatic retry on rate limits\nconst AUTO_RETRY_REGISTRY = [\n 'smart-folder2table',\n 'any-file2json-converter',\n // Add more as needed\n];\n\nconst canAutoRetry = AUTO_RETRY_REGISTRY.some(name =>\n workflowName.toLowerCase().includes(name.toLowerCase())\n);\n\nlet errorType = 'unknown';\nlet severity = 'medium';\nlet isRetryable = false;\nlet suggestedAction = 'manual_review';\n\n// Auth errors (401, 403, credential issues)\nif (errorMsg.includes('401') || errorMsg.includes('403') ||\n errorMsg.includes('unauthorized') || errorMsg.includes('forbidden') ||\n errorMsg.includes('credential') || errorMsg.includes('token expired') ||\n errorMsg.includes('invalid_grant') || errorMsg.includes('authentication')) {\n errorType = 'auth_error';\n severity = 'critical';\n suggestedAction = 'refresh_credentials';\n isRetryable = false;\n}\n// Rate limits (429)\nelse if (errorMsg.includes('429') || errorMsg.includes('rate limit') ||\n errorMsg.includes('too many requests') || errorMsg.includes('quota') ||\n errorMsg.includes('throttl')) {\n errorType = 'rate_limit';\n severity = 'high';\n suggestedAction = 'wait_and_retry';\n isRetryable = true;\n}\n// Network/timeout errors\nelse if (errorMsg.includes('timeout') || errorMsg.includes('etimedout') ||\n errorMsg.includes('econnrefused') || errorMsg.includes('econnreset') ||\n errorMsg.includes('network') || errorMsg.includes('enotfound') ||\n errorMsg.includes('socket hang up') || errorMsg.includes('502') ||\n errorMsg.includes('503') || errorMsg.includes('504')) {\n errorType = 'network_error';\n severity = 'high';\n suggestedAction = 'retry_with_backoff';\n isRetryable = true;\n}\n// LLM Schema errors (specific pattern - retryable because LLMs are non-deterministic)\nelse if (errorMsg.includes('llm output did not match required schema')) {\n errorType = 'llm_schema_error';\n severity = 'high';\n suggestedAction = 'auto_retry_8h';\n isRetryable = true;\n}\n// Parse errors (JSON, schema) - general case, NOT retryable\nelse if (errorMsg.includes('json') || errorMsg.includes('parse') ||\n errorMsg.includes('unexpected token') || errorMsg.includes('schema') ||\n errorMsg.includes('syntax error') || errorName.includes('syntaxerror')) {\n errorType = 'parse_error';\n severity = 'medium';\n suggestedAction = 'check_llm_output';\n isRetryable = false;\n}\n// Validation errors\nelse if (errorMsg.includes('required') || errorMsg.includes('missing') ||\n errorMsg.includes('invalid') || errorMsg.includes('validation') ||\n errorMsg.includes('not found') || errorMsg.includes('does not exist')) {\n errorType = 'validation_error';\n severity = 'medium';\n suggestedAction = 'check_input_data';\n isRetryable = false;\n}\n// Resource errors\nelse if (errorMsg.includes('memory') || errorMsg.includes('disk') ||\n errorMsg.includes('storage') || errorMsg.includes('out of space')) {\n errorType = 'resource_error';\n severity = 'critical';\n suggestedAction = 'check_resources';\n isRetryable = false;\n}\n\n// ====== LLM Cost Estimation by Provider ======\n// Rate limits are per API provider, so we track them separately:\n// - Groq: 5 req/min (bottleneck causing 429 errors)\n// - Google Gemini: Separate limit for OCR (not typically the bottleneck)\nconst LLM_COST_BY_NODE = {\n // Early failures - most work remaining\n 'Gmail Trigger': { groq_base: 1, groq_per_att: 1, google_per_att: 1 },\n 'Clean Email object': { groq_base: 1, groq_per_att: 1, google_per_att: 1 },\n\n // Classification stage (Groq)\n 'subject-classifier-LM': { groq_base: 1, groq_per_att: 1, google_per_att: 0 },\n\n // Attachment processing stage\n 'any-file2json-converter': { groq_base: 0, groq_per_att: 1, google_per_att: 1 },\n 'Create Attachment Profile':{ groq_base: 0, groq_per_att: 1, google_per_att: 1 },\n\n // Invoice extraction stage (Groq only)\n 'Accountant-concierge-LM': { groq_base: 0, groq_per_att: 1, google_per_att: 0 },\n\n // smart-table-fill (Groq, batched) - 4 batches typical\n 'Extract Data from String':{ groq_base: 0, groq_per_att: 4, google_per_att: 0 },\n \"Call 'smart-table-fill'\": { groq_base: 0, groq_per_att: 4, google_per_att: 0 },\n 'smart-table-fill': { groq_base: 0, groq_per_att: 4, google_per_att: 0 },\n\n // Default for unknown nodes\n '_default': { groq_base: 1, groq_per_att: 1, google_per_att: 1 }\n};\n\nconst cost = LLM_COST_BY_NODE[record.failed_node] || LLM_COST_BY_NODE['_default'];\nconst expected_groq_requests = cost.groq_base + (attachmentCount * cost.groq_per_att);\nconst expected_google_requests = attachmentCount * cost.google_per_att;\n\n// Package LLM cost estimate as single JSON field\n// Different workflows have different cost structures, so keep as flexible package\nconst llmCostEstimate = {\n attachment_count: attachmentCount,\n groq: expected_groq_requests,\n google: expected_google_requests\n};\n\n// ====== Extract Retry Timing for Rate Limits ======\nlet retryAfterSeconds = 60; // default\nif (errorType === 'rate_limit') {\n // Pattern: \"retry in 55.251598518s\" or \"retry in 55 seconds\"\n const retryMatch = errorMsgOriginal.match(/retry in (\\d+(?:\\.\\d+)?)\\s*s/i);\n if (retryMatch) {\n retryAfterSeconds = Math.ceil(parseFloat(retryMatch[1]));\n }\n}\n\nreturn [{\n json: {\n ...record,\n error_type: errorType,\n severity: severity,\n is_retryable: isRetryable,\n suggested_action: suggestedAction,\n retry_params: JSON.stringify(llmCostEstimate),\n can_auto_retry: canAutoRetry,\n retry_after_seconds: retryAfterSeconds\n }\n}];" }, "type": "n8n-nodes-base.code", "typeVersion": 2, "position": [ -320, 592 ], "id": "fc3a9fa9-042f-4b1c-a01e-6dad08795d7e", "name": "Prepare & Classify Error" }, { "parameters": { "operation": "appendOrUpdate", "documentId": { "__rl": true, "value": "YOUR_FAILEDITEMS_SHEET_ID", "mode": "list", "cachedResultName": "007_Error-handler.n8n-sheet", "cachedResultUrl": "https://docs.google.com/spreadsheets/d/YOUR_FAILEDITEMS_SHEET_ID/edit?usp=drivesdk" }, "sheetName": { "__rl": true, "value": "gid=0", "mode": "list", "cachedResultName": "FailedItems", "cachedResultUrl": "https://docs.google.com/spreadsheets/d/YOUR_FAILEDITEMS_SHEET_ID/edit#gid=0" }, "columns": { "mappingMode": "autoMapInputData", "value": {}, "matchingColumns": [ "execution_id" ], "schema": [ { "id": "timestamp", "displayName": "timestamp", "required": false, "defaultMatch": false, "display": true, "type": "string", "canBeUsedToMatch": true }, { "id": "workflow_id", "displayName": "workflow_id", "required": false, "defaultMatch": false, "display": true, "type": "string", "canBeUsedToMatch": true }, { "id": "workflow_name", "displayName": "workflow_name", "required": false, "defaultMatch": false, "display": true, "type": "string", "canBeUsedToMatch": true }, { "id": "execution_id", "displayName": "execution_id", "required": false, "defaultMatch": false, "display": true, "type": "string", "canBeUsedToMatch": true, "removed": false }, { "id": "execution_mode", "displayName": "execution_mode", "required": false, "defaultMatch": false, "display": true, "type": "string", "canBeUsedToMatch": true }, { "id": "failed_node", "displayName": "failed_node", "required": false, "defaultMatch": false, "display": true, "type": "string", "canBeUsedToMatch": true }, { "id": "retry_of", "displayName": "retry_of", "required": false, "defaultMatch": false, "display": true, "type": "string", "canBeUsedToMatch": true }, { "id": "error_message", "displayName": "error_message", "required": false, "defaultMatch": false, "display": true, "type": "string", "canBeUsedToMatch": true }, { "id": "error_stack", "displayName": "error_stack", "required": false, "defaultMatch": false, "display": true, "type": "string", "canBeUsedToMatch": true }, { "id": "error_name", "displayName": "error_name", "required": false, "defaultMatch": false, "display": true, "type": "string", "canBeUsedToMatch": true }, { "id": "started_at", "displayName": "started_at", "required": false, "defaultMatch": false, "display": true, "type": "string", "canBeUsedToMatch": true }, { "id": "status", "displayName": "status", "required": false, "defaultMatch": false, "display": true, "type": "string", "canBeUsedToMatch": true }, { "id": "retry_count", "displayName": "retry_count", "required": false, "defaultMatch": false, "display": true, "type": "string", "canBeUsedToMatch": true }, { "id": "error_type", "displayName": "error_type", "required": false, "defaultMatch": false, "display": true, "type": "string", "canBeUsedToMatch": true }, { "id": "severity", "displayName": "severity", "required": false, "defaultMatch": false, "display": true, "type": "string", "canBeUsedToMatch": true }, { "id": "is_retryable", "displayName": "is_retryable", "required": false, "defaultMatch": false, "display": true, "type": "string", "canBeUsedToMatch": true }, { "id": "suggested_action", "displayName": "suggested_action", "required": false, "defaultMatch": false, "display": true, "type": "string", "canBeUsedToMatch": true }, { "id": "retry_input", "displayName": "retry_input", "required": false, "defaultMatch": false, "display": true, "type": "string", "canBeUsedToMatch": false } ], "attemptToConvertTypes": false, "convertFieldsToString": false }, "options": {} }, "type": "n8n-nodes-base.googleSheets", "typeVersion": 4, "position": [ 352, 592 ], "id": "44879a1e-9691-43e0-978f-2ed0026a8572", "name": "Append to FailedItems", "credentials": { "googleSheetsOAuth2Api": { "name": "" } } }, { "parameters": { "jsCode": "// Format Telegram Alert with Severity Emoji\n// Outputs FULL message (no truncation) - chunking handled downstream\n// Includes LLM cost estimates for rate-limit awareness\n\nconst esc = (s) => String(s).replace(/_/g, '\\\\_');\n\nconst severityEmoji = {\n critical: '\\uD83D\\uDD34', // Red circle\n high: '\\uD83D\\uDFE0', // Orange circle\n medium: '\\uD83D\\uDFE1', // Yellow circle\n low: '\\uD83D\\uDFE2' // Green circle\n};\n\nconst emoji = severityEmoji[$json.severity] || '\\u26AA'; // White circle default\n\n// Parse LLM cost estimate JSON\nconst llmCost = JSON.parse($json.retry_params || '{}');\n\n// Build full message with failed node prominently displayed\n// Added attachment count and LLM estimates for rate limit visibility\nconst msg = `${emoji} ${esc($json.error_type).toUpperCase()}\n\\u274C Workflow: ${esc($json.workflow_name)}\n\\uD83D\\uDCCD Node: ${esc($json.failed_node || 'unknown')}\n---\nError: ${esc($json.error_message)}\n---\nExecution: ${$json.execution_id}\nSeverity: ${$json.severity}\nRetryable: ${$json.is_retryable ? 'Yes' : 'No'}\nAction: ${esc($json.suggested_action)}\nAttachments: ${llmCost.attachment_count || 0}\nLLM calls: Groq=${llmCost.groq || 0} | Google=${llmCost.google || 0}\nStack: ${esc($json.error_stack || 'N/A')}`;\n\nreturn [{\n json: {\n fullMessage: msg\n }\n}];" }, "type": "n8n-nodes-base.code", "typeVersion": 2, "position": [ 576, 592 ], "id": "e7f5d957-8e4d-49d7-a9a0-d00fc403e7f0", "name": "Format Telegram Alert" }, { "parameters": { "jsCode": "// Split Long Message into Chunks\n// Telegram max message length is ~4096 chars, using 4000 to be safe\n\nconst message = $json.fullMessage || 'No error message';\nconst maxLength = 4000;\nconst chunks = [];\n\nfor (let i = 0; i < message.length; i += maxLength) {\n chunks.push(message.substring(i, i + maxLength));\n}\n\nreturn chunks.map((chunk, idx) => ({\n json: {\n chunk: chunk,\n part: idx + 1,\n total: chunks.length\n }\n}));" }, "type": "n8n-nodes-base.code", "typeVersion": 2, "position": [ 800, 592 ], "id": "fbc7df72-a695-42f1-b1a5-29f9055b93fd", "name": "Split Long Message" }, { "parameters": { "options": {} }, "type": "n8n-nodes-base.splitInBatches", "typeVersion": 3, "position": [ 1024, 592 ], "id": "ff1ddec4-fe63-4059-ba4f-b8fb2b711ef0", "name": "Loop Over Chunks" }, { "parameters": { "chatId": "YOUR_CHAT_ID_1", "text": "={{ $json.total > 1 ? '[' + $json.part + '/' + $json.total + '] ' : '' }}{{ $json.chunk }}", "additionalFields": { "appendAttribution": false } }, "type": "n8n-nodes-base.telegram", "typeVersion": 1.2, "position": [ 1248, 592 ], "id": "cf9f8582-c986-49e3-9ddc-d35bbb196100", "name": "Send Telegram Alert", "credentials": { "telegramApi": { "name": "" } } }, { "parameters": { "conditions": { "options": { "caseSensitive": true, "leftValue": "", "typeValidation": "strict", "version": 2 }, "conditions": [ { "id": "resolver-failure-check", "leftValue": "={{ $json.workflow_name }}", "rightValue": "8-hour Task Resolver", "operator": { "type": "string", "operation": "equals" } } ], "combinator": "and" }, "options": {} }, "type": "n8n-nodes-base.if", "typeVersion": 2.2, "position": [ 128, 592 ], "id": "300a706d-0ff2-4945-ae32-b1b0efcbd59e", "name": "Is Resolver Failure?" }, { "parameters": { "chatId": "YOUR_CHAT_ID_1", "text": "=\ud83d\udea8\ud83d\udea8\ud83d\udea8 CODE RED \ud83d\udea8\ud83d\udea8\ud83d\udea8\n\n\u26a0\ufe0f ERROR HANDLER DOWN \u26a0\ufe0f\n\nThe 8-hour Task Resolver has FAILED!\nThis means automatic error recovery is broken.\n\nFailed Node: {{ $json.failed_node }}\nError: {{ $json.error_message }}\nExecution: {{ $json.execution_id }}\n\n\ud83d\udd25 IMMEDIATE ATTENTION REQUIRED \ud83d\udd25", "additionalFields": { "appendAttribution": false } }, "type": "n8n-nodes-base.telegram", "typeVersion": 1.2, "position": [ 128, 784 ], "id": "ac4c8411-3c5f-4d31-bd8e-6d108fe297f3", "name": "CODE RED Alert", "credentials": { "telegramApi": { "name": "" } } }, { "parameters": { "conditions": { "options": { "caseSensitive": true, "leftValue": "", "typeValidation": "strict", "version": 2 }, "conditions": [ { "id": "rate-limit-check", "leftValue": "={{ $json.error_type }}", "rightValue": "rate_limit", "operator": { "type": "string", "operation": "equals" } }, { "id": "auto-retry-check", "leftValue": "={{ $json.can_auto_retry }}", "rightValue": true, "operator": { "type": "boolean", "operation": "true", "singleValue": true } } ], "combinator": "and" }, "options": {} }, "type": "n8n-nodes-base.if", "typeVersion": 2.2, "position": [ -96, 592 ], "id": "eh-if-rate-limit-auto-retry", "name": "IF: Rate Limit Auto-Retry?" }, { "parameters": { "jsCode": "// Extract Config for Execute Workflow retry\n// Gets original Config values from failed execution's runData\nconst execution = $('Error Trigger').first().json.execution || {};\nconst runData = execution.data?.resultData?.runData || {};\n\n// Extract Config node output from smart-folder2table\nconst configData = runData['Config']?.[0]?.data?.main?.[0]?.[0]?.json || {};\n\nreturn [{\n json: {\n // Pass original config values\n folder_id: configData.folder_id || '',\n spreadsheet_id: configData.spreadsheet_id || '',\n data_sheet_name: configData.data_sheet_name || 'Sheet1',\n source_file_column: configData.source_file_column || 'source_file',\n match_column: configData.match_column || 'source_file',\n batch_size: configData.batch_size || 10,\n schema_sheet_name: configData.schema_sheet_name || 'Description_hig7f6',\n // Add new rate limit param from error classification\n rate_limit_wait_seconds: $('Prepare & Classify Error').first().json.retry_after_seconds || 60\n }\n}];" }, "type": "n8n-nodes-base.code", "typeVersion": 2, "position": [ -96, 976 ], "id": "eh-extract-config-for-retry", "name": "Extract Config for Retry" }, { "parameters": { "amount": "={{ $('Prepare & Classify Error').first().json.retry_after_seconds }}", "unit": "seconds" }, "type": "n8n-nodes-base.wait", "typeVersion": 1.1, "position": [ 128, 976 ], "id": "eh-wait-rate-limit", "name": "Wait (Rate Limit)" }, { "parameters": { "workflowId": { "__rl": true, "value": "={{ $('Prepare & Classify Error').first().json.workflow_id }}", "mode": "id" }, "workflowInputs": { "mappingMode": "defineBelow", "value": { "folder_id": "={{ $json.folder_id }}", "spreadsheet_id": "={{ $json.spreadsheet_id }}", "data_sheet_name": "={{ $json.data_sheet_name }}", "source_file_column": "={{ $json.source_file_column }}", "match_column": "={{ $json.match_column }}", "batch_size": "={{ $json.batch_size }}", "schema_sheet_name": "={{ $json.schema_sheet_name }}", "rate_limit_wait_seconds": "={{ $json.rate_limit_wait_seconds }}" }, "matchingColumns": [], "schema": [], "attemptToConvertTypes": false, "convertFieldsToString": false }, "options": { "waitForSubWorkflow": true } }, "type": "n8n-nodes-base.executeWorkflow", "typeVersion": 1.2, "position": [ 352, 976 ], "id": "eh-execute-smart-folder2table", "name": "Retry Workflow (Execute)" }, { "parameters": { "chatId": "YOUR_CHAT_ID_1", "text": "=\u23f3 Rate limit hit in {{ $('Prepare & Classify Error').first().json.workflow_name }}\n\nRestarting via Execute Workflow after {{ $('Prepare & Classify Error').first().json.retry_after_seconds }}s delay...\n\nFailed node: {{ $('Prepare & Classify Error').first().json.failed_node }}\nrate_limit_wait_seconds={{ $('Prepare & Classify Error').first().json.retry_after_seconds }}\n\nResumability: Already-processed files will be skipped.", "additionalFields": { "appendAttribution": false } }, "type": "n8n-nodes-base.telegram", "typeVersion": 1.2, "position": [ 576, 976 ], "id": "eh-auto-retry-alert", "name": "Send Auto-Retry Alert", "credentials": { "telegramApi": { "name": "" } } }, { "parameters": { "content": "## Auto-Retry for Rate Limits\n\n**Registry-based:** only workflows in `AUTO_RETRY_REGISTRY` (in the `Prepare & Classify Error` code node) get immediate retry; others go through the normal flow \u2192 8-hour resolver. Current registry: smart-folder2table, any-file2json-converter.\n\n**How:** extract original Config values from the failed execution's runData \u2192 wait \"retry in Xs\" from the error message (default 60s) \u2192 re-run the SAME workflow (dynamic workflow_id) with the original config + rate_limit_wait_seconds.\n\n**Resumability:** restarted workflows check the sheet and skip already-processed files.", "height": 300, "width": 940 }, "type": "n8n-nodes-base.stickyNote", "typeVersion": 1, "position": [ -160, 912 ], "id": "eh-sticky-auto-retry", "name": "Sticky Note - Auto-Retry" }, { "parameters": { "content": "### If stops here\n\nSet document \u2192 your FailedItems spreadsheet,\nsheet \u2192 `FailedItems`,\nmatch column \u2192 `execution_id`", "height": 140, "width": 220, "color": 3 }, "type": "n8n-nodes-base.stickyNote", "typeVersion": 1, "position": [ 352, 432 ], "id": "eh-sticky-faileditems-setup", "name": "Sticky Note - FailedItems Setup" }, { "parameters": { "conditions": { "options": { "caseSensitive": false, "leftValue": "", "typeValidation": "loose", "version": 2 }, "conditions": [ { "id": "runner-down-check", "leftValue": "={{ ($json.execution?.error?.message || '').toLowerCase() }}", "rightValue": "task request timed out|task runner|no runner|not ready|at capacity|task broker", "operator": { "type": "string", "operation": "regex" } } ], "combinator": "and" }, "options": {} }, "id": "b1f00d00-0000-4000-8000-000000000001", "name": "Runner/Infra Down?", "type": "n8n-nodes-base.if", "typeVersion": 2.2, "position": [ -544, 592 ] }, { "parameters": { "chatId": "YOUR_CHAT_ID_1", "text": "=\ud83d\udea8 n8n ALERT \u2014 infra / task-runner failure\n\nA workflow failed with a Code-node / task-runner timeout, so the normal error pipeline (which starts with Code nodes) cannot run and would NOT alert you.\n\nWorkflow: {{ $json.workflow?.name || $json.workflow?.id }}\nFailed node: {{ $json.execution?.error?.node?.name }}\nError: {{ $json.execution?.error?.message }}\nExecution: {{ $json.execution?.id }}\n\nLikely cause: external task runner is down or version-drifted vs primary.\nFix: Railway \u2192 redeploy n8n-runner + primary.", "additionalFields": { "appendAttribution": false } }, "id": "b1f00d00-0000-4000-8000-000000000002", "name": "\ud83d\udea8 Runner-Proof Alert", "type": "n8n-nodes-base.telegram", "typeVersion": 1.2, "position": [ -320, 400 ], "credentials": { "telegramApi": { "name": "" } } }, { "parameters": { "content": "## \ud83d\udee1\ufe0f Runner-proof alert branch\n\nThe normal pipeline starts with **Code nodes** (Prepare & Classify Error). When the **task runner is down**, those Code nodes time out (~60s) and this workflow dies BEFORE reaching any Telegram node \u2014 so you get total silence.\n\nThis IF checks the raw error message for runner/timeout signatures. If matched, it sends a Telegram alert built with **expressions only** (no Code node \u2192 runs in n8n's main process \u2192 survives a runner outage).\n\nFALSE \u2192 normal rich pipeline (unchanged).", "height": 320, "width": 560, "color": 4 }, "id": "b1f00d00-0000-4000-8000-000000000003", "name": "Sticky Note - Runner-Proof Branch", "type": "n8n-nodes-base.stickyNote", "typeVersion": 1, "position": [ -384, 240 ] }, { "parameters": { "operation": "send", "sendTo": "YOUR_ALERT_EMAIL", "subject": "=\ud83d\udea8 n8n infra/runner failure \u2014 {{ $json.workflow?.name || $json.workflow?.id }}", "emailType": "text", "message": "=\ud83d\udea8 n8n ALERT \u2014 infra / task-runner failure\n\nA workflow failed with a Code-node / task-runner timeout, so the normal error pipeline (Code nodes) cannot run and would NOT alert you.\n\nWorkflow: {{ $json.workflow?.name || $json.workflow?.id }}\nFailed node: {{ $json.execution?.error?.node?.name }}\nError: {{ $json.execution?.error?.message }}\nExecution: {{ $json.execution?.id }}\n\nLikely cause: external task runner down or version-drifted vs primary.\nFix: Railway \u2192 redeploy n8n-runner + primary.", "options": {} }, "id": "b1f00d00-0000-4000-8000-000000000004", "name": "\ud83d\udce7 Runner-Proof Email", "type": "n8n-nodes-base.gmail", "typeVersion": 2.1, "position": [ -96, 400 ], "credentials": { "gmailOAuth2": { "name": "" } } }, { "parameters": { "operation": "send", "sendTo": "YOUR_ALERT_EMAIL", "subject": "=\u26a0\ufe0f n8n workflow failure alert", "emailType": "text", "message": "={{ $json.fullMessage }}", "options": {} }, "id": "b1f00d00-0000-4000-8000-000000000005", "name": "\ud83d\udce7 Email Alert", "type": "n8n-nodes-base.gmail", "typeVersion": 2.1, "position": [ 576, 784 ], "credentials": { "gmailOAuth2": { "name": "" } } } ], "connections": { "Error Trigger": { "main": [ [ { "node": "Runner/Infra Down?", "type": "main", "index": 0 } ] ] }, "IF: Rate Limit Auto-Retry?": { "main": [ [ { "node": "Extract Config for Retry", "type": "main", "index": 0 } ], [ { "node": "Is Resolver Failure?", "type": "main", "index": 0 } ] ] }, "Extract Config for Retry": { "main": [ [ { "node": "Wait (Rate Limit)", "type": "main", "index": 0 } ] ] }, "Wait (Rate Limit)": { "main": [ [ { "node": "Retry Workflow (Execute)", "type": "main", "index": 0 } ] ] }, "Retry Workflow (Execute)": { "main": [ [ { "node": "Send Auto-Retry Alert", "type": "main", "index": 0 } ] ] }, "Append to FailedItems": { "main": [ [ { "node": "Format Telegram Alert", "type": "main", "index": 0 } ] ] }, "Format Telegram Alert": { "main": [ [ { "node": "Split Long Message", "type": "main", "index": 0 }, { "node": "\ud83d\udce7 Email Alert", "type": "main", "index": 0 } ] ] }, "Split Long Message": { "main": [ [ { "node": "Loop Over Chunks", "type": "main", "index": 0 } ] ] }, "Loop Over Chunks": { "main": [ [], [ { "node": "Send Telegram Alert", "type": "main", "index": 0 } ] ] }, "Send Telegram Alert": { "main": [ [ { "node": "Loop Over Chunks", "type": "main", "index": 0 } ] ] }, "Is Resolver Failure?": { "main": [ [ { "node": "CODE RED Alert", "type": "main", "index": 0 }, { "node": "Append to FailedItems", "type": "main", "index": 0 } ], [ { "node": "Append to FailedItems", "type": "main", "index": 0 } ] ] }, "Runner/Infra Down?": { "main": [ [ { "node": "\ud83d\udea8 Runner-Proof Alert", "type": "main", "index": 0 }, { "node": "\ud83d\udce7 Runner-Proof Email", "type": "main", "index": 0 } ], [ { "node": "Prepare & Classify Error", "type": "main", "index": 0 } ] ] }, "Prepare & Classify Error": { "main": [ [ { "node": "IF: Rate Limit Auto-Retry?", "type": "main", "index": 0 } ] ] } }, "meta": { "templateCredsSetupCompleted": true } }