Chat with Open-Source LLM via Hugging Face

The workflow JSON

Copy or download the full n8n JSON below. Paste it into a new n8n workflow, add your credentials, activate. Full import guide →

{
  "meta": {
    "templateCredsSetupCompleted": true
  },
  "nodes": [
    {
      "id": "27e5f0c0-ba88-4c28-b3be-99c973be15cb",
      "name": "Sticky Note",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        -480,
        -140
      ],
      "parameters": {
        "width": 1083,
        "height": 357,
        "content": "## This is an example of basic LLM Chain connected to an open-source model\n### The Chain is connected to the Mistral-7B-Instruct-v0.1 model, but you can change this\n\nPlease note the initial prompt that guides the model:\n```\nYou are a helpful assistant.\nPlease reply politely to the users.\nUse emojis and a text.\nQ: {{ $json.input }}\nA: \n```\n\nThis way the model \"knows\" that it needs to answer the question right after the `A: `.\n\nSince Hugging Face node is this is an inference mode, it does not support LangChain Agents at the moment. Please use [Ollama Chat Model](https://docs.n8n.io/integrations/builtin/cluster-nodes/sub-nodes/n8n-nodes-langchain.lmchatollama/) node for that"
      },
      "typeVersion": 1
    },
    {
      "id": "4756d5a8-7027-4942-b214-a5ff8310869a",
      "name": "When chat message received",
      "type": "@n8n/n8n-nodes-langchain.chatTrigger",
      "position": [
        -200,
        280
      ],
      "parameters": {
        "options": {}
      },
      "typeVersion": 1.1
    },
    {
      "id": "20a36351-8579-4ac6-9746-526b072aeaa6",
      "name": "Basic LLM Chain",
      "type": "@n8n/n8n-nodes-langchain.chainLlm",
      "position": [
        20,
        280
      ],
      "parameters": {
        "messages": {
          "messageValues": [
            {
              "message": "=You are a helpful assistant. Please reply politely to the users. Use emojis and a text."
            }
          ]
        }
      },
      "typeVersion": 1.5
    },
    {
      "id": "9b88e307-3ad5-4167-8c5f-e5827f7444ac",
      "name": "Hugging Face Inference Model",
      "type": "@n8n/n8n-nodes-langchain.lmOpenHuggingFaceInference",
      "position": [
        120,
        440
      ],
      "parameters": {
        "model": "mistralai/Mistral-7B-Instruct-v0.1",
        "options": {
          "maxTokens": 512,
          "temperature": 0.8,
          "frequencyPenalty": 2
        }
      },
      "credentials": {
        "huggingFaceApi": {
          "name": "<your credential>"
        }
      },
      "typeVersion": 1
    }
  ],
  "connections": {
    "When chat message received": {
      "main": [
        [
          {
            "node": "Basic LLM Chain",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Hugging Face Inference Model": {
      "ai_languageModel": [
        [
          {
            "node": "Basic LLM Chain",
            "type": "ai_languageModel",
            "index": 0
          }
        ]
      ]
    }
  }
}

Credentials you'll need

Each integration node will prompt for credentials when you import. We strip credential IDs before publishing — you'll add your own.

• huggingFaceApi

About this workflow

This workflow demonstrates how to connect an open-source model to a Basic LLM node.

Source: https://n8n.io/workflows/1980/ — original creator credit. Request a take-down →

More AI & RAG workflows → · Browse all categories →

Related workflows

Workflows that share integrations, category, or trigger type with this one. All free to copy and import.