> ## Documentation Index
> Fetch the complete documentation index at: https://vastai-80aa3a82-ltxv2-serverless.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Langflow + Ollama

<script
  type="application/ld+json"
  dangerouslySetInnerHTML={{
__html: JSON.stringify({
"@context": "https://schema.org",
"@type": "HowTo",
"name": "How to Use Langflow with Ollama on Vast.ai",
"description": "A step-by-step guide to setting up and using Langflow's node-based agent builder with Ollama for running open weight language models on Vast.ai's GPU instances.",
"step": [
  {
    "@type": "HowToStep",
    "name": "Setup Account and Find the Template",
    "text": "Setup your Vast account and add credit by reviewing the quickstart guide. Find the Langflow template in the recommended templates page. Before loading it, click the pencil icon to open the template editor."
  },
  {
    "@type": "HowToStep",
    "name": "Configure Template Variables",
    "text": "In the template editor, configure OLLAMA_MODEL (the most important variable to choose which model should be downloaded when instance starts) and LANGFLOW_ARGS (allows you to pass alternative startup arguments to langflow application). When finished, click 'Create & Use' button to save your copy of the template."
  },
  {
    "@type": "HowToStep",
    "name": "Choose a GPU and Rent Instance",
    "text": "The most important consideration when picking an instance is VRAM. For best performance, model weights must fit into GPU VRAM with room for context window. You can use multi-GPU setups as an alternative to single high VRAM instances. Find a suitable instance and click the 'Rent' button to start the loading phase."
  },
  {
    "@type": "HowToStep",
    "name": "Access the Instance",
    "text": "After a short time, click the 'Open' button to access the Instance Portal. This page gives easy access to both Langflow application and Ollama API. Click Langflow's 'Launch Application' button. Note: It will take time for Langflow and Ollama to install and for the Ollama model to download. Monitor loading status in Instance Portal 'Logs' tab."
  },
  {
    "@type": "HowToStep",
    "name": "Create and Configure Workflow in Langflow",
    "text": "After opening Langflow, click 'Create first flow'. Select a workflow template (e.g., Content Generation -> Blog Writer). Replace the default Language Model node with the Ollama component from the left side menu. Configure Ollama node with Base URL 'http://localhost:11434', select your Ollama model, and reconnect the nodes. Customize workflow settings (URL, instructions, etc.) and click 'Run flow' button to execute."
  }
]
})
}}
/>

Langflow is a node-based agent builder you can use from your web browser.  While it integrates with many frontier language models it also has a fantastic Ollama integration which makes it really easy to use with open weight models as well as custom fine-tunes.

We have two templates you can choose for this guide.  The **Langflow template** provides both Ollama and Langflow installed within the instance.  You can also use the [**Ollama standalone template**](https://cloud.vast.ai/?ref_id=62897\&creator_id=62897\&name=Ollama) to integrate with a local langflow installation via [ssh local port forwarding](/documentation/instances/sshscp#Yj5Wh).  The choice is yours. For this guide we will use the Langflow bundled template.

Before moving on with the guide,** Setup your Vast account and add credit**. Review the [quickstart guide](/documentation/get-started/quickstart) to get familar with the service if you do not have an account with credits loaded.

## Initial Setup

Let's get started with the configuration - There is not much you need to change here but it's a good idea to create a customized version of the template so Ollama automatically downloads your preferred model.

### Find the Template

You can find the Langflow template in our [recommended templates](https://cloud.vast.ai/templates/) page.  Before loading it up, click the pencil icon to open up the template editor

<Frame caption="Langflow Template">
  <img src="https://mintcdn.com/vastai-80aa3a82-ltxv2-serverless/vO1ArlvzWra2Xqg3/images/use-cases-ai-agents.webp?fit=max&auto=format&n=vO1ArlvzWra2Xqg3&q=85&s=13c3977d877619d79d209f6f6eb7b46d" alt="Langflow template card" width="800" height="518" data-path="images/use-cases-ai-agents.webp" />
</Frame>

### Custom configuration

In the template editor you'll find two really useful configuration variables.

* `OLLAMA_MODEL` is the most important variable.  Here you can choose which model should be downloaded when the instance starts.
* `LANGFLOW_ARGS`allows you to pass alternative startup arguments to the langflow application.  The defaults should be fine for this demo, but you are free to change these as you need.

<img src="https://mintcdn.com/vastai-80aa3a82-ltxv2-serverless/EVv-IFlz-DsWmGpt/images/use-cases-ai-agents-2.webp?fit=max&auto=format&n=EVv-IFlz-DsWmGpt&q=85&s=b961a47cb48058a993ebe11b3c726b51" alt="" width="942" height="172" data-path="images/use-cases-ai-agents-2.webp" />

When you have finished entering your settings click the '**Create & Use**' button to save your copy of the template.

You'll be taken to the search interface where you can choose an appropriate GPU instance to run your model.  You can access your custom template in future from the 'My Templates' section of the templates page.

## Starting the Instance

It's now time to use your template to start a GPU instance.

### Choose a GPU

The most important consideration when picking an instance to run language models is the VRAM.  For best performance, your model weights must fit into the GPU VRAM with room left over for the context window.&#x20;

You do not have to use a single GPU when running LLMs - Sometimes a multi-GPU setup can be as effective of better than a single high VRAM instance.

### Rent an Instance

When you have found a suitable instance it's time to click the '**Rent**' button.  This will start the loading phase.

<Note>
  If you are not sure which instance to choose - Try one.  There is no minimum rental period and if it is not suitable you are able to destroy that instance and start another, paying only for the time the instance was in the 'running' state
</Note>

## Accessing the Instance

After a short time, your instance will be ready to access.  Simply click the 'Open' button to get started.

<img src="https://mintcdn.com/vastai-80aa3a82-ltxv2-serverless/EVv-IFlz-DsWmGpt/images/use-cases-ai-agents-3.webp?fit=max&auto=format&n=EVv-IFlz-DsWmGpt&q=85&s=68b5556d55fff4e76262d4fed555b81b" alt="" width="917" height="225" data-path="images/use-cases-ai-agents-3.webp" />

You will now find the Instance Portal has opened.&#x20;

<Frame caption="Instance Portal">
  <img src="https://mintcdn.com/vastai-80aa3a82-ltxv2-serverless/EVv-IFlz-DsWmGpt/images/use-cases-ai-agents-4.webp?fit=max&auto=format&n=EVv-IFlz-DsWmGpt&q=85&s=3734291fe722e08af44387b728704ecd" alt="Instance Portal" width="1149" height="726" data-path="images/use-cases-ai-agents-4.webp" />
</Frame>

This page gives you easy access to both the Langflow application and the Ollama API.  Click Langflow's 'Launch Application' button.&#x20;

<Note>
  It will take some time for Langflow and Ollama to be installed and for the Ollama model to download.  You can monitor the loading status in the Instance Portal 'Logs' tab&#x20;
</Note>

## Getting Started with Langflow

After opening Langflow, click the '**Create first flow**' button.

<img src="https://mintcdn.com/vastai-80aa3a82-ltxv2-serverless/vO1ArlvzWra2Xqg3/images/use-cases-ai-agents-5.webp?fit=max&auto=format&n=vO1ArlvzWra2Xqg3&q=85&s=b58b84535dbc14d8ef78d232777ff801" alt="" width="800" height="749" data-path="images/use-cases-ai-agents-5.webp" />

While Langflow is extremely powerful, for this example we will create a simple blow post writer.

Select **Content Generation** -> **Blog Writer**

<img src="https://mintcdn.com/vastai-80aa3a82-ltxv2-serverless/vO1ArlvzWra2Xqg3/images/use-cases-ai-agents-6.webp?fit=max&auto=format&n=vO1ArlvzWra2Xqg3&q=85&s=8839e1590c0e21512c9f18e21d755c06" alt="" width="1242" height="665" data-path="images/use-cases-ai-agents-6.webp" />

Initially, the flow will look like this

<Frame caption="Blog Writer Defaults">
  <img src="https://mintcdn.com/vastai-80aa3a82-ltxv2-serverless/vO1ArlvzWra2Xqg3/images/use-cases-ai-agents-7.webp?fit=max&auto=format&n=vO1ArlvzWra2Xqg3&q=85&s=454c6c8b81f43c62a3facc7f52dfa1ce" alt="blog writer flow" width="1280" height="455" data-path="images/use-cases-ai-agents-7.webp" />
</Frame>

We will need to replace the Language Model with the Ollama alternative to make use of the GPU and avoid having to make API calls to external services.&#x20;

Click on the **Language Model** node and using the three dot icon, choose **Delete.**

<Frame caption="Delete Language Model">
  <img src="https://mintcdn.com/vastai-80aa3a82-ltxv2-serverless/vO1ArlvzWra2Xqg3/images/use-cases-ai-agents-8.webp?fit=max&auto=format&n=vO1ArlvzWra2Xqg3&q=85&s=428f29a53e57ccc4f302aeffc2578822" alt="delete language model" width="800" height="938" data-path="images/use-cases-ai-agents-8.webp" />
</Frame>

Next, from the left side menu, select the **Ollama** component and drag it to the space created by deleting the original language model component.

<Frame caption="Add Ollama Node">
  <img src="https://mintcdn.com/vastai-80aa3a82-ltxv2-serverless/vO1ArlvzWra2Xqg3/images/use-cases-ai-agents-9.webp?fit=max&auto=format&n=vO1ArlvzWra2Xqg3&q=85&s=63e3850c2d67ece2af8eae70c82d2354" alt="Add Ollama node" width="800" height="843" data-path="images/use-cases-ai-agents-9.webp" />
</Frame>

Now that is in place it must be configured to communicate with the Ollama API. Enter `http://localhost:11434` in the Base URL field.  You'll need to then select your Ollama model and re-connect the nodes as shown below.

<Frame caption="Ollama Node Connected">
  <img src="https://mintcdn.com/vastai-80aa3a82-ltxv2-serverless/EVv-IFlz-DsWmGpt/images/use-cases-ai-agents-10.webp?fit=max&auto=format&n=EVv-IFlz-DsWmGpt&q=85&s=dc28b8cec9482939b173ce902e600c1e" alt="Ollama node connected" width="1280" height="466" data-path="images/use-cases-ai-agents-10.webp" />
</Frame>

<Note>
  If the model field does not immediately show your available models, simply toggle the 'Tool Mode Enabled' switch.
</Note>

### Configuring the Workflow

You could run this node immediately, but first let's make some minor modifications.

Change the **URL** in the **URL node** to `https://vast.ai`and set the **Depth** to `2`

Change the **Text** in the **Instructions node** to `Use the references above for style to write a new blog/tutorial about how Vast.ai can empower people who want to leverage affordable GPU resources`

### Run the Workflow

Simply click the **Playground** button followed by the **Run flow** button and wait for the agent to learn about the subject matter and write a blog post.  It'll only take a few seconds.

<Frame caption="Completed Blog Post">
  <img src="https://mintcdn.com/vastai-80aa3a82-ltxv2-serverless/EVv-IFlz-DsWmGpt/images/use-cases-ai-agents-11.webp?fit=max&auto=format&n=EVv-IFlz-DsWmGpt&q=85&s=98b50cbe8c69de549891af8fc5ea0ca8" alt="Completed Blog Post" width="846" height="893" data-path="images/use-cases-ai-agents-11.webp" />
</Frame>

## Advanced Usage

This short guide serves only as an introduction to Langflow, but it is extremely capabale and easy to use with some practice.  We recommend that you check out the excellent [documentation](https://docs.langflow.org/about-langflow) to assist you in creating complex projects.

Remember, any *Language Model* component can be replaced with the *Ollama* component, and any *Agent* component can be configured to use *Ollama* as a custom provider.