> ## Documentation Index
> Fetch the complete documentation index at: https://vastai-80aa3a82-ltxv2-serverless.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# CUDA

<script
  type="application/ld+json"
  dangerouslySetInnerHTML={{
__html: JSON.stringify({
"@context": "https://schema.org",
"@type": "HowTo",
"name": "How to Set Up CUDA Programming on Vast.ai",
"description": "A comprehensive guide to setting up and running CUDA applications on Vast.ai's cloud platform, including template selection, environment configuration, and development best practices.",
"step": [
  {
    "@type": "HowToStep",
    "name": "Select the Right Template",
    "text": "Navigate to the Templates tab and search for the NVIDIA CUDA template. Choose this template for a standard CUDA development environment with pre-configured security features, TLS, authentication, and Jupyter notebook integration. Alternatively, create a custom CUDA template if you need specific CUDA or Python versions."
  },
  {
    "@type": "HowToStep",
    "name": "Edit the Template and Select Launch Mode",
    "text": "Edit the template to use Jupyter launch mode for browser-based development behind firewalls, or SSH launch mode for VSCode Remote-SSH, lower latency, local terminal, and advanced features like tmux. Jupyter is ideal for persistent sessions, while SSH is better for IDE integrations and file transfers."
  },
  {
    "@type": "HowToStep",
    "name": "Create Your Instance",
    "text": "Select your desired GPU configuration from the Search tab based on computational needs. For CUDA development, consider: Minimum 16GB RAM, 10GB storage (CUDA Toolkit core ~2GB, development files ~3-4GB, source code ~4GB), 4+ CPU cores for compilation, and 100+ Mbps network for remote development. Rent the GPU of your choice."
  },
  {
    "@type": "HowToStep",
    "name": "Connect to Your Instance",
    "text": "Go to the Instances tab to see your instance being created. If using Jupyter launch mode, click the OPEN button for notebook support. If using SSH launch mode, click Open Terminal Access, copy the SSH connect string (ssh -p PORT root@IP), and execute it in your terminal (Mac/Linux/PowerShell) or use Windows Putty tools."
  },
  {
    "@type": "HowToStep",
    "name": "Set Up Your Development Environment",
    "text": "The base environment includes CUDA toolkit, Python with ML libraries, and development utilities (gcc, make). Install additional dependencies with 'apt-get update && apt-get install -y cuda-samples'. Navigate to your workspace with 'cd ${WORKSPACE}'. Set up CUDA environment variables by adding to ~/.bashrc: 'export PATH=/usr/local/cuda/bin:$PATH' and 'export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH', then run 'source ~/.bashrc'."
  }
]
})
}}
/>

# CUDA Programming on Vast.ai

## Introduction

This guide walks you through setting up and running CUDA applications on Vast.ai's cloud platform. You'll learn how to set up a CUDA development environment, connect to your instance, and develop CUDA applications efficiently using NVIDIA's development tools.

## Prerequisites

* A Vast.ai account
* Basic familiarity with CUDA programming concepts
* Basic knowledge of Linux command line
* [(Optional) Install TLS Certificate for Jupyter](/documentation/instances/jupyter)
* [(Optional) SSH client installed on your local machine and SSH public key added the Keys section at cloud.vast.ai](/documentation/instances/sshscp)
* [(Optional) Vast-cli installed on your local machine for command-line management](/cli/get-started)
* [(Optional) Docker knowledge for customizing development environments](https://docs.docker.com/get-started/)

## Setup

### 1. Selecting the Right Template

Navigate to the [Templates tab](https://cloud.vast.ai/templates/) to view recommended templates.

Search for [NVIDIA CUDA](https://cloud.vast.ai?ref_id=62897\&template_id=61e14a0dd1f97aa0aa6719d20bc9b02e) template if:

* You need a standard CUDA development environment
* You want pre-configured security features (TLS, authentication)
* You require Jupyter notebook integration
* You need additional development tools like Tensorboard

[Make a custom CUDA template](/documentation/templates/creating-templates) if:

* You need a specific CUDA or Python version
* You have special library requirements
* You want to minimize image size for faster instance startup

### 2. Edit the Template and Select Template

You can edit the template to use Jupyter launch mode if:&#x20;

* You're behind a corporate firewall that blocks SSH
* You prefer browser-based development
* You want persistent terminal sessions that survive browser disconnects
* You need quick access without SSH client setup
* You want to combine CUDA development with notebook documentation
* You plan to switch between multiple terminal sessions in the browser

You can edit the template to use SSH launch mode if:&#x20;

* You're using [VSCode Remote-SSH](https://code.visualstudio.com/docs/remote/ssh) or other IDE integrations
* You need lowest possible terminal latency
* You prefer using your local terminal emulator
* You want to use advanced terminal features like tmux
* You're doing extensive command-line development
* You need to transfer files frequently using scp or rsync

### 2. Create Your Instance

Select your desired GPU configuration based on your computational needs from the [Search tab](https://cloud.vast.ai/create/). For CUDA development, consider:&#x20;

* System Requirements:&#x20;
  * RAM: Minimum 16GB for development tools
  * Storage: 10GB is usually sufficient
    * CUDA Toolkit core: \~2GB
    * Development files and builds: \~3-4GB
    * Room for source code and dependencies: \~4GB
  * CPU: 4+ cores recommended for compilation
  * Network: 100+ Mbps for remote development

&#x20;Rent the GPU of your choice.

### 3. Connecting to Your Instance

Go to [Instances tab](https://cloud.vast.ai/instances/) to see your instance being created. There are multiple ways to connect to your instance:

* If Jupyter launch mode is selected in your template:
  * Click the "OPEN" button or "Jupyter" button on your instance card&#x20;
  * Access a full development environment with notebook support
* If you selected SSH launch mode:
  * Click Open Terminal Access button&#x20;
  * Copy Direct ssh connect string contents that looks like this "ssh -p 12345 root\@123.456.789.10 -L 8080:localhost:8080"
  * You take the ssh command and execute in your terminal in your [Mac or Linux-based computer or in Powershell](/documentation/instances/sshscp)
  * You can use [Powershell or Windows Putty tools](/documentation/instances/sshscp) if you have a Windows computer

## Installation

### Setting Up Your Development Environment

1. The base environment includes:
   * CUDA toolkit and development tools
   * Python with common ML libraries
   * Development utilities (gcc, make, etc.)
2. Install additional CUDA dependencies:

```bash theme={null}
apt-get update
apt-get install -y cuda-samples
```

### Configuring Your Workspace

1. Navigate to your workspace:

```bash theme={null}
cd ${WORKSPACE}
```

1. Set up CUDA environment variables:

```bash theme={null}
echo 'export PATH=/usr/local/cuda/bin:$PATH' >> ~/.bashrc
echo 'export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH' >> ~/.bashrc
source ~/.bashrc
```

## Troubleshooting

### Common Issues and Solutions

CUDA not found:

* Check if GPU is detectable: `nvidia-smi`

```linux theme={null}
nvidia-smi

```

If output like "No devices were found" shows up,  report the machine after clicking on the wrench icon and rent a different machine.

## Best Practices

### Development Workflow

* Code Organization
  * Keep source files in `${WORKSPACE}`
  * Use version control for code management
  * Maintain separate directories for builds and source
* Performance Optimization
  * Use proper CUDA stream management
  * Optimize memory transfers
  * Profile code using NVIDIA tools

## Advanced Topics

### Custom Environment Setup

Create a provisioning script for custom environment setup:

```bash theme={null}
#!/bin/bash
. /venv/main/bin/activate
pip install additional-packages
wget custom-tools.tar.gz
```

### Remote Development Setup

Configure VS Code or other IDEs for [remote development](https://code.visualstudio.com/docs/remote/ssh):

* Use SSH port forwarding for secure connections
* Configure development tools to use remote CUDA compiler
* Set up source synchronization using Syncthing

## Conclusion

You now have a fully configured CUDA development environment on Vast.ai. This setup provides the flexibility of cloud GPU resources with the convenience of local development.

## Additional Resources

* [NVIDIA CUDA Documentation](https://docs.nvidia.com/cuda/)
* [Vast.ai Documentation](https://vast.ai/docs/)
* [CUDA Sample Projects](https://github.com/NVIDIA/cuda-samples)