Running a Local LLM with Ollama and Claude Code in an LXD Sandbox

Posted By: JoshPosted On:

Initial LXD Setup

# Install LXD
sudo pacman -S lxd

# Enable LXD socket
sudo systemctl enable lxd.socket
sudo systemctl start lxd.socket

# Enable unprivileged container execution
sudo usermod -v 1000000-1000999999 -w 1000000-1000999999 root

# Add your user to the lxd group if desired
sudo usermod -aG lxd $(whoami)

# Perform first-time LXD initialization
# (I used the defaults for everything but later ran out of space with the
# 30GiB default storage size so had to resize it to 100GiB. Suggest
# starting with 100GiB immediately - even more if you want to experiment
# with many different models)
sudo lxd init

# Create and start container (I named mine "ai")
lxc launch ubuntu:24.04 ai

lxc exec ai -- /bin/bash

# within the container do:
useradd -m josh -s /bin/bash -G sudo,video,render
passwd josh

# back outside the container, I made a permanent alias called "ai" to run
# bash as user "josh" in my ai container:
alias ai='lxc exec ai -- su - josh'

AMD GPU Passthrough Setup

This section is AMD-specific.

sudo pacman -S base-devel
git clone https://aur.archlinux.org/amd-container-toolkit.git
cd amd-container-toolkit
makepkg -si
sudo amd-ctk cdi generate --output=/etc/cdi/amd.json

# First I tried:
# lxc config device add ai gpu0 gpu gputype=physical id=amd.com/gpu=all
# But that was giving this error:
# Error: Invalid devices: Device validation failed for "gpu0": Invalid CDI vendor ("amd.com")
# Even though the /etc/cdi/amd.json file looked good and had ok permissions.
# I don't know why that wasn't working, but I did this instead and it worked:

lxc config device add ai gpu0 gpu

# Note that the 44 here is the ID of the "video" group in /etc/group
# within the container.
lxc config device add ai kfd unix-char path=/dev/kfd gid=44 mode=0660

Install ROCm

Mostly following https://rocm.docs.amd.com/projects/install-on-linux/en/latest/install/quick-start.html, within the ai container as user “josh” do:

wget https://repo.radeon.com/amdgpu-install/7.2/ubuntu/noble/amdgpu-install_7.2.70200-1_all.deb
sudo apt install ./amdgpu-install_7.2.70200-1_all.deb
sudo apt update
sudo apt install python3-setuptools python3-wheel
sudo usermod -a -G render,video $LOGNAME # Add the current user to the render and video groups
sudo apt install rocm

# Run rocminfo in container to verify it lists info and not an error
rocminfo

I also followed the “AMDGPU driver installation” section of instructions from that page but I’m not sure that was actually necessary in the container. I also installed rocm-core and rocminfo in my host Arch system but not sure those were necessary either.

Share Local Project Folder With Container

Distilling instructions from “Add a shared host directory to an LXC/LXD container (read-write mode)” section of https://www.cyberciti.biz/faq/how-to-add-or-mount-directory-in-lxd-linux-container/

From host as user:

# Allow root (running lxd daemon) to remap UID 1000 (local user ID)
echo "root:1000:1" | sudo tee -a /etc/subuid /etc/subgid

# Tell lxc to remap user IDs
# Host user ID 1000 is "josh"
# Container user ID 1001 is "josh"
lxc config set ai raw.idmap "both 1000 1001"

lxc restart ai

lxc config device add ai prog disk source=/home/josh/prog path=/home/josh/prog

Install Ollama and Claude Code

Now for the easy part!

In ai container as josh:

curl -fsSL https://ollama.com/install.sh | sh
curl -fsSL https://claude.ai/install.sh | bash
ollama pull qwen3-coder # Or whatever model you want!
ollama launch claude
Comments are closed.