Distro Hopping Diaries: The Workstation

Part three of three. Part one was the why, part two was the install. This one is the rig becoming a workstation.

The machine has been running 26.04 for a few days now. @ and @home on btrfs, Timeshift snapshotting daily, dual 4K monitors at the right scaling, both GPUs visible to nvidia-smi. That’s a working desktop. It is not yet a working AI/ML workstation.

Let’s make it so.

One scope note up front. This post is the foundation: driver, container runtime, GPU passthrough, CUDA, PyTorch training, and local LLM inference. Not AI dev tooling like Cursor — that’s its own question, with its own answer, in a different post. The thing I want from this post is a machine that can compile a CUDA kernel, train a neural network, and serve a current-generation multimodal language model at a respectable token rate.

The Plan

The mental model is a triad. Three layers, each doing one job, each verifiable on its own.

Infrastructure is the bottom layer: NVIDIA driver on the host, Docker, NVIDIA Container Toolkit, NGC as the source of pre-built CUDA containers. The driver is the only CUDA-flavored thing on the host — nvcc, cuDNN, NCCL, PyTorch all live in containers. Each project pulls the CUDA version it needs through its container; the host stays clean. This is how I’d run things on a server, and I want the same layout on a personal machine.

Training is the middle layer: PyTorch in an NGC container, model weights and datasets on the dev drive, GPU passthrough via --gpus 'device=1' pinning to the 5070 Ti. The MNIST CNN is the canonical first deep-learning workload — simple enough that anyone recognizes it, complete enough to verify the whole training loop (forward pass, loss computation, backward pass, optimizer step, evaluation).

Inference is the top layer: Ollama on the host serving local models, also pinned to the 5070 Ti. Different runtime, different optimization profile, different scope — but the same GPU underneath. If both training and inference work on the same card without fighting each other, the rig is real.

Three demonstrations will close out the post:

  1. CUDA itself — vectorAdd.cu compiled and run inside an nvidia/cuda container. Proves the infrastructure layer.
  2. Deep learning — MNIST CNN trained in PyTorch from inside an NGC container. Proves the training layer.
  3. LLM inference — Gemma 4 served via Ollama on the host, pinned to the 5070 Ti. Proves the inference layer.

If all three work, the rig is real. The rest is project-specific.

Storage: A Subvolume for Dev Work

The machine has two NVMe drives. The 1 TB primary is @/@home from part two — the OS, dotfiles, scratch space. The 2 TB secondary has been sitting unformatted since the install. That’s the dev drive: project source, datasets, model weights, container images. Anything that can grow without warning lives here, because it’s exactly the kind of growth that fills an OS drive silently.

Same recipe as part two: LUKS, btrfs, a subvolume, same passphrase as the OS drive so systemd’s keyctl helper unlocks both with one prompt at boot.

sudo wipefs -a /dev/nvme0n1
sudo cryptsetup luksFormat --type luks2 /dev/nvme0n1
sudo cryptsetup open /dev/nvme0n1 cryptdev
sudo mkfs.btrfs -L dev /dev/mapper/cryptdev

Mount, create a subvolume, unmount:

sudo mount -o noatime /dev/mapper/cryptdev /mnt
sudo btrfs subvolume create /mnt/@dev
sudo umount /mnt

I always create at least one subvolume even when I only need one, because btrfs subvolume list reads better with named entries than with unnamed filesystem roots, and because adding more subvolumes later is cheaper if the discipline is already there.

Wire crypttab and fstab to mount it at boot:

LUKS_UUID=$(sudo blkid -s UUID -o value /dev/nvme0n1)
BTRFS_UUID=$(sudo blkid -s UUID -o value /dev/mapper/cryptdev)
echo "cryptdev UUID=$LUKS_UUID none luks" | sudo tee -a /etc/crypttab
echo "UUID=$BTRFS_UUID /data/dev btrfs defaults,noatime,subvol=@dev 0 2" | sudo tee -a /etc/fstab

The none keyfile entry tells systemd to prompt at boot, but Ubuntu’s initramfs caches the first successful unlock and reuses it for subsequent LUKS volumes with matching passphrases. One prompt, both volumes unlocked.

Mount, take ownership, symlink:

sudo mkdir -p /data/dev
sudo systemctl daemon-reload
sudo mount /data/dev
sudo mkdir -p /data/dev/{projects,datasets,models,docker,ollama,experiments}
sudo chown -R $USER:$USER /data/dev
ln -s /data/dev ~/Developer

The symlink is the one cosmetic thing I really wanted: ~/Developer/projects, ~/Developer/models, ~/Developer/docker all resolve through /data/dev. The home directory layout reads naturally; the actual storage lives where it belongs. Same trick I’ve used on every workstation since the early Mac days — Developer in the home directory, real bytes elsewhere.

Reboot test: one passphrase prompt, both volumes unlock, findmnt /data/dev confirms.

The Inspection Mounts

A small thing I should have done years ago and finally did this round: mount each btrfs filesystem at its top-level subvolume too, so the full subvolume tree is browsable.

sudo mkdir -p /storage/filesystem/{os,dev} /storage/backup/{os,dev}

In /etc/fstab, two more lines — same UUIDs as the existing @ and @dev mounts, but with subvolid=5 to mount the filesystem root:

UUID=XXXX /storage/filesystem/os   btrfs defaults,noatime,subvolid=5 0 0
UUID=XXXX /storage/filesystem/dev  btrfs defaults,noatime,subvolid=5 0 0

Now ls /storage/filesystem/os shows @, @home, timeshift-btrfs/ as siblings. ls /storage/filesystem/dev shows @dev. When I want to browse a Timeshift snapshot or peek at how Docker is laying out its image layers, no flags needed — just cd. The /storage/backup/{os,dev} directories are pre-staged for the backup HDDs that come online in a separate post.

Docker, and the Storage Detour

Docker on Ubuntu 26.04 is mostly straightforward. Docker’s apt repo has a resolute suite published, so the install reads exactly like every other Ubuntu Docker install I’ve done in the last five years:

sudo apt install -y ca-certificates
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg \
    -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] \
  https://download.docker.com/linux/ubuntu $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt update
sudo apt install -y docker-ce docker-ce-cli containerd.io \
    docker-buildx-plugin docker-compose-plugin

What I wanted before starting Docker for the first time: image data on the 2 TB drive, btrfs as the storage driver. Two settings in /etc/docker/daemon.json and we’d be done:

{
  "data-root": "/data/dev/docker",
  "storage-driver": "btrfs"
}

I wrote that file, started Docker, ran docker info. And here’s where the day got interesting.

 Storage Driver: overlayfs
 Docker Root Dir: /var/lib/docker

Neither setting took effect. Both fields had silently reverted to defaults.

The first guess — that daemon.json wasn’t being read — was wrong. dockerd --help confirmed the default config path was /etc/docker/daemon.json, and the systemd unit wasn’t passing any override flags. The file was being read. The settings were just being ignored.

The journal told the real story:

level=info msg="Docker daemon" commit=d329809 containerd-snapshotter=true storage-driver=overlayfs version=29.4.2

Docker 29 (and 28 before it) defaults to containerd-snapshotter mode. In this mode, Docker delegates image storage to containerd’s snapshotter system rather than using its own graph drivers. The storage-driver: btrfs and data-root keys in daemon.json belong to the legacy graph-driver path; under containerd-snapshotter they’re silently bypassed. Storage decisions move into containerd’s /etc/containerd/config.toml instead.

Configuring containerd’s btrfs snapshotter on Ubuntu 26.04 turns out to be doable in principle — the package supports it, the docs describe how. But Ubuntu 26.04 ships containerd 2.2.3, which has a documented regression in the btrfs snapshotter’s interaction with the new mount-manager plugin.

So here’s the choice: configure the modern path on a known-broken version, or step back to the legacy Docker-managed btrfs storage driver, which has been stable for years.

I went legacy. Disable containerd-snapshotter explicitly, let Docker manage storage directly:

{
  "data-root": "/data/dev/docker",
  "storage-driver": "btrfs",
  "features": {
    "containerd-snapshotter": false
  }
}

The features block is the important addition — without it, daemon.json is parsed correctly and silently ignored. With it, Docker’s legacy graph driver path takes over and data-root and storage-driver mean what they say.

sudo systemctl stop docker docker.socket
sudo rm -rf /var/lib/docker /var/lib/containerd /data/dev/docker/*
sudo systemctl start docker
sudo docker info | grep -E "Storage Driver|Docker Root Dir"
 Storage Driver: btrfs
 Docker Root Dir: /data/dev/docker

There it is. The image cache lives on @dev, btrfs subvolumes for image layers will be visible via btrfs subvolume list, and daemon.json does what its keys say.

Add myself to the docker group, and there’s a small Ubuntu 26.04 surprise waiting:

sudo usermod -aG docker $USER
newgrp docker
Command 'newgrp' not found, but can be installed with:
sudo apt install util-linux-extra

newgrp got split out of util-linux into util-linux-extra in this release. A two-package install or a logout-and-back-in fixes it. Minor, but the kind of small papercut a fresh distro release always brings. Worth knowing.

Verify Docker is up and working:

docker run --rm hello-world
Hello from Docker!
This message shows that your installation appears to be working correctly.
...

NVIDIA Container Toolkit

Docker can talk to GPUs only with help. NVIDIA Container Toolkit is what registers a GPU runtime with Docker so --gpus works.

The toolkit ships from NVIDIA’s apt repo, not Ubuntu’s archive — this is the one part of the stack where you go to NVIDIA directly:

curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | \
    sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
    sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
    sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
sudo apt update
sudo apt install -y nvidia-container-toolkit
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

The verification is nvidia-smi, but running inside a container:

docker run --rm --gpus all nvidia/cuda:12.6.0-base-ubuntu24.04 nvidia-smi

That command pulls a small CUDA base image from Docker Hub, hands the container access to all GPUs via NVIDIA’s runtime, and runs nvidia-smi inside it. The output should match what nvidia-smi shows on the host. It does:

+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 595.58.03              Driver Version: 595.58.03      CUDA Version: 13.2     |
+-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 4060 Ti     Off | 00000000:05:00.0  On   |                  N/A |
| 30%   40C    P8              3W / 165W  |     612MiB / 16380MiB  |      0%      Default |
+-----------------------------------------+------------------------+----------------------+
|   1  NVIDIA GeForce RTX 5070 Ti     Off | 00000000:0C:00.0 Off   |                  N/A |
|  0%   44C    P8             24W / 300W  |      15MiB / 16303MiB  |      0%      Default |
+-----------------------------------------+------------------------+----------------------+

Both cards visible from inside the container, full path through Docker → containerd → nvidia-container-runtime → CUDA library injection → host driver. This is the foundation. Everything else builds on it.

About the dual-GPU split: the 4060 Ti is at PCIe address 05:00.0 and drives both 4K displays. The 5070 Ti is at 0C:00.0 and is for compute. From here on out, every container that uses the GPU gets pinned to the 5070 Ti via --gpus 'device=1'. The 4060 Ti keeps GNOME and the cursor responsive while the 5070 Ti pegs at 100%.

The CUDA Hello-World

Vector add. Two arrays of floats, sum them on the GPU, copy the result back. Thirty lines of CUDA C, no dependencies, the canonical first GPU program since approximately forever.

#include <stdio.h>
#include <cuda_runtime.h>

__global__ void vectorAdd(const float *A, const float *B, float *C, int n) {
    int i = blockDim.x * blockIdx.x + threadIdx.x;
    if (i < n) C[i] = A[i] + B[i];
}

int main() {
    int n = 999999999;  // ~1B elements, ~4GB per array. Stays under INT_MAX.
    size_t bytes = n * sizeof(float);
    float *h_A = (float*)malloc(bytes);
    float *h_B = (float*)malloc(bytes);
    float *h_C = (float*)malloc(bytes);
    for (int i = 0; i < n; i++) { h_A[i] = i; h_B[i] = 2*i; }

    float *d_A, *d_B, *d_C;
    cudaMalloc(&d_A, bytes); cudaMalloc(&d_B, bytes); cudaMalloc(&d_C, bytes);
    cudaMemcpy(d_A, h_A, bytes, cudaMemcpyHostToDevice);
    cudaMemcpy(d_B, h_B, bytes, cudaMemcpyHostToDevice);

    int threads = 256;
    int blocks = (n + threads - 1) / threads;

    int iterations = 999999;
    for (int iter = 0; iter < iterations; iter++) {
        vectorAdd<<<blocks, threads>>>(d_A, d_B, d_C, n);
    }
    cudaDeviceSynchronize();
    cudaMemcpy(h_C, d_C, bytes, cudaMemcpyDeviceToHost);

    printf("Test PASSED\n");
    free(h_A); free(h_B); free(h_C);
    cudaFree(d_A); cudaFree(d_B); cudaFree(d_C);
    return 0;
}

A note on the size and the loop: my first version did a million elements once, finished in microseconds, and the nvidia-smi watch in the other terminal never had a chance to refresh fast enough to catch any GPU activity. The 5070 Ti was so quick the screenshot opportunity passed before the screenshot tool fired. Bumping it to a hundred million elements and a thousand iterations still wasn’t enough — the card chewed through that in roughly a second. I ended up at a billion elements and a million iterations to keep the GPU pegged long enough for screenshots.

Compile and run inside the container, pinned to the 5070 Ti:

docker run --rm \
    --gpus 'device=1' \
    -v $(pwd):/work -w /work \
    nvidia/cuda:12.6.0-devel-ubuntu24.04 \
    bash -c "nvcc vectorAdd_loop.cu -o vectorAdd_loop && ./vectorAdd_loop"
==========
== CUDA ==
==========
CUDA Version 12.6.0
...
Test PASSED

The devel tag is what brings nvcc along — the base image has the runtime libraries but not the compiler. --gpus 'device=1' passes through only the 5070 Ti, by its CUDA index.

In the other terminal, watch -n 0.5 nvidia-smi, captured during the run:

| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 4060 Ti     Off | 00000000:05:00.0  On   |                  N/A |
| 30%   46C    P8              4W / 165W  |    1008MiB / 16380MiB  |      0%      Default |
+-----------------------------------------+------------------------+----------------------+
|   1  NVIDIA GeForce RTX 5070 Ti     Off | 00000000:0C:00.0 Off   |                  N/A |
| 40%   61C    P1            193W / 300W  |   12802MiB / 16303MiB  |    100%      Default |
+-----------------------------------------+------------------------+----------------------+

The 5070 Ti is genuinely working: 100% GPU utilization, P1 power state, 193 W out of 300 W cap, fan ramped to 40%, 12.8 of 16.3 GB VRAM holding three float arrays of about a billion elements each. The 4060 Ti is at P8, 4 W, 0% utilization, just keeping gnome-shell and Xwayland and 1Password and nautilus responsive. The dual-GPU split is doing exactly what it was designed to do: compute on one card, display on the other, with no fight between them.

On to the second leg of the triad.

PyTorch and MNIST

The CUDA experiment proves the toolchain. The PyTorch experiment proves the framework, and proves a real training loop works — gradient descent, weights updating, accuracy climbing. That’s the bar for “this is an AI/ML workstation,” not “this can run inference on a model someone else trained.”

NVIDIA publishes monthly PyTorch container releases on NGC (the NVIDIA GPU Cloud catalog). The image tag I want is nvcr.io/nvidia/pytorch:26.04-py3 — and yes, that’s the April 2026 NGC release, and yes, the version number coincidentally matches the Ubuntu version. NGC’s YY.MM-py3 tagging just happens to collide with Ubuntu’s YY.MM versioning. Two unrelated calendars. Use it.

The MNIST CNN is ~30 lines of PyTorch:

import torch
import torch.nn as nn
import torch.optim as optim
from torchvision import datasets, transforms
from torch.utils.data import DataLoader
import time

print(f"CUDA available: {torch.cuda.is_available()}")
print(f"GPU count: {torch.cuda.device_count()}")
for i in range(torch.cuda.device_count()):
    print(f"  GPU {i}: {torch.cuda.get_device_name(i)}")

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
print(f"Using device: {device}")

transform = transforms.Compose([
    transforms.ToTensor(),
    transforms.Normalize((0.1307,), (0.3081,))
])

train_loader = DataLoader(
    datasets.MNIST('/data', train=True, download=True, transform=transform),
    batch_size=128, shuffle=True)
test_loader = DataLoader(
    datasets.MNIST('/data', train=False, transform=transform),
    batch_size=1000)

class CNN(nn.Module):
    def __init__(self):
        super().__init__()
        self.conv1 = nn.Conv2d(1, 32, 3, 1)
        self.conv2 = nn.Conv2d(32, 64, 3, 1)
        self.fc1 = nn.Linear(9216, 128)
        self.fc2 = nn.Linear(128, 10)
    def forward(self, x):
        x = torch.relu(self.conv1(x))
        x = torch.relu(self.conv2(x))
        x = torch.max_pool2d(x, 2)
        x = torch.flatten(x, 1)
        x = torch.relu(self.fc1(x))
        return self.fc2(x)

model = CNN().to(device)
optimizer = optim.Adam(model.parameters())
criterion = nn.CrossEntropyLoss()

start = time.time()
for epoch in range(1, 6):
    model.train()
    for data, target in train_loader:
        data, target = data.to(device), target.to(device)
        optimizer.zero_grad()
        loss = criterion(model(data), target)
        loss.backward()
        optimizer.step()
    model.eval()
    correct = 0
    with torch.no_grad():
        for data, target in test_loader:
            data, target = data.to(device), target.to(device)
            pred = model(data).argmax(dim=1)
            correct += pred.eq(target).sum().item()
    acc = 100. * correct / len(test_loader.dataset)
    print(f"Epoch {epoch}: test accuracy {acc:.2f}%")

print(f"Total training time: {time.time() - start:.1f}s")

Two convolutional layers, two fully-connected layers, five epochs, Adam optimizer. Genuinely nothing fancy. This is everyone’s first deep learning model since Yann LeCun wrote it down in 1998.

Run it inside the container, pinned to the 5070 Ti:

docker run --rm \
    --gpus 'device=1' \
    -v $(pwd):/workspace \
    -v $(pwd)/data:/data \
    -w /workspace \
    nvcr.io/nvidia/pytorch:26.04-py3 \
    python train_mnist.py
==========
== PyTorch ==
==========

NVIDIA Release 26.04 (build 297444911)
PyTorch Version 2.12.0a0+0291f96
...
CUDA available: True
GPU count: 1
  GPU 0: NVIDIA GeForce RTX 5070 Ti
Using device: cuda
Epoch 1: test accuracy 98.50%
Epoch 2: test accuracy 98.61%
Epoch 3: test accuracy 98.75%
Epoch 4: test accuracy 98.98%
Epoch 5: test accuracy 98.98%
Total training time: 42.4s

42 seconds. Five epochs. 98.98% test accuracy on a model with no tuning, no augmentation, no learning-rate schedule. PyTorch saw the 5070 Ti, not the 4060 Ti, because --gpus 'device=1' passed through only that card and the container’s torch.cuda.device_count() reflects what the container can see, not what the host has.

The nvidia-smi watch during training tells a more nuanced story than the CUDA hello-world did:

|   1  NVIDIA GeForce RTX 5070 Ti     Off | 00000000:0C:00.0 Off   |                  N/A |
| 31%   42C    P1             58W / 300W  |    1257MiB / 16303MiB  |      7%      Default |
+-----------------------------------------+------------------------+----------------------+

7% GPU utilization. 1.2 GB VRAM. 58 W out of 300 W. The 5070 Ti is barely awake.

This is what training a tiny model on a big GPU looks like, and it’s worth being honest about. MNIST is small. The CNN is small. The training data fits in memory many times over. The bottleneck isn’t compute — it’s the CPU feeding batches across the PCIe bus to a GPU that processes them faster than they can arrive. A 16 GB Blackwell card is so far past what this workload needs that most of the time it’s waiting.

That’s the right outcome for the test: I want to verify that training works, not that training saturates the card. The GPU was used. Gradients were computed on it. Weights were updated on it. The accuracy climbed in the way it should. If the same code on the same hardware misbehaves later, I’ll know it’s a regression, not setup.

When the model is bigger — fine-tuning a transformer, training something with a real parameter count — utilization will climb to where it should be. This experiment was about the foundation, not the ceiling.

Ollama, and Reading the Timing Right

Inference is the third leg of the triad. Different runtime, different concerns. Where training is bandwidth-and-compute hungry and PyTorch wants every kernel optimized, inference is mostly memory-bandwidth-bound — at small batch sizes the GPU spends most of its time pulling weights through cache, and the constraints are model-fits-in-VRAM and how fast the bus can stream data.

Install Ollama via the upstream script. Redirect model storage to the dev drive before pulling anything, via a systemd drop-in:

[Service]
Environment="OLLAMA_MODELS=/data/dev/ollama"
Environment="CUDA_DEVICE_ORDER=PCI_BUS_ID"
Environment="CUDA_VISIBLE_DEVICES=1"

OLLAMA_MODELS keeps model files off the OS drive — at 5-10 GB per model, this matters fast. The two CUDA_* variables pin Ollama to the 5070 Ti specifically.

Pull a model and run a prompt:

ollama pull gemma4
ollama run gemma4 --verbose "Write a 400-word essay on dark matter"

Ollama’s --verbose flag prints a structured timing summary after each response. First prompt to a freshly-loaded model:

total duration:        8.210s
load duration:         3.034s
prompt eval count:     28 token(s)
prompt eval rate:      1963.48 tokens/s
eval count:            716 token(s)
eval duration:         4.651s
eval rate:             153.94 tokens/s

Second run, same model, still loaded:

total duration:        6.918s
load duration:         0.262s
prompt eval rate:      2000.89 tokens/s
eval count:            946 token(s)
eval rate:             153.45 tokens/s

load duration drops from 3 seconds on the cold first run to 0.26 seconds once the model is resident — a 12× drop. That’s the OS-level dance of pulling the model file from NVMe through the page cache, then transferring 9.6 GB from DRAM to VRAM over PCIe. Cold-start latency tells you about disk and PCIe, not about the GPU.

eval rate is the steady-state generation speed — actual tokens per second once the model is producing output. That number is rock-steady at 153-154 tokens per second across runs. That’s roughly 2× what an M-series MacBook does on the same model. The 5070 Ti is doing exactly what a 16 GB Blackwell card with 896 GB/s memory bandwidth should do.

gemma4:latest is a 9.6 GB model on disk — about 3× the VRAM footprint of gemma3:4b (3.3 GB) — and it’s hitting roughly the same generation rate as the smaller model. 153 tokens/s on Gemma 4, 160 tokens/s on Gemma 3 4B, basically the same number despite the much larger working set. (Gemma 4’s flagship multimodal models use a Mixture-of-Experts architecture — 26B total parameters with 4B activated per token — which is part of why per-token speed stays competitive with much smaller dense models.)

This is also the answer to a question the post hasn’t asked yet: how big a model can this rig actually serve usefully? The answer is “anything up to about 12-13 GB after counting KV cache and runtime overhead” — which covers the 4B and 9.6B models I’ve tried, and probably a good chunk of the 13B-parameter quantized models out there. Above that, you start paying the cost of partial CPU offload and the experience falls off a cliff.

There’s a nice symmetry to the triad here. Training proves that gradients can flow on this card. Inference proves that the same card can serve a real LLM at speeds that are useful for actual interactive work. Same hardware, two completely different workloads, both healthy.

Closing the Series

That closes the trilogy. Three posts, three weekends, one working AI/ML workstation.

Part one was the why. Part two was the install — TPM detour, btrfs conversion, Timeshift, nouveau. Part three was the workstation — Docker on btrfs, NVIDIA Container Toolkit, CUDA in a container, MNIST training to 99% on the 5070 Ti, and Gemma 4 serving at 154 tokens/s. The bet paid out. There’s a working machine on my desk now that I can use to learn.

One bonus post still to come — out soon. Backups: btrfs everywhere, btrbk on a systemd timer, two 4 TB HDDs as local point-in-time recovery. Its own post: Distro Hopping Diaries: Bonus.

The NAS half of the backup story — connecting these to a ZFS-mirrored NAS, itself due for a fresh 26.04 install — is its own post (or two), coming when it’s ready. HDD prices are up 30-50% on AI data center demand, so any hardware refresh is on hold.

For now, the trilogy is done. The rig is real. Everything else when it’s ready.