Set up ZFS pools + Docker pull-through cache on mdmz-node-9d38d3 #1

New issue

Open

opened 2026-04-22 22:10:22 +00:00 by opadmin · 0 comments

opadmin commented

2026-04-22 22:10:22 +00:00

Owner

Background

mdmz-node-9d38d3 (172.24.0.103) has significant unused storage with ZFS partition labels already set:

Pool Label	Disks	Size	Type
`bulk-storage`	sda	10.9T	Seagate HDD
`fast-storage`	sdg, sdi (mirror)	2x 1.8T	Samsung 870 EVO SSD
`cache-storage`	sdb, sdc, sdd, sde, sdh	5x 224G	Toshiba/PNY SSD
(unlabeled)	sdf	224G	Toshiba SSD (spare?)

Partitions exist with zfs_member type but ZFS is not installed on the NixOS config and no pools are imported.

Tasks

1. Enable ZFS in NixOS configuration

Add boot.supportedFilesystems = [ "zfs" ]; to the node's NixOS config
Set networking.hostId (required for ZFS)
Rebuild and reboot

2. Import/create ZFS pools

zpool import bulk-storage
zpool import fast-storage
zpool import cache-storage

If pools don't import cleanly, create them:

fast-storage: mirror of sdg1 + sdi1 (1.8T usable, redundant)
cache-storage: raidz1 of sdb1,sdc1,sdd1,sde1,sdh1 (~900G usable)
bulk-storage: single vdev sda1 (10.9T, backups/archives)

3. Deploy Docker registry pull-through cache

Create ZFS dataset: fast-storage/registry
Deploy registry:2 as a K8s deployment on node-9d38d3 (nodeSelector)
Configure as pull-through cache for https://registry-1.docker.io
Update dind daemon.json to add registry mirror

4. Use ZFS for game server PVCs

Create ZFS datasets for game server world data
Set up a CSI driver or local-path override pointing to ZFS mounts

Impact

Docker image pulls cached locally — no repeated downloads of base images
Game server world data on fast, checksummed ZFS storage
Backups can use ZFS snapshots + send/receive

## Background `mdmz-node-9d38d3` (172.24.0.103) has significant unused storage with ZFS partition labels already set: | Pool Label | Disks | Size | Type | |------------|-------|------|------| | `bulk-storage` | sda | 10.9T | Seagate HDD | | `fast-storage` | sdg, sdi (mirror) | 2x 1.8T | Samsung 870 EVO SSD | | `cache-storage` | sdb, sdc, sdd, sde, sdh | 5x 224G | Toshiba/PNY SSD | | (unlabeled) | sdf | 224G | Toshiba SSD (spare?) | Partitions exist with `zfs_member` type but ZFS is **not installed** on the NixOS config and no pools are imported. ## Tasks ### 1. Enable ZFS in NixOS configuration - Add `boot.supportedFilesystems = [ "zfs" ];` to the node's NixOS config - Set `networking.hostId` (required for ZFS) - Rebuild and reboot ### 2. Import/create ZFS pools ```bash zpool import bulk-storage zpool import fast-storage zpool import cache-storage ``` If pools don't import cleanly, create them: - `fast-storage`: mirror of sdg1 + sdi1 (1.8T usable, redundant) - `cache-storage`: raidz1 of sdb1,sdc1,sdd1,sde1,sdh1 (~900G usable) - `bulk-storage`: single vdev sda1 (10.9T, backups/archives) ### 3. Deploy Docker registry pull-through cache - Create ZFS dataset: `fast-storage/registry` - Deploy `registry:2` as a K8s deployment on node-9d38d3 (nodeSelector) - Configure as pull-through cache for `https://registry-1.docker.io` - Update dind daemon.json to add registry mirror ### 4. Use ZFS for game server PVCs - Create ZFS datasets for game server world data - Set up a CSI driver or local-path override pointing to ZFS mounts ## Impact - Docker image pulls cached locally — no repeated downloads of base images - Game server world data on fast, checksummed ZFS storage - Backups can use ZFS snapshots + send/receive