I stood up a 3-node etcd v3 cluster on tiny NanoPi Neo boards running Debian. It’s now humming along at:

  • node1 — 192.168.8.242
  • node2 — 192.168.8.194
  • node3 — 192.168.8.103

Below is exactly what worked — including the dead-ends I hit and the fixes that made it stable.

Why etcd here?

I wanted a lightweight, reliable key-value store for small services in the lab. etcd’s Raft consensus and simple v3 API make it a great fit—even on constrained ARM boards—so long as you keep the config minimal and the storage tidy.

What I get from etcd:
- Consistency first: Strongly consistent reads/writes via Raft.
- Simple ops: One binary, clear CLI (etcdctl), easy snapshots/restores.
- Watches & prefixes: Native change streams and hierarchical-ish keys.
- Small footprint: Fine for NanoPi Neo as long as you trim the backend size and compact/defrag periodically.

Hardware & gotchas (ARM/Neo)

These NanoPi Neo boards are tiny 32-bit ARM systems. etcd runs fine with a few tweaks:

  • Arch flag (required): add Environment=ETCD_UNSUPPORTED_ARCH=arm in the systemd unit (etcd 3.4+ on armv7 requires it).
  • Listen vs advertise:
  • Listen on all interfaces: --listen-client-urls http://0.0.0.0:2379 and --listen-peer-urls http://0.0.0.0:2380
  • Advertise the node’s own IP: --advertise-client-urls http://<IP>:2379, --initial-advertise-peer-urls http://<IP>:2380
  • Open ports: allow TCP 2379 (client) and 2380 (peer) between nodes.
  • Binary path: on Debian, etcd is usually at /usr/bin/etcd (match this in ExecStart).
  • Storage wear & size: SD cards are fragile—keep the backend small:
    Environment=ETCD_QUOTA_BACKEND_BYTES=100000000 (~100 MB), and schedule compaction/defrag.
  • Memory/limits: keep default snapshot counts; add LimitNOFILE=65536 in [Service].
  • Clock sync: consensus hates clock drift—ensure systemd-timesyncd or chrony is active.
  • Quick probes:
  • Listeners: ss -lntp | egrep '2379|2380'
  • Health: curl -s http://<IP>:2379/health
  • Logs: journalctl -u etcd -n 80 --no-pager

Install

On all three nodes (Debian):


```bash
sudo apt update
sudo apt install -y etcd-client etcd-server   # installs /usr/bin/etcd and /usr/bin/etcdctl

which etcd etcdctl         # -> /usr/bin/etcd /usr/bin/etcdctl
etcd --version
etcdctl version
sudo mkdir -p /var/lib/etcd
sudo systemctl stop etcd 2>/dev/null || true
sudo systemctl disable etcd 2>/dev/null || true

The bootstrap that actually works (single → three)

Bringing all three up at once is easy to mis-sequence. The reliable path is: start node1 alone → add members → join node2 & node3.

0) Stop & wipe stale state (all nodes)

```bash
sudo systemctl stop etcd 2>/dev/null || true
sudo rm -rf /var/lib/etcd/*
Start node1 as a single-member cluster

Create /etc/systemd/system/etcd.service on node1 (192.168.8.242):

[Unit]
Description=etcd - single-node bootstrap
After=network.target
Wants=network-online.target

[Service]
Type=notify
User=root
Environment=ETCD_UNSUPPORTED_ARCH=arm
# Environment=ETCD_QUOTA_BACKEND_BYTES=100000000

ExecStart=/usr/bin/etcd \
  --name=node1 \
  --data-dir=/var/lib/etcd \
  --initial-advertise-peer-urls http://192.168.8.242:2380 \
  --listen-peer-urls http://0.0.0.0:2380 \
  --listen-client-urls http://0.0.0.0:2379 \
  --advertise-client-urls http://192.168.8.242:2379 \
  --initial-cluster node1=http://192.168.8.242:2380 \
  --initial-cluster-state new \
  --initial-cluster-token etcd-cluster-1
Restart=always
RestartSec=5
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

Bring it up and verify:

sudo systemctl daemon-reload
sudo systemctl enable --now etcd
curl -s http://192.168.8.242:2379/health

2) From node1, add node2 & node3 to the cluster
export ETCDCTL_API=3
ENDPOINTS="http://192.168.8.242:2379"

etcdctl --endpoints="$ENDPOINTS" member add node2 --peer-urls=http://192.168.8.194:2380
etcdctl --endpoints="$ENDPOINTS" member add node3 --peer-urls=http://192.168.8.103:2380

3) Join node2 and node3 as existing members

Create /etc/systemd/system/etcd.service on each, swapping name/IP.

node2 (192.168.8.194):

[Unit]
Description=etcd - cluster member (node2)
After=network.target
Wants=network-online.target

[Service]
Type=notify
User=root
Environment=ETCD_UNSUPPORTED_ARCH=arm

Environment=ETCD_QUOTA_BACKEND_BYTES=100000000

ExecStart=/usr/bin/etcd \
--name=node2 \
--data-dir=/var/lib/etcd \
--initial-advertise-peer-urls http://192.168.8.194:2380 \
--listen-peer-urls http://0.0.0.0:2380 \
--listen-client-urls http://0.0.0.0:2379 \
--advertise-client-urls http://192.168.8.194:2379 \
--initial-cluster node1=http://192.168.8.242:2380,node2=http://192.168.8.194:2380,node3=http://192.168.8.103:2380 \
--initial-cluster-state existing \
--initial-cluster-token etcd-cluster-1
Restart=always
RestartSec=5
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

node3 (192.168.8.103):

[Unit]
Description=etcd - cluster member (node3)
After=network.target
Wants=network-online.target

[Service]
Type=notify
User=root
Environment=ETCD_UNSUPPORTED_ARCH=arm

Environment=ETCD_QUOTA_BACKEND_BYTES=100000000

ExecStart=/usr/bin/etcd \
--name=node3 \
--data-dir=/var/lib/etcd \
--initial-advertise-peer-urls http://192.168.8.103:2380 \
--listen-peer-urls http://0.0.0.0:2380 \
--listen-client-urls http://0.0.0.0:2379 \
--advertise-client-urls http://192.168.8.103:2379 \
--initial-cluster node1=http://192.168.8.242:2380,node2=http://192.168.8.194:2380,node3=http://192.168.8.103:2380 \
--initial-cluster-state existing \
--initial-cluster-token etcd-cluster-1
Restart=always
RestartSec=5
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

Start both:

sudo systemctl daemon-reload
sudo systemctl enable --now etcd

4) Quick sanity
export ETCDCTL_API=3
ALL="http://192.168.8.242:2379,http://192.168.8.194:2379,http://192.168.8.103:2379"

etcdctl --endpoints="$ALL" endpoint health
etcdctl --endpoints="$ALL" member list