> For the complete documentation index, see [llms.txt](https://docs.soniclabs.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.soniclabs.com/sonic/node-deployment/node-hardening/tier-5-stay-current.md).

# Tier 5: Stay Current

Hardened releases, OS patching and containers.

**Goal:** benefit from hardening done in Sonic itself, and keep the rest of the server patched.

## 5.1 Use hardened releases

Besides fixing bugs, new releases can turn off features that are risky to expose. The fewer features an attacker can reach, the fewer places a bug can hide.

For example, starting with [v2.2.5](https://github.com/0xsoniclabs/sonic/releases/tag/v2.2.5), the node **rejects custom JavaScript tracers** (code supplied inside an RPC request) unless you turn them on. Built-in tracers (`callTracer`, `prestateTracer`, etc.) keep working.

{% hint style="info" %}
**Depend on custom JavaScript tracers?** After the upgrade, they fail with `custom tracer is not permitted`. Read [Tier 1.2](https://docs.soniclabs.com/sonic/node-deployment/node-hardening/tier-1-essentials.md) before you turn them back on.
{% endhint %}

To confirm your node rejects custom JavaScript tracers (run it against the node directly, on `127.0.0.1`):

```bash
curl -s -H 'content-type: application/json' -d '{"jsonrpc":"2.0","id":1,"method":"debug_traceCall","params":[{"to":"0x0000000000000000000000000000000000000000"},"latest",{"tracer":"{result:function(){return 1},fault:function(){}}"}]}' \
  http://127.0.0.1:18545
```

✅ Expected: an `"error"` saying `custom tracer is not permitted`.

❌ A `"result"` of `1` means custom tracers are on. Upgrade, or turn them off ([Tier 1.2](https://docs.soniclabs.com/sonic/node-deployment/node-hardening/tier-1-essentials.md)).

⚠️ If the error says the method doesn't exist or isn't available, `debug` is turned off on this node, and the test tells you nothing. Run it on a node that has `debug` enabled.

**Static builds.** Check how your binary is built:

```bash
file /usr/local/bin/sonicd
```

If it says `statically linked` or `static-pie linked`, re-run [Tier 2](https://docs.soniclabs.com/sonic/node-deployment/node-hardening/tier-2-lock-the-node-in-a-box.md) step 2.0. The `ExecPaths=` line will shrink to just the binary.

## 5.2 Keep the operating system patched

Many attacks chain a bug in the node with a bug in the operating system to gain more control.

Run a supported LTS release (recommended: **Ubuntu 24.04 LTS**, see [Before you start](https://docs.soniclabs.com/sonic/node-deployment/node-hardening/before-you-start.md) step 3), and plan the move to the next one before standard support for yours ends. Enable automatic security updates:

```bash
sudo apt install unattended-upgrades && sudo dpkg-reconfigure -plow unattended-upgrades   # Debian/Ubuntu
sudo dnf install dnf-automatic && sudo systemctl enable --now dnf-automatic.timer         # RHEL/Fedora
```

Schedule regular reboots so kernel updates actually take effect.

**Validators:** a reboot takes the validator offline until the node is back in sync (with a named pipe, until someone sends the password). Plan reboots like upgrades, and check the log afterwards ([Tier 2](https://docs.soniclabs.com/sonic/node-deployment/node-hardening/tier-2-lock-the-node-in-a-box.md), test (a)).

## 5.3 If you run the node in containers

Containers aren't a security boundary by default, and the systemd protections from [Tier 2](https://docs.soniclabs.com/sonic/node-deployment/node-hardening/tier-2-lock-the-node-in-a-box.md) don't apply inside them. If you use containers, apply the same ideas:

* Use a **minimal image** containing only the `sonicd` binary (and its libraries), with no shell, no package manager, no download tools. Don't use the repository's experimental `Dockerfile` image in production.
* Run as a non-root user, with a read-only filesystem except the data volume.
* Drop all privileges and block privilege gain.
* Make sure nothing can be run from the data folder (the container version of [Tier 2A](https://docs.soniclabs.com/sonic/node-deployment/node-hardening/tier-2-lock-the-node-in-a-box.md)): put the data folder on its own disk, mounted with `noexec,nosuid,nodev` in `/etc/fstab`. The container inherits these options.

Example:

```bash
docker run -d --name sonic \
  --user "$(id -u sonic):$(id -g sonic)" \
  --read-only --tmpfs /tmp:rw,noexec,nosuid \
  --cap-drop=ALL --security-opt no-new-privileges \
  -v /var/lib/sonic:/var/lib/sonic \
  -p 5050:5050 -p 5050:5050/udp -p 127.0.0.1:18545:18545 \
  <your-minimal-sonic-image> --datadir=/var/lib/sonic --http.addr=0.0.0.0 ...
```

* **Inside the container**, the node must listen on `0.0.0.0`, or Docker can't forward traffic to it. Control who can reach it with the `-p` address instead: `-p 127.0.0.1:18545:18545` makes RPC reachable only from the host itself.
* **Published ports bypass `ufw`.** Docker edits the firewall directly, so a port published with `-p 18545:18545` is open to the world whatever `ufw` says. Publish RPC on `127.0.0.1` or a private IP, and rely on your cloud firewall too.
* **Validators:** no RPC flags, and publish only the peer-to-peer port (`-p 5050:5050 -p 5050:5050/udp`).

**Block the private network (the container version of** [**Tier 2B**](https://docs.soniclabs.com/sonic/node-deployment/node-hardening/tier-2-lock-the-node-in-a-box.md)**).** Container networks can reach your private network by default. Docker reserves a firewall chain called `DOCKER-USER` for your own rules:

```bash
# Containers on the default Docker network may not open connections to internal networks
for NET in 10.0.0.0/8 172.16.0.0/12 192.168.0.0/16 100.64.0.0/10 169.254.0.0/16; do
  sudo iptables -I DOCKER-USER -i docker0 -d "$NET" -m conntrack --ctstate NEW -j DROP
done
# Only if your DNS server has a private IP (see "Checking DNS" in Tier 2): allow it.
# Never allow 169.254.169.254: on GCP it's also the metadata service. Give the
# container another resolver with --dns instead.
# sudo iptables -I DOCKER-USER -i docker0 -d 10.0.0.2 -j RETURN
```

* If you use your own Docker network (e.g. with Docker Compose), replace `docker0` with that network's interface (`ip link | grep br-`).
* These rules disappear on reboot. Put the commands in a small service like `sonic-guard` ([Tier 3.2](https://docs.soniclabs.com/sonic/node-deployment/node-hardening/tier-3-get-alerted.md)), with `After=docker.service`.
* Also apply the [Tier 4.3](https://docs.soniclabs.com/sonic/node-deployment/node-hardening/tier-4-control-access.md) network rules in your cloud firewall.

## 5.4 Rebuild instead of repair

Keep a documented, repeatable way to build a node server from scratch (image, configuration management, scripts). After any suspected compromise, and periodically anyway, rebuild rather than patching a server you can't fully trust.

**Validators:** the rebuilt server uses the same signing key, so stop the old one for good before the new one starts. Never run both, not even for a minute: that's double-signing. After a suspected compromise, follow the validator steps in [Tier 3.4](https://docs.soniclabs.com/sonic/node-deployment/node-hardening/tier-3-get-alerted.md) first.

## ✅ Tier 5 checklist

* [ ] Running v2.2.5 or later; with `debug` on, custom JavaScript tracers off and the test request rejected
* [ ] Automatic OS security updates on; regular reboots
* [ ] OS on a supported LTS release (recommended: Ubuntu 24.04 LTS)
* [ ] Containers (if used): minimal image, non-root, read-only, no capabilities, noexec data folder, `DOCKER-USER` rules, RPC published only on localhost or a private IP
* [ ] Servers can be rebuilt from scratch from documented steps (validators: old server stopped for good before the new one starts)
