Daemon
This is what I imagine Daemon looks like.
(Image by Victoria_Borodinova from Pixabay)
Part 2 of “The Network” which is a series of posts where I live the principle that security through obscurity is not the way. Part 1 covering the mail server can be found here: Talon
The DNS server for the nmrc.org domain is daemon.nmrc.org aka Daemon. While my registrar-of-many-years Gandi (gandi.net) functions as the main DNS server (in fact there is a main and two backups), the main DNS server in my heart is Daemon.
The history of how the whole domain and idea of servers at home started with Talon, but Daemon’s history is still sort of interesting. It was the original gateway server between the wild and woolly Internet and the internal network. Functioning as the main firewall as well as the main DNS server (friends provided backup DNS services), it was also a method of remote access when traveling - particularly when having to play remote admin.
Later on, as Gandi began to offer DNS services, I opted to set things up there as DNS backup. A recent drive failure on Daemon prompted a hardware upgrade, and Gandi was promoted to the main DNS services, at least for now.
With the advent of my UniFi UDM Pro, Daemon no longer functions as the gatekeeper between private and public networks, it remains as just a DNS server, out on the perimeter as a bastion host like the other servers. At some point it will once again serve double duty as most of my servers do, but for now this is it.
The Server Itself
We’ll start with the basics. With the recent drive failure on Daemon, I took that opportunity to replace the entire system with more modern hardware.
Case - NZXT H210
Power Supply - EVGA 650W P2
Motherboard - MSI B450-I Mini ATX
Memory - Corsair Vengeance LPX 16GB DDR4
OS drive - Western Digital WD Blue WD10JPVX Sata 1TB
Okay, it actually looks like this…..
The hardware choices were based upon a couple of factors, not the least of which is availability, which has been impacted in large part due to the pandemic. This isn’t me saying I am not happy - in fact I am quite pleased - but given the situation I didn’t always get my first pick.
Recovery
Important note - as this was a hardware failure that prompted the entire build-from-scratch thing, I decided to test out the “break glass” scenario of loading on a fresh OS and recovering with the data, and not a full drive restore. I prefer this method, and I’m dropping the whole old school full drive thing I’ve done for ages out of habit. This was less stressful as I could get the main function of the server up and operating (DNS) quickly, and then take my time with the remainder of the more general settings.
Configuration
The operating system is currently Ubuntu Server 21.04. DNS is handled by BIND9, although I am seriously planning on moving to PowerDNS at some point in the future (more on that later).
What DNS server should the DNS server itself be pointed at? Well, experience had taught me it should point at the “nearest” upstream server, so Daemon uses some specific servers. Here is that info along with the rest of the configuration:
DNS - 68.94.156.1, 68.94.157.1. These are the ISP's (AT&T) main DNS servers, so routing-wise they are the closest.
NTP - NTP server is set to time.google.com, it supports leap smearing which is standard for NMRC servers. Using Ubuntu's timesyncd, excerpt from
/etc/systemd/timesyncd.conf:
[Time]
NTP=time1.google.com time2.google.com time3.google.com time4.google.com
FallbackNTP=ntp.ubuntu.comLogging - Currently logging is to local host only. Eventually will change to a dedicated location, perhaps a dedicated server just for this task.
Firewall - This is handled with ufw, with SSH (port 22) allowed from vortex, DNS (port 53) requests from anywhere including TCP for "larger" requests.
Patching information - Automated, set to update, upgrade, and autoremove old packages via settings in
/etc/apt/apt.conf.d/50unattended-upgradesand/etc/apt/apt.conf.d/20auto-upgrades. Auto reboot if needed at 2am.Remote access - This is implemented using SSH. Two-factor authentication is used via Duo Security’s Duo Linux. Not only are things configured to use specific public keys, but port forwarding and tunneling are disabled. Also, even though Duo’s Ubuntu package only supports the latest LTS version 20.04, it runs fine on 21.04 and is even automatically updated.
Backup details - Awaiting automated access to internal NAS, currently manual.
Break glass plan - Minimal. Console access to the server is available so repairs, incident investigations, etc can be performed without network access. Worse case scenario is wipe/reload OS from scratch plus manual restoration of select data. As stated earlier, this was the case for the Daemon upgrade of hardware, so this will be the direction moving forward.
The main service is DNS, so here is the important info:
DNS - DNS records for the nmrc.org domain
$TTL 604800
@ IN CAA 0 issue "letsencrypt.org"
@ IN SOA daemon.nmrc.org. thegnome.nmrc.org. (
2021081200 ; serial
43200 ; Refresh period
7200 ; Retry interval
2419200 ; Expire time
86400 ; Negative caching TTL
)
;
@ IN NS daemon.nmrc.org.
@ IN A 162.196.226.75
@ IN MX 5 talon.nmrc.org.
daemon IN A 162.196.226.77
talon IN A 162.196.226.75
vortex IN A 162.196.226.74
rigor-mortis IN A 162.196.226.76
blackhole IN A 162.196.226.73
www IN A 162.196.226.76
web IN CNAME www
dragon IN A 162.196.226.78
_dmarc IN TXT "v=DMARC1; p=reject"
default._domainkey IN TXT "v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDorm4fA6kdVeQty3S5LvhjJKp2H7aeKkw+UVP5vssEqBUZ/VS0ph7Ou/DF0RxDricc7uGkqc0hxbe5k1pImvWD0p/ga61VM/54kozgiTKZ0yAX9UkdF5o8fC/Mb1PP5Bm+I9t5I2yyxRzXVTTQOATQEyhhurqWlgIVZv8Y9drBBQIDAQAB"
@ IN TXT "exploring the world one class C at a time"
@ IN TXT "v=spf1 mx a ip4:162.196.226.72/29 -all"The main problem with this set of domain records is that a very few number of domains that regularly send email to NMRC do not do proper MX lookups. They simply look up nmrc.org instead of looking up an MX record (this took ages to figure out). However web browsers given nmrc.org still look for that address, and this meant there were situations where mail was trying to be delivered to the web server. To resolve this, a web server was added to the mail server that redirects all web traffic to www.nmrc.org. Ugly, but it works.
This further complicated the SPF record, as I had to add ip4:65.70.17.128/29 to allow mail messages coming from other hosts, but mainly I had to adjust OpenDKIM (via /etc/opendkim/TrustedHosts on talon) to allow normal outbound email leaving the private network behind vortex to actually get signed and properly delivered. All of this allowed DMARC to be adjusted for p=reject, helping to lock down email even further. So SPF, DKIM, and DMARC are all functioning and properly locking down email.
What About DNSSEC?
This is where it gets complicated. Before, I actually used to have Daemon set up as the main DNS server with the Gandi server as backups. It is possible to set up DNSSEC with this setup, but I’d have to get a backup DNS server that was not a Gandi server.
Gandi does have some decent APIs I can take advantage of, and as Gandi is currently the main DNS server I could set up DNSSEC with them at the click of a button, but I’d still like to include Daemon in that scenario. However, that choice would involve either learning and experimenting with their APIs or using PowerDNS instead of Bind. Of course using PowerDNS could open up some additional possibilities, so I’m leaning toward the second option, but as of this writing I am still wanting to wait until all hardware for all servers is updated - then the more advanced configurations will commence.
Why All This?
There is one simple reason to do all of this work and to expend so much time and effort - knowledge. It is one thing to read up on various protocols, it is quite another to go at it hands-on. With the NMRC domain I am managing email for myself and others, and doing this whole email thing right also means DNS has to be functioning as well. The insight I’ve gained has helped immeasurably in my career, and at this point I wouldn’t have it any other way.
