TLS certificates for locally hosted webservices

It’s 2024, and there’s generally no excuse for any website to be served over
HTTP instead of HTTPS. While SSL/TLS certificates used to be expensive and
difficult to manage, ever since Let’s Encrypt launched in 2016 it’s become
almost unnervingly simple to generate valid certificates for your own
websites for the exceedingly low price of Free.

But what about locally hosted websites? That is, websites that are hosted on
your local machine, or another machine on your LAN (Local Area Network).
These websites might be accessed via something like http://192.168.1.123:8093

There’s nothing terribly wrong about this; if you’re following tutorials to
set up your home lab, you’ll likely have a whole bunch of services that look
like this. But there are also several reasons why you might not want to
access them via HTTP and the LAN IP:

Your connection is not private

Did you know that in Chrome, you can type thisisunsafe to get past this screen?

DNS

Just a quick note here about how to make something like https://subdomain.mydomain.com
point to your non-web-accessible server at 192.168.1.123.

It’s actually very simple - remember that DNS just resolves a domain name to an IP address.
While it may be better to run your own local DNS server,
there’s really nothing stopping you from using something like Cloudflare DNS
to let the whole world know that some domain should resolve to a local IP.

It doesn’t make your local LAN any more accessible to the outside internet.

HSTS preload list

By the way, there’s another reason why you might need a TLS certificate.
Originally, I was happy to just set up DNS and access my sites via HTTP.

But there’s this thing called the HSTS preload list
which is hard-coded into modern browsers. Any domain that appears on this
list is forced to upgrade to HTTPS, even if it was originally accessed via
HTTP and the webserver didn’t request an upgrade.

The surprising part is that Google added all of the TLDs that it owns
to the HSTS preload list.
This means that if you have a .dev or a .you domain or any of the other
50+ TLDs that Google owns, you literally must have a valid TLS certificate
for your websites.

All .dev domains are preloaded

Thanks Google. I mean, it's a good thing, but still...

Standard Let’s Encrypt setup

The problem is that the most stock-standard method of deploying Let’s Encrypt
certificates requires that your website is accessible to the public internet,
so that Let’s Encrypt’s servers can access a file on your webserver and prove
that you have control over that webserver. The process looks something like this:

  1. You install certbot on your webserver, which automates the whole process
  2. certbot puts a file on your webserver which contains a special token,
    making it accessible at http://<YOUR_DOMAIN>/.well-known/acme-challenge/<TOKEN>
  3. Let’s Encrypt’s servers access that file, verifies its authenticity, and
    issues you a certificate
  4. certbot can even update your webserver configuration to use this
    certificate to properly handle TLS termination

This is called the HTTP-01 challenge, and you can issue a certificate for a
particular domain using this method. The main benefit is that the entire
process is entirely automated, since certbot can modify your webserver
to pass the challenge. It can even handle renewals, which is amazing.

Some sub-optimal workarounds

Temporarily make your local webserver public accessible

After letting certbot and Let’s Encrypt complete the HTTP-01 challenge
successfully, you could turn off your port forwarding and you would have a
valid certificate…for 90 days. Have fun doing that every 90 days for
certificate renewal, not to mention how icky is feels to have your private
webservers open to attack, even temporarily.

Be your own Certificate Authority

Instead of letting Let’s Encrypt or some other recognised CA sign your
certificates, you could become your own CA and sign certificates for your
services. But this means that every device that you want to use with your
services has to recognise your new CA by installing the CA root certificate
on it.

Good luck doing that for your computer(s), mobile devices, your spouse’s
devices, any guests that come over…

Confusing advice out there

The main reason I wrote this post is because it’s surprisingly difficult to
find the (really quite simple) solution to this problem. There’s even a
page from
the official Let’s Encrypt website that talks about certificates for
localhost that
seems relevant and is often linked to, but really isn’t.

That page talks about the very specific case about a native app deployed
alongside a webserver to be run on localhost, and the difficulties of
getting a valid certificate issued in this case.

DNS-01 challenge

The DNS-01 challenge is
another method of proving ownership of a domain to issue a certificate. In
fact, it’s actually a stronger proof than the HTTP-01 challenge because it
proves that you own the entire domain (including all sub-domains), instead
of just a single domain name. This means you can issue wildcard certificates
(eg, for *.mydomain.com).

The beauty of this challenge type is that it doesn’t require your webserver
to be publicly accessible; instead you need to add a special record to a TXT
entry on your DNS server to prove ownership.

It’s no secret that this challenge type exists. It’s just rarer - perhaps
because it comes with the somewhat-significant downside of needing to store
API keys for your DNS provider on your webserver to allow certbot to
automate the challenge. Additionally, your DNS provider actually
needs to have API support for creating DNS records. Cloudflare is, once
again, fantastic on this front (and free for simple use cases like ours).

Of course, because our webserver is local and not accessible to the internet,
the attack surface is much reduced and keeping API credentials on the webserver
probably isn’t a big deal.

Putting it all together

Here’s the setup I ended up with:

  • DNS records that all resolve to the same local IP address on Cloudflare
    (because I haven’t gotten around to setting up a local DNS server yet)
  • An nginx reverse proxy at that IP, which looks
    at the Host header and forwards traffic to other local webservices
  • Certbot set up alongside nginx to automate TLS certificate renewals using
    the DNS-01 challenge

I’m running nginx in a Docker container because
it makes everything so easy. I even found an off-the-shelf container that
already has nginx and certbot set up. Here’s the docker compose file:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
services:
  nginx:
    # https://github.com/JonasAlfredsson/docker-nginx-certbot
    image: jonasal/nginx-certbot:5.0.1
    container_name: nginx
    restart: 'unless-stopped'
    volumes:
      # The final certificates are stored here
      # You must also provide your DNS provider's API key as a file here
      # See https://github.com/JonasAlfredsson/docker-nginx-certbot/blob/master/docs/certbot_authenticators.md
      - ./letsencrypt:/etc/letsencrypt
      # Makes it easy to configure nginx
      - ./conf.d:/etc/nginx/conf.d
    ports:
      - "80:80"
      - "443:443"
    environment:
      - CERTBOT_EMAIL=<your email>
      # Tells cerbot to use DNS-01, specifically with Cloudflare
      - CERTBOT_AUTHENTICATOR=dns-cloudflare

Here’s the nginx config:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
server { 
    # Listen to port 443 on both IPv4 and IPv6. 
    listen 443 ssl; 
    listen [::]:443 ssl; 
 
    # Domain names this server should respond to 
    server_name subdomain.mydomain.com; # certbot_domain:*.mydomain.com 
 
    # Load the certificate files 
    ssl_certificate         /etc/letsencrypt/live/wildcard.mydomain.com.dns-cloudflare/fullchain.pem; 
    ssl_certificate_key     /etc/letsencrypt/live/wildcard.mydomain.com.dns-cloudflare/privkey.pem; 
    ssl_trusted_certificate /etc/letsencrypt/live/wildcard.mydomain.com.dns-cloudflare/chain.pem; 
 
    ssl_dhparam /etc/letsencrypt/dhparams/dhparam.pem; 
 
    proxy_http_version 1.1; 
    proxy_set_header   Upgrade $http_upgrade; 
    proxy_set_header   Connection $http_connection; 
    proxy_set_header   Host $http_host; 
    proxy_set_header   X-Real-IP $remote_addr; 
    proxy_set_header   X-Forwarded-For $proxy_add_x_forwarded_for; 
    proxy_set_header   X-Forwarded-Proto $scheme; 
 
    location / { 
        proxy_pass http://192.168.1.123:9876; 
    } 
}

Note the special # certbot_domain:*.mydomain.com comment. This is a hint
that the scripts in this particular Docker image understand, so that they’ll
ask certbot to use this domain and issue a wildcard certificate instead of
using the server_name and generating a certificate for each subdomain. See
the issue
and the docs
and the example.

The scripts actually automatically add those ssl_* directives. It’s pretty
neat.

After this simple config, you can now access https://subdomain.mydomain.com
and everything works perfectly.