As the title says, I want to know the most paranoid security measures you’ve implemented in your homelab. I can think of SDN solutions with firewalls covering every interface, ACLs, locked-down/hardened OSes etc but not much beyond that. I’m wondering how deep this paranoia can go (and maybe even go down my own route too!).
I used to have all VMs in my QEMU/KVM server on their own /30 routed network to prevent spoofing. It essentially guaranteed that a compromised VM couldn’t give itself the IP of say, my web server and start collecting login creds. Managing the IP space got painful quick.
full disk encryption on everything except the router (no point in encrypting the router)
the server doesn’t have a display connected for obvious reasons, so I’m manually unlocking it via ssh on each boot
obviously, the SSH keys are different, so the server has a different IP in initrd. That said, I still don’t have any protection against malicious modification of initrd or UEFI
the server scans all new SSL certificates in realtime using certspotter and notifies me of any new certificates issued for my domains that it doesn’t know about (I use Cloudflare so it triggers relatively often, but I still do checks on who the issuer is)
firewall blocks outgoing 25 so nobody can impersonate my mailserver
I use a similar setup, but use a USB for my boot drive that has the lvs partition encryption keyfile. I find it much handier since my computer is not near my server. I can boot and then walk upstairs and it is ready, and remove the USB later.
Then there is no way to brute force the decryption or get a password out of me. Also, when the USB is removed and put in a safe place, there is no way to modify the boot partition or UEFI either.
Then I have a password encryption on my data harddrives that I don’t know the password to, but is on my password manager.
The thing about being paranoid about this stuff is that I probably focused on the wrong thing. A smash & grab is completely protected against, but that is like a 0.1% chance anyway and a 0.1% chance on top of that 0.1% chance that it would be targeted enough that they would even try to decrypt it.
Full disk encryption is really only usefully at all for an unpowered system. Network hardening will probably take care of 99.99% of attack attempts where encryption is 0.01%.
Even for a laptop, if it gets stolen in public, it is still running and can have the keys extracted or break into the running system if someone really wants to hack it. They wouldn’t even try to reboot and break the disk encryption probably…
Too much info, but I guess I am just rambling about how dumb my approach probably is 😅
You might be interested in setting up network bound encryption via Clevis and Tang.
I use a hidden pi zero in my house acting as a Tang server. It’s great being able to reboot any of my encrypted servers without having to manually unlock disks.
there’s not much to know about it, I use Cloudflare simply because its routing is better than direct IP connections for many places on Earth. I can’t fully use Cloudflare anyway because I host many non-web services.
After reading this thread I’m apparently not paranoid enough.
Internet facing services are on their own firewalled vlan (dmz), behind a rev proxy, and I have crowdsec running on the proxy and router.
Anything that can get away with putting up on a vps I have (e.g. this Lemmy server). But some things have storage/compute requirements I’m not willing to shell out for.
Mine’s pretty simple, I have a “don’t open ports until ABSOLUTELY NECESSARY” policy, wireguard works well enough for everything else I need to access remotely. I also keep SSH disabled on any machine that has direct access to the internet.
Would you have to compromise on your security according to your threat model if you ran VMs rather than dedicated devices? I’m no security engineer and I don’t know if KVM/QEMU can fit everyones needs, but AWS uses XCP-ng, and unless they’re using a custom version of it, all changes are pushed upstream. I’d definitely trust AWS’ underlying virtualisation layer for my VMs, but I wonder if I should go with XCP or KVM or bhyve.
This is my personal opinion, but podman’s networking seems less difficult to understand than Docker. Docker was a pain the first time I was reading about the networking in it.
Really like your setup. Do you have any plans to make it more private/secure?
I used VMs some time ago but never managed to look deeper into separation of bare metal vs VMs. Hence I can’t assess this reasonably.
Docker got me interested when it started and after discovering its networking capabilities I never looked back.
Basically I’m trying to minimize the possibility that by intercepting one dockerized service the attacker is able to start interacting with all devices. And I have lots of devices because of a fully automated house. ;)
My paranoia will ensure the constant growth of privacy and security :)
Guessing it is more a habit from back in time when ssl certification wasn’t common. Panic of MITM attacks, friends sharing their trusted access to other friends, etc. all contributed to my actual status of paranoia.
Don’t make me reconsider my cybersec approach ;)
How do you all that have your services on your LAN accessing it over wireguard when external pass the wife/kids/family test? If I had to have my wife activate a VPN before she could access our nextcloud or bitwarden, she’d just never use it
that should not be the case because wireguard only ‘runs’ when it sends or receives packets. try setting the keepalive time a bit higher, 5 minutes maybe.
always on they wouldnt know about it and if the connection failed or the wg service crashed on their phone then the services wouldn’t work. It adds a complexity that you don’t want when you’re trying to pass the wife test. Plus yes battery.
no wireguard should not decrease batterylife (see my other comment)
we use wg eith always on without any problems. sometimes it stops on one phone but l9oking for the key icon and clicking the action button in the navigationcontrolmenuthingy is quite easy
Only remote access by wireguard and ssh on non standard port with key based access.
Fail2ban bans after 1 attempt for a year. Tweaked the logs to ban on more strict patterns
Logs are encrypted and mailed off site daily
System updates over tor connecting to onion repos.
Nginx only has one exposed port 443 that is accessible by wireguard or lan. Certs are signed by letsencrypt. Paths are ip white listed to various lan or wireguard ips.
Only allow one program with sudo access requiring a password.
Every other privelaged action requires switching to root user.
I dont allow devices I dont admin on the network so they go on their own subnet. This is guests phones and their windows laptops.
Well I dont trust closed source software and do what I can to avoid it when I can. At least foss can be audited. Also all the linux devices on the main network are devices I admin.
Fail2ban yes; one year, however, is IMO a bit excessive.
Most ISP IP assignments do tend to linger - even with DHCP the same IP will be re-assigned to the same gateway router for quite a number of sequential times - but most IPs do eventually change within a few months. I personally use 3 months as a happy medium for any blacklist I run. Most dynamic IPs don’t last this long, almost all attackers will rotate through IPs pretty quickly anyhow, and if you run a public service (website, etc.), blocking for an entire year may inadvertently catch legitimate visitors.
Plus, you also have to consider the load such a large blocklist will have on your system, if most entries no longer represent legitimate threat actors, you’ll only bog down your system by keeping them in there.
Fail2ban can be configured to allow initial issues to cycle back out quicker, while blocking known repeat offenders for a much longer time period. This is useful in keeping block lists shorter and less resource-intensive to parse.
With Debian it’s just the apt-tor package, and the project maintains an official list at… onion.debian.org iirc?
I don’t know if serving onion traffic is more expensive for Debian/mirror maintainers so idk if this is something everybody should use
Onion repositories are package repositories hosted on tor hidden services. The connection goes through six hops and is end to end encrypted. In addition to further legitimizing the tor network with normal everyday usage it has the benefit of hiding what packages have been installed on a system.
Here are some notes about them if you want to read more.
It would be funny if that were the case. I was just hoping to be a little more paranoid from you lot and maybe improve on the things I’ve thought about
Yeah, just having a little fun in the role of a paranoid admin. My setup isn’t worth mentioning since it fits my threat model (i.e. nobody gives a shit about my network, just don’t be the low hanging fruit) but I’m interested in other replies. Hope you get some useful responses here.
No, honestly I’m not an attacker, but your local bank. We just need your help to update our systems. Please provide us the following credentials to continue using our phish- *ugh* services.
I am clearly not paranoid enough. For a while I was running an open source router inline between the network AP and the fiber to Ethernet box and running nids but the goddamn thing kept crapping out every few days so i took it back out until I can find a more stable solution.
I have plans if I can ever get around to it. I want the smart TV, printer and other shitty things on a separate network from the more trusted devices. I don’t know how yet but I would like to set up 802.1X for the trusted stuff.
You could not connect the TV and printer to the network but instead attach them to raspberry Pi or similar devices. This allows you full control and stops them calling home and spying.
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !selfhosted@lemmy.world
A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don’t control.
Rules:
Be civil: we’re here to support and learn from one another. Insults won’t be tolerated. Flame wars are frowned upon.
No spam posting.
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it’s not obvious why your post topic revolves around selfhosting, please include details to make it clear.
Don’t duplicate the full text of your blog or github here. Just post the link for folks to click.
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
No trolling.
Resources:
selfh.st Newsletter and index of selfhosted software and apps
One day I will setup my security onion, but I’m procrastinating
I understand some of these words.
all buzz :P
deleted by creator
Acronyms, initialisms, abbreviations, contractions, and other phrases which expand to something larger, that I’ve seen in this thread:
15 acronyms in this thread; the most compressed thread commented on today has 16 acronyms.
[Thread #493 for this sub, first seen 6th Feb 2024, 16:55] [FAQ] [Full list] [Contact] [Source code]
I used to have all VMs in my QEMU/KVM server on their own /30 routed network to prevent spoofing. It essentially guaranteed that a compromised VM couldn’t give itself the IP of say, my web server and start collecting login creds. Managing the IP space got painful quick.
I use a similar setup, but use a USB for my boot drive that has the lvs partition encryption keyfile. I find it much handier since my computer is not near my server. I can boot and then walk upstairs and it is ready, and remove the USB later.
Then there is no way to brute force the decryption or get a password out of me. Also, when the USB is removed and put in a safe place, there is no way to modify the boot partition or UEFI either.
Then I have a password encryption on my data harddrives that I don’t know the password to, but is on my password manager.
The thing about being paranoid about this stuff is that I probably focused on the wrong thing. A smash & grab is completely protected against, but that is like a 0.1% chance anyway and a 0.1% chance on top of that 0.1% chance that it would be targeted enough that they would even try to decrypt it.
Full disk encryption is really only usefully at all for an unpowered system. Network hardening will probably take care of 99.99% of attack attempts where encryption is 0.01%.
Even for a laptop, if it gets stolen in public, it is still running and can have the keys extracted or break into the running system if someone really wants to hack it. They wouldn’t even try to reboot and break the disk encryption probably…
Too much info, but I guess I am just rambling about how dumb my approach probably is 😅
You might be interested in setting up network bound encryption via Clevis and Tang. I use a hidden pi zero in my house acting as a Tang server. It’s great being able to reboot any of my encrypted servers without having to manually unlock disks.
I know about it, but it kinda defeats the purpose (the purpose being police raid protection)
Do you recommend any resources about this? I’d be interested in learning how to implement this.
I’m using the recently merged Clevis module for NixOS. There was a recent talk at FOSDEM about it.
Thanks, I’d like to know more about your public-facing setup using cloudflare
there’s not much to know about it, I use Cloudflare simply because its routing is better than direct IP connections for many places on Earth. I can’t fully use Cloudflare anyway because I host many non-web services.
After reading this thread I’m apparently not paranoid enough.
Internet facing services are on their own firewalled vlan (dmz), behind a rev proxy, and I have crowdsec running on the proxy and router.
Anything that can get away with putting up on a vps I have (e.g. this Lemmy server). But some things have storage/compute requirements I’m not willing to shell out for.
Mine’s pretty simple, I have a “don’t open ports until ABSOLUTELY NECESSARY” policy, wireguard works well enough for everything else I need to access remotely. I also keep SSH disabled on any machine that has direct access to the internet.
Do you use a KVM to interact with machines that can access the Internet?
No, as it’s just my main desktop, my laptop and an isolated PiHole VM
I’m somewhat paranoid therefore running several isolated servers. And it’s still not bulletproof and will never be!
Would you have to compromise on your security according to your threat model if you ran VMs rather than dedicated devices? I’m no security engineer and I don’t know if KVM/QEMU can fit everyones needs, but AWS uses XCP-ng, and unless they’re using a custom version of it, all changes are pushed upstream. I’d definitely trust AWS’ underlying virtualisation layer for my VMs, but I wonder if I should go with XCP or KVM or bhyve.
This is my personal opinion, but podman’s networking seems less difficult to understand than Docker. Docker was a pain the first time I was reading about the networking in it.
Really like your setup. Do you have any plans to make it more private/secure?
I used VMs some time ago but never managed to look deeper into separation of bare metal vs VMs. Hence I can’t assess this reasonably.
Docker got me interested when it started and after discovering its networking capabilities I never looked back.
Basically I’m trying to minimize the possibility that by intercepting one dockerized service the attacker is able to start interacting with all devices. And I have lots of devices because of a fully automated house. ;) My paranoia will ensure the constant growth of privacy and security :)
Why would you rotate passsord though?
Rather choose something random and strong than changing it every 6th moon.
Rotating passwords only for web services. Vaultwarden does make it easy. Not all services allow 2FA.
Sounds still excessive but that’s what the thread is here for.
Would probably understand it more if I knew more aspects.
Cheers to more cybersec :)
Guessing it is more a habit from back in time when ssl certification wasn’t common. Panic of MITM attacks, friends sharing their trusted access to other friends, etc. all contributed to my actual status of paranoia.
Don’t make me reconsider my cybersec approach ;)
You could always add another layer ;D
How do you all that have your services on your LAN accessing it over wireguard when external pass the wife/kids/family test? If I had to have my wife activate a VPN before she could access our nextcloud or bitwarden, she’d just never use it
Is always on not an option?
Always on wireguard kills battery life on mobile for me so I guess that’s a no.
that should not be the case because wireguard only ‘runs’ when it sends or receives packets. try setting the keepalive time a bit higher, 5 minutes maybe.
It also breaks android auto for me.
always on they wouldnt know about it and if the connection failed or the wg service crashed on their phone then the services wouldn’t work. It adds a complexity that you don’t want when you’re trying to pass the wife test. Plus yes battery.
no wireguard should not decrease batterylife (see my other comment) we use wg eith always on without any problems. sometimes it stops on one phone but l9oking for the key icon and clicking the action button in the navigationcontrolmenuthingy is quite easy
My most paranoid config is disabling Ipv4
That’s it. If someone wants to attack me, they will need to adopt IPv6!
And find your IP in a /56 or /64 range (depending on what your ISP gives you). Good luck.
Only remote access by wireguard and ssh on non standard port with key based access.
Fail2ban bans after 1 attempt for a year. Tweaked the logs to ban on more strict patterns
Logs are encrypted and mailed off site daily
System updates over tor connecting to onion repos.
Nginx only has one exposed port 443 that is accessible by wireguard or lan. Certs are signed by letsencrypt. Paths are ip white listed to various lan or wireguard ips.
Only allow one program with sudo access requiring a password. Every other privelaged action requires switching to root user.
I dont allow devices I dont admin on the network so they go on their own subnet. This is guests phones and their windows laptops.
Linux only on the main network.
I also make sure to backup often.
Is that a security benefit?
I guess it cuts the attack surface profile down a bit?
If big corporations hoovering your data should be on everyone’s threat list, then yea, i’d say its a huge benefit.
Well I dont trust closed source software and do what I can to avoid it when I can. At least foss can be audited. Also all the linux devices on the main network are devices I admin.
Fail2ban yes; one year, however, is IMO a bit excessive.
Most ISP IP assignments do tend to linger - even with DHCP the same IP will be re-assigned to the same gateway router for quite a number of sequential times - but most IPs do eventually change within a few months. I personally use 3 months as a happy medium for any blacklist I run. Most dynamic IPs don’t last this long, almost all attackers will rotate through IPs pretty quickly anyhow, and if you run a public service (website, etc.), blocking for an entire year may inadvertently catch legitimate visitors.
Plus, you also have to consider the load such a large blocklist will have on your system, if most entries no longer represent legitimate threat actors, you’ll only bog down your system by keeping them in there.
Fail2ban can be configured to allow initial issues to cycle back out quicker, while blocking known repeat offenders for a much longer time period. This is useful in keeping block lists shorter and less resource-intensive to parse.
My block list is very small actually due to the non standard ssh port. Everything else goes through wireguard.
If it was open to the public then yes I’d have to reconsider the ban length.
That makes a lot more sense for your setup, then.
How does this help, assuming your DNS isn’t being spoofed?
Please see my reply below with links.
Thanks, never thought of that before. I’ll certainly try it, great way to help the network!
Can you explain why you use onion repos? I’ve never heard of that, and I’ve heard of kind of a lot of things.
With Debian it’s just the apt-tor package, and the project maintains an official list at… onion.debian.org iirc?
I don’t know if serving onion traffic is more expensive for Debian/mirror maintainers so idk if this is something everybody should use
Onion repositories are package repositories hosted on tor hidden services. The connection goes through six hops and is end to end encrypted. In addition to further legitimizing the tor network with normal everyday usage it has the benefit of hiding what packages have been installed on a system.
Here are some notes about them if you want to read more.
https://blog.torproject.org/debian-and-tor-services-available-onion-services/
https://www.whonix.org/wiki/Onionizing_Repositories
That is very interesting, thanks!
That’s pretty neat. I might start doing that, just for kicks.
Nice try, attacker trying to get me to do their reconnaissance work for them. I’m on to you.
Spfff me, never Anyway please tell me more about your IP adress and your private keys.
Ok my ip address is 192.168.3.200.
Im gonna hack u now
Hey its me your friend Aaron how are you. Hey i was just wandering, what is your credit card details again? just wandering
My private key has a 3 in it
“I’m in”
Hackerman
It would be funny if that were the case. I was just hoping to be a little more paranoid from you lot and maybe improve on the things I’ve thought about
Yeah, just having a little fun in the role of a paranoid admin. My setup isn’t worth mentioning since it fits my threat model (i.e. nobody gives a shit about my network, just don’t be the low hanging fruit) but I’m interested in other replies. Hope you get some useful responses here.
I’d love to play paranoid admin over my network. Thanks!
No, honestly I’m not an attacker, but your local bank. We just need your help to update our systems. Please provide us the following credentials to continue using our phish- *ugh* services.
Credit card number: _____________
CVV: ___
Expiration date: ______
I am clearly not paranoid enough. For a while I was running an open source router inline between the network AP and the fiber to Ethernet box and running nids but the goddamn thing kept crapping out every few days so i took it back out until I can find a more stable solution.
I have plans if I can ever get around to it. I want the smart TV, printer and other shitty things on a separate network from the more trusted devices. I don’t know how yet but I would like to set up 802.1X for the trusted stuff.
You could not connect the TV and printer to the network but instead attach them to raspberry Pi or similar devices. This allows you full control and stops them calling home and spying.
NAT 🥴