Asking broadly like this is akin to asking for a guide on how to cook, it’s generally too broad for there to be a single guide. You first need to figure out what your goals are (you state one already, you’d like it to be externally accessible), determine what services you want to host, and then start looking at how to do so.
The advice I’d give is to start with a solid base, you’ll need something to self host on and it really shouldn’t be the PC you use for other things. Get it setup to run a virtualization OS such as proxmox and use that as your starting point. Then do a lot of reading. I spend probably three to four times as much time reading about the service I’m planning to deploy compared to actually doing the work to deploy it. Lastly, plan. You should have a solid plan in the beginning of how you want your service to work (what will be external vice internal only, how will you setup the networking stack to do that, are you going to have a domain, and will you use subdomains or folders to divide services, what does your IP space look like, will you host your own firewall to make the networking more controlled or fight with your ISPs router, do you want to use docker, kubernetes, or maybe full VMs for each service, do you want/need a UI to manage things from or are you comfortable with CLI, etc). These answers will lead you to guides for various services as well as service specific forums where help is more focused.
I use protonmail with their family plan, it’s not terribly priced when you consider it comes with calendar, vpn, and drive storage as well. The biggest annoyance is probably that you have to use their mobile apps due to the encryption and they are not the greatest, but it does encrypt everything which I find outweighs the forced use of just their app.
Backblaze B2. Any software that is S3 compatible can use B2 as the target and it’s reasonably priced for the service. I backup all the PCs and services to a Synology NAS and then backup that to B2 (everything except my Plex media, that would be pricy and it’s easy enough to re-rip from disc if needed).
This is pretty much it, Plex offers far more client apps that are full featured and they make it super easy to setup and use both as an admin and a user. Especially for things like OTA TV where they provide the guide data once it’s setup (which is why it’s a paid option). I’d move to JellyFin in a heartbeat if they’d support OTA and DVR playback on AppleTV.
I also have a decent amount of video data for Plex (not nearly 50TB, but more than I want I pay to backup). I figure if worst comes to worst I can rip DVD/BluRays again (though I’d rather not) so I only backup file storage from my NAS that my laptops and desktop backup to. It’s just not worth the cost to backup data that’s fairly easy to replace.
If jellyfin could record and playback OTA TV on my Apple TV I’d switch tomorrow, but it seems the team is either unable to or unwilling to work on that feature which is core to how my household uses Plex. The only maybe solution is Infuse which is paid and closed source so is no better really than using Plex in that regard.
Like most things in the world, your use case is not the only use case and as such a solution that checks all the boxes for you will not check all the boxes for everyone.
I would recommend prowlarr instead of jackett for indexer management, and pihole as at least an additional blocking service but in reality it’s really all you need for use at home. I’d also strongly encourage use of a VPN on your *arr download services. I use a separate box to run Plex and then have my *arrs all running on their own VM inside if it to provide separation and allow be to more easily segregate the network traffic (as someone that doesn’t really know docker that well it “just works” for me. Also probably worth looking at how to store your media on an external target, it’s easy to quickly accumulate 10s of TBs of media and trying to store that all on the server locally is asking for trouble. Better to set everything up on a NAS to start.