Afaik you need some external resource to coordinate the punch. The STUN protocol is purpose built for this, and both clients need to be able to reach a STUN server to coordinate which port and public IP they’ll try to connect to each other on. I assume this does something similar but with p2p network tech instead of a STUN server.
So it doesn’t need any port forwarding, and works on CGNAT? How the “NAT hole punching” works? Both clients connect to something on IPFS?
Afaik, for DHT with torrent, clients need to know at least one tracker, what is the “tracker” here? Something on IPFS? Who am I sending my IP addresses?
How much overhead does this add to speed? I love with Wireguard, that it’s barely noticeable, really close to p2p speeds, OpenVPN was awful in this regard.
First off great find. I didn’t think to check the AUR. I personally wouldn’t use it as that version is 3 years out of date but its existence means that it might be entirely possible to get a non Nix version. I’m not sure I fully understand why it needs Nix OS but what do I know.
It is all libp2p magic
There have been lots if talks on libp2p and Nat traversal. I suggest you check them out. How it actually works is pretty complex and requires someone more knowledgeable than me to explain. One way it works is that both devices start a TCP connection at the same time which gets the proper ports to open up.
AUR packages ending with"-git" or “-svn” always pull the latest commit from source. The version number means that was the last time the packager had to change something on the PKGBUILD script, not the actual version which would be installed.
Where should I look? Where were these talks? I’m interested.
It says it connects to a “Hole Punch Coordination (DCUtR - Direct Connection Upgrade through Relay)”. So for NAT traversal to work, you need a third party, this relay. As I expected. I guess you can self host this, but than you could just host a wireguard server. I guess if you are on a locked down network where you cannot connect to any relay (e.g. how the Chinese Great Firewall works technically they could block it) you can’t initiate a connection behind a NAT.
Nonetheless it seems interesting, but no magic here. Maybe the big difference that the relay servers are distributed, so no central authority to block easily.
That doesn’t match my experience with AUR at all. Usually it pulls a specific git revision and checks the hash. This also ensures that the build shouldn’t suddenly fail to some extent.
Though it’s entirely possible that it’s not like this for all packages, though I find it kind of counterintuitive since your package manager wouldn’t know when to perform an update in this case.
In the PKGBUILD file you can list sources (line 12,13) and their respective checksums (line 14,15). In this PKGBUILD there are 2 sources: the first is the systemd unit file, it’s coming from the package’s AUR repo, not from upstream, you can see its checksum. The second source is the actual source, and you can see, it’s checksum is ‘SKIP’ so it shouldn’t be checked.
With these kind of packages you can’t get notified if there is an update available, but if you install it again with your favorite AUR helper it would update itself for the latest version. It calculates version number from the latest commit hash, before building and installing, so if that is the same it won’t update again.
Nix just calls the *.nix files, it’s still go under the hood. PKGBUILD is similar to the flake.nix and package.nix files to me, but I have no experience with nix.
DHT is autonomous and does not require a tracker. Usually it is only used as a fallback as a regular tracker is quicker. It’s p2p, and is split accross people hosting it.
Nothing too major about how it’s usually used, but the yaml spec does allow arbitrary code execution when parsing a file and relies on the parser to have that feature disabled: https://en.m.wikipedia.org/wiki/YAML#Security
That’s why for python, yaml.save_load() is a thing. That’s fine for your local config files and may even be a feature for you, but it shouldn’t be used to exchange information between services.
My general view is similar, yaml is better if it should be written by humans, json is better if it should be written and read only by a machine. but hyprspace uses json for configuration, so I don’t really understand cellardoor’s comment
I’ve never used Yggdrasil but it looks like a standalone project. It also appears have a smaller team and a little less funding but don’t know for sure.
That’s the funniest thing about this whole conversation: I do. Quite regularly. It works fine. Better than HTTP for my usecase. No clue what the fuck you people are on about.
Didn’t realize we all were now incapable of looking out the window. That seems like something an absolutely incapable person would do because they’re way too lazy.
Is IPFS something your family and friends check on regularly? I don’t even know what it is.
Considering your reluctance to give any information about your assertion that such a project using it becomes useless, I’m not sure you know what it is either :P
I haven’t checked it out in years. From my understanding, IPFS aims to be a distributed filesystem that kinda works like Bittorent. If you access a file, you then seed it. Last time I checked it out, the project was jumping on the crypto bandwagon… Just checked out their website now, and don’t know WTF it is.
I say might be because - while the required bootstrap peers include a bunch of ones based on bootstrap.libp2p.io - there is a long list of hardcoded ip addresses and I don’t recognize any of them.
So those might be libp2p.io ip addresses, but they might also be IPFS ip addresses, or even belong to someone else altogether. (Edit: There are WHOIS tools online like https://lookup.icann.org/en that can be used to look these up and figure out who they belong to if you are really curious, but I can’t be bothered to do that right now.)
In any case, it looks like the way this works is that from a peer, libp2p tries to look up additional peers, and so on. So at most IPFS would be used as a way to get a listing, but once the desired peer is found, IPFS is cut out of the picture for that particular connection and NAT hole punching is used to establish a direct connection between peers instead (as per the linked wikipedia article, https://en.wikipedia.org/wiki/Hole_punching_(networking )
Tailscale… is not that good. The underlying wireguard is robust, but tailscale control plane is completely proprietary, as well as their DERP servers that it too often uses completely needlessly. They can also block you off from downloading it, updating, or logging in, if you happen to be in a wrong country.
I’m myself looking for an alternative to it, but having trouble finding something I could share with non tech savvy friends while not being as complex on my end as, say, open/strongswan ais. Any suggestions welcome.
Have you considered having Headscale on a cheap VPS? We are actually doing that and it is pretty capable. IIRC, you can configure not to use the tailscale servers at all, and use your own public VPS for coordination. Bonus point, tailscale hired the Headscale developer and maintainer, and they are allowed to work on Headscale while on their payroll. The team looks very much into FOSS.
Yep. That’s the number one contender. Well right after overriding default DERP’s with my own VPS machines. I’ll definitely try it out over some weekend.
One of my other concerns with this and other solutions suggested is the reliance on wireguard which can be subject to fingerprinting and censorship. Do you happen to know if it’d be possible to swap out Headscale’s implementation of wireguard to amnezia? I’ll have to do my homework anyway, but who knows, maybe there are some pitfalls to avoid.
Oh, never heard of amnezia. Never needed actually. But it looks like a good improvement on Wireguard. I will need a separate setup to test it out and currently I’m away from home with no clue when I will return. If I happen to find anything, I will definitely ping you.
In the HN page you linked many people mentioned v2ray. Have you tried that? How good is it?
In the HN page you linked many people mentioned v2ray. Have you tried that? How good is it?
Nope, haven’t actually read the comments, just sent the article as reference to the issue. It does indeed sound quite promising. Think it’d be nice to have even if as just a fallback, so I’ll try that too, whenever I get a moment.
Yeah I don’t understand how this is different than headscale, but I’m very much not savvy on the pipes and tubes that make the Internet go round. Can anyone explain?
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !selfhosted@lemmy.world
A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don’t control.
Rules:
Be civil: we’re here to support and learn from one another. Insults won’t be tolerated. Flame wars are frowned upon.
No spam posting.
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it’s not obvious why your post topic revolves around selfhosting, please include details to make it clear.
Don’t duplicate the full text of your blog or github here. Just post the link for folks to click.
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
No trolling.
Resources:
selfh.st Newsletter and index of selfhosted software and apps
Wireguard is p2p.
EDIT: I guess the point is it’s doing peer discovery without static public IPs or DNS. Pretty cool!
Or port forwarding. You have to open a udp port for wireguard
Technically you can nat punch with wire guard
How do I learn this power? Don’t you still need at least one server exposed?
Afaik you need some external resource to coordinate the punch. The STUN protocol is purpose built for this, and both clients need to be able to reach a STUN server to coordinate which port and public IP they’ll try to connect to each other on. I assume this does something similar but with p2p network tech instead of a STUN server.
Interesting, it’s on AUR, I will try it.
So it doesn’t need any port forwarding, and works on CGNAT? How the “NAT hole punching” works? Both clients connect to something on IPFS?
Afaik, for DHT with torrent, clients need to know at least one tracker, what is the “tracker” here? Something on IPFS? Who am I sending my IP addresses?
How much overhead does this add to speed? I love with Wireguard, that it’s barely noticeable, really close to p2p speeds, OpenVPN was awful in this regard.
First off great find. I didn’t think to check the AUR. I personally wouldn’t use it as that version is 3 years out of date but its existence means that it might be entirely possible to get a non Nix version. I’m not sure I fully understand why it needs Nix OS but what do I know.
It is all libp2p magic
There have been lots if talks on libp2p and Nat traversal. I suggest you check them out. How it actually works is pretty complex and requires someone more knowledgeable than me to explain. One way it works is that both devices start a TCP connection at the same time which gets the proper ports to open up.
AUR packages ending with"-git" or “-svn” always pull the latest commit from source. The version number means that was the last time the packager had to change something on the PKGBUILD script, not the actual version which would be installed.
Where should I look? Where were these talks? I’m interested.
Edit: I found the whitepaper about hole punching: https://research.protocol.ai/publications/decentralized-hole-punching/
It says it connects to a “Hole Punch Coordination (DCUtR - Direct Connection Upgrade through Relay)”. So for NAT traversal to work, you need a third party, this relay. As I expected. I guess you can self host this, but than you could just host a wireguard server. I guess if you are on a locked down network where you cannot connect to any relay (e.g. how the Chinese Great Firewall works technically they could block it) you can’t initiate a connection behind a NAT.
Nonetheless it seems interesting, but no magic here. Maybe the big difference that the relay servers are distributed, so no central authority to block easily.
That doesn’t match my experience with AUR at all. Usually it pulls a specific git revision and checks the hash. This also ensures that the build shouldn’t suddenly fail to some extent.
Though it’s entirely possible that it’s not like this for all packages, though I find it kind of counterintuitive since your package manager wouldn’t know when to perform an update in this case.
It’s documented in the wiki, they are called VCS packages, and it’s not the usual, they work a bit differently: https://wiki.archlinux.org/title/VCS_package_guidelines
You can see in this instance, that it skips the sha checking for upstream source, in line 15 of the PKGBUILD it says ‘SKIP’: https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=hyprspace-git#n15
sha1sums parameter is documented in the wiki: https://wiki.archlinux.org/title/PKGBUILD#sha1sums
In the PKGBUILD file you can list sources (line 12,13) and their respective checksums (line 14,15). In this PKGBUILD there are 2 sources: the first is the systemd unit file, it’s coming from the package’s AUR repo, not from upstream, you can see its checksum. The second source is the actual source, and you can see, it’s checksum is ‘SKIP’ so it shouldn’t be checked.
With these kind of packages you can’t get notified if there is an update available, but if you install it again with your favorite AUR helper it would update itself for the latest version. It calculates version number from the latest commit hash, before building and installing, so if that is the same it won’t update again.
The PKGBUILD looks like it is just building via go. I’m not sure how you would configure it without Nix. I’ll try building it.
Nix just calls the *.nix files, it’s still go under the hood. PKGBUILD is similar to the flake.nix and package.nix files to me, but I have no experience with nix.
DHT is autonomous and does not require a tracker. Usually it is only used as a fallback as a regular tracker is quicker. It’s p2p, and is split accross people hosting it.
YAML?? (╯°□°）╯︵ ┻━┻)
Nothing too major about how it’s usually used, but the yaml spec does allow arbitrary code execution when parsing a file and relies on the parser to have that feature disabled: https://en.m.wikipedia.org/wiki/YAML#Security
That’s why for python,
yaml.save_load()is a thing. That’s fine for your local config files and may even be a feature for you, but it shouldn’t be used to exchange information between services.My general view is similar, yaml is better if it should be written by humans, json is better if it should be written and read only by a machine. but hyprspace uses json for configuration, so I don’t really understand cellardoor’s comment
Xml has entered the chat
Yeah I agree. Although recently I’ve become partial to toml… In the end I’ll use what’s common in the ecosystem I’m developing in
nit: you mean
yaml.safe_load().Oh yeah, of course.
Hey did you know that any JSON file is also a valid YAML file? I bet you’ll love YAML a lot more now that you have this information
Toml is superior to all.
Careful. The yaml cult will come after you in a long and formless column, and only self destruct when one of them is a step too far to the left.
Sounds relatively similar to Yggdrasil
Not quite
What are some key differences?
It uses libp2p
I’ve never used Yggdrasil but it looks like a standalone project. It also appears have a smaller team and a little less funding but don’t know for sure.
Fair, Yggdrasil is mainly intended for research in internet-scale routing through a mesh network and less as a finished product.
Never heard of libp2p before, but apparently it’s used by IPFS? Looks pretty interesting indeed.
Is this made by the same guy who does hyprland?
@semperverus @possiblylinux127 No, this other person has a working ‘e’ key on their keyboard.
Eh what its hyprspace. The title is incorrect but the link says hypr
*but relies on IPFS.
Useless.
Why is it useless?
Lol. You checked on IPFS lately? Different times. Different world.
Do you have more information? Haven’t looked into it for a while. What happened?
IPFS is like a dead Multiplayer game, or an Onion network. Check it out.
That which is asserted without evidence may be dismissed without evidence.
Go ahead and try to use it then.
That’s the funniest thing about this whole conversation: I do. Quite regularly. It works fine. Better than HTTP for my usecase. No clue what the fuck you people are on about.
Didn’t realize we all were now incapable of looking out the window. That seems like something an absolutely incapable person would do because they’re way too lazy.
Is IPFS something your family and friends check on regularly? I don’t even know what it is.
Considering your reluctance to give any information about your assertion that such a project using it becomes useless, I’m not sure you know what it is either :P
I haven’t checked it out in years. From my understanding, IPFS aims to be a distributed filesystem that kinda works like Bittorent. If you access a file, you then seed it. Last time I checked it out, the project was jumping on the crypto bandwagon… Just checked out their website now, and don’t know WTF it is.
I sure wonder how this is supposed to function, any explanation anywhere, like a diagram or something?
It relies on libp2p not ipfs. ipfs uses libp2p as its transport
“A Lightweight VPN Built on top of IPFS + Libp2p”
Seems like both at a glance
Tell me what I’m misunderstanding here.
So I dug into the source code a bit to see how it’s used. It turns out that IPFS might actually optional, as per the log line on https://github.com/hyprspace/hyprspace/blob/master/p2p/node.go#L213 (“Getting additional peers from IPFS API”)
The list of required bootstrap peers is hardcoded in the same file, but a few lines above, specifically at https://github.com/hyprspace/hyprspace/blob/master/p2p/node.go#L181
I say might be because - while the required bootstrap peers include a bunch of ones based on bootstrap.libp2p.io - there is a long list of hardcoded ip addresses and I don’t recognize any of them.
So those might be libp2p.io ip addresses, but they might also be IPFS ip addresses, or even belong to someone else altogether. (Edit: There are WHOIS tools online like https://lookup.icann.org/en that can be used to look these up and figure out who they belong to if you are really curious, but I can’t be bothered to do that right now.)
In any case, it looks like the way this works is that from a peer, libp2p tries to look up additional peers, and so on. So at most IPFS would be used as a way to get a listing, but once the desired peer is found, IPFS is cut out of the picture for that particular connection and NAT hole punching is used to establish a direct connection between peers instead (as per the linked wikipedia article, https://en.wikipedia.org/wiki/Hole_punching_(networking )
deleted by creator
What about Tailscale? I know it’s Proprietary software, but still.
Tailscale… is not that good. The underlying wireguard is robust, but tailscale control plane is completely proprietary, as well as their DERP servers that it too often uses completely needlessly. They can also block you off from downloading it, updating, or logging in, if you happen to be in a wrong country.
I’m myself looking for an alternative to it, but having trouble finding something I could share with non tech savvy friends while not being as complex on my end as, say, open/strongswan ais. Any suggestions welcome.
Have you considered having Headscale on a cheap VPS? We are actually doing that and it is pretty capable. IIRC, you can configure not to use the tailscale servers at all, and use your own public VPS for coordination. Bonus point, tailscale hired the Headscale developer and maintainer, and they are allowed to work on Headscale while on their payroll. The team looks very much into FOSS.
Yep. That’s the number one contender. Well right after overriding default DERP’s with my own VPS machines. I’ll definitely try it out over some weekend.
One of my other concerns with this and other solutions suggested is the reliance on wireguard which can be subject to fingerprinting and censorship. Do you happen to know if it’d be possible to swap out Headscale’s implementation of wireguard to amnezia? I’ll have to do my homework anyway, but who knows, maybe there are some pitfalls to avoid.
Oh, never heard of amnezia. Never needed actually. But it looks like a good improvement on Wireguard. I will need a separate setup to test it out and currently I’m away from home with no clue when I will return. If I happen to find anything, I will definitely ping you.
In the HN page you linked many people mentioned v2ray. Have you tried that? How good is it?
Nope, haven’t actually read the comments, just sent the article as reference to the issue. It does indeed sound quite promising. Think it’d be nice to have even if as just a fallback, so I’ll try that too, whenever I get a moment.
I use zerotier personally
https://netbird.io/ maybe?
Headscale worked for me, but I get the non-tech saavy friends part doesn’t quite jive with it as a solution.
Still, anyone wanna ditch Tailscale and only use it for hosting sites across proxies? Headscale is great.
Yeah I don’t understand how this is different than headscale, but I’m very much not savvy on the pipes and tubes that make the Internet go round. Can anyone explain?
Tailscale is actually a lot more open than you think. The agents are all foss and there is a self hostable version.
This reminds me of nebula although nebula does require a central server to coordinate hosts.
removed by mod
I hope you aren’t expecting people to just randomly click a Google docs link.
This is highly sus
removed by mod
Isn’t that same as Tor?
Not in the least