Iroh uses QUIC connections and uses the EndpointId, the public ed25519 key, in the TLS handshake for authentication. This makes it impossible for a server to try and machine-in-the-middle the connection.
But headscale is a community project. The iroh relay code is by number0 just like iroh itself and lives in the same MIT and Apache2 licensed repository. You can even embed it into your webserver if you have a special use case - it is very modular.
iroh is an open source library. The relay servers are open source too but number0 runs public, rate limited, relay servers that can be used by everyone. The commercial offerings are for dedicated relay servers and more insight into your network.
It's extremely subtle, fooled me initially too. The `fn handle_connection` takes a different argument, so rust `Derefs` the `ConnectionRef` into `Connection` for the first example. A bit too subtle to my liking.
Oh wow. Ok. Subtle and error prone. This screams for a more ergonomic API like not making Connection cloneable or doing as_ref instead of Deref or not decoupling the lifetime when you do a clone.
That's interesting, because the connection to the relay server is established using HTTP1.1 over TLS. Followed by a WebSocket upgrade. It should look like any other webserver connection on the internet. Could be worth investigating your network conditions and filing an issue for this.
There might be some confusion here, holepunching is a core functionality of iroh. There are still some firewall configurations that iroh can not yet holepunch and that can still be improved, but in general the holepunching works rather well.
I'm not sure I fully understand this window logic question. QUIC does MTU discovery, so if the link supports bigger datagrams the MTU will go up. Unreliable datagrams using RFC9221 can be sent up to the MTU size minus the QUIC packet overhead. So if your link supports >1500 bytes then you should be able to send datagrams >1500 bytes using iroh.
Fragmenting datagrams (or IP packets) is generally not a good idea. All protocol designs have been moving away from this the past few decades. If you want unreliable messages of larger than the MTU maybe taking some inspiration from Media-over-QUIC is a good idea. They use one uni-directional QUIC stream per message and include some metadata at the start of each stream to explain how old it is. If a stream takes too long to read to end-of-stream and you already have a newer message in a new uni-directional stream you can cancel the previous streams (using something like SendStream::reset or RecvStream::stop in Quinn API terms, depending on which side detects the message is no longer needed earlier). Doing this will stop QUIC from retransmitting the lost data from the message that's being slow to receive.
Right, I should have been more clear about that. Window logic was perhaps the wrong term, since I don't care about resends.
The use case I have in mind is for realtime data synchronization. Say we want to share a state larger than 1500 bytes, then we have to come up with a clever scheme to compress the state or do partial state transfer, which could require knowledge of atomic updates or even database concepts like ACID, which feels over-engineered.
I'd prefer it if the protocol batched datagrams for me. For example, if we send a state of 3000 bytes, that's 2 datagrams at an MTU of 1500. Maybe 1 of those 2 fails so the message gets dropped. When we send a state again, for example in a game that sends updates 10 times per second, maybe the next 2 datagrams make it. So we get the most recent state in 3 datagrams instead of 4, and that's fine.
I'm thinking that a large unreliable message protocol should add a monotonically increasing message number and index id to each datagram. So sending 3000 bytes twice might look like [0][0],[0][1] and [1][0],[1][1]. For each complete message, the receiver could inspect the message number metadata and ignore any previous ones, even if they happen to arrive later.
Looks like UDP datagram loss on the internet is generally less than 1%:
So I think this scheme would generally "just work" and hiccup every 5 seconds or so when sending 10 messages per second at 2 datagrams each and a 99% success rate, and the outage would only last 100 ms.
We might need more checklist items:
( ) Doesn't provide a way to get the last known Maximum Transmission Unit (MTU)
And optionally:
( ) Doesn't provide a way to get large unreliable message number metadata
As one of the iroh developers I must say thank you for creating ZeroTier! It absolutely was part of the inspiration and it's seamless functioning continues to amaze me daily. Something that continues to drive me to strive for as seamless an experience in iroh.
I love the fact we can make different tools learning from each other and approaching making p2p usable in different ways.
