Interesting. A year ago I was looking for something exactly like this for distributing data between multiple servers. Everything required a ton of overhead or was too big to use. I ended up just using json. I did discover that Brotli can compress 3 gigs of json down into just 70 megs nearly instantly.
This post leads me to piggyback and see what people think of lambdabuffers (which are not my work but something I became aware of through the Haskell community).
I’d love to see benchmarks testing the two, and out of curiosity also including compressed JSON docs to take into account the impact of payload volume.
Nevertheless, I think there are two major features that differentiate protobuff and fleece, which are:
fleece is implemented as an appendable data structure, which might open the door to some usages,
protobuf supports more data types than the ones supported by JSON, which may be a good or bad thing depending on the perspective.
In the end, if the world survived with XML for so long, I’d guess we can live with minor gains just as easily.
"Appendable” seems like a positive spin on the (…)
I don’t think your take makes sense. It’s a write-only data structure which supports incremental changes. By design it tracks state and versioning. You can squash it if you’d like but others might see value in it.
It’s not serialized from what I understand so it doesn’t need parsing. It’s sort of a structure+pointer dump. We’ll see how well that translates to other languages than C though.
You probably wouldn’t. The main difference is that protobuf is structured while fleece is unstructured, so you would use it in places where you don’t want to (or can’t) tie yourself to a schema outright.
Q: Why didn’t you write this in $NEW_LANGUAGE instead of crufty C++?
A: I probably should have! $NEW_LANGUAGE is deservedly attracting a lot of attention for its combination of safety, readable syntax, and support for modern programming paradigms. I’ve been trying out $NEW_LANGUAGE and want to write more code in it. But for this I chose C++ because it’s supported on all platforms, lots of people know how to use it, and it still supports high-level abstractions (unlike C.)
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !programming@programming.dev
Welcome to the main community in programming.dev! Feel free to post anything relating to programming here!
Cross posting is strongly encouraged in the instance. If you feel your post or another person’s post makes sense in another community cross post into it.
Hope you enjoy the instance!
Rules
Rules
Follow the programming.dev instance rules
Keep content related to programming in some way
If you’re posting long videos try to add in some form of tldr for those who don’t want to watch videos
Interesting. A year ago I was looking for something exactly like this for distributing data between multiple servers. Everything required a ton of overhead or was too big to use. I ended up just using json. I did discover that Brotli can compress 3 gigs of json down into just 70 megs nearly instantly.
Brotli aged like fine wine. Almost 9 years of being in public domain.
https://opensource.googleblog.com/2015/09/introducing-brotli-new-compression.html
This post leads me to piggyback and see what people think of lambdabuffers (which are not my work but something I became aware of through the Haskell community).
I don’t have much experience with similar tools but that looks quite interesting, thanks for sharing!
No problem! I plan to teach for them when/if iterop becomes difficult when sending data between WASM, Haskell, Plutus, and Purescript.
The main question I would have is why use it instead of protobuf? Having native support for binary values aside.
Of lambdabuffers for that matter.
I’d love to see benchmarks testing the two, and out of curiosity also including compressed JSON docs to take into account the impact of payload volume.
Nevertheless, I think there are two major features that differentiate protobuff and fleece, which are:
In the end, if the world survived with XML for so long, I’d guess we can live with minor gains just as easily.
“Appendable” seems like a positive spin on the “truncated YAML-file is frighteningly often valid” problem…
I don’t think your take makes sense. It’s a write-only data structure which supports incremental changes. By design it tracks state and versioning. You can squash it if you’d like but others might see value in it.
It’s not serialized from what I understand so it doesn’t need parsing. It’s sort of a structure+pointer dump. We’ll see how well that translates to other languages than C though.
You probably wouldn’t. The main difference is that protobuf is structured while fleece is unstructured, so you would use it in places where you don’t want to (or can’t) tie yourself to a schema outright.
What’s the difference between this and CBOR?
Lol
As someone who clicked partly to check if it was Rust-based, I think that’s a 100% fair call-out lol
Same 😂
I see what you did there…