My use case is different (B2 for offsite data backups) but I went down the path of rclone and it has been working out very well. A lot better than I expected. I read their official docs (installation, usage, the backblaze setup and crypt) to get started. Played at the command line for a few minutes and realize how quick and easy it was. Wrote a script to automate it and off to the races I went!
What’s your use-case? What do you want to achieve?
Using blob storages as filesystems doesn’t work well and could - with B2’s pricing structure - became excessively expensive. Blob storages are designed for easily writing and reading individual blobs. Filesystems are designed for random access, listing, traversal, etc.
It’s for storing a few terabytes of fairly static media (for the most part, write-once). The codebases using it don’t natively support object storage (and will be in Docker containers).
It’s on a Hetzner server, and Backblaze (even after the price increase) will be a lot cheaper than normal drives, although their storage box option is probably better value over about two GB.
I plan to switch over later when it makes sense to - the nice thing about Backblaze is that it scales with your storage, whereas with Hetzner you have to jump from 1 TB to 5 TB.
But the thing is: B2 is cheap for storage, but retrieval and traversal are very expensive. And if that happens transparently on the filesystem (because you accidentally run grep or the service in question regularly hashes the files or something), you would implicitly download everything stored. And IIRC retrieval costs ten times the storage costs… each time.)
Backblaze recently revised B2 storage, it’s gone up by $1 and they offer free egress equal to I believe 2x or 3x the data you have stored with them. Let me try find a link to this…
I’ve had better experiences with rclone mount vs s3fs, but the experience isn’t great. It’s not a posix filesystem so stuff like file locking won’t work. Which makes it dangerous to run databases (incl SQLite) on top of.
Rclone has options to enable a local cache as well. It helps with performance, but I haven’t tested the performance too much yet.
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !selfhosted@lemmy.world
A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don’t control.
Rules:
Be civil: we’re here to support and learn from one another. Insults won’t be tolerated. Flame wars are frowned upon.
No spam posting.
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it’s not obvious why your post topic revolves around selfhosting, please include details to make it clear.
Don’t duplicate the full text of your blog or github here. Just post the link for folks to click.
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
No trolling.
Resources:
selfh.st Newsletter and index of selfhosted software and apps
My use case is different (B2 for offsite data backups) but I went down the path of rclone and it has been working out very well. A lot better than I expected. I read their official docs (installation, usage, the backblaze setup and crypt) to get started. Played at the command line for a few minutes and realize how quick and easy it was. Wrote a script to automate it and off to the races I went!
What’s your use-case? What do you want to achieve?
Using blob storages as filesystems doesn’t work well and could - with B2’s pricing structure - became excessively expensive. Blob storages are designed for easily writing and reading individual blobs. Filesystems are designed for random access, listing, traversal, etc.
It’s for storing a few terabytes of fairly static media (for the most part, write-once). The codebases using it don’t natively support object storage (and will be in Docker containers).
It’s on a Hetzner server, and Backblaze (even after the price increase) will be a lot cheaper than normal drives, although their storage box option is probably better value over about two GB.
I have a similar use case and whilst I’m not quite there yet I’m intending to go with a storage box
I plan to switch over later when it makes sense to - the nice thing about Backblaze is that it scales with your storage, whereas with Hetzner you have to jump from 1 TB to 5 TB.
Oh I know, I had the same thoughts
Object storage does really bad as a filesystem and it may very seriously be cheaper for you to have a 5TB storage box over 2TB of object storage
S3 has no list operation, so any time the computer wants to “list” a directory in object storage that is expensive
But the thing is: B2 is cheap for storage, but retrieval and traversal are very expensive. And if that happens transparently on the filesystem (because you accidentally run
grepor the service in question regularly hashes the files or something), you would implicitly download everything stored. And IIRC retrieval costs ten times the storage costs… each time.)This is no longer the case. Starting in October egress is free up to 3x the volume stored with them.
Oh wow. Good to know! Thanks!
Yup: Backblaze Product and Pricing Updates
Backblaze recently revised B2 storage, it’s gone up by $1 and they offer free egress equal to I believe 2x or 3x the data you have stored with them. Let me try find a link to this…
https://www.backblaze.com/blog/2023-product-announcement/
I’ve had better experiences with rclone mount vs s3fs, but the experience isn’t great. It’s not a posix filesystem so stuff like file locking won’t work. Which makes it dangerous to run databases (incl SQLite) on top of.
Rclone has options to enable a local cache as well. It helps with performance, but I haven’t tested the performance too much yet.