Does anyone know why postgresql is so broken when it comes to upgrades? Why it doesn’t do an in-place upgrade of the DB automatically when starting the new version?
I’ve had enough problems with postgresql that I basically avoid anything using it unless I have no other options.
That’s the thing I don’t like about Postgres either. The performance is significantly better than with MariaDB but Postgres is such a pain for non-enterprise use.
Same with crash recovery, Postgres just can’t recover if the WAL is corrupted. MariaDB will happily fix itself but Postgres will just sit there and wait until somebody babysits it.
So you better spin up a second Postgres container, run pg_resetwal, restart the database and terminate any open transactions manually with a 2 page query you hopefully wrote down. Might reindex all tables as well to be sure.
I have a separate “postgres unfuck” script by now.
I like PSQL far more than Maria DB, but it is the most stupid software for upgrades. It is the reason that, whenever I can’t use SQLite, I use a NoSQL DB like Mongo - any single executable NoSQL that contains the entire DB to a single directory seems to be the common factor. Sometimes you might hit an API change, but I think the number of times I’ve had a production application break because of a NoSQL DB server software upgrade is still at 0.
I’ve spend more than a decade supporting both Postgres and MongoDB in production.
While they each have quirks, I prefer the quirks of Postgres.
I just spent a massive amount of time retooling code to deal with a MongoDB upgrade. The code upgrade is so complex because that’s where the schema is defined. No wonder MongoDB upgrades are easier— the database has externalized a lot of complexity that now becomes some coders problem to deal with.
MongoDB does have that annoying quirk where it creates several huge files even with a small amount of data in the DB itself. But at least it can be upgraded.
SQLite is really the only one I’ve used that doesn’t bother me in some way.
Agreed, SQLite 💗. It does have limitations when you need to scale with remote connections and concurrency; then you have to start bringing in layers, and it’s really not designed for that. For those jobs, it’s just better IMO to reach to something designed for that use case to begin with.
For minor version upgrades, the database remains binary compatible. Nothing to do.
The dump/restore required during major upgrades allows format changes which enable new features and performance improvements without dragging around cruft forever to stay backwards compatible.
For professionals running PostgreSQL clusters in production there is a way to cycle in the new server version with zero user-visible downtime.
pg_dump needs to connect to a running PostgreSQL instance.
But if you upgrade the binaries and try to start up, you can’t because the old data format doesn’t work. Because you can’t start up, pg_dump can’t connect.
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !selfhosted@lemmy.world
A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don’t control.
Rules:
Be civil: we’re here to support and learn from one another. Insults won’t be tolerated. Flame wars are frowned upon.
No spam posting.
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it’s not obvious why your post topic revolves around selfhosting, please include details to make it clear.
Don’t duplicate the full text of your blog or github here. Just post the link for folks to click.
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
No trolling.
Resources:
selfh.st Newsletter and index of selfhosted software and apps
Does anyone know why postgresql is so broken when it comes to upgrades? Why it doesn’t do an in-place upgrade of the DB automatically when starting the new version?
I’ve had enough problems with postgresql that I basically avoid anything using it unless I have no other options.
That’s the thing I don’t like about Postgres either. The performance is significantly better than with MariaDB but Postgres is such a pain for non-enterprise use.
Same with crash recovery, Postgres just can’t recover if the WAL is corrupted. MariaDB will happily fix itself but Postgres will just sit there and wait until somebody babysits it.
So you better spin up a second Postgres container, run
pg_resetwal, restart the database and terminate any open transactions manually with a 2 page query you hopefully wrote down. Might reindex all tables as well to be sure.I have a separate “postgres unfuck” script by now.
I like PSQL far more than Maria DB, but it is the most stupid software for upgrades. It is the reason that, whenever I can’t use SQLite, I use a NoSQL DB like Mongo - any single executable NoSQL that contains the entire DB to a single directory seems to be the common factor. Sometimes you might hit an API change, but I think the number of times I’ve had a production application break because of a NoSQL DB server software upgrade is still at 0.
I’ve spend more than a decade supporting both Postgres and MongoDB in production.
While they each have quirks, I prefer the quirks of Postgres.
I just spent a massive amount of time retooling code to deal with a MongoDB upgrade. The code upgrade is so complex because that’s where the schema is defined. No wonder MongoDB upgrades are easier— the database has externalized a lot of complexity that now becomes some coders problem to deal with.
MongoDB does have that annoying quirk where it creates several huge files even with a small amount of data in the DB itself. But at least it can be upgraded.
SQLite is really the only one I’ve used that doesn’t bother me in some way.
Agreed, SQLite 💗. It does have limitations when you need to scale with remote connections and concurrency; then you have to start bringing in layers, and it’s really not designed for that. For those jobs, it’s just better IMO to reach to something designed for that use case to begin with.
For minor version upgrades, the database remains binary compatible. Nothing to do.
The dump/restore required during major upgrades allows format changes which enable new features and performance improvements without dragging around cruft forever to stay backwards compatible.
For professionals running PostgreSQL clusters in production there is a way to cycle in the new server version with zero user-visible downtime.
It’s just weird that it doesn’t do the dump-restore automatically when it detects data from an old version.
I think there is a catch-22.
pg_dump needs to connect to a running PostgreSQL instance.
But if you upgrade the binaries and try to start up, you can’t because the old data format doesn’t work. Because you can’t start up, pg_dump can’t connect.
Yeah they would need to make it backwards compatible.
Fantastic, thanks for this. Any reason you didn’t go to 17?
No particular reason. Just chose one and went with it.