I also struggled to get it to work with an Intel Celeron N5100. To get it to work I followed the instructions in the Jellyfin documentation. It seems like there are some additional steps for some versions of Intel CPUs, could it be that yours is affected? After enabling “Low-Power Encoding” it worked as expected.
Yes, I have it installed as a Docker container in a Debian 12 machine. My Docker compose file is something like this:
The group numbers were obtained following Jellyfin’s documentation.
You also need to configure Jellyfin from Menu > Playback. In “Hardware acceleration” I selected “Intel QuickSync (QSV)”. I have selected all but AV1 (not supported by my CPU) from “Enable hardware decoding from:” and from “Hardware encoding options:” I have enable all 3 of them.