When you hit that config need the next step is light weight VM + pcie passthru.

Pretty much the exact reason containerized environments were created.