If you’re lucky you just set it to the wrong version, mine uses 10.3.0 (see below).
I tried running the docker container first as well but gave up since there are seperate versions for cuda and rocm which comes packaged with this as well and therefor gets unnecessary big.
I am running it on Fedora natively. I installed it with the setup script from the top of the docs:
curl -fsSL https://ollama.com/install.sh | sh
After that i created a service file (also stated in the linked docs) so that it starts at boot time (so i can just boot my pc and forget it without needing to login).
The crucial part for the GPU in question (RX 6700XT) was this line under the [service] section:
Environment="HSA_OVERRIDE_GFX_VERSION=10.3.0"
As you stated, this sets the environment variable for rocm.
Also to be able to reach it from outside of localhost (for my server):
oh man… I’m such a dumb dumb … didn’t even try 10.3.0, now I did and the docker version works and is extremely fast, compared to a CPU… Thank you so much.
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !selfhosted@lemmy.world
A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don’t control.
Rules:
Be civil: we’re here to support and learn from one another. Insults won’t be tolerated. Flame wars are frowned upon.
No spam posting.
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it’s not obvious why your post topic revolves around selfhosting, please include details to make it clear.
Don’t duplicate the full text of your blog or github here. Just post the link for folks to click.
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
No trolling.
Resources:
selfh.st Newsletter and index of selfhosted software and apps
If you’re lucky you just set it to the wrong version, mine uses 10.3.0 (see below).
I tried running the docker container first as well but gave up since there are seperate versions for cuda and rocm which comes packaged with this as well and therefor gets unnecessary big.
I am running it on Fedora natively. I installed it with the setup script from the top of the docs:
curl -fsSL https://ollama.com/install.sh | shAfter that i created a service file (also stated in the linked docs) so that it starts at boot time (so i can just boot my pc and forget it without needing to login).
The crucial part for the GPU in question (RX 6700XT) was this line under the [service] section:
Environment="HSA_OVERRIDE_GFX_VERSION=10.3.0"As you stated, this sets the environment variable for rocm. Also to be able to reach it from outside of localhost (for my server):
Environment="OLLAMA_HOST=0.0.0.0"For me it was 0.0.0.0:11434
oh man… I’m such a dumb dumb … didn’t even try 10.3.0, now I did and the docker version works and is extremely fast, compared to a CPU… Thank you so much.
Glad i could help ;)