Yes, mostly https://gpt4all.io/ only to find out that even the “uncensored” models are bullshit and won’t even provide you with a Windows XP Pro key. That’s kind of my benchmark for models nowadays. :P
It’s good for me because I’m piss poor at programming. In my defense, I’m not a programmer or even programmer adjacent. I do see how it wouldn’t be useful to a pro. It also has occasionally given me garbage advice that an expert would spot right away while I had to figure out in my own that it was ‘hallucinating’ again. There’s nothing better for learning than troubleshooting, though!
I can absolutely see it getting useful for a pro. It’s already a better version of IDE templates. If you have to write boilerplate code this can already do that. It’s a huge time saver for the things you’d have to go look up to remember how to do and piece together yourself.
Example: today I wanted a quick way to serve my current working directory over HTTP so I could do some quick web work. I asked ChatGPT to write me a bash function I could stick in my profile to do this, and I told it to pick a random unused port. That would have taken me much longer had I went to lookup how to do that all. The only hint I gave it was to use the Python builtin module for serving http.
There’s a project called Tabby that your can host as a server on a machine that has a GPU, and has a VSCode extension that connects to the server.
The default model is called starcoder, and it’s the small version, 1B parameters. The downside is that it’s not super smart (but still an improvement over built in tools), but since it’s such a small model, I’m getting sub-second processing times.
I haven’t tried any of them but I did just listen to a podcast the other week where they talk about LlamaGPT vs Ollama and other related tools. If you’re interested it’s episode 540: Uncensored AI on Linux by Linux Unplugged
Dbzero Lemmy has a relationship with the Horde AI shared LLM group. My primary use is for chat roleplay but they have streamlined guides to hosting your own models for personal or horde use. One of the primary interfaces is SillyTavern but they integrate numerous models
I tired a bunch, but current state of the art is text-generation-webui, which can load multiple models and has a workflow similar to stable-diffusion-webui.
Absolutely yes. You can try GPT4ALL which works on any decent CPU computer (the minimum I managed to run it with is a 2018 6 core 2.0ghz ARM64 processor) and has a lot of built in models. You can also import uncensored models (like the TheBloke ones on Huggingface ).
I also tried AutoGPT some times ago which is quite complex and cool.
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !selfhosted@lemmy.world
A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don’t control.
Rules:
Be civil: we’re here to support and learn from one another. Insults won’t be tolerated. Flame wars are frowned upon.
No spam posting.
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it’s not obvious why your post topic revolves around selfhosting, please include details to make it clear.
Don’t duplicate the full text of your blog or github here. Just post the link for folks to click.
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
No trolling.
Resources:
selfh.st Newsletter and index of selfhosted software and apps
Checkout ollama.
There’s a lot of models you can pull from the official library.
Using ollama, you can also run external gguf models found on places like huggingface if you use a modelfile with something as simple as
I’m actively using ollama with docker to run llama2:13b model. It’s generally works fine but heavy on resources as expected.
Yes, mostly https://gpt4all.io/ only to find out that even the “uncensored” models are bullshit and won’t even provide you with a Windows XP Pro key. That’s kind of my benchmark for models nowadays. :P
Will it tell you how to make meth?
If you want to be able to use your models from everywhere sefurely, then koboldcpp on the ai horde is your best option. Super easy to set up
deleted by creator
It’s good for me because I’m piss poor at programming. In my defense, I’m not a programmer or even programmer adjacent. I do see how it wouldn’t be useful to a pro. It also has occasionally given me garbage advice that an expert would spot right away while I had to figure out in my own that it was ‘hallucinating’ again. There’s nothing better for learning than troubleshooting, though!
deleted by creator
I can absolutely see it getting useful for a pro. It’s already a better version of IDE templates. If you have to write boilerplate code this can already do that. It’s a huge time saver for the things you’d have to go look up to remember how to do and piece together yourself.
Example: today I wanted a quick way to serve my current working directory over HTTP so I could do some quick web work. I asked ChatGPT to write me a bash function I could stick in my profile to do this, and I told it to pick a random unused port. That would have taken me much longer had I went to lookup how to do that all. The only hint I gave it was to use the Python builtin module for serving http.
I’ve found it’s pretty good for translating between steps so to speak.
Converted some bash to python relatively quickly by giving it snippets and fixing errors as it made them.
I also had success generating an ansible playbook based on my own previously written install instructions for SillyTavern and llama.cpp.
I could do both of those tasks myself, but thar would be more difficult than having a mostly correct translation and fixing some errors.
You should make sure you are running a model that fits in your vram, for me it runs faster than any online LLM I’ve tried.
There’s a project called Tabby that your can host as a server on a machine that has a GPU, and has a VSCode extension that connects to the server.
The default model is called starcoder, and it’s the small version, 1B parameters. The downside is that it’s not super smart (but still an improvement over built in tools), but since it’s such a small model, I’m getting sub-second processing times.
deleted by creator
Not with success but I’m using huggingface since a couple of days. You may want to have a look into it
I’ve played around with a few of them. I’ve found LM Studio the most robust and user friendly.
deleted by creator
Mixtral is an amazing one that isn’t super slow or require incredible hardware foe a decent speed.
In general this guy has really good videos/tutorials for the latest tools.
Here is an alternative Piped link(s):
Mixtral
Piped is a privacy-respecting open-source alternative frontend to YouTube.
I’m open-source; check me out at GitHub.
I haven’t tried any of them but I did just listen to a podcast the other week where they talk about LlamaGPT vs Ollama and other related tools. If you’re interested it’s episode 540: Uncensored AI on Linux by Linux Unplugged
“Uncensored” models are bullshit everything but uncensored. Just ask them for a Windows XP Pro key and you’ll see how uncensored they really are.
Recntly started using HuggingChat 🤗
Huggingchat for image generation is beautiful beautiful nightmare fuel.
I seriously love it.
I do image generation on AUTOMATIC1111
Really happy that i switched the text ai to something more opened that CloseAI
Dbzero Lemmy has a relationship with the Horde AI shared LLM group. My primary use is for chat roleplay but they have streamlined guides to hosting your own models for personal or horde use. One of the primary interfaces is SillyTavern but they integrate numerous models
ollama + codellama works perfect, I use it from neovim with a plug-in called gen-nvim I think
I tired a bunch, but current state of the art is
text-generation-webui, which can load multiple models and has a workflow similar tostable-diffusion-webui.https://github.com/oobabooga/text-generation-webui
I’ve tried both this and https://github.com/jmorganca/ollama. I liked the latter a lot more; just can’t remember why.
GUI for ollama is a separate project: https://github.com/ollama-webui/ollama-webui
Absolutely yes. You can try GPT4ALL which works on any decent CPU computer (the minimum I managed to run it with is a 2018 6 core 2.0ghz ARM64 processor) and has a lot of built in models. You can also import uncensored models (like the TheBloke ones on Huggingface ).
I also tried AutoGPT some times ago which is quite complex and cool.