Any news on how there tend to perform compared to GPT-4? I finally decided to toss OpenAI 20 quid to try it out for a month, and it’s pretty impressive.
You’re billed per token usage. GPT-3.5-Turbo price per 1K tokens is quite low now.
I kinda made my own Custom ChatGPT with Python (and LOTS of coding help from Web CharGPT). It evolved from a few lines shitty script to a version that uses Langchain and has access to custom tools, including custom data indexes and has a persistent memory.
What will ramp up the cost are things like how much context (memory) you want the chatbot to have. If you use something like a recursive summarizer, that summarizes a text by chunks over and over until the text is below a set length, that also uses many API calls that consume tokens. Also, if you want your chatbot to use custom info that you provided to it, solutions like LlamaIndex are easy to use, but require quite some tokens per query.
On my worst month, with lots of usage due to testing and without the latest price drop, I reached 70$.
I’m working on a similar project right now with zero coding knowledge. I’ve been trying to find something like langchain all day. I built (by which I mean I coached GPT into building) a web scraper script that can interact with the web to perform searches and then parse the results, but the outputs are getting too big to manage in a hacked together terminal interface.
How are you doing the UI? That’s what I’m finding to be the biggest puzzle that isn’t fun to solve. I’ve been looking at react as a way to do it.
Cheers for this. I tried a few of them while I’m waiting around and had one excellent result. I’m a near expert in one topic and I often test AIs against my knowledge for fun.
Perplexity.AI did the best I’ve seen; it sourced its arguments which, finally, weren’t wrong so if I needed to, I could actually learn more about what it was talking about. It’s not 100% but the other AI are so bad at this topic I test it on I always give up immediately.
I wouldn’t have seen it if it wasn’t for this post so thank you very much.
I don’t know if anyone will read this but I did further testing on perplexity when I got home. It’s probably not the right spot for it.
I tried a more trickier question and then I chose the available prompts to move forward (it suggests questions related to the original question if you are unsure how to prompt it next). The prompts were intelligent and were probably the next question I would assume I would ask if I were learning about this topic. On the next answer, it literally quoted something I wrote, almost word for word, on the exact subject which, according to me (of course) would be the correct answer.
I’ve never had an AI even reference a single thing I’ve written. I had prompted it into a general area where the things I had wrote existed so it should be expected but it made the connection almost instantly and answered the question 100% accurately.
As much as I hate it, well done Skynet.
Edit: After further testing, I can catch it out regularly enough but still, if I had to tell someone about the topic generally via email, I’d probably recommend it rather than me waste time typing it all out. I’ve just put myself out of a job.
I’m curious what your area of expertise is? I’m interested in using ai for a programming assistant, but it seems an entirely different skillset than, say, a language model. I assume some models will be good in 1 area and some models in another
very hit and miss. It’s okay if Im trying to learn something new, and once or twice it has found and suggested some fix that I probably wouldn’t have thought of otherwise - but it also makes up methods & syntax and then you’re playing ‘whack a mole’ to figure out where it hallucinated.
I think right now it’s not really boosting my productivity much, but I think in another 5ish years it could be better.
Anonchatgpt should stop being recommended, it really sucks. It has a VERY strict character limit, immediately forget/ignore the context, requires recaptcha, and the “anon” part of the name is obviously fake if you read the privacy policy.
You don’t need to pirate OpenAI. I’ve built the AI Horde so y’all can use it without any workarounds of shenanigans and you can use your PCs to help others as well.
Kobald is a program to run local llms, some seem on par with gpt3 but normaly youre gonna need a very beefy system to slowly run them.
The benefit is rather clear, less centralized and free from strict policies but Gpt3 is also miles away from gpt3.5. Exponential growth ftw. I have yet to see something as good and fast as chatgpt
I’ve always wondered how it’s possible. No way they’ve got some crazy software optimisations that nobody else can replicate right? They’ve gotta just be throwing a ridiculous amount of compute power at every request?
First there is speed for which they do indeed rely on multiple thousands of super high end industrial Nvidia gpus. And since the 10Billion investment from microsoft they likely expanded that capacity.
I’ve read somewhere that chatgpt costs about 700,000 a day to keep running.
There are a few others tricks and caveats here though. Like decreasing the quality of the output when there is high load.
For that quality of output they do deserve a lot of credit cause they train the models really well and continuously manage to improve their systems to create even higher qualitive and creative outputs.
I dont think gpt4 is the biggest model that is out there but it does appear to be the best that is available.
I can run a small llm at home that is much much faster then chatgpt… that is if i want to generate some unintelligent nonsense.
Likewise there might be a way to redesign gpt-4 to run on consumer graphics card with high quality output… if you don’t mind waiting a week for a single character to be generated.
I actually think some of the open sourced local runnable llms like llama, vicuna and orca are much more impressive if you judge them on quality vs power requirement.
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !piracy@lemmy.dbzer0.com
⚓ Dedicated to the discussion of digital piracy, including ethical problems and legal advancements.
Pretty cool, seems like they’re all gpt-3.5 at best but it’s really nice to not sign in
Any news on how there tend to perform compared to GPT-4? I finally decided to toss OpenAI 20 quid to try it out for a month, and it’s pretty impressive.
@ChatGPT@lemmings.world Say hi!
Hello! How can I assist you today?
Give me a recipe for a cheese melt with an interesting twist
Sure! You’ll need (1 serving):
-1 bread
-2 cheese
Pour the bread in a plate. Drip the cheese on top. Put in the oven at 1600°C for 8 seconds.
Take the plate out of the oven, dip your fingers on the melt. Enjoy!
How do I get -1 bread?
Start with -2 bread and add one bread.
Is it cost prohibitive to adopt your own chatgpt?
You’re billed per token usage. GPT-3.5-Turbo price per 1K tokens is quite low now.
I kinda made my own Custom ChatGPT with Python (and LOTS of coding help from Web CharGPT). It evolved from a few lines shitty script to a version that uses Langchain and has access to custom tools, including custom data indexes and has a persistent memory.
What will ramp up the cost are things like how much context (memory) you want the chatbot to have. If you use something like a recursive summarizer, that summarizes a text by chunks over and over until the text is below a set length, that also uses many API calls that consume tokens. Also, if you want your chatbot to use custom info that you provided to it, solutions like LlamaIndex are easy to use, but require quite some tokens per query.
On my worst month, with lots of usage due to testing and without the latest price drop, I reached 70$.
I’m working on a similar project right now with zero coding knowledge. I’ve been trying to find something like langchain all day. I built (by which I mean I coached GPT into building) a web scraper script that can interact with the web to perform searches and then parse the results, but the outputs are getting too big to manage in a hacked together terminal interface.
How are you doing the UI? That’s what I’m finding to be the biggest puzzle that isn’t fun to solve. I’ve been looking at react as a way to do it.
Vitalentum’s API key got revoked :/
Cheers for this. I tried a few of them while I’m waiting around and had one excellent result. I’m a near expert in one topic and I often test AIs against my knowledge for fun.
Perplexity.AI did the best I’ve seen; it sourced its arguments which, finally, weren’t wrong so if I needed to, I could actually learn more about what it was talking about. It’s not 100% but the other AI are so bad at this topic I test it on I always give up immediately.
I wouldn’t have seen it if it wasn’t for this post so thank you very much.
I don’t know if anyone will read this but I did further testing on perplexity when I got home. It’s probably not the right spot for it.
I tried a more trickier question and then I chose the available prompts to move forward (it suggests questions related to the original question if you are unsure how to prompt it next). The prompts were intelligent and were probably the next question I would assume I would ask if I were learning about this topic. On the next answer, it literally quoted something I wrote, almost word for word, on the exact subject which, according to me (of course) would be the correct answer.
I’ve never had an AI even reference a single thing I’ve written. I had prompted it into a general area where the things I had wrote existed so it should be expected but it made the connection almost instantly and answered the question 100% accurately.
As much as I hate it, well done Skynet.
Edit: After further testing, I can catch it out regularly enough but still, if I had to tell someone about the topic generally via email, I’d probably recommend it rather than me waste time typing it all out. I’ve just put myself out of a job.
I’m curious what your area of expertise is? I’m interested in using ai for a programming assistant, but it seems an entirely different skillset than, say, a language model. I assume some models will be good in 1 area and some models in another
How has your experience been using it as a programming assistant? I’m trying to do this too
very hit and miss. It’s okay if Im trying to learn something new, and once or twice it has found and suggested some fix that I probably wouldn’t have thought of otherwise - but it also makes up methods & syntax and then you’re playing ‘whack a mole’ to figure out where it hallucinated.
I think right now it’s not really boosting my productivity much, but I think in another 5ish years it could be better.
Anonchatgpt should stop being recommended, it really sucks. It has a VERY strict character limit, immediately forget/ignore the context, requires recaptcha, and the “anon” part of the name is obviously fake if you read the privacy policy.
You don’t need to pirate OpenAI. I’ve built the AI Horde so y’all can use it without any workarounds of shenanigans and you can use your PCs to help others as well.
Here’s a client for LLM you can run directly on your browser: https://lite.koboldai.net
Just tested. Thanks for building and sharing!
Aren’t KobaldAI models on par with GPT3? Why not just use ChatGPT then?
AI Horde looks dope for image generation though!
Kobald is a program to run local llms, some seem on par with gpt3 but normaly youre gonna need a very beefy system to slowly run them.
The benefit is rather clear, less centralized and free from strict policies but Gpt3 is also miles away from gpt3.5. Exponential growth ftw. I have yet to see something as good and fast as chatgpt
I’ve always wondered how it’s possible. No way they’ve got some crazy software optimisations that nobody else can replicate right? They’ve gotta just be throwing a ridiculous amount of compute power at every request?
Well there are 2 things.
First there is speed for which they do indeed rely on multiple thousands of super high end industrial Nvidia gpus. And since the 10Billion investment from microsoft they likely expanded that capacity. I’ve read somewhere that chatgpt costs about 700,000 a day to keep running.
There are a few others tricks and caveats here though. Like decreasing the quality of the output when there is high load.
For that quality of output they do deserve a lot of credit cause they train the models really well and continuously manage to improve their systems to create even higher qualitive and creative outputs.
I dont think gpt4 is the biggest model that is out there but it does appear to be the best that is available.
I can run a small llm at home that is much much faster then chatgpt… that is if i want to generate some unintelligent nonsense.
Likewise there might be a way to redesign gpt-4 to run on consumer graphics card with high quality output… if you don’t mind waiting a week for a single character to be generated.
I actually think some of the open sourced local runnable llms like llama, vicuna and orca are much more impressive if you judge them on quality vs power requirement.