Outside of English, ChatGPT makes up words, fails logic tests, and can't do basic information retrieval.

Outside of English, ChatGPT makes up words, fails logic tests, and can’t do basic information retrieval.

JWBananas
link
fedilink
English
31Y

regular expressions on crack

How does it go?

“I know! I’ll use an LLM!”

“Now you have three problems.”

True. When you have a hammer, everything looks like a nail.

I haven’t actually implemented this yet, but I’ve been thinking about making a local file search program using an LLM. It would enable me to search for things in ways that are absolutely impossible with language-naive tools.

Here are a few examples of tasks I have in the past wanted to do, but was not able to:

  • Find references to the board game ‘go’ but please for the love of god do not return every case of the verb ‘go’. Also include passages that refer to the game but don’t mention it by name.

  • Find all references to foods, eating, or meals.

  • Find all the dialog lines of a specific character.

You can’t do any of that with simple search, or even with regular expressions. You need general language awareness.

I know that LLMs will not be perfect at these tasks, either (at least not the current ones), but I think they could be quite effective.

Create a post

A nice place to discuss rumors, happenings, innovations, and challenges in the technology sphere. We also welcome discussions on the intersections of technology and society. If it’s technological news or discussion of technology, it probably belongs here.

Remember the overriding ethos on Beehaw: Be(e) Nice. Each user you encounter here is a person, and should be treated with kindness (even if they’re wrong, or use a Linux distro you don’t like). Personal attacks will not be tolerated.

Subcommunities on Beehaw:


This community’s icon was made by Aaron Schneider, under the CC-BY-NC-SA 4.0 license.

  • 1 user online
  • 153 users / day
  • 312 users / week
  • 620 users / month
  • 2.26K users / 6 months
  • 1 subscriber
  • 3.36K Posts
  • 67.7K Comments
  • Modlog