It writes more informative commits than I could ever make so I’m just reading what it says and mostly copy/pasting completely most of the time, I write all of the changes I’ve made into an LLM with a large context window and it write a very detailed commit not just with a title but with bullet points describing each of the changes precisely
Okay, so to be honest, at first, I didn’t understand all the ❌negativity, but I shared this with a friend to get her take on the issue, and she 🕵️♀️clued me into the fact that webpack already does this with copilot and pull requests, and the results . . . speak for themselves.
Wow😅. I didn’t think it would be that bad. It seems that every example I find is just incorrect. I’ll look at the code. It will be a two line change, but the summary will be difficult to follow and often says things that are not part of the code changes. Then, there are also contradictions which make the pull requests harder to follow than if you just read the code with no other context. Darn it. I really thought this was a 🧊cool idea.
I’m definitely going to be sticking to writing my own commits as always.
y’all need to speak for your own companies. obviously some companies will not allow it, and I’d be personally skeptical of allowing it if I ran a company - but I also work at a place that effectively has given a quiet go-ahead to use it, with objectively talented engineers regularly making use of LLMs for boilerplate and other aspects of work.
obviously, there’s some calculus on when to use it, and you better damn inspect your outputs, but treating as a blanket rule that OP is a terrible employee at their company when you don’t know the company is rude as hell and uncalled for.
I think a lot of people in this thread are just upset/projecting because this is the first real hint that they’re not as much of a special-boy-programmer as they think. OP’s use case is fairly limited in scope, using the LLM for something it is actually pretty good for, and never implied he doesn’t check the output. They’ll never admit it, and will deflect, but they’re just worried.
Precision > concision && accuracy > concision. Just use your own wording as the commit message. I’d rather see an account of a code change from the viewpoint of the change’s author than a shorter reformulation, even if that reformulation did come from a human who knew the problem space and wasn’t prone to making shit up on the fly.
I’d rather see no commit message than an AI-generated one.
Also if I wasn’t misinterpreting OP, it sounded from the post I was responding to like OP provided a summary to the LLM along with code. If OP’s writing a summary anyway, why not just proofread that and use that as the commit message rather than involving an LLM in the middle of the process?
Even in a hypothetical where the company hired human tech writers to write commit messages for developers, I’d rather have in the commit message what the developer had to say rather than the possible misinterpretation of the tech writer.
Of course he knows why he made the changes. He made them. But computers are much faster as typing and with a sophisticated enough LLM you can offload some gruntwork. I’d argue if you’re not utilizing all the tools at your disposal, you’re not performing like you should.
we are talking about programmers/software developers. Speed typing is in our nature. If you can’t type 50 characters faster than writing the prompt to get the LLM to spat out a commit meesage, then I will question your competence.
Also the opposite is true, when the Boss realise that major parts or grunt work as you say can be automated by utilising LLM, then they would be inclined to employ that and reduce your wage. Because do you really deserve 6 figures when a computer can do the grunt work for their expensive human resources?
If you were on my team and I knew you were doing this, aside from all the other issues everybody else has mentioned in this thread, I’d be going out of my way not just to check every single one of your git commits (both code changes and commit messages) for inaccuracies but also also to find every possible reason to nit pick everything you committed.
You shouldn’t be using LLMs to write your git commits (messages or code changes.) They hallucinate. But if you are going to use them, you need to spend much more time proofreading what they output than you’d spend writing it yourself. Check every single word for errors. (And, honestly, make the text fit with the way other commits by your team read (assuming you are on a team).)
In short, if you’re going to use LLMs, DO NOT TRUST ANYTHING THEY GIVE YOU. And don’t be surprised if you get negative blowback from others for using them at all. Keep in mind what can happen if you trust LLMs.
IMHO, the provided link is largely irrelevant to this topic. It is about lawyers who used ChatGPT as a search enginehttps://youtu.be/oqSYljRYDEM?t=1436, which is not what it is for, and it will tell you that over and over again. The lawyers in question were not even “trusting ChatGPT”. They blatantly and actively disregarded ChatGPT telling them that it was not a search engine and could not provide legal advice https://youtu.be/oqSYljRYDEM?t=1466
This topic is about using LLMs to generate natural language describing code changes that it is provided with which is not only completely different than using an LLM as a defacto search engine, but it is also something LLMs are actually meant to do: autocomplete. This topic is more akin to using LLMs to write title headings for legal documents which are already basically complete as is than it is akin to the link provided.
You realize you don’t need to list all your changes in the commit message, right? Anyone can blame or diff said changes.
The commit message is meant to be used for the high level stuff, the intent, representing / connecting progress towards a larger work item, and other important context from outside the codebase. Insert other reasons that aren’t saying literally what was changed if you feel I have missed something.
Also one should use their time better if they are spending so much time writing commit messages they feel the need to automate it. Commit messages are rarely read ever again (once merged, lets say), it is not okay to be spending a lot of time on them. That’s not an excuse to write bad commit messages, but you have to balance the time cost with expected utility.
And an addendum to the above. Describing what you did without reasoning, context, or other information that isnt captured within the changes itself makes your commit messages entirely useless. It makes IDE-inserted in-line blame information useless as well. Thus you are now wasting all the time you spend on commit messages, even if you spend less because it’s automated.
Yeah, good commit messages are about intent and context of a change - not what the change itself is. We can look at the diff for that. Just write a single line or two summarizing what the commit does, and everything else should be adding context on top that doesn’t directly exist in the codebase.
If a glorified autocomplete algorithm can write more informative and concise commit messages than you, the actual author behind the code, I think you need to sit down and think long and hard what that actually implies.
That an AI might be better at writing documentation than the average dev, who is largely inept at writing good documentation?
Understandably, as technical writing isn’t exactly a focus point or career growing thing for most devs. If it was, we would be writing much better code as well.
I’ve seen my peers work, they could use something like this. I’d welcome it.
I disagree. I am capable of writing a good commit message, I just don’t really want to. Depending on the change, formulating a concise text that includes all relevant information can be quite time consuming.
I can travel to Italy on foot. Will I do that? Of course not. What does it imply? That I’m incapable of moving my body? Of course not! It just means I’m too lazy to do it and that there are faster ways to get to the same goal.
I disagree. I am capable of writing a good commit message, I just don’t really want to. Depending on the change, formulating a concise text that includes all relevant information can be quite time consuming.
formulating a concise text that includes all relevant information
how do you write Pull-Requests? Just half-assing it? Writing good commit messages help you write a good PR as well (as it can be just a summary of the things you commited) But yeah, you do you.
There is a saying, which I don’t generally agree with. Those who can, do. Those who can’t, teach. That said, writing good code and describing code effectively are two different skills, and there is no guarantee any given individual will have both.
Why do you need to spend a “considerable amount of time crafting commit messages?” That feeding your code into an LLM and getting it to summarise for you is faster? I don’t understand how this could possibly streamline anyone’s workflow? What do your commit messages normally look like?
So what I get from this is that some people need to be forced to write decent commit messages.
Echoing what others have mentioned, commit messages need to document why something was changed and put it into the context of the project. You should do this even for private projects, just so 1) you build good habits and 2) if you let the project rest for a while you don’t need to figure put everything from the start again.
it’d be interesting to see some examples of what your script came up with - I’m a bit skeptical of what an AI would come up with in terms of a commit message, and I’d think you’d need a pretty complex system to get commit messages to be maximally useful. I’ve found LLMs can stray towards being too high level and struggle if you ask more specific questions.
but I could also see it as being helpful for a sort of audit log for what changed, and I don’t think it would be too harmful, as long as you’re checking what the LLM is generating and making sure there’s corresponding code changes, that it’s not hallucinating etc.
hard to tell without examples - perhaps you could expand your post with some?
an aside, sorry you got such an overwhelmingly toxic response. the amount of angry people on this platform who feel the need to morally educate everyone around them objectively sucks and makes it a really unpleasant place to be.
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !programming@programming.dev
Welcome to the main community in programming.dev! Feel free to post anything relating to programming here!
Cross posting is strongly encouraged in the instance. If you feel your post or another person’s post makes sense in another community cross post into it.
Hope you enjoy the instance!
Rules
Rules
Follow the programming.dev instance rules
Keep content related to programming in some way
If you’re posting long videos try to add in some form of tldr for those who don’t want to watch videos
Is this an open source project?
Okay, so to be honest, at first, I didn’t understand all the ❌negativity, but I shared this with a friend to get her take on the issue, and she 🕵️♀️clued me into the fact that webpack already does this with copilot and pull requests, and the results . . . speak for themselves.
Wow😅. I didn’t think it would be that bad. It seems that every example I find is just incorrect. I’ll look at the code. It will be a two line change, but the summary will be difficult to follow and often says things that are not part of the code changes. Then, there are also contradictions which make the pull requests harder to follow than if you just read the code with no other context. Darn it. I really thought this was a 🧊cool idea.
I’m definitely going to be sticking to writing my own commits as always.
Sounds like an incoming resume generating event.
so it reads proprietary code?
I mean, if it’s self-hosted it’s probably fine
I don’t think that Lead and Projectmanager will love to hear that. At all.
y’all need to speak for your own companies. obviously some companies will not allow it, and I’d be personally skeptical of allowing it if I ran a company - but I also work at a place that effectively has given a quiet go-ahead to use it, with objectively talented engineers regularly making use of LLMs for boilerplate and other aspects of work.
obviously, there’s some calculus on when to use it, and you better damn inspect your outputs, but treating as a blanket rule that OP is a terrible employee at their company when you don’t know the company is rude as hell and uncalled for.
I think a lot of people in this thread are just upset/projecting because this is the first real hint that they’re not as much of a special-boy-programmer as they think. OP’s use case is fairly limited in scope, using the LLM for something it is actually pretty good for, and never implied he doesn’t check the output. They’ll never admit it, and will deflect, but they’re just worried.
deleted by creator
Great, I still write “updated stuff”. Maybe I’ll try doing that instead.
Why don’t you know WHY you did something to the codebase that you now want to commit?
Relying on an LLM for that sounds like a crutch that you need to get rid of ASAP!
Im more in favour in writting “my” commit message myself and let LLMs refotmulate and make concise.
Precision > concision && accuracy > concision. Just use your own wording as the commit message. I’d rather see an account of a code change from the viewpoint of the change’s author than a shorter reformulation, even if that reformulation did come from a human who knew the problem space and wasn’t prone to making shit up on the fly.
The problem is people are lazy and most places I’ve been, peoeple make bad commit messages and often very non informative.
I’d rather see no commit message than an AI-generated one.
Also if I wasn’t misinterpreting OP, it sounded from the post I was responding to like OP provided a summary to the LLM along with code. If OP’s writing a summary anyway, why not just proofread that and use that as the commit message rather than involving an LLM in the middle of the process?
Even in a hypothetical where the company hired human tech writers to write commit messages for developers, I’d rather have in the commit message what the developer had to say rather than the possible misinterpretation of the tech writer.
Of course he knows why he made the changes. He made them. But computers are much faster as typing and with a sophisticated enough LLM you can offload some gruntwork. I’d argue if you’re not utilizing all the tools at your disposal, you’re not performing like you should.
we are talking about programmers/software developers. Speed typing is in our nature. If you can’t type 50 characters faster than writing the prompt to get the LLM to spat out a commit meesage, then I will question your competence.
Also the opposite is true, when the Boss realise that major parts or grunt work as you say can be automated by utilising LLM, then they would be inclined to employ that and reduce your wage. Because do you really deserve 6 figures when a computer can do the grunt work for their expensive human resources?
If you were on my team and I knew you were doing this, aside from all the other issues everybody else has mentioned in this thread, I’d be going out of my way not just to check every single one of your git commits (both code changes and commit messages) for inaccuracies but also also to find every possible reason to nit pick everything you committed.
You shouldn’t be using LLMs to write your git commits (messages or code changes.) They hallucinate. But if you are going to use them, you need to spend much more time proofreading what they output than you’d spend writing it yourself. Check every single word for errors. (And, honestly, make the text fit with the way other commits by your team read (assuming you are on a team).)
In short, if you’re going to use LLMs, DO NOT TRUST ANYTHING THEY GIVE YOU. And don’t be surprised if you get negative blowback from others for using them at all. Keep in mind what can happen if you trust LLMs.
IMHO, the provided link is largely irrelevant to this topic. It is about lawyers who used ChatGPT as a search engine https://youtu.be/oqSYljRYDEM?t=1436, which is not what it is for, and it will tell you that over and over again. The lawyers in question were not even “trusting ChatGPT”. They blatantly and actively disregarded ChatGPT telling them that it was not a search engine and could not provide legal advice https://youtu.be/oqSYljRYDEM?t=1466
This topic is about using LLMs to generate natural language describing code changes that it is provided with which is not only completely different than using an LLM as a defacto search engine, but it is also something LLMs are actually meant to do: autocomplete. This topic is more akin to using LLMs to write title headings for legal documents which are already basically complete as is than it is akin to the link provided.
You realize you don’t need to list all your changes in the commit message, right? Anyone can blame or diff said changes.
The commit message is meant to be used for the high level stuff, the intent, representing / connecting progress towards a larger work item, and other important context from outside the codebase. Insert other reasons that aren’t saying literally what was changed if you feel I have missed something.
Also one should use their time better if they are spending so much time writing commit messages they feel the need to automate it. Commit messages are rarely read ever again (once merged, lets say), it is not okay to be spending a lot of time on them. That’s not an excuse to write bad commit messages, but you have to balance the time cost with expected utility.
And an addendum to the above. Describing what you did without reasoning, context, or other information that isnt captured within the changes itself makes your commit messages entirely useless. It makes IDE-inserted in-line blame information useless as well. Thus you are now wasting all the time you spend on commit messages, even if you spend less because it’s automated.
Yeah, good commit messages are about intent and context of a change - not what the change itself is. We can look at the diff for that. Just write a single line or two summarizing what the commit does, and everything else should be adding context on top that doesn’t directly exist in the codebase.
If a glorified autocomplete algorithm can write more informative and concise commit messages than you, the actual author behind the code, I think you need to sit down and think long and hard what that actually implies.
Don’t ever call me a glorified autocomolete algorithm again
deleted by creator
And what does it imply?
That an AI might be better at writing documentation than the average dev, who is largely inept at writing good documentation?
Understandably, as technical writing isn’t exactly a focus point or career growing thing for most devs. If it was, we would be writing much better code as well.
I’ve seen my peers work, they could use something like this. I’d welcome it.
it implies that the author of the code has no idea what they were doing.
I disagree. I am capable of writing a good commit message, I just don’t really want to. Depending on the change, formulating a concise text that includes all relevant information can be quite time consuming.
I can travel to Italy on foot. Will I do that? Of course not. What does it imply? That I’m incapable of moving my body? Of course not! It just means I’m too lazy to do it and that there are faster ways to get to the same goal.
how do you write Pull-Requests? Just half-assing it? Writing good commit messages help you write a good PR as well (as it can be just a summary of the things you commited) But yeah, you do you.
There is a saying, which I don’t generally agree with. Those who can, do. Those who can’t, teach. That said, writing good code and describing code effectively are two different skills, and there is no guarantee any given individual will have both.
Why do you need to spend a “considerable amount of time crafting commit messages?” That feeding your code into an LLM and getting it to summarise for you is faster? I don’t understand how this could possibly streamline anyone’s workflow? What do your commit messages normally look like?
deleted by creator
So already not imperative. I hope the model writes better message than you.
So what I get from this is that some people need to be forced to write decent commit messages.
Echoing what others have mentioned, commit messages need to document why something was changed and put it into the context of the project. You should do this even for private projects, just so 1) you build good habits and 2) if you let the project rest for a while you don’t need to figure put everything from the start again.
I’m really curious to see examples of commit messages.
And more details on how you achieved it, I’d like to give it a go myself, as some of my coworkers commit messages are less than stellar.
Does it write in the conventional commit formalism ?
deleted by creator
Wait, it can split commits ? I thought you “just” used a LLM to write the messages, but it can also act on the commits ?
How did you achieve this?
deleted by creator
it’d be interesting to see some examples of what your script came up with - I’m a bit skeptical of what an AI would come up with in terms of a commit message, and I’d think you’d need a pretty complex system to get commit messages to be maximally useful. I’ve found LLMs can stray towards being too high level and struggle if you ask more specific questions.
but I could also see it as being helpful for a sort of audit log for what changed, and I don’t think it would be too harmful, as long as you’re checking what the LLM is generating and making sure there’s corresponding code changes, that it’s not hallucinating etc.
hard to tell without examples - perhaps you could expand your post with some?
an aside, sorry you got such an overwhelmingly toxic response. the amount of angry people on this platform who feel the need to morally educate everyone around them objectively sucks and makes it a really unpleasant place to be.
This assumes anyone ever reads them.