[HN Gopher] Gitlab AI is going head to head with GitHub Copilot
___________________________________________________________________
Gitlab AI is going head to head with GitHub Copilot
Author : sh_tomer
Score : 107 points
Date : 2023-06-29 17:58 UTC (5 hours ago)
(HTM) web link (about.gitlab.com)
(TXT) w3m dump (about.gitlab.com)
| ianhawes wrote:
| This looks very interesting, but as an FYI using LLMs to do
| machine translation is a terrible waste of resources. The example
| on their landing page has a "translate.py" sample which, albeit
| handy, is not something I would do beyond basic string
| translations.
| lucasmullens wrote:
| > using LLMs to do machine translation is a terrible waste of
| resources.
|
| Could you elaborate on that? LLMs seem perfectly suited for
| language tasks like translation. They don't seem particularly
| expensive either, especially compared to hiring a person.
| nonameiguess wrote:
| There are already machine-translation services trained and
| created specifically for that purpose. While it's an amazing
| realization that LLMs can do this and do it pretty well
| without having to be trained specifically for this one
| purpose, training something to do literally all text
| generation tasks is expensive compared to training something
| specifically to do language to language translation.
|
| For a maybe more obvious example, say that LLMs ever got good
| enough to do arbitrary precision arithmetic on numbers up to
| hundreds of digits. Would that be a good use of one when
| calculators can already do this and are far cheaper to
| produce? I guess it makes no difference from a free-tier
| consumer's perspective, but it's still more expensive even if
| you aren't personally paying the expense.
| og_kalu wrote:
| GPT-4 is a much much better translator than Google
| Translate and the like. You should absolutely be using GPT
| for translations especially for distant language pairs that
| quickly devolve into nonsense with Google Translate, Deepl
| etc
|
| https://www.reddit.com/r/Korean/comments/13lkh6c/gpt4_is_fa
| r...
|
| https://github.com/ogkalu2/Human-parity-on-machine-
| translati...
| sacred_numbers wrote:
| The quality difference is substantial. I don't care if it's
| wasteful to use something that has many uses for a
| supposedly narrow task (although I don't see translation as
| a particularly narrow task anymore than I see writing as a
| narrow task). I would gladly waste untold trillions of
| floating point operations for a 1% increase in translation
| quality. From my experiments, though, it's much higher than
| 1% increase in translation quality. And regardless of how
| wasteful the compute is, it's actually cheaper in terms of
| dollars. Using GPT-3.5 to translate Korean to English would
| cost about $11 per million words, based on the average
| characters per token of the small sample of text I gave it.
| DeepL (the best translation service I could find) costs $25
| per million characters, or for my sample text, about $64
| per million words. At $11 per million words I can have
| GPT-3.5 perform multiple translation passes and use it's
| own judgment to pick the best translation and STILL save
| money compared to DeepL.
| lolinder wrote:
| The original paper[0] that laid the foundation for modern
| LLMs was demonstrated on machine translation tasks. It's
| one of the primary use cases these architectures were
| designed for. What other types of models do you have in
| mind that outperform them?
|
| [0] "Attention Is All You Need"
| https://arxiv.org/pdf/1706.03762.pdf
| senko wrote:
| LLMs _are_ the machine-translation services created
| specifically for that purpose[0], it just turned out they
| 're very good at many other things!
|
| Your analogy would be like saying why use a computer to
| multiply numbers if you can calculate them using
| calculator, which is much cheaper. Sure, but if you already
| have a computer, no need to use a dedicated calculator as
| wel.
|
| [0] https://nlp.seas.harvard.edu/annotated-
| transformer/#results
| blibble wrote:
| it's not doing that, it has generated a dictionary with some
| values
|
| presumably to be used later to lookup words (crap approach, but
| it's an LLM, what do you expect)
|
| but it didn't bother to write any code, it's just data
|
| (plus it even managed to screw up the dictionary with double
| braces)
| renewiltord wrote:
| Going to be interesting. Github had the advantage that they were
| offering Copilot into an audience of people who just defaulted to
| them. Gitlab on the other hand is for people who are into license
| wars, etc. so the AI product is going to be offered to a hostile
| audience.
| lolinder wrote:
| These days Gitlab's main customer is the enterprise. They
| pushed away a lot of their open source users when they heavily
| curtailed the free tier.
| RadiozRadioz wrote:
| Crucial: do they train on GPL code? If I am to use this tool, I
| must abide by the license terms of the training data. Even if it
| is found that the GPL does not cover LLM responses as derivative
| works, I would prefer to be on the safe side and refuse to use
| models trained on software with non-permissive licenses unless I
| am building Free Software.
| colordrops wrote:
| Unless the LLM is cutting and pasting GPL code, I don't see how
| this differs from an engineer learning to code by referencing
| open source projects then creating their own project with its
| own licensing. I'd imagine these models could be tuned or at
| least a check could be added that no licensed code is being
| returned.
| wizzwizz4 wrote:
| If someone learns to program by _memorising_ and then _re-
| using_ blocks of code, they 're absolutely doing it wrong -
| but, also, they should be able to attribute their code.
|
| Most of the techniques I used, I've invented myself (or
| learnt from the standard documentation). When I use a
| technique that I _haven 't_ invented myself, I look up where
| it came from. Half the time, my version is actually radically
| different (and my attribution is mistaken); the other half,
| I've remembered an inferior version, so I steal the better
| version _and then attribute it appropriately_.
|
| That's one way it's different. There are others. Really,
| though, we should be asking the question "in what way is this
| the same as humans learning?", expecting answers that would
| convince an education specialist.
| colordrops wrote:
| > Most of the techniques I used, I've invented myself
|
| This is a very strong statement considering most code is
| just rearranging existing patterns. Unless you are doing
| cutting edge academic research, I'm very skeptical of your
| claim.
| wizzwizz4 wrote:
| You missed a bit.
|
| > (or learnt from the standard documentation)
|
| This means my Python code is mostly Pythonic. (The rest
| is kinda idiosyncratic, but I don't often get
| complaints.) But also:
|
| > _considering most code is just rearranging existing
| patterns._
|
| I am an outspoken critic of "design patterns". They have
| their place, if you're working with legacy tooling like
| C++, Java or Rust, but if most of your work is
| rearranging existing patterns, you have long outgrown
| your tooling and you need a better programming language.
| (Or you're copy-paste programming and need to _learn_
| your tooling first.)
|
| I _am_ doing academic research, but that 's besides the
| point. So far, I've learned that it's rather hard to be
| cutting-edge if you don't look at other people's work:
| you end up re-inventing _all_ the wheels, and any insight
| you may have brought is lost in the noise.
| olddustytrail wrote:
| LLMs don't memorize and reuse. They don't really have a
| memory of any kind. I think the problem is they're _more_
| different from human learning than you think.
| mitthrowaway2 wrote:
| The LLM's trained parameters are a lossy memory of their
| training data.
| olddustytrail wrote:
| No, they aren't.
| meowface wrote:
| The same argument could be used to defend image generation
| models. (And, personally, I'm favorable to that argument for
| both.)
| cj wrote:
| It's a blurry line, I think.
|
| Imagine a prompt "Photo of person, Shutterstock ID 132456,
| with blue eyes instead of brown eyes, watermark removed"
|
| If the prompt returns Shutterstock photo #123456 without
| the watermark (and with the different color eyes) but
| otherwise a near identical photo, I think most people would
| agree the output shouldn't be free to use without buying
| the original photo license from shutterstock.
|
| To a certain extent, we're betting that these models won't
| accept or reply to prompts that are that specific (e.g.
| referencing a specific image for sale on shutterstock by
| its ID number). Or even just providing the photo in the
| prompt and asking the model to remove the watermark and
| upscale the image to a higher resolution.
|
| I'm fearful that LLM's will become (or already are?) an
| easy copyright bypass tool that can be abused, in the
| example above, to put companies like shutterstock out of
| business.
|
| This is the sort of problem regulation might help with.
|
| I haven't read OpenAI's TOS, but I'm curious who owns the
| output of the model and whether OpenAI is transferring
| copyright/licensing liability onto the user or if OpenAI is
| representing that output from the model is 100% free to be
| used in any way the user wants.
| colordrops wrote:
| You could say the same of an artist. If you ask an artist
| to paint a facimile with modifications, they would. That
| doesn't mean all of their output is "tainted" by
| copyright just because they learned to paint by
| referencing existing works.
| OkayPhysicist wrote:
| I feel like those kinds of issues could be easily solved
| in court: "What prompt was used to generate this image?
| Oh, shameless copyright laundering? Rule in favor of the
| plaintiff. Case Dismissed"
| GordonS wrote:
| Huh, I hadn't thought of it that way, interesting.
| jraph wrote:
| Even with permissive license, you'd need to respect
| attribution.
| hospitalJail wrote:
| I'm so torn on GPL...
|
| It prevents bad actors like Apple from ripping off people's
| philanthropic labor, but it also prevents me from ripping off
| people's labor. It also focuses effort onto the FOSS project.
|
| I like PyQts solution of having GPL or buy a commercial
| license.
|
| I suppose I still like MIT/Apache style the best. Even if
| someone rips them off, we didn't lose progress.
| mattl wrote:
| Why do you want to rip off people's labor? What's stopping
| you from incorporating it into your own work?
| hospitalJail wrote:
| I mean:
|
| Compile already working code, slap my logo on it, spend
| millions of dollars marketing it with young good looking
| adults subliminally letting you know that you aren't cool
| unless you give me money.
| GuB-42 wrote:
| You can do that with GPL code.
|
| Make your logo a trademark so that others can't use it.
| It doesn't violate GPL because trademarks are not
| copyright, and make sure your marketing campaign drives
| home the point that you are the real deal and all others
| are ripoffs (including the original).
|
| I mean, people manage to sell bottled water to people who
| have perfectly good and 1000 times cheaper tap water.
| ENGNR wrote:
| At least you're honest!
| justinclift wrote:
| Did you miss that they're being sarcastic? ;)
| andrewl-hn wrote:
| Most non-GPL licenses require attribution. So, there are two
| real choices: either we consider that AI generated code doesn't
| infringe copyright and we are free to use any code:
| proprietary, GPL, AGPL, or we have to attribute the generated
| code to all sources, and thus in practical terms your program
| has to carry a few thousand copies of MIT, BSD, Apache, etc
| licenses with different copyright headers.
| jedbrown wrote:
| Even MIT licensed code requires you to preserve the copyright
| and permission notice.
|
| If a human did what these language models are doing (output
| derivative works with the copyright and license stripped), it
| would be a license violation. When humans want to create a new
| implementation with clean IP, they have one team study the IP-
| encumbered code and write a spec, then a different team writes
| a new implementation according to the spec. LM developers could
| have similar practices, with separately-trained components that
| create an auditable intermediate representation and
| independently create new code based on that representation. The
| tech isn't up to that task and the LM authors think they're
| going to get away with laundering what would be plagiarism if a
| human did it.
| cj wrote:
| Has anyone been able to create a prompt that GPT4 replies to
| with copyrighted content (or content extremely similar to the
| original content)?
|
| I'm curious how easy or difficult it is to get GPT to spit
| out content (code or text) that could be considered obvious
| infringement.
|
| Tempted to give it half of some closed-source or restrictive
| licensed code to see if it auto-completes the other half in a
| manner that is obviously recreating the original work.
| jameshart wrote:
| I can reproduce when prompted all the lyrics to Bohemian
| Rhapsody, but my doing so isn't automatically copyright
| infringement. It would depend on where, when, how, in front
| of what audience, and to what purpose I was reciting them
| as to whether it was irrelevant to copyright law, protected
| under some copyright use case, civilly infringing, or
| criminally infringing copyright abuse.
|
| The same applies to GPT. It could reproduce Bohemian
| Rhapsody lyrics in the course of answering questions and
| there's no automatic breach of copyright that's taking
| place. It's okay for GPT to know how a well known song
| goes.
|
| If copilot 'knows how some code goes' and is able to
| complete it, how is that any different?
| throwaway38249 wrote:
| OK, it can exist without breaking any laws, but if you
| can't release anything it helps you write, what's the
| point?
| littlestymaar wrote:
| I don't know about GPT-4 but you could get ChatGPT to spit
| Carmac's Fast Inverse square root with the comments and all
| (I can't find the tweet though...)
|
| Edit: it wasn't ChatGPT but Copilot see
| https://twitter.com/mitsuhiko/status/1410886329924194309
| jameshart wrote:
| There's a clear separation between the training process which
| looks at code and outputs nothing but weights, and the
| generation process which takes in weights and prompts and
| produces code.
|
| The weights _are_ an intermediate representation that
| contains nothing resembling the original code.
| zacmps wrote:
| But the original content is frequently recoverable.
|
| You can't just take copyrighted code, base 64 it, sent it
| to someone, have them decode it, and claim there was no
| copyright violation.
|
| From my (admittedly vague) understanding copyright law
| cares about the lineage of data, and I don't see how any
| reasonable interpretation could consider that the lineage
| doesn't pass through models.
|
| IANAL
| __loam wrote:
| I think this view is incredibly dangerous to any kind of
| skills mastery. It has the potential to completely destroy
| the knowledge economy and eventually degrade AI due to a
| dearth of training data.
| axus wrote:
| It reminds me of people needing to do a "clean room
| implementation" without ever seeing similar code. I feel
| like a human being who read a bunch of code and then
| wrote something similar without copy/paste or looking at
| the training data should be protected, and therefore an
| AI should too.
| jameshart wrote:
| Okay, that's an argument from consequences, but is the
| view factually _wrong_?
| vkou wrote:
| The neurons in my brain when I plagiarize are just
| arrangements of atoms that contain nothing that resembles
| orginal code/text passages/etc.
| jameshart wrote:
| The trained weights of a GPT model are a frozen, static,
| transmissible representation. They're not equivalent to
| the live state of a brain.
| szundi wrote:
| Pretty equivalent to the snapshot of a live brain. Those
| inside it are even called neurons and neural network
| jameshart wrote:
| No, they are the weights that are used to configure a
| neural network. They're a map of how to build a useful
| brain, not a neural state.
| nonameiguess wrote:
| I guess we'll see what the legal system says, but it's nearly
| impossible for me to imagine how anyone could ever justify
| saying a system trained on potentially millions of separate
| open-source projects can then be said to universally produce
| derivative works of any specific project it trained on.
|
| On the other hand, if a developer using this tool then goes and
| tells it "please write me a C library in the style of GNU
| libc," then yeah, that is skirting a fine line. But just don't
| do that.
| johndough wrote:
| GitHub Copilot copying Quake's fast inverse square root
| function is a famous example where Copilot copied GPL-
| licensed code only given the comment //
| fast inverse square root
|
| https://news.ycombinator.com/item?id=27710287
| Spivak wrote:
| I mean if you ask the tool to produce a specific algorithm
| by it's very famous name then that's on you. If you ask
| DALL-E to draw Pikachu you can't be suprised Pikachu when
| it's a copyright violation.
| lgas wrote:
| I don't know, it seems like a bit more of a gray area
| here than with Pikachu. For example, if I said "write me
| an implementation of fast inverse square root in rust"...
| and it did... that's certainly not a copyright violation.
| And if I said "ok, now port this rust code to C..." and
| it did, then that's certainly not a violation. But then
| why should I be penalized because the language I want it
| in happens to be the original language it was written in?
| ipaddr wrote:
| A character for character copy including comments? That
| should be penalized under current law.
|
| Why not songs, software, entire books and tv shows?
| ipaddr wrote:
| You can't play both sides. If you ask it for something
| with a copyright and it gives it to you, it has broken
| the law governing this. Ask it for underage porn and it
| gives it to you.. it broke the law.
| AlotOfReading wrote:
| Doesn't have to be famous. I gave ChatGPT lines 48-67 of
| drivers/hwmon/sht21.c [1] and asked it to complete the
| function. Prompt: Complete the function
| for an sht21 driver [code omitted]
|
| What it returned was the copyrighted function, comments
| and all: static inline int
| sht21_rh_ticks_to_per_cent_mille(int ticks) {
| ticks &= ~0x0003; /* clear status bits / /*
| Formula RH = -6 + 125 * SRH / 2^16 from data sheet 6.1,
| * optimized for integer fixed point (3 digits) arithmetic
| */ return ((15625 * ticks) >> 13) - 6000;
| }
|
| Notice how it even included a comment referencing a
| specific datasheet!
|
| This driver is hardly "famous" or even "notable", because
| those aren't things LLMs understand. The prompt simply
| contains enough context to be distinctive and the sht21.c
| is an old, stable driver in each of the many kernel trees
| included in its training set.
|
| Regurgitation isn't a particularly rare thing with LLMs,
| most cases just aren't this obvious.
|
| [1] https://github.com/torvalds/linux/blob/c6b0271053e7a5
| ae57511...
| frankgrecojr wrote:
| https://docs.gitlab.com/ee/user/project/repository/code_sugg...
|
| > Google Vertex AI Codey APIs are not trained on private non-
| public GitLab customer or user data.
| whoisthemachine wrote:
| An impressive iteration on this would be a model that would say
| "here's my solution, and here's what it's inspired by, you may
| need to update your software's license and/or distribution if
| you use this solution."
| JoshTriplett wrote:
| GitLab seems to be a lot more responsive to feedback. It'd be
| worth asking them if they can publicly document their training
| set and provide a list of licenses and copyright notices.
| dnsmichi wrote:
| GitLab team member here.
|
| The training data is documented in https://docs.gitlab.com/ee
| /user/project/repository/code_sugg...
|
| AI Transparency is important, all available AI features
| provide documentation for training data, and are built with
| privacy first.
|
| The GitLab Duo announcement adds more feature details and
| plans. https://about.gitlab.com/blog/2023/06/22/meet-gitlab-
| duo-the...
|
| The AI/ML blog series provides insights on experiments, and
| features being built.
| https://about.gitlab.com/blog/2023/04/24/ai-ml-in-
| devsecops-...
| JoshTriplett wrote:
| "Codey was fine-tuned on a large dataset of high quality,
| permissively licensed code from external sources" is not
| sufficient information to be able to provide attribution
| and licensing information.
|
| Would it be possible to get a complete list of sources and
| licenses?
| curtis3389 wrote:
| Okay, but how much are they charging?
|
| It's getting really annoying how many sites force you into a
| trial just to find out how much it'll cost when it ends.
|
| EDIT: Is this even positioned to compete with Copilot? What
| editors are there plugins for? There is surprisingly little
| information on the site.
| john_cogs wrote:
| Code Suggestions can be used in GitLab's Web IDE and VS Code
| and Microsoft Visual Studio when you have the corresponding
| GitLab extension installed:
| https://docs.gitlab.com/ee/user/project/repository/code_sugg...
|
| We offer experimental support for additional editors:
| https://docs.gitlab.com/ee/user/project/repository/code_sugg...
| lolinder wrote:
| That doesn't answer the question of what the pricing will be.
| I'm assuming it's not going to be free. Will it be a
| standalone subscription, or another thing that gets bundled
| into the main tiers to justify the ever-increasing prices?
| john_cogs wrote:
| Code Suggestions is free while in Beta.
|
| When GA, it will be included in our $9 per user per month
| AI add-on.
| ollieglass wrote:
| Seems to be $99 per month.
|
| Signing up for the free trial funnels me to the trial of Gitlab
| Ultimate. So assuming that you need an Ultimate subscription to
| use it after the trial, that's the price. Pricing is here
| https://about.gitlab.com/pricing/
|
| In contrast, Copilot is $10 per month
| https://github.com/features/copilot#pricing
| john_cogs wrote:
| Please see this comment:
| https://news.ycombinator.com/item?id=36527397
| [deleted]
| frankgrecojr wrote:
| Have they said what LLM they use?
| SparkyMcUnicorn wrote:
| Google's Codey API.
|
| https://docs.gitlab.com/ee/user/project/repository/code_sugg...
| smallpipe wrote:
| What nimrod added a chatbot to this page ? It's downright
| unusable on mobile, it just keeps popping up with shit. Even
| closing it creates two more notifications that use up two thirds
| of the screen
| ases wrote:
| All the code examples on this page are doubling any brackets ([],
| {}, ()). How have they managed that? Not a fantastic first
| impression of its capabilities...
| john_cogs wrote:
| GitLab team member here. Thanks for flagging.
|
| Our web team is working to resolve this issue here:
| https://gitlab.com/gitlab-com/marketing/digital-experience/b...
| Chiron1991 wrote:
| Not only that, but the Golang example is full of errors.
| Parameter definitions don't have colons between name and type.
| The map for seen elements is declared as a, but later
| referenced to as m. The append instruction references uniques,
| which is undefined. The return statement also references
| uniques.
| blibble wrote:
| ah yes, so a typical example of generative AI output
| mmcclure wrote:
| I'm going to give them the benefit of the doubt and assume this
| is a marketing site problem and not a product problem, but
| either way, not a good look. Reminds me of Google's big Bard
| announcement including a wrong answer in the image.
| SparkyMcUnicorn wrote:
| Gitlab is using Google's Vertex Codey models/API for the code
| completions.
|
| Showcasing innacuracies in AI responses must be a Google
| requirement.
| Xeamek wrote:
| ...something something do not editorialize titles something
| something...
| [deleted]
| lee101 wrote:
| [dead]
| ezekg wrote:
| Is the Go example even valid? I've never seen that (arr: [[]]int)
| syntax.
| dharmab wrote:
| Neither is the Python example
| ezekg wrote:
| Well, that's kind of embarrassing... the JS example is also
| all sorts of messed up.
| [deleted]
| esafak wrote:
| We need LLMs to respect the grammar of the language they are
| generating in.
| Dudester230602 wrote:
| So a follower, not a leader.
| ldehaan wrote:
| [dead]
| coolgoose wrote:
| So whos code is used for training?
| dnsmichi wrote:
| GitLab team member here.
|
| The training data is publicly documented, and AI features are
| built with privacy first.
|
| For all URLs please check my comment in
| https://news.ycombinator.com/item?id=36526159
| elforce002 wrote:
| Nice. I really like the transparency. That's one of the main
| reasons I didn't use github copilot.
| dnsmichi wrote:
| Thanks!
|
| AI for self-managed instances is also focussing on privacy,
| while bringing more efficiency to teams.
| https://about.gitlab.com/blog/2023/06/15/self-managed-
| suppor...
| dgrin91 wrote:
| I wish them luck. While I'm not a huge fan of copilot coding AIs,
| I understand how they are going to be a killer feature for many
| and I worry that it would create a big moat for Github. I hope
| that Gitlab is able to close that gap because I really like
| Gitlab.
| cassianoleal wrote:
| Codeium [0] seems like a worthy competitor to Copilot already.
|
| [0] https://codeium.com/
| luxurytent wrote:
| Don't sleep on Cody either https://about.sourcegraph.com/cody
| michaelmior wrote:
| I've been using Codeium with Vim for a few months and it
| works great :)
| whimsicalism wrote:
| editorialized title
| phil917 wrote:
| I just canceled my Copilot subscription last night. I definitely
| never saw the productivity boost that I've seen so many claim.
|
| I actually feel like the suggestions seemed to get worse during
| my month of using it for some reason.
|
| Towards the end, it started suggesting these large blocks of code
| (another issue I had with the interface as well) which were very
| not relevant to what I was attempting to write.
|
| All in all, I'm very underwhelmed from my first experience with
| "AI enhanced" coding.
| lostmsu wrote:
| Just today Copilot saved me close to 1 hour of typing and
| replaced it essentially with ~1 minute of pressing Tab.
|
| I was hooking IAudioClient COM class to capture and silence
| arbitrary app's audio, and as soon as I wrote signatures for
| its members Copilot was able to generate skeletal stub
| implementation with logging as well as hooking code totaling
| about 150 lines of Rust.
| Tronno wrote:
| My experience was very similar. Copilot became a sort of
| improved autocomplete, that would save copy/pasting repeated
| variable names into tests, logs, etc, but offered no other
| significant improvements. Its suggestions were frequently
| incorrect, often not obviously so.
| benabbottnz wrote:
| > _that would save copy /pasting repeated variable names into
| tests, logs, etc_
|
| This is all it takes for me to get value out of my
| subscription. It's saving me time by anticipating my next
| test / block of code based on what I've previously written,
| and that saved time really adds up.
|
| Sure it's not "Jesus take the wheel" level AI yet, but time
| is money, and so is my sanity.
| jsight wrote:
| TBH, using this as a fancy smart autocomplete doesn't seem like
| the best idea. I can't believe that I'm suggesting something
| that sounds like Clippy... but...
|
| It really needs to be something where you can rattle off a list
| of requirements and have it build the code for you. The code
| context is necessary, but not sufficient.
| bgarbiak wrote:
| It's coming up: https://github.com/features/preview/copilot-x
| nikeee wrote:
| I don't get the thing with automated PR and commit
| messages. They mostly describe _what_ was done and not why.
|
| Also, if you enhance your available information with AI,
| why not just write no message at all (or a short one as
| always) and use the AI when _looking_ at the commits/PRs?
| The AI will certainly be better at this in a later stage
| because they themselves will get better and they will have
| more context due to being able to look at the commits that
| followed the PR/commit.
|
| There is no point in generating the commit message when
| committing.
|
| When auto-generated messages will become a commodity, there
| will be issues with it. For example, if the message doesn't
| fit with the actual commit contents (semantically or
| syntactically), I have to think hard whether this is an AI-
| message-generator-bug, or the author missed something or
| wether I am missing something. This is not cool and makes
| reading commit messages harder.
| __loam wrote:
| I'm excited to continue to be disappointed.
| 62951413 wrote:
| Are there other people who find that IntelliJ's autocomplete
| suggestions and shortcuts (including live templates&Co) already
| do 80% of what Copilot is trying?
| SkyPuncher wrote:
| Yea, but that 20% is ultra powerful. It handles the fuzzy
| things really well and it pattern matches even better.
|
| If I write a comment describing what I'm doing, it will
| generate multiple lines of correct code
| roflyear wrote:
| People love to cargo-cult. It's wild. It was okay but I'm
| happier with ChatGPT for one-off things I don't know how to do
| "give me a sql query that does this obscure thing" etc..
| lolinder wrote:
| It's okay that it didn't fit well into your workflow, but
| accusing those who use it of cargo culting isn't
| constructive. Other people don't have the same workflow as
| you do, and as simonw noted above, some of us actually
| _adjusted_ our workflow to get the most out of Copilot.
| gavinray wrote:
| To offer a counter opinion: while I can live without ChatGPT
| and cancelled my OpenAI plus membership, I cannot live without
| Copilot and would be willing to pay a lot to keep it.
|
| There are a lot of tricks to optimizing output from Copilot
| ("Copilot-Driven Development"), I wrote a little about what
| I've discovered here:
|
| https://gavinray97.github.io/blog/a-day-without-a-copilot#co...
| endisneigh wrote:
| Same. It's funny reading comments on here. Even gpt4 I think is
| just ok. Maybe the people who see these huge increases are more
| junior?
| simonw wrote:
| I'm definitely not more junior, and I'm seeing massive
| productivity improvement from Copilot and GPT-4 - but I'm
| finding it takes a lot of expertise to get the best results,
| both in my ability as a programmer and in terms of knowing
| the best ways to use the AI tools.
|
| Learning how to get the best results of them takes a great
| deal of experimentation.
| lolinder wrote:
| This has been my experience as well. I actually avoid
| recommending Copilot to juniors because I think using it
| requires a _deeper_ understanding of what you 're trying to
| do than is required to just write the code by hand. When
| you have that understanding it can be a huge time saver,
| but it's not something you can just pick up and magically
| ask to solve your problems for you.
| pjot wrote:
| I'm curious what your workflow is like switching between
| copilot and gpt-4. I typically have an open window for
| each, though this can feel more cumbersome than necessary
| at times.
| simonw wrote:
| I actually use the ChatGPT UI more than copilot - I write
| code with it while I'm out walking the dog, by describing
| what I want, then copy and paste it into my text editor
| on my laptop when I get home.
|
| I'm increasingly using my LLM CLI tool for quick lookups,
| and sometimes to make changes to existing code too - see
| https://github.com/simonw/symbex/releases/tag/1.0
| SparkyMcUnicorn wrote:
| Copilot Chat is pretty great. Highly recommend signing up
| for the waitlist.
|
| https://github.com/github-
| copilot/chat_waitlist_signup/join
| coolsunglasses wrote:
| I've been working as a programmer for almost 15 years and I
| work on fairly touchy (perf, reliability-wise) networked
| systems in a programming language with a good type system. I
| find Copilot to be very valuable even if I have to give it a
| little bump or edit what it produces sometimes.
| simonw wrote:
| Did you change the way you worked to help facilitate it at all?
|
| I get great results from Copilot, but that's because I do a
| bunch of things to help it work for me.
|
| One example: I'll often paste in a big chunk of text - the
| class definition for an ORM model for example - then use
| Copilot to write code that uses that class, then delete the
| code I pasted in again later.
|
| Or I'll add a few lines of comments describing what I'm about
| to do, and let Copilot write that code for me.
|
| Did you try the trick where you paste in a bunch of code and
| then start writing tests for it and Copilot spits out the test
| code for you?
|
| A few more notes about how I've used Copilot here:
|
| - https://til.simonwillison.net/gpt3/writing-test-with-copilot
|
| - https://til.simonwillison.net/gpt3/reformatting-text-with-
| co...
|
| I've also used Copilot to take educated guesses at things -
| like an inline mini-ChatGPT - with good results:
|
| - https://til.simonwillison.net/gpt3/guessing-amazon-urls
|
| As with so many other AI tools, Copilot is desperately lacking
| detailed documentation. It's not at all obvious how to get the
| most out of it.
| SparkyMcUnicorn wrote:
| Using Copilot has taught me how to write better code
| comments. And the improved comment quality benefits everyone,
| not just the AI.
|
| Even something as simple as adding a doc block for a method
| can boost the quality quite a bit.
|
| I could live without it, but I'd also keep paying if the
| price increased.
| leothelion_ wrote:
| I agree. It has made me better at articulating what I want
| to do, what a function does, and something as granular as
| what a particular loop hopes to achieve
___________________________________________________________________
(page generated 2023-06-29 23:02 UTC)