https://meta.stackexchange.com/questions/388401/new-blog-post-from-our-ceo-prashanth-community-is-the-future-of-ai Stack Exchange Network Stack Exchange network consists of 181 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers. Visit Stack Exchange [ ] Loading... 1. + Tour Start here for a quick overview of the site + Help Center Detailed answers to any questions you might have + What's Meta? How Meta is different from other sites + About Us Learn more about Stack Overflow the company, and our products. 2. 3. current community + Stack Exchange + Meta Stack Exchange help chat your communities Sign up or log in to customize your list. more stack exchange communities company blog 4. 5. Log in 6. Sign up Meta Stack Exchange is a question and answer site for meta-discussion of the Stack Exchange family of Q&A websites. It only takes a minute to sign up. Sign up to join this community [ano] Anybody can ask a question [ano] Anybody can answer [an] The best answers are voted up and rise to the top Meta Stack Exchange 1. Home 2. 1. Public 2. Questions 3. Tags 4. Users 5. Unanswered 3. Teams Stack Overflow for Teams - Start collaborating and sharing organizational knowledge. [teams-illo-free-si] Create a free Team Why Teams? 4. Teams 5. Create free Team Teams Q&A for work Connect and share knowledge within a single location that is structured and easy to search. Learn more about Teams New blog post from our CEO Prashanth: Community is the future of AI Ask Question Asked yesterday Modified today Viewed 11k times -111 Today, we released a blog post from Stack Overflow CEO Prashanth Chandrasekar about AI and its intersection with communities. Because of the timely nature and the content of it, I'm reproducing it here for comments or questions from community members. You can find the original blog post here. Senior members of the company staff will be reviewing everything here and will respond to questions, though it may be 24 hours or so before answers are ready, just because of the amount of interest that this is likely to get. Please leave any questions that you have below as individual answers. Community is the future of AI By Prashanth Chandrasekar Throughout history, great thinkers have made predictions about how new technology would reshape the way in which humans work and live. With every paradigm shift, some jobs grow, some change, and some are lost. John Maynard Keynes wrote in 1930 that new technology meant humans would be working 30 hours a week or less, and that the main challenge would be what to do with all our free time. So far, predictions of this nature haven't exactly come true. As new technology empowers us, we push ourselves to new heights and reach for previously unattainable goals. Over nearly 15 years, Stack Overflow has built the largest online community for coders to exchange knowledge, a place where anyone with an internet connection can ask or answer questions, free of charge, and learn from their peers. Stack Overflow for Teams, our enterprise SaaS product, is trusted by over 15,000 organizations to serve as their internal knowledge bases. With the recent advent of dramatically improved artificial intelligence, many industries are wondering how technologies like ChatGPT will change their business. For software development, the answer seems more immediate than most. Even before the latest wave of AI, a third of the code being written on popular code repositories was authored by an AI assistant. Today, sophisticated chatbots, built on top of cutting edge large language models (LLM), can write functional code for a website based on nothing more than a photo of a rough sketch drawn on a napkin. They can answer complex queries about how to build apps, help users to debug errors, and translate between different languages and frameworks in minutes. At Stack Overflow, we've had to sit down and ask ourselves some hard questions. What role do we have in the software community when users can ask a chatbot for help as easily as they can another person? How can our business adapt so that we continue to empower technologists to learn, share, and grow? It's worth reflecting on an important property of technological progress. The Jevons Paradox shows us that, as innovation allows us to do more, we settle on a new normal, moving the goal posts for what we expect of people and organizations, then competing to see who can find new ways to pull ahead of the pack. For knowledge work, as the cost of an action diminishes, we often do more of it. Abstracting away repetitive or tedious tasks frees technologists up to make new discoveries or progress innovation. If new AI systems make it possible to create software simply by chatting with a computer, my prediction is that, far from the job of programmer disappearing, we'll end up with millions of new software developers, as workers from fields like finance, education, and art begin making use of AI-powered tools that were previously inaccessible to them. We are enthusiastic about welcoming this next generation of developers and technologists, providing them with a community and with solutions, just as we have for the last 15 years. We've got a dedicated team working on adding GenAI to Stack Overflow and Stack Overflow for Teams and will have some exciting news to share this summer. I've heard similar sentiments expressed recently by Microsoft founder Bill Gates, by Geoff Hinton, the godfather of the neural network approach that produced today's AI revolution, and by Stephen Wolfram, a pioneer across computer science and mathematics. Each sees in today's AI the potential for the loss of certain jobs, yes, but also, if history is a guide, a future in which a great variety of more highly skilled work becomes available to an even larger group of people. Just as tractors made farmers more productive, we believe these new generative AI tools are something all developers will need to use if they want to remain competitive. Given that, we want to help democratize knowledge about these new AI technologies, ensuring that they are accessible to all, so that no developers are left behind. I talk to developers of varying experience levels all of the time, and I've been hearing anecdotes of novice programmers building simple web apps with the help of AI. Most of these stories, however, don't begin and end with an AI prompt. Rather, the AI provides a starting point and some initial momentum, and the human does additional research and learning to finish the job. The AI can debug some errors, but is stymied by others. It can suggest a good backend service, but often can't solve all the points of friction that arise when integrating different services. And of course, when a problem is the result not of instructions from a machine, but human error, the best answers come from other people who have experienced the same issues. For more experienced programmers, AI will be an amplifier of their existing skill, making them more ambitious in their projects. The result, as Jevons would predict, is that they spend more time with AI, but also more time creating new ideas, researching new topics, and asking new questions that had not occurred to them before. They feel empowered to reach farther beyond their traditional skillset and to push the boundaries in terms of the kind of work they want to take on. We are excited about what we can bring to the fast moving arena of generative AI. One problem with modern LLM systems is that they will provide incorrect answers with the same confidence as correct ones, and will "hallucinate" facts and figures if they feel it fits the pattern of the answer a user seeks. Grounding our responses in the knowledge base of over 50 million asked and answered questions on Stack Overflow (and proprietary knowledge within Stack Overflow for Teams) helps users to understand the provenance of the code they hope to use. We want to help coders stay in the flow state, allowing them to create with the latest tools with the confidence that they will be able to document and understand the provenance, source, and context of the code being generated. Community and reputation will also continue to be core to our efforts. If AI models are powerful because they were trained on open source or publicly available code, we want to craft models that reward the users who contribute and keep the knowledge base we all rely on open and growing, ensuring we remain the top destination for knowledge on new technologies in the future. AI systems are, at their core, built upon the vast wealth of human knowledge and experiences. They learn by training on data - for example open-source code and Stack Overflow Q&A. It is precisely this symbiotic relationship between humans and AI that ensures the ongoing relevance of community-driven platforms like Stack Overflow. Allowing AI models to train on the data developers have created over the years, but not sharing the data and learnings from those models with the public in return, would lead to a tragedy of the commons. It might be in the self-interest of each developer to simply turn to the AI for a quick answer, but unless we all continue contributing knowledge back to a shared, public platform, we risk a world in which knowledge is centralized inside the black box of AI models that require users to pay in order to access their services. As the AI landscape continues to evolve, the need for communities that can nurture, inform, and challenge these technologies becomes paramount. These platforms will not only offer the necessary guidance to refine AI algorithms and models but also serve as a space for healthy debate and exchange of ideas, fostering the spirit of innovation and pushing the boundaries of what AI can accomplish. Our thesis on community as the center of a safe, productive, and open future for AI also offers some exciting prospects for our business. Stack Overflow for Teams, our enterprise, private version of Stack Overflow, helps to power a community-driven knowledge base inside of 15K+ organizations like Box, Microsoft, and Liberty Mutual. Decades of institutional knowledge, shaped and curated by subject matter experts and experienced teams, allows the employees at these organizations to more easily collaborate, improving productivity and trust. Incorporating generative AI technologies into the organizations using Stack Overflow for Teams will allow us to layer a conversational interface on top of this wealth of information. We believe this could lead to tremendous productivity gains: from new hires being able to onboard more quickly, to speed up developer workflows, as users are able to quickly ask questions and retrieve answers tapping into the company's history, documentation and Q&A. The example above is just one of many possible applications of GenAI to our Stack Overflow public platform and Stack Overflow for Teams, and they have energized everyone at our company. We'll be working closely with our customers and community to find the right approach to this burgeoning new field and I've tasked a dedicated team to work full time on such GenAI applications. I'll continue to share updates through channels such as my quarterly CEO blog, but I'll be back in touch soon to announce something big on this topic. In the meantime, thank you to our community and customers for continuing to help us on our mission to empower the world to develop technology through collective knowledge. * discussion * featured * blog * announcements * company Share Improve this question Follow edited 18 hours ago philipxy's user avatar philipxy 42822 gold badges66 silver badges1212 bronze badges asked yesterday Philippe's user avatar PhilippeStaffModPhilippe 13.3k66 gold badges2929 silver badges4646 bronze badges 27 * 25 Anyhow, my personal feeling is that AI is a huge wave washing the world these days. And it's not a good thing. It's going to cause enormous damage in the long run, not end of the world as some say, but damage that will take lots of years to fix. So while I understand the desire to jump on the worldwide AI wagon, I'm sad SE will also take part in it. :( - Shadow The Spring Wizard yesterday * 28 How will the LLM be trained to ignore all of the out of date and plain wrong answers on the SE network? - DavidPostill yesterday * 10 If you have questions, to ensure that they are seen and responded to, please add them as an answer, not just a comment. - Philippe StaffMod yesterday * 13 Also Y'all. I posted an answer and am going to bed. I expect answers in the morning, and will be reviewing and clearing out anything that should be an answer in about 7-8 hours if someone hasn't beaten me to it. You've been warned ;) - Journeyman Geek Mod yesterday * 16 You've probably heard this a lot of times before, but you'll get fewer downvotes than people unhappy with this, due to this site having its own pool of reputation. - Andreas detests censorship yesterday * 62 So let me get this straight; we've deleted tens of thousands of CGPT answers just to have another CGPT clone introduced to the site itself? You could at least take us out to dinner before you screw us that hard. CGPT already exists; what's the point in introducing a technology we've so far distanced ourselves from precisely over quality concerns, and when either the same tech or a better tech exists elsewhere on the internet? - Zoe stands with Ukraine yesterday * 3 @ZoestandswithUkraine As a moderator on an affected site - I think your views are super important, and it would be a shame if it was missed. I'd appreciate it if you'd convert your comment to a post too. - Journeyman Geek Mod 23 hours ago * 9 @JourneymanGeek Based on the wording, it doesn't really matter where I point it out. If I write an answer, I'm going to have to expand it because answers are more serious than comments. However, this isn't a feedback post. Note how the question explicitly asks for questions that'll be responded to; this isn't a feedback post, it just looks like one. This is an early announcement, and no amount of negative feedback will change anything, because this isn't a feedback post. I'm not going to spend more time on something that will almost guaranteed end up being ignored or at the very least not - Zoe stands with Ukraine 22 hours ago * 29 It seems that SE CEO is as detached from the Community as he can possibly be. - Resistance Is Futile 12 hours ago * 17 This seems very much in the same vein as the cryptocurrency craze of 2020/2021, just with GPT: everyone is doing it, so let's do it too! Stack Exchange does have a massive Q&A count in its 50 million Q&A... the problem is that some 30 million (if not more) are... low quality or duplicates. Stack Exchange content is probably a great corpus of data for training big models, for sure, but I'd caution Prashanth (if he ever visits Q&A and reads comments) to not overestimate the content the company is working with here (lest we end up with yet another gen AI suffering from accuracy problems. - TylerH 7 hours ago * 5 I mean, i get that, but that seems a bit too aspirational to me, given what the company has promised and delivered previously. -\_ (tsu)_/- It's more likely we'll just get clippy. - Kevin B 6 hours ago * 8 How much of this announcement itself was written by ChatGPT? It has the same bland emptiness as ChatGPT-generated text, and it is difficult to even read because I keep feeling at first like I'm missing what's being communicated, only to realise that nothing is. Out of 1,548 words the only thing I got out of it is "we're going to start incorporating ChatGPT into Stack Overflow but we're not really sure how yet". And... no, Stack Overflow does not want that. - kaya3 5 hours ago * 6 @Trilarion It shouldn't be so hard for a company like Stack Overflow to beat Google at making a better in-site search for its own content than what Google can do. SO has access to more data and better domain knowledge of the content, and should have no problem attracting enough talent to develop such a thing, given its overall reputation as a company. - TylerH 5 hours ago * 4 @kaya3 likely none (at least normal checks corroborate the idea it's not) [unless you meant it ironically]. That's marketing - a human-powered model that had been capable of generating mostly meaningless text long before LLMs became a thing :) You seem to have extracted all there is that is meaningful, the rest is just inconsequential fluff. - Oleg Valter is with Ukraine 4 hours ago * 4 @Braiam It should fairly obvious how a company focused on broad internet searching has different concerns than a company focused on searching its own system. If Google wanted to dedicate a team to making a site search for questions on SO, for example, then yes they could probably out-compete even SO's own best concerted effort. But they aren't doing that, because it's not their job or concern, and much of what is useful for searching SO (all the parameters and keywords one can use for searching) are simply not supported in Google search because they only apply to SE sites. - TylerH 3 hours ago | Show 12 more comments 23 Answers 23 Sorted by: Reset to default [Highest score (default) ] 119 I'm not quite as excited as Prashanth to be participating in training AI, I participate here because I like teaching people, and I'm not particularly impressed by the job ChatGPT is currently doing of that, despite all the existing material it's been trained on. In particular, I've recently seen a new flavor of question here on sites like Biology.SE that fits the format: "ChatGPT told me XYZ, can you explain this to me?" This pattern isn't new, "My professor/ teacher/textbook told me XYZ and I'm confused" is pretty common. The difference is, no one should be learning from ChatGPT right now. It's not capable of understanding, it's a method of generating carefully crafted BS that appears to be knowledgeable without actually having knowledge. From generating citations to completely fake papers to thinking Romania is land-locked to lifting plagiarized text, right now one of the biggest challenges to everyone else is filtering and fact-checking this BS. While versions of AI-assisted code writing have been around for quite awhile, the latest iterations are something quite different and I fear the hype will drive expectations for productivity that are not achievable without creating massive problems and security risks where the people paying the consequences will not be the same as the people collecting the benefits. I don't really get what "Community is the future of AI" is supposed to mean, but for now, I fear it means that Our Community is not being treated respectfully by purveyors and users of recent AI technology releases. Share Improve this answer Follow answered yesterday Bryan Krause's user avatar Bryan KrauseBryan Krause 4,40822 gold badges1717 silver badges2525 bronze badges 12 * 30 I don't understand why people expect a language model to be able to reason about. It's not intelligent by any stretch. - Braiam yesterday * 12 @Braiam I think it's largely because advocates for the technology keep telling them they should and how they're missing out if they don't use this new technology for everything. - Bryan Krause yesterday * 6 @Braiam It's misinformation; or, "fake news", if you wanna call it that. People uneducated in the field are told by people they believe have the knowledge, that AI is "these things", and then they believe it; it's as simple as that. - Andreas detests censorship yesterday * 1 @JosephGarvin: Congratulations, I guess...? How does this work as a response to this answer though? - Makoto yesterday * 5 If I recall correctly, what was risible - & telling - was ChatGPT's claiming Romania is a land-locked country bordered by the Black Sea. - Scortchi - Reinstate Monica yesterday * 2 @Scortchi-ReinstateMonica That's right, thank you - I had trouble finding a recounting of this particular example to link to. Basically, the example showed that ChatGPT 'knew' what sorts of words belong in a paragraph talking about land-locked countries: if I recall, this included how it presents hardships and results in a strong-willed people, and also it knew to discuss borders of land-locked countries, which for Romania included a body of water, but has no actual concept of the meaning of "land-locked". - Bryan Krause yesterday * Re "creating massive problems and security risks": This has (allegedly) already happened (Samsung semiconductor leak). - This_is_NOT_a_forum yesterday * 4 @Makoto I suppose Joseph is responding to my suggestion that people shouldn't use ChatGPT for learning, which I stand by. It seems like he merely wants to write code that doesn't error, which I suppose you can do with a trial and error sequence of ask ChatGPT, try, and then maybe ask on SO if it fails. But, I'd argue that's not really the same thing as learning, and while it might be okay for, say, modifying a plot where the result is evident, you're likely to make errors in data analysis if you try to "learn" statistics from ChatGPT, because those sorts of errors don't stop program execution. - Bryan Krause yesterday * 2 @BryanKrause: Yes, ChatGPT's proclivity for confabulation ("hallucination") is rightly notorious, but its proclivity for self-contradiction is equally worth underlining. For non-sequiturs too: I've been told that snails don't eat mice because they are from different taxonomic groups. - Scortchi - Reinstate Monica 23 hours ago * 5 @Scortchi-ReinstateMonica Indeed, but these are all symptoms of the same underlying issue that ChatGPT and similar models don't know anything except the statistics of words, and while this turns out to be sufficient to generate a somewhat believable mirage it isn't a substitute. It's a cute trick when you know what the right answer is, dangerous when you don't. - Bryan Krause 23 hours ago * 2 Here's an example (from a month ago) of ChatGPT getting mixed up with simple numerical facts. From scottaaronson.blog/?p=7094# comment-1947593 It first gives some correct info about Fibonacci numbers, but then it says: "Two consecutive natural numbers that have the same parity are 8 and 9, both of which are odd" - PM 2Ring 19 hours ago * 3 All the companies eager to hump the latest buzzword aren't necessarily using ChatGPT but a far inferior AI though. So while your concerns about ChatGPT are valid, rest assured that nothing equally bad will reach SO, but more likely something hundred times worse. As seen on 100% of all sites using a "support" AI bot to "help" users. Some 20 years after MS Clippy, these bots have not evolved at all. - Lundin 12 hours ago Add a comment | 102 This entire announcement makes me question the CEO's understanding of what his company's platforms actually are. There's a reason AI has been banned on several of the sites. We're actively fighting it over on Stack Overflow. This worries me. Don't get me wrong; I love AI/ML; I find it highly interesting, and I do look forward to the development of it into the future, but it has no place on SE. Honestly, my first impression was: "wtf?" This seems so detached from the reality at the SE sites, which is highly worrisome, for someone that should know the reality, and is in charge of their future. Please do not add GenAI to Stack Overflow. You're free to experiment with it, but don't pollute Stack Overflow with it. We're not interested in fact-checking AI content; we are interested in generating the content ourselves. layer a conversational interface on top of this wealth of information But why...? This is a distraction. These sites exist to easily let us find answers to the questions we seek, not be bombarded with noise. What purpose does this serve? I find this post very demotivating. I'm not sure if there's any point in continuing to moderate and curate Stack Overflow. You have made me insecure about my efforts to SE. Are they lasting? Will my work be undone? I don't know. And a simple <> in response, is not enough to cure that lack of trust I have. Other problems I take issue with: * MSE is a second class citizen to the blog * The CEO doesn't post this himself on MSE * You don't link to the MSE discussion from the blog * [S:The MSE discussion is not visible in the sidebar on any of the other sites and Meta sites. As such, it's invisible, because there's no link to it from the blog. * My comment on the blog linking to this MSE discussion takes forever to be accepted; many people will have missed it by the time it finally gets put there. Also, it's not my job to do that; it's yours.:S] Resolved one day after being featured. Share Improve this answer Follow edited 5 hours ago answered yesterday Andreas detests censorship's user avatar Andreas detests censorshipAndreas detests censorship 45111 gold badge33 silver badges1010 bronze badges New contributor Andreas detests censorship is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct. 9 * 1 @Andreasdetestscensorship ask chatgpt to remove all the marketing speaks :) - Braiam yesterday * 1 @Braiam I was under the impression that ChatGPT was designed to add them, not remove them. - Andreas detests censorship yesterday * NotTheDroids seems to be able to achieve that meta.stackexchange.com/a/388405/213575 - Braiam yesterday * @Braiam Not a great attempt, but sure, I guess it worked somewhat. With the advent of generative AI, I guess we'll see more of it. We'll have to use a disregarding AI to strip it out. - Andreas detests censorship yesterday * 24 This is gonna sound hella pessimistic, but...the CEO has been pretty far out of touch for a while now. Even the commitment to read and participate on Meta once a quarter seems to have become little more than a to-do item than something that is really meant to build that understanding. - Makoto yesterday * 4 @Makoto I think you meant <>, not <>. :/ - Andreas detests censorship yesterday * 1 "I'm not sure if there's any point in continuing to moderate and curate Stack Overflow" I wouldn't take this that far. Seems like an overreaction to me. It's not like they're going to delete posts you've written, or undo moderation activities you've done. Though it might mean more content to moderate. But who knows? They haven't given us much detail yet. - starball 22 hours ago * 2 @starball "Though it might mean more content to moderate." And that's the essence. Flooding the site with this content is undoing the effort we put into keeping the sites clean and tidy. - Andreas detests censorship 22 hours ago * 3 eh. if AI-generated content gets added in to the Q&A pages, there's a good chance that I'll userscript it out of my UI (like I've already done with the blogposts and the new related questions section under unanswered questions). something something if I can't see it, it doesn't exist. - starball 22 hours ago Add a comment | 75 Do not even think about incorporating something like ChatGPT into this site, or anywhere on the network. To do so would be a slap in the face of all of the real contributors and their hard work and diligence to ensure that the content that is being produced comes from actual subject matter experts and those knowledgeable in the field. The attitude from the tech industry is that AI is this panacea that will solve problems or make things easier, and I have the impression that this perspective is not changed within Stack Overflow. This worries me; y'all seem eager to solve a problem that y'all have had next to no success in quantifying. One cannot claim to be supportive of creative minds or individuals when these AI models are generated and built on top of so much unlicensed, unprivileged data. (This is why I have reservations on things like GitHub Copilot, for instance.) It really doesn't take a lot of snooping around to make this observation. I will absolutely not consent to having my data processed in pursuit of any AI integration, and I'll be looking at what my options are to prevent that in the coming weeks, should you move forward with this approach. Share Improve this answer Follow answered yesterday Makoto's user avatar MakotoMakoto 48.2k1313 gold badges8787 silver badges197197 bronze badges 15 * 2 "The attitude from the tech industry is that AI is this panacea that will solve problems or make things easier", I mean, it will allow me to write BS that is asked of me. Stuff that I know will have zero impact on the business but everyone loves to ask others. - Braiam yesterday * 10 The CC BY-SA license all your contributions have been labelled with, permits SE to use your contributions as they see fit. Legally, you already consented to have your data used in the pursuit of AI integration, and you cannot revoke this license, so you can't prevent it. - Andreas detests censorship yesterday * 6 @Andreasdetestscensorship: I guess the option I'd have is to cease further contributions. There's too many ethical matters at stake for me on this one. - Makoto yesterday * 1 @Makoto How believable and trustworthy is the idea of ceasing contributions? We still remember the highly turbulent years. I logged off for 2,5 years, because of it, but just came back 1-2 months ago. And now there's this. I'm not sure if I have faith in our ability to make these demands, and following up on them. - Andreas detests censorship yesterday * 5 @Andreasdetestscensorship Not as they see fit. As the terms of the license permit. - Bryan Krause yesterday * 2 @BryanKrause "You are free to: Share -- copy and redistribute the material in any medium or format; Adapt -- remix, transform, and build upon the material for any purpose, even commercially. Under the following terms: [attribution, same-license]". Seems pretty free to me. - Andreas detests censorship yesterday * 3 @Andreasdetestscensorship Attribution and same-license are extremely important terms that limit and restrict many uses. - Bryan Krause yesterday * @BryanKrause Those terms are very much not a huge problems for SE creating a tool for SE's use. - Andreas detests censorship yesterday * 4 @Andreasdetestscensorship what do you mean? If the AI uses text I have written in its output or even to get to its output, then the license requires it to attribute me. I don't see how that is even possible here, which would make it impossible to train AI using SE content. - terdon yesterday * 1 @terdon While collecting source data, their automated process compiles a list of all contributors. The link to this list is attached to all output data of the model trained with this source data. - Andreas detests censorship yesterday * 5 @terdon all of the big LLMs are almost certainly trained in part on Stack Overflow content. I don't think the legal issues here are settled at all, but in any case the problem is already present with the current models, the SE model would be nothing new in this regard. - Mad Scientist yesterday * 3 @MadScientist exactly. I am just pointing out that using it within SE doesn't magically remove the need for attribution. Andreas, I don't see how that can actually meet the requirement to reference the specific source by name. But IANAL, and I suspect you're not either, so let's wait and see what kind of monstrosity SE try to foist on us and deal with it then. I bet you it will not have a way of clearly indicating that the text it generates came from that post, but maybe I'll be wrong. - terdon yesterday * @Andreasdetestscensorship: I mean I'm already doing it. I don't ask new questions. I seldom answer questions. I can stop editing and voting on things too with the help of a few extensions that I can install on my desktop and phone. This isn't a challenge. - Makoto yesterday * 1 "The attitude from the tech industry" Rather, the attitude of the marketing departments of the tech industry. And from there CEOs can listen to their marketing department or their engineering department. Even the AI itself, if asked, is far more skeptical towards the use of AI than the marketeers. So it would seem to be that we can outsource the decision-making about whether or not to implement AI in a product from the CEO to an AI and thereby get better results. - Lundin 12 hours ago * 2 @Lundin except that is not AI. It's a language model. We've had this problem before. We call self driving instead of driving assistance, autoaim instead of aim helper; over promise and under deliver. While much of the crap that is asked today would be able to generate an adequate response by language model, and therefore less crap being asked, is a good thing(tm), not all of it can be generated in a way that is useful. - Braiam 4 hours ago Add a comment | 63 Assuming that this isn't SO/SOFT only - what does this actually mean for the communities - and what are the benefits for us? There are fundamental parts of the network that have not really gotten attention - including some relatively 'trivial' feature requests over design changes (like better access to meta and chat) , and reimplementation of features that have been ignored, or put on the back burner due to lack of resources (notifications), or other priorities. We've also have a significant amount of workload from folks 'naively' using ChatGPT and other such LLM/ML tools. I'm not purely sceptical of these tools in context, but I'm wondering looking at the costs of other organisations doing this - with massive GPGPU farms and such. It is incredibly expensive - and well, when features and requests that directly benefit the community now are delayed, I'd ask if this is the right call - and would be concerned that resources that could be put to more direct use is diverted to a moonshot. I've been here long enough to see the rather painful effects that the company getting a bet like this wrong can have. Pivoting to the new hot technology while it's hot makes sense for a startup, but without a solid idea of what the benefits are is concerning. So practically, what are the benefits to the communities, and what are the guarantees that if this fails, that there wouldn't be a large negative impact to staffing and resources as has happened in the past? Critically - is there a plan to turn 'ideas' into a revenue source and clear goals for this team to meet, as opposed to a sense of FOMO? At Stack Overflow, we've had to sit down and ask ourselves some hard questions. What role do we have in the software community when users can ask a chatbot for help as easily as they can another person? As an active user of the site? Experience, and the synthesis of knowledge. Humans make links and put their collected knowledge together. I've even occasionally gotten bespoke answers to unusual issues because someone found it interesting. ChatGPT writes like a 13-year-old trying to do a literature review. We are excited about what we can bring to the fast moving arena of generative AI. One problem with modern LLM systems is that they will provide incorrect answers with the same confidence as correct ones, and will "hallucinate" facts and figures if they feel it fits the pattern of the answer a user seeks. Yes, we noticed. We've even banned the use of many of these tools on many of our sites. They're basically emitting entirely plausible garbage, and that's worse than complete garbage. . Grounding our responses in the knowledge base of over 50 million asked and answered questions on Stack Overflow (and proprietary knowledge within Stack Overflow for Teams) helps users to understand the provenance of the code they hope to use. We want to help coders stay in the flow state, allowing them to create with the latest tools with the confidence that they will be able to document and understand the provenance, source, and context of the code being generated. While you have a more focused corpus of information, it doesn't necessarily mean that the generative AI/LLM models you use will actually understand what you're doing. Maybe this might lead to better search, but I feel like outside the lowest hanging fruit, unless you've got something up your sleeve OpenAI and Google doesn't have, it's hopelessly optimistic. I'd also question how feeding information into an LLM model works with our licence. Would GenAI-processed results still be under the CC-BY-SA licence and how would you properly attribute the original source of the question? Incorporating generative AI technologies into the organizations using Stack Overflow for Teams will allow us to layer a conversational interface on top of this wealth of information. My previous job was with a vendor for the local government. Outside the 'data cannot leave the country', the very idea of feeding our data to an external source would freak my former boss out. There's a news article of information leaking as well at Samsung, including confidential source code. This might not be as good an idea as it seems. --------------------------------------------------------------------- RESPONSE by Philippe: The goal is to leverage the latest technology to add value for the Stack Overflow community. We are working closely with community members throughout this process to experiment with GenAI to build solutions to solve for historical pain points with the site. We would love to explore the following areas with community members and will remain open to feedback as we test and learn about these subjects: + Guided question asking to improve question quality + Tooling for the site and content moderation tooling to reduce the load on moderators + Content recommendations to improve discovery and lower duplication + Personalization of search experiences to improve content discovery + Trends and insights on emerging topics in the community to highlight new knowledge contributions and subject matter experts For example, historically, community members have manually provided feedback to newer community members asking questions on the site. Leveraging GenAI to give newer community members real-time feedback on asking questions that are appropriate on Stack Overflow might reduce the load on community members. We will also work with the Moderators and power users on the site to test and refine the application of these AI solutions going forward: we are committed to building, updating, and curating high-quality knowledge on our site and ensuring these voices are an important part of the feedback cycle as we test these newer solutions to existing community challenges. -- Philippe Share Improve this answer Follow edited 6 hours ago Philippe's user avatar PhilippeStaffMod 13.3k66 gold badges2929 silver badges4646 bronze badges answered yesterday Journeyman Geek's user avatar Journeyman GeekModJourneyman Geek 148k3535 gold badges265265 silver badges590590 bronze badges 1 * We won't know what it really means until summer I am guessing, when they will actually share what Prashanth teased in the post. - TylerH 4 hours ago Add a comment | 57 feature-request Please provide an official interpretation of the CEO's wording for regular Meta Stack Exchange users. The above post, as it was originally intended for a different audience, uses a communication style that might be magnificent for the target audience and the publishing platform, but the "storytelling methodology" has resulted in the writer including a lot of stuff that looks to be irrelevant for this context and space. AFAIK, most of Meta Stack Exchange's regular users aren't CEOs, aren't top decision makers handling advertising budgets, aren't finance market stock buyers, aren't top-notch tech influencers. Most users are "regular" folks interested in having one or more healthy Q& A communities. For many of these communities, Stack Overflow is the template for the Q&A model and it informs their initial and ongoing operation. It's clear that without Stack Overflow the other sites that support these communities can't exist, that is not being questioned. While many of the current communities were proposed and supported by Stack Overflow users from their conception to their launch, each community has developed their own culture and workings. This observation is relevant as most communities are being affected by ChatGPT and have reviewed the possibility of banning generated text content. Many of them have done this. What are the official implications and repercussions of the current CEO's words for the Stack Exchange communities? I'm not asking that you write down the roadmap or describe the features that are coming as it's clear that you will do that when you are ready. I'm not asking you to add content, but rather communicate the same content in a way that is easier to understand for the regular audience of this space. Share Improve this answer Follow edited yesterday Oleg Valter is with Ukraine's user avatar Oleg Valter is with Ukraine 7,78333 gold badges1818 silver badges4646 bronze badges answered yesterday Ruben's user avatar RubenRuben 10.7k22 gold badges1919 silver badges5656 bronze badges 7 * 15 I certainly don't struggle to read through the marketing "mud", but what a waste of time it is. - Andreas detests censorship yesterday * 17 I wholeheartedly agree with the criticism about the writing style. Why do people write things in such a roundabout way if they could straightforward present what all this is about in an as concise as possible manner? It always gives the impression that it wants to look like more than it really is in the end. What a pity. I feel like I really need an AI to understand what some people are saying. - Trilarion yesterday * 5 @Trilarion Obviously because the intention is to not be straightforward. If they get straight to the point, people will have a definitive answer, and take a stand. This writing style keeps everyone on the edge, hoping it isn't as bad as it really is. It's manipulative. - Andreas detests censorship yesterday * 9 Apparently they accidentally posted the unedited ChatCEO output. PS Some hallucinations: "we want to craft models that reward the users who contribute" ?? "It is precisely this symbiotic relationship between humans and AI that ensures the ongoing relevance of community-driven platforms like Stack Overflow." ?? - philipxy 20 hours ago * 2 This! I really want to know if this blog is "I'm modern and with the AI times and a brilliant tech innovator CEO visionary invest now and gimme lots of money" or "I actually want to do stuff with LLMs". I really hope it's the first - Erik A 13 hours ago * 1 You can always vote to close as 'needs details or clarity'... - user3840170 11 hours ago * 2 My best effort at a non-marketing version: "Stack Overflow (the company) has spun up a new team of employees dedicated to training a LLM (and performing product research in general) based on network Q&A content with the goal of increasing user satisfaction as well as bringing in more users to the network. We will share more about what we have been working on some time this summer." - TylerH 4 hours ago Add a comment | 55 I've tasked a dedicated team to work full time on such GenAI applications. Let's get things straight. The company has the budget to waste on "GenAI applications" (whatever that is supposed to mean), but doesn't for: * keeping Traducir open-source (in case anyone happens to not know why it's important: it powers community-lead efforts for providing translation strings for localized versions of Stack Overflow); * fixing issues and improving the public Stack Exchange API (I am not even counting MSE and MSO requests, just the Stack Apps ones); * fixing issues reported on MSO and MSE, as well as implementing feature requests (MSO, MSE); The list is far from being exhaustive. In light of the following statement: Community and reputation will also continue to be core to our efforts. How does syphoning limited resources from the enormous list of problems and improvements that can be implemented across the network to something that is considered wasted effort at best by many members of the said community (the general tone of the discussion under the post is proof enough of that) correlate with the claim that the community is at the core of the company's efforts? Share Improve this answer Follow edited 15 hours ago answered yesterday Oleg Valter is with Ukraine's user avatar Oleg Valter is with UkraineOleg Valter is with Ukraine 7,78333 gold badges1818 silver badges4646 bronze badges Add a comment | 53 Currently about 60 sites in the Stack Exchange network ban the use of ChatGPT in them. Now that Stack Exchange it taking a leap towards AI, does it mean it will also override those bans and allow ChatGPT (and other AI generation programs) posts all around the network? (Personally I really hope that they'll let the bans stay, but have to ask.) --------------------------------------------------------------------- RESPONSE by Philippe: The existing bans on ChatGPT content are intended to do a few things: 1. prevent plagiarism on the sites, since ChatGPT does not attribute content. 2. prevent a tidal wave of content that LOOKS like it might be correct but isn't. There remains a need for both of those things, and we will support those SE sites that choose to continue to ban answers that are created by generative AI programs like ChatGPT. Internally the work that we are doing toward integrating AI solutions is geared toward using versions of this technology to support the community's efforts and goals. We are committed to building solutions that benefit the community, with the quality of the content on our sites always at the front of mind. We are committed to building trust in the collective knowledge of the community on the site. This means we will highlight where knowledge came from and prohibit the creation of knowledge that looks superficially solid but isn't accurate (such as using AI to create answers that have not been reviewed, edited or verified by a community member). We also will not be using AI to create unattributed answers. -- Philippe Share Improve this answer Follow edited 6 hours ago Philippe's user avatar PhilippeStaffMod 13.3k66 gold badges2929 silver badges4646 bronze badges answered yesterday Shadow The Spring Wizard's user avatar Shadow The Spring WizardShadow The Spring Wizard 164k2929 gold badges400400 silver badges801801 bronze badges 19 * 2 Okay, first and foremost let me not knee-jerk to this. This is a legitimate question. - Makoto yesterday * 19 But if they decide to override those bans, I'm going to hit the door. There'll be no sense in trying to save what's left. Worse, it'd be damn near impossible to do so. - Makoto yesterday * 2 just seems like a silly question to me. The bans that are in place are banning a particular abuse of chatgpt that isn't allowed anyway. generative ai existing on the site won't suddenly make plagiarizing content allowed. - Kevin B yesterday * 17 @KevinB: Given that the type of clientele a ban on ChatGPT is meant to prevent, said clientele seeing/reading "AI on Stack Overflow is OK now" isn't going to exactly take the time to really parse out what that means in specifics... - Makoto yesterday * 2 @KevinB I beg to differ. When a company officially leading AI projects, having half their sites, including the main one, ban it doesn't look good. And in business, what matters most is appearance. I can totally see this happening. - Shadow The Spring Wizard yesterday * @Makoto well, I won't quit SE, but yeah that would make lots of good people to leave. - Shadow The Spring Wizard yesterday * @Makoto You're welcome to stay in the chat rooms, though. And I'm sure you'll continue finding answers here, until they're overgrown by the jungle created by AI. - Andreas detests censorship yesterday * @Andreasdetestscensorship I don't think SE will actually use AI to generate posts. More likely something more subtle, e.g. using it to determine questions quality, finding things out of place (voting rings) and more like this. It would be useful, yes, but like I said in the answer, the point is they add water to the wave that will drown the world, even if those water are useful in a way. - Shadow The Spring Wizard yesterday * 2 @ShadowTheSpringWizard How does generative AI help with that? - Andreas detests censorship yesterday * @Andreasdetestscensorship oh, missed that. blush. Still, I doubt they'll really use it to generate posts. So maybe help them write blog posts. - Shadow The Spring Wizard yesterday * 4 @ShadowTheSpringWizard Oh, no. More posts about blockchains and the next successful crypto wallet thing. - Andreas detests censorship yesterday * 3 "Currently 89 sites in the Stack Exchange network ban the use of ChatGPT in them." ... I wonder if the CEO was even aware of this fact when he wrote this blog post. - Travis J 17 hours ago * 1 @ShadowTheSpringWizard - I agree. Can't come up with a better way to do it though. Open to suggestions! - Philippe StaffMod 6 hours ago * 2 Yeah, but I assure you that won't get built today, so working with what we've got.... - Philippe StaffMod 6 hours ago * 1 @Philippe You could use the comments for feedback: "Upvote this comment if you like the response, downvote this comment if you don't like the response". (yes, this is a joke) - Bryan Krause 4 hours ago | Show 4 more comments 32 Disclaimer: I am personally fully against AI-generated answers/ solutions on SE. While this answer is somewhat harsh, I do feel the criticisms within it are fair. --------------------------------------------------------------------- ... John Maynard Keynes wrote in 1930 that new technology meant humans would be working 30 hours a week or less, and that the main challenge would be what to do with all our free time. ... Can we get a citation for that "30 hours a week or less"? NPR and The Guardian both say 15 hours, not 30. Additionally, assuming you are referring to this document written by Keynes, it says "Three-hour shifts or a fifteen-hour week may put off the problem for a great while." (page five). ... Even before the latest wave of AI, a third of the code being written on popular code repositories was authored by an AI assistant. Again, according to what? I haven't looked, but what specifically does "popular code repositories" actually mean? And how is this (un-named) source determining how much of the code is AI-assisted? Is it some statistic from a product like GitHub Copilot? Also, when exactly do you consider the latest wave of AI to have started? When GH Copilot was released? Or when ChatGPT was released? Or some other point in time? You make a claim, but it is largely unclear and contains no source at all. Today, sophisticated chatbots, built on top of cutting edge large language models (LLM), can write functional code for a website based on nothing more than a photo of a rough sketch drawn on a napkin. They can answer complex queries about how to build apps, help users to debug errors, and translate between different languages and frameworks in minutes. At Stack Overflow, we've had to sit down and ask ourselves some hard questions. What role do we have in the software community when users can ask a chatbot for help as easily as they can another person? How can our business adapt so that we continue to empower technologists to learn, share, and grow? That's more fair. I've seen the project about generating the website - I agree, it's pretty interesting. At the same time, though, AI-generated code from chatbots has many limitations, perhaps most notably that it is often flat-out wrong, even if it sounds confident/ correct. This is a large part of why SO and other SE sites have banned ChatGPT-generated answers. It's worth reflecting on an important property of technological progress. The Jevons Paradox shows us that, as innovation allows us to do more, we settle on a new normal, moving the goal posts for what we expect of people and organizations, then competing to see who can find new ways to pull ahead of the pack. For knowledge work, as the cost of an action diminishes, we often do more of it. Abstracting away repetitive or tedious tasks frees technologists up to make new discoveries or progress innovation. Fair enough, that's reasonable. If new AI systems make it possible to create software simply by chatting with a computer, my prediction is that, far from the job of programmer disappearing, we'll end up with millions of new software developers, as workers from fields like finance, education, and art begin making use of AI-powered tools that were previously inaccessible to them. That's quite the run-on sentence. We are enthusiastic about welcoming this next generation of developers and technologists, providing them with a community and with solutions, just as we have for the last 15 years. We've got a dedicated team working on adding GenAI to Stack Overflow and Stack Overflow for Teams and will have some exciting news to share this summer. At risk of being harsh/rude, WHY?! Please no. A very large amount of work and effort has been put into finding and removing AI-content on SO, and the rest of the network (when it is banned on a site). ChatGPT answers are often wrong and not wanted. I cannot speak for others, but I certainly do not welcome that, nor am I excited for that in any way. Harsh, but that feels like you are flat-out oblivious to the fact that (overall) the community does not want ChatGPT answers (per the vote-counts on the ChatGPT ban announcement on MSO a.k.a. the absolute highest scoring post on MSO). [image caption] Community members and AI must work together to share knowledge and solve problems No, we certainly don't. SO and SE has been solving problems and sharing knowledge for many years without ChatGPT. It is certainly not a "must". I'm not alone in thinking AI might lead to an explosion of new developers. I've heard similar sentiments expressed recently by ... Bill Gates, ... Geoff Hinton, ... and by Stephen Wolfram ... Each sees in today's AI the potential for the loss of certain jobs, yes, but also, if history is a guide, a future in which a great variety of more highly skilled work becomes available to an even larger group of people. Just as tractors made farmers more productive, we believe these new generative AI tools are something all developers will need to use if they want to remain competitive. Given that, we want to help democratize knowledge about these new AI technologies, ensuring that they are accessible to all, so that no developers are left behind. Sure, some developers are excited about it. I also find it interesting. But when you say "we want to help democratize knowledge about these new AI technologies", does that mean you want to share information about those technologies? I don't have an issue with that, and doing so is good (assuming it is on-topic, of course)... but actually adding AI into SE is different. Can you clarify which you're referring to? I talk to developers of varying experience levels all of the time, and I've been hearing anecdotes of novice programmers building simple web apps with the help of AI. Most of these stories, however, don't begin and end with an AI prompt. Rather, the AI provides a starting point and some initial momentum, and the human does additional research and learning to finish the job. The AI can debug some errors, but is stymied by others. It can suggest a good backend service, but often can't solve all the points of friction that arise when integrating different services. And of course, when a problem is the result not of instructions from a machine, but human error, the best answers come from other people who have experienced the same issues. Yes. Exactly. Sometimes, it does it correctly. Sometimes, it can't. The best answers do come from other people, not chatbots. For more experienced programmers, AI will be an amplifier of their existing skill, making them more ambitious in their projects. The result, as Jevons would predict, is that they spend more time with AI, but also more time creating new ideas, researching new topics, and asking new questions that had not occurred to them before. They feel empowered to reach farther beyond their traditional skillset and to push the boundaries in terms of the kind of work they want to take on. I'm not convinced of your first sentence, but that's just my personal viewpoint, though. One problem with modern LLM systems is that they will provide incorrect answers with the same confidence as correct ones, and will "hallucinate" facts and figures if they feel it fits the pattern of the answer a user seeks. Grounding our responses in the knowledge base of ... questions on Stack Overflow (and ... knowledge within Stack Overflow for Teams) helps users to understand the provenance of the code they hope to use. ... That's a good goal... but, how exactly will you "ground ... responses in the knowledge base of 50 million asked and answered questions"? Specifically, how will you ensure only correct answers? This is way easier said than done - if it was practically do-able, it would have likely already been done. Community and reputation will also continue to be core to our efforts. If AI models are powerful because they were trained on open source or publicly available code, we want to craft models that reward the users who contribute and keep the knowledge base we all rely on open and growing, ensuring we remain the top destination for knowledge on new technologies in the future. Yep, the community is very important. But could you elaborate on this a bit? Like, community members will get rewarded when a chatbot answer based on theirs is marked helpful? What exactly does this mean? AI systems are, at their core, built upon the vast wealth of human knowledge and experiences. They learn by training on data - for example open-source code and Stack Overflow Q&A. It is precisely this symbiotic relationship between humans and AI that ensures the ongoing relevance of community-driven platforms like Stack Overflow. Allowing AI models to train on the data developers have created over the years, but not sharing the data and learnings from those models with the public in return, would lead to a tragedy of the commons. It might be in the self-interest of each developer to simply turn to the AI for a quick answer, but unless we all continue contributing knowledge back to a shared, public platform, we risk a world in which knowledge is centralized inside the black box of AI models that require users to pay in order to access their services. I'm still not convinced of any benefit from adding AI content into SO, even if it is free. Also, what "symbiotic" relationship? The one in which SO flat-out banned it? Not exactly what I'd call symbiotic at all As the AI landscape continues to evolve, the need for communities that can nurture, inform, and challenge these technologies becomes paramount. These platforms will not only offer the necessary guidance to refine AI algorithms and models but also serve as a space for healthy debate and exchange of ideas, fostering the spirit of innovation and pushing the boundaries of what AI can accomplish. Yes. Innovation is good, as is progress. Productive discussion of those technologies is also good. The next paragraph is little more than an ad for SO Teams, so I'm omitting it. Incorporating generative AI technologies into the organizations using Stack Overflow for Teams will allow us to layer a conversational interface on top of this wealth of information. We believe this could lead to tremendous productivity gains: from new hires being able to onboard more quickly, to speed up developer workflows, as users are able to quickly ask questions and retrieve answers tapping into the company's history, documentation and Q&A. Would it be limited for SO for Teams? Or would it go to actual SO? Not trying to be overly harsh, but can you elaborate on what information a chatbot would be able to give that wouldn't be accessible by searching? The example above is just one of many possible applications of GenAI to our Stack Overflow public platform and Stack Overflow for Teams, and they have energized everyone at our company. We'll be working closely with our customers and community to find the right approach to this burgeoning new field and I've tasked a dedicated team to work full time on such GenAI applications. I'll continue to share updates through channels such as my quarterly CEO blog, but I'll be back in touch soon to announce something big on this topic. In the meantime, thank you to our community and customers for continuing to help us on our mission to empower the world to develop technology through collective knowledge. NO please. I do NOT want generative AI in the public SO. Also, is everyone at the company actually excited about this? I suppose maybe, but it seems like an exaggeration. Not going to name any names, but what about the previous statement (from a staff member, on MSE), that SE would "work internally progresses on identifying these posts and making our systems more resilient to issues like this in the future" (referring to ChatGPT-answers). That answer made me think SE was going to help communities deal with AI-answers. Can you clarify if that answer is still true? Also, my answers on SE are intended for humans, not for training a machine learning model with. As someone that has personally put in a large amount of effort and time into dealing with AI-generated answers on SE, I'm extremely disappointed to hear that you may incorporate generative AI models into SO. Share Improve this answer Follow edited yesterday answered yesterday cocomac's user avatar cocomaccocomac 2,33722 gold badges88 silver badges3434 bronze badges 3 * 8 "a third of the code being written on popular code repositories was authored by an AI assistant" Another problem with that is that amount of code is not a good quality metric. The two-thirds of code not being authored by an AI assistant are probably the more interesting parts. - Trilarion yesterday * 2 for lazy readers like me, would you be okay with narrowing down the quoted sections? For some of the quoted sections, it's hard to tell when skimming what part of the quoted section you're responding to. If you want to keep it for context, that's fine though. - starball yesterday * 1 @starball I've narrowed it down some, once I'm at my laptop again I'll see if I can trim the quotes any further (without removing the meaning). But yep, the quotes weren't clear about what specifically I was referring to - cocomac yesterday Add a comment | 24 Note that I've flagged ~800 GPT answers on Stack Overflow. I was honestly hoping for a bit more information about how Stack Exchange as a company (rather than the individual sites) plans to handle the deluge of unverified AI answers that all too often turn out to be hallucinations. Instead, I ended up not quite sure what was being announced, even after reading it through twice. I do turn to ChatGPT myself for its ability to re-word and summarize information, so I decided to ask it to summarize the post to see if there was something I missed: Prashanth Chandrasekar, CEO of Stack Overflow, believes that artificial intelligence (AI) will not replace programmers but rather will create millions of new software developers from fields like finance, education and art who previously didn't have access to AI-powered tools. Sophisticated chatbots, built on top of cutting-edge large language models, can already write functional code and answer complex queries about how to build apps, debug errors, and translate between different languages and frameworks in minutes. While AI provides a starting point and some initial momentum, humans do the additional research and learning to finish the job. AI will amplify existing programming skills and help programmers spend more time creating new ideas, researching new topics, and asking new questions that had not occurred to them before. Stack Overflow plans to add GenAI to its products, democratize knowledge about new AI technologies, and provide a community and solutions to the next generation of developers and technologists. (emphasis added) This was really in line with my personal reading of the post. Since everything but the last sentence seems to me to be rather commonly accepted, I guess the "announcement" can be boiled down to the final sentence, which still leaves things quite open. Share Improve this answer Follow edited yesterday answered yesterday NotTheDr01ds's user avatar NotTheDr01dsNotTheDr01ds 1,08533 silver badges1111 bronze badges 12 * 1 seems like it did a rather poor job of summarizing the post. - Kevin B yesterday * 8 @KevinB It's pretty much what I got out of the post after reading it several times. Did you have a different takeaway? - NotTheDr01ds yesterday * 6 " we're adding GenAI to SO and SOfT " - Kevin B yesterday * 5 @KevinB Ah, so we did pretty much have the same takeaway, it seems ;-) - NotTheDr01ds yesterday * Just curious, how did you find all those 800 answers? You have some script? - Shadow The Spring Wizard yesterday * 6 @ShadowTheSpringWizard There's several techniques for identifying CGPT (including watching bountied questions, or searching for recurring phrases), but it does help that there's an absurd quantity. 800 posts was the daily volume at one point (and that wasn't even close to the peak volume, though I don't remember the numbers) - Zoe stands with Ukraine yesterday * 1 @Zoe huh, and I thought I was spending lots of time on SE... :D - Shadow The Spring Wizard yesterday * 2 @ShadowTheSpringWizard To be clear, the numbers are for Stack Overflow only. I don't know what the numbers were network-wide. If you had visited and browsed new posts on SO daily, you too would've found a lot of CGPT, especially in December - Zoe stands with Ukraine yesterday * 4 @ShadowTheSpringWizard While I'm aware of scripts to help detect, I use a far more "manual" method that I honestly would prefer stay "loosely guarded". At this point, I'm of the belief that we have some users (a small minority of the total GPT answers) that are actively working to evade detection, and I'd like to not give them any more ideas on how to do so ;-) - NotTheDr01ds yesterday * 2 I'll throw out one of the more "obvious" methods, though, in hopes of at least slowing it down. When I do spot an answer that I might suspect is GPT-based, it's a dead giveaway if the user posted a detailed answer just 3 minutes after their previous (also GPT) answer :-). E.g. 20 long answers in 2 hours is pretty suspect. - NotTheDr01ds yesterday * Thanks for the ChatGPT summary, I guess. ;) But I'm still not clear on what "Community is the future of AI" is supposed to mean... - PM 2Ring 19 hours ago * @NotTheDr01ds I see. One thing though, if one changes AI generated answer so it becomes something else, it might become valid. But that's really case-by-case thing. - Shadow The Spring Wizard 15 hours ago Add a comment | 23 Incorporating generative AI technologies into the organizations using Stack Overflow for Teams will allow us to layer a conversational interface on top of this wealth of information. We believe this could lead to tremendous productivity gains. I think it'd be more beneficial and productive to work on better search functionality, and better possible-duplicate detection in the Ask Question module Generative AI is working with past information- probably taking from our past Q&A. 1. If it can answer a new question, I'd have suspicions about the new-ness of the question. 2. If it can't (likely when faced with a new question), then you should just leave it to your community of experts who actually know what they're talking about. In either case, what's important on the asker's side is to search, and what would be more useful to them (and anyone else looking for possible duplicates of a question) is better search functionality. I'm confused about whether this is coming to SO/SE or just SO for Teams Incorporating generative AI technologies into the organizations using Stack Overflow for Teams will allow us to layer a conversational interface on top of this wealth of information. This paragraph talks about integrating AI into SO for Teams and doesn't mention SO, but the next paragraph mentions both: The example above is just one of many possible applications of GenAI to our Stack Overflow public platform and Stack Overflow for Teams. So which is it? Are you just adding AI to SO for Teams? Or also to SO (/ the rest of the public Stack Exchange network)? I don't care much about SO for Teams. But if AI is coming to SO or the rest of the Stack Exchange sites, I really hope it's put in a completely separate area in each network site than regular Q&A. For example, I don't want to see AI-written answers under question posts. Either that, or you've got to (not really, but it sure would be nice if you'd) talk with us and think real hard about * How to make sure bad AI-generated content is easy for people to vote down (I'm thinking of rep thresholds) * How that content is going to age over time. What would you do when the language model updates? What would you do with already-written answers with older language models? * What moderation burden is this going to put on the community and its moderators? --------------------------------------------------------------------- For the rest of my thoughts, see also my answers to * Could ChatGPT be a viable way to answer people's questions? * The future role of Stack Exchange vs. emerging AIs --------------------------------------------------------------------- various other commentary: We are excited about what we can bring to the fast moving arena of generative AI The community has already been bringing it since inception. We write content licensed under CC-BY-SA. The language models are using our community-written content (whether the way they use it is allowed under CC-BY-SA, I don't know. I'm not a lawyer). AI systems are, at their core, built upon the vast wealth of human knowledge and experiences. They learn by training on data - for example open-source code and Stack Overflow Q&A. It is precisely this symbiotic relationship between humans and AI that ensures the ongoing relevance of community-driven platforms like Stack Overflow. So yes. Agreed :) and with the rest of that paragraph. Except that I wouldn't say "symbiotic". Maybe "parasitic" (with Stack Exchange being the host organism). Even when people "contribute" AI-generated answers to questions here, it's often a hallucination (and against the site rules because of the hallucinations and other reasons like moderation and curation burden). Our thesis on community as the center of a safe, productive, and open future for AI also offers some exciting prospects for our business Most of what was written up to that point was pretty grounded / concrete (at least- I thought so). But I don't really get this statement. It seems a bit wishy-washy to me. I suppose community is important in that that's where voting and editing comes in (curation), but I'd say the bigger source of value in what we do here is freely contribute expertise. The experts are (in my current view) the starting point of what makes Stack Exchange valuable. I don't think it's true to say that all the experts here are here for community. They might be attracted to here because of the standard of quality and name-recognition of the large network sites though (not sure what either of those things has to do directly with community). Or they might be here for magic internet points and/or to alleviate an inferiority complex. Regardless of what is central to the future of AI, what is really central to Stack Exchange (which is what I care about) is fact-based Q&A. To say community is central to Stack Exchange (not that anyone has said that here) could be misleading, since we're not a discussion platform and we don't do chit-chat (except in chat subdomain sites). Side note: Another problem I've seen with ChatGPT answers on SO is that they often have a bunch of "buffer info"- info that may be useful, but really isn't necessary in the context of the question. One of the selling-points of Stack Exchange (see the tour page) is that we avoid noise. Share Improve this answer Follow edited 19 hours ago answered yesterday starball's user avatar starballstarball 6,35222 gold badges99 silver badges4646 bronze badges 9 * 1 "I don't want to see AI-written answers under question posts." Many of us fully agree with you. However, the CEO didn't quite say that AI would be used to generate answers. OTOH, he didn't tell us exactly what it will be used for, either. - PM 2Ring 18 hours ago * 8 IMHO, there is room for LLM-based tools on the network. Eg, a search tool with a conversational interface that can help find related questions and dupes. A more intelligent wizard to guide question creation. A tool that can rephrase questions and answers to eliminate typos and awkward grammar, although of course there's a risk that the LLM will alter the meaning, which may be difficult for the author to detect, especially if English isn't their native language, or they're using unfamiliar jargon. - PM 2Ring 18 hours ago * 2 LLMs are capable none of these things. All it could do is gaslight visitors. - chx 16 hours ago * 1 @chx LLMs can do useful processing of language, but you do need to prompt them appropriately, and be aware of their limitations. It also helps if you have some idea of how they work, and realise that they're primarily operating on syntax, with only a very tenuous grasp of semantics. I said more about that here. Unfortunately, many people get misled by GPT output, mistakingly thinking that it knows what it's talking about. - PM 2Ring 15 hours ago * @PM2Ring most importantly, if you have no idea what the answer is, then the generated content can be helpful or dangerous. Thus you also need to be at least generally aware of what the answer is. At least for technical topics. You don't need to if you're trying to generate ideas for a story or something. - VLAZ 15 hours ago * 1 LLMs can do useful processing of language <= absolutely can not. They can provide a word which is most likely to follow previous words. Nothing about language there. Finding meaning in the word salad it spouts is apophenia, nothing more. - chx 15 hours ago * 3 @VLAZ Sure. Using GPT to actually answer technical questions is madness. It'd be safer to get financial advice from the Wolf of Wall Street. ;) And if I were a programming teacher, I'd forbid my students from looking at the code it regurgitates. I learned to be a good coder by studying the code of excellent programmers. ChatGPT has read a lot of code, both good and bad, but since it doesn't really know what it's doing, we have to classify its behaviour as cargo-cult coding. - PM 2Ring 15 hours ago * 3 @chx Here's a crude analogy. ChatGPT is like a person doing a jigsaw puzzle. You give it a prompt of a few joined pieces and it adds more pieces, one by one. The picture on the jigsaw consists of a bunch of words written in English, and the jigsaw pieces are cut so that when you join them together the result will be syntactically correct English. However, the guy doing the puzzle doesn't understand English. When we see the result, it looks impressive because we understand English, but ChatGPT just knows how to stick bits of jigsaw together. - PM 2Ring 15 hours ago * 2 FWIW, I posted that analogy in this chat conversation: chat.stackexchange.com/transcript/message/63357211#63357211 - PM 2Ring 15 hours ago Add a comment | 22 I came to this announcement to post an answer after noticing some worrisome exchanges about "SE is going to use generated content" in the chat rooms. I came here to verify those rumors and to post a concrete answer to what the actual "plan" was. Yet after reading the "announcement" multiple times... I don't yet know what the plan is. I can't see what you are talking about. I can't see what your ideas are. I can't see what you are going to do. Nor do other users apparently since most answers start with a "if you mean to...". This announcement could be summarized as a haiku and no actual meaning would be lost. GanAI is future Many cool guys say that too Let's jump the vagon All I can see is that (as Zoe pointed out in a comment) this actually feels more like an "early announcement" than a "we want your feedback on our plan" post. If anything, because if you wanted actual feedback we would at least need to know WHAT the plan actually is. Oh, yes, you then say that "soon" you will be telling us what the "big plan" is, but that would only mean that we can't have any question now, right? Also, does this "soon" mean "after we already committed to it, you will get to know when we have already decided and your feedback won't matter anymore"? What is the point of asking us to "leave any questions that you have below" NOW if we actually have to GUESS YOUR INTENT because you didn't tell us what this is all about??? tripleee speculated about three different scenarios in their post. For starters... I guess it would be nice to know if any of these actually fit what your are planning to do. Share Improve this answer Follow edited 9 hours ago answered 14 hours ago SPArcheon's user avatar SPArcheonSPArcheon 19k22 gold badges4444 silver badges9191 bronze badges 2 * 4 What we got: "I'll continue to share updates through channels such as my quarterly CEO blog, but I'll be back in touch soon to announce something big on this topic." it's a s u r p r i s e !! - starball 14 hours ago * 4 << What is the point of asking question NOW>>. Well, because it's not a question, but a statement. They don't seem to want feedback. - Andreas detests censorship 14 hours ago Add a comment | 17 Please do not integrate this algorithm into the site. It plagiarizes The algorithm will reproduce whole sections of one to one content without citation nor warning. It then strings these sections together with transitional words such as however, additionally, or furthermore. It cannot generate Aside from the issue of reproducing existing works without citing them, the algorithm cannot actually generate legitimate content which fails at its own primary goal. Instead, any time the algorithm generates content, it simply goes with the most likely to occur series of content; as opposed to the correct series of content. This is dangerous at worst, and "hallucinating" at best. It acts as an authority The algorithm presents all of these flaws, the issues with citation and of [S:lying:S] hallucinating, as if they are canon; as if each statement, code, or demonstration were 100% accurate and trustworthy. Integrating something which presents content in such a manner is negligent for a place aimed at being a repository of knowledge. --------------------------------------------------------------------- There is no need to integrate an algorithm which has these traits. It is dangerous, negligent, and ethically questionable. Share Improve this answer Follow answered 20 hours ago Travis J's user avatar Travis JTravis J 31.3k44 gold badges7272 silver badges138138 bronze badges 11 * 3 You are assuming that they will include the current version of ChatGPT as-is, which is likely not the case. There is a lot of ongoing research into how to solve the issues you have brought up. Perplexity AI comes to mind, which cites its sources and is thus (in theory) no worse than human-generated content. - The Guy with The Hat 19 hours ago * 3 GPT is a lossy compressor: the compression ratio for GPT-3 is ~50, which is the ratio of the size of the training data to the size of the parameters. (We don't have the numbers for vanilla ChatGPT or GPT-4). I agree that it doesn't generate, it stochastically re-generates. And it only chooses the most likely token when its temperature parameter is zero (and there's only 1 most likely token). But the temperature is usually set to a non-zero value, which allows it to produce more "interesting" sequences. - PM 2Ring 19 hours ago * 5 @TheGuywithTheHat - It is all based on the same algorithm though, the transformer algorithm: "a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely". This algorithm leans more towards simply producing existing language as opposed to learning concepts and applying them; it is also entirely the basis of today's "chatbots" which all suffer the same drawbacks. - Travis J 19 hours ago * 3 @TravisJ I am aware of how they work, my career is in deep learning and I have used GPT as early as 2019. A transformer, while technically an "algorithm", is not the algorithm. A transformer or transformer-based model is only a single part of the entire system that creates and uses the LLM. While practically all LLMs are currently based on transformers, differences in how they are trained and used can give vastly different results. It is entirely feasible for Stack Overflow to develop a cross between ChatGPT and Google, with most of the benefits and few of the drawbacks of each of them. - The Guy with The Hat 16 hours ago * 1 @TheGuywithTheHat - The training will only effect what it can reproduce, and the use will only effect the style of reproduction. However, it is still just that, reproduction. There is no creation, no error checking, and that is by design. Too many corners were cut to create this in a narrow scope, and tweaking minor aspects of the design won't alter the core functionality. It needs to be able to dog food its own productions, and that gets back into more complicated Neural Networks and away from this Transformer algorithm. - Travis J 16 hours ago * 4 There are a lot of assumptions in your comment. I think that MSE comments are not the place for an in-depth discussion. "it is still just [reproduction]" -- Does that matter? "There is no creation" -- It depends on how you define creation, and the model's output is still equally useful regardless of how you define the word. "no error checking" -- ChatGPT does not currently have error checking, but there is no fundamental reason that error checking cannot be built into a system with an LLM. "and that is by design" -- for current ChatGPT, yes, but not necessarily for other/future LLMs. - The Guy with The Hat 15 hours ago * 1 Assumptions? Perhaps your bias, unintended or not, may be at play here. It absolutely does matter when it is purely reproduction but does not cite the source. That is the core issue of plagiarism. I feel like I am just repeating my points. Without error checking, combining multiple aspects of fact for a wild assertion (its version of producing, not necessarily creating) is a very large problem with the outputs being given. Again, yes, this is all by design. - Travis J 6 hours ago * From a practical perspective: I tried ChatGPT and it was helpful in 1 out of 4 cases, so better than nothing but not better than say the average human. It's definitely not an expert but its grammar is impeccable. - Trilarion 6 hours ago * @TravisJ Let me summarize: You are correct about the current state of ChatGPT. You are not correct about LLM systems in general. - The Guy with The Hat 5 hours ago * @TheGuywithTheHat - I am not talking directly about ChatGPT so much as the underlying technology that fuels it, which as you say you are familiar with. That technology is being used as the underpinning for the current trend of using or training LLM's. If there were an LLM that was treated more as its broader category of NLP (where comprehension has a heavier weight) and it used a real neural net (as opposed to this simplified version), then perhaps I would agree; but I haven't seen any of those, and certainly the entire market now is transformer. - Travis J 4 hours ago * @TravisJ GPT is absolutely a real neural net; there's no fundamental reason that transformers are any less "real" than other types of NNs. Additionally, even GPT-3 uses more than just a basic attention layer. Additionally, OpenAI hasn't released the model architecture for GPT-4, so we don't know what is used in the latest model. Additionally, even if we assume that the model is "dumb", there's no reason that it can't be included as part of a larger system that does do explicit error checking, citing sources, and other "smart" things. - The Guy with The Hat 3 hours ago Add a comment | 15 Welcome to the enshittification of Stack Overflow! No one at the helm is reading or cares about this page. Make no mistake, within less than a year new visitors will get a first answer or at least a question rewording suggestion provided by a ChatGPT-alike. Share Improve this answer Follow edited yesterday This_is_NOT_a_forum's user avatar This_is_NOT_a_forum 6,68244 gold badges3434 silver badges5555 bronze badges answered yesterday chx's user avatar chxchx 1,13666 silver badges1515 bronze badges 8 * 7 Jon Skeet only wrote the first 1000 answers on C# himself. Afterwards he secretly trained a LLM on himself and let it run. - Trilarion yesterday * 1 "new visitors will get a first answer or at least a question rewording suggestion provided by a ChatGPT-alike" This is not inherently a bad thing. It could end up being bad if SO doesn't implement it in a good way, but I don't think we should assume that yet. - The Guy with The Hat yesterday * 1 That is an interesting, if quite long, analysis. For example, "Attention is like cryptocurrency: a worthless token that is only valuable to the extent that you can trick or coerce someone into parting with "fiat" currency in exchange for it." and "This is just what Twitter has done as part of its march to enshittification: thanks to its "monetization" changes, the majority of people who follow you will never see the things you post. I have ~500k followers on Twitter and my threads used to routinely get hundreds of thousands or even millions of reads. Today, it's hundreds..." - This_is_NOT_a_forum yesterday * 4 @TheGuywithTheHat It is 100% to be devastatingly bad. How many of these transitions have we suffered through to think it will be otherwise? Slashdot, digg, twitter, even Google, the list is endless. - chx 17 hours ago * 1 @chx Can you explain how either of those would be bad? In the first case, the user at least gets an initial answer. (They can follow up if it fails, and we can all laugh at the dumb AI.) In the second case, the community gets a better-formulated question. Of course SO might fumble this by instead treating moderators as unpaid filters for AI content, but that's not what your specific examples are about. - lofidevops 4 hours ago * 1 I commented on other answers multiple times: problem is, ChatGPT answers inherently have zero information value, it's a stochastic parrot but apophenia makes people think there is something, they are essentially gaslit. The outcome will be a massive influx of garbage and as time goes on, harder and harder to filter garbage. - chx 4 hours ago * 1 @chx Ok, so your concern is that AI generated garbage will in fact lead to more low quality questions along the lines of "AI told me to rm -rf and now it's broken"? I think that's a fair concern. I am hoping they will use asker-driven gamification rather than unpaid labor to solve it, I guess we'll see. The enshittification article is great btw. I am hoping the economics aren't the same here - lofidevops 4 hours ago * Also, apophenia! I think you've just explained humans to me. - lofidevops 4 hours ago Add a comment | 14 The GenAI hype-train manages to (just about) clear the bar for usefulness set by its illustrious recent predecessors (web3.0 and metaverse). But regardless of its own merits, it's important to recognise that it is currently a hype-train. Public interest is high, the news both tech and non-tech are excitedly reporting about it left and right and the average Joe is lapping that up. But this level of public giddiness isn't sustainable - there will be something else new and exciting that will come along soon and capture the public's interest and generative AI will continue but it won't be grabbing every third headline any more. This isn't a new phenomenon - humans have been getting bored with the latest and greatest for a very long time, just ask NASA how quickly the general public lost interest in the moon-landings. So if SO Inc is wanting to get in on this area - are they prepared for what happens in 3, 6, or 9 months when it's not headline news any more? Is this something that is genuinely in the best interests of the company and the network and would add real value without that hype? Because if the answer is no - then I'd question the wisdom of throwing any real resources at this, and instead cash in on the hype by writing a few blog posts, have some people discuss it on the podcast or whatever. Sure you probably lower your ceilings on potential rewards doing that - but you avoid throwing millions at something and having it turn out to be a boondoggle. Share Improve this answer Follow edited yesterday terdon's user avatar terdon 23.2k66 gold badges4848 silver badges8282 bronze badges answered yesterday motosubatsu's user avatar motosubatsumotosubatsu 2,96511 gold badge88 silver badges1818 bronze badges 1 * 1 Related: JourneymanGeek's answer post touches on the costs of running something like this. - starball yesterday Add a comment | 14 I must be a very unlucky person. These things that seem to sprout like flowers in other people's IT gardens just don't happen to me. Every question I ask the AI comes with errors at best, that's when the AI decides to be creative and starts spewing a bunch of information taken from I don't know where. I'm not saying that AI is useless or anything like that. Far from it, I like AI and I like the promise of social progress that it brings to humanity but what I mean is that it's premature for us to have this type of integration in the community that is the foundation of activities directly and indirectly related to Information Technologies. Asked about the possibility of its participation in the ChatGPT communities, ChatGPT responded: My participation in the ChatGPT communities is limited to providing responses to your questions and engaging in conversation with you. As a language model, I am not capable of actively participating in the same way as humans do. However, I am here to assist you in any way I can and provide you with the information and insights you need. I am convinced that the training of AI is currently not robust enough to meet the needs of our communities. An error made with the endorsement of AI has the potential to spread virally and be incorporated into various systems used around the world. I believe that any AI system must be tested, improved, specialized, and retested before it is made available to our public, and that its responses should be finally curated by members of the Stack Exchange community. Without being polite, I feel like they're saying to me: -Okay, thank you for these years of unpaid work but we don't need you anymore. From now on, we'll have a little system that will work for free just like you, but with one difference, the system it's a bit imprecise and doesn't complain when it sees the mistakes we've made and in fact, it doesn't even see them. Share Improve this answer Follow edited yesterday answered yesterday Augusto Vasques's user avatar Augusto VasquesAugusto Vasques 34633 silver badges1111 bronze badges 5 * it must be my mistake but I edited it and the system republished the answer - Augusto Vasques yesterday * Don't feel guilty, this is something that commonly happens. Apparently you deleted the original post. While there is no rule about how handle this cases, some time is better to keep the oldest post instead of the newest. - Ruben yesterday * 4 "I am convinced that the training of AI is currently not robust enough to meet the needs of our communities." It's not just the training. It's the design itself of the AI that limits it. No amount of training will fix the issues inherent in the model. - Andreas detests censorship yesterday * 3 To be fair, any specialized AI creation from Stackoverflow Inc. would likely take some time to be fully operational. By then, the concern about the reliability of such automated systems could already have been lowered. Starting to work on that now would allow StackOverflow to gain important experience and get a stake in that market (whenever it will come to its full potential). - Trilarion yesterday * 4 Re "I must be a very unlucky person": No, it only takes about 10 minutes to realise ChatGPT is a pathological liar when it comes to factual things. Or it gives very bad advice. Do keep the transcripts as evidence to prove that this is actually the case. But it can be put to good use (e.g., to generate ideas) - This_is_NOT_a_forum yesterday Add a comment | 13 I agree with the general consensus that ChatGPT in its current state can be dangerous when used incorrectly (which is especially likely with inexperienced coders). However, when used correctly, it could be helpful. Grounding our responses in the knowledge base of over 50 million asked and answered questions on Stack Overflow (and proprietary knowledge within Stack Overflow for Teams) helps users to understand the provenance of the code they hope to use. This is the key point that I think a lot of people here are missing. LLMs could point users to existing answers, which doesn't give it an opportunity to hallucinate information. They can also easily solve simple issues and provide tailored guidance for question askers. Many ChatGPT answers are not easy to verify, but that doesn't mean that all of them are like that. If I make a simple mistake such as forgetting a return statement, ChatGPT can see that and point it out to me. I see the potential for a system where an LLM can respond to particularly simple questions before they are even posted, reducing the flood of low-quality questions. There are currently LLM solutions that aim for trustworthy answers that cite sources (perplexity.ai comes to mind). These are of course only as good as the training data, but that training data is by definition as reliable as the answers given by humans. In this way, it could be used as a smarter search engine that is customized for Stack Overflow. Another safe way to integrate LLMs into SO is by providing question-asking guidance to new users. It could be possible for an LLM to say, for example, "Hey, it looks like you are encountering an error, but you haven't included a specific error message in your question. If you haven't already, could you copy-paste the error you're getting directly into your question?" Overall I don't think it will be easy to use LLMs/GenAI in a non-disruptive and safe way, but I am cautiously optimistic that it will be possible to do so. One thing to keep in mind is the self-driving car evaluation problem: do we aim for perfection, or do we just aim for being better than the average humans who are currently doing the task? Perfection is ideal, but better-than-human performance is still worth using. Share Improve this answer Follow answered yesterday The Guy with The Hat's user avatar The Guy with The HatThe Guy with The Hat 6,53333 gold badges2929 silver badges5454 bronze badges 15 * 4 "It could be possible for an LLM to say, for example, "Hey, it looks like you are encountering an error, but you haven't included a specific error message in your question. If you haven't already, could you copy-paste the error you're getting directly into your question?"" Sounds like a very useful application. But I'm not sure it is on top of the priority list. - Trilarion yesterday * 3 @Trilarion Sure, that's why I said multiple times that there are good and bad ways that GenAI could be used, but never speculated on exactly how SO will end up using it. - The Guy with The Hat yesterday * 3 Given all this garbage is capable of is spitting out the next words most likely to follow, @Trilarion's thinking is just wishful thinking. There is nothing and as long as there is not a fundamental change in AI, there will be nothing. Seeing any value or meaning in the output is apophenia. Understandable but still. - chx 16 hours ago * 1 @chx It doesn't matter if the text on your screen was written by a human or an AI, the content is still equally correct or incorrect regardless of its source. If we could train an AI to correctly predict the exact word that I am going to use next, that word is then equivalent to that of a human. There are currently shortcomings with the reliability of LLMs, but they are not insurmountable. - The Guy with The Hat 15 hours ago * 3 @chx I mostly agree, but not fully. It's not totally apophenia / Eliza effect. The network does manage to catch some conceptual / semantic structure as it's learning syntactic structures. GPT-4 is even more impressive than ChatGPT (3.5); OTOH, it's more dangerous because it can be harder to detect when it's uttering nonsense. - PM 2Ring 15 hours ago * 2 It doesn't matter if the text on your screen was written by a human or an AI, the content is still equally correct or incorrect regardless of its source....A Human make errors, and I would be surprised if you never saw any error on any answers on SO/SE before. But at the very least, a Human know how to understand and discern. Based on what you're saying, a simple markov chain model is basically at the same level intellectually as everyone on SO. - Nord The Star Wizard 13 hours ago * 1 "it can be harder to detect when it's uttering nonsense" It is not. What it always utters is neither nonsense nor correct: it has zero information value. Putting information value to it , judging whether it's correct is incorrect is utterly pointless. The best thing you can do is ignore. Might as well listen to your parrot at home, that's what this is: a stochastic parrot. - chx 13 hours ago * 1 @NordTheStarWizard "a simple markov chain model is basically at the same level" -- This is a massive straw man. LLMs are way way way more complex than a markov chain. You correctly point out that humans make mistakes, so there's no need for perfection for LLMs to surpass humans in Q&A, especially if we confine it to only simple questions. There already exist AIs that can provide sources for their claims, and AIs that you can ask to double-check their own work. There's no reason that we can't make an AI that will search additional sources if it is not yet confident enough in its answer. - The Guy with The Hat 13 hours ago * 1 @chx Does SO answers written by humans have value? What if we search google and get snippets of those answers? What if we make an AI that takes the most relevant snippets and concatenates them into a single paragraph? What if the AI then processes the snippets to make them flow smoothly from one to the other? What if we then streamline the process so that the AI does those last two steps simultaneously? It seems like you are arguing that the original answers have value, but the answer produced by the AI has no value. At what point does it lose value? - The Guy with The Hat 13 hours ago * 1 You never mentioned complexity being an issue, though? You only said, "It doesn't matter if the text on your screen was written by a human or an AI, the content is still equally correct or incorrect regardless of its source.". Given even non-complex algorithms such as the "AI" in games, or even markov chain could count here. It does not matter if it's made by a Human, so this is just me taking literally what you said. Also, I never said that LLMs making "mistakes" (which would count as giving them some form of intelligence, which they do not) would be on the same level as Humans. - Nord The Star Wizard 12 hours ago * (cont2) We understand and can infer context without always needing to look things up or having a large dataset. Context understanding is only relevant to Humans. chatbot might give the impression it "understand", or even infer context based on prompt and the statistical properties of it's dataset, but that isn't the same thing. Are you truly sure statistical theory/properties are being used in Humans brain? - Nord The Star Wizard 12 hours ago * @NordTheStarWizard If you want to argue semantics, let's say that I said "LLMs are at a way higher level than markov chains." However, I never claimed that markov chains could not generate a correct answer. I think that they could generate one, it's just unlikely. In comparison, an LLM is much more likely to generate a correct answer, and I see no reason that the probability of an LLM giving a correct answer couldn't exceed the probability of a human giving a correct answer. My point about mistakes is that it doesn't matter whether an AI can correct its mistakes if it makes very few of them. - The Guy with The Hat 12 hours ago * 1 @NordTheStarWizard I am also not claiming that LLM definitely will exceed humans, I'm merely claiming that it is within the realm of possibility. We should not yet assume that it is impossible, and should instead try to see if it is possible. I am not sure that "statistical theory/properties are being used in Humans brain," but I am not sure that that is not the case. If it is, then an AI would be able to emulate it. Even if it is not, then it could still be possible for an AI to simulate it. It is difficult to prove a negative, so for now we should make an attempt to prove the positive. - The Guy with The Hat 12 hours ago * Who really knows what the future of AI will be? Oh wait, I can ask a LLM about that. - Trilarion 12 hours ago * 11 This pretty much aligns with my thoughts on it: perhaps the biggest chore on SO for the past decade or so has been pointing folks to answers that already exist - a task a LLM could in theory do well enough. Not as well as a SME, but perhaps significantly better than a random person using search, and more importantly tirelessly and without complaint. And using computers for soul-deadening chores is always a good use of their time. - Shog9 9 hours ago Add a comment | 11 Quite simply, if people want to use some kind of AI assistance to answer their questions quickly, then they can use those. If they want an actual human, often expert, to answer their questions, they will log into Stack Overflow. I don't understand what the benefits of mushing these two approaches together would be. Stack Overflow's unique selling point is that it (should be at least) expert humans answering questions using their personal experience. Why risk killing this to add something people can do on another site^1? ^1 I know the answer is $$$ Share Improve this answer Follow answered 7 hours ago user438383's user avatar user438383user438383 25211 silver badge88 bronze badges 1 * Re "what the benefits of mushing these two approaches together would be": Perhaps they want the users back that were lost to ChatGPT (users who just want an answer ASAP, any answer, no matter the quality, human or AI-generated). They would count in the KPIs.. - This_is_NOT_a_forum 31 mins ago Add a comment | 10 I am not here to debug ChatGPT. I am not here to debug generative LLMs in general. Share Improve this answer Follow answered 6 hours ago JonathanZ supports MonicaC's user avatar JonathanZ supports MonicaCJonathanZ supports MonicaC 19855 bronze badges New contributor JonathanZ supports MonicaC is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct. Add a comment | 9 Let me try to synthesize an analysis of some possible interpretations and my (and I expect, the broader community's) attitude to these variations. * "Yikes, no!" Interpretation: "We will start polluting the site with ChatGPT content based on its current capabilities, and with our usual rigorous testing and interaction with the broader user community" (that's sarcasm for "quickly, surprisingly, with bad timing, and without proper thought or coordination"). * "Mmmmmmaybe..." Interpretation: "We want to jump on the current hype bandwagon and spend months on an expensive project which will probably ultimately be cancelled. But if it's successful, we expect to see some sort of next-gen AI which actually doesn't try to answer when it doesn't know, which could really boost the site's capabilities and your user experience. For the time being, though, don't be nervous; this is just something we put out to hopefully instill some new market confidence in our stock evaluation." (The last sentence only illustrates how bad I am at market-speak, I'm afraid. I hope you get the idea.) * "Yes! You're on the right track!" Interpretation: "Our focus is on reducing the number of badly articulated beginner questions by answering them before they are actually published on the site. We have a proof of concept for a new question model which shows some promise already." Share Improve this answer Follow answered 17 hours ago tripleee's user avatar tripleeetripleee 9,97544 gold badges3030 silver badges5959 bronze badges 7 * 3 Related to "Yes!": The Guy With The Hat's post, PM 2Ring's comment - starball 16 hours ago * We should be reducing the number of badly articulated beginner questions by refusing to publish them on the site until they have been reviewed by humans from a more restricted pool (at least, "logged-in users" rather than the general public; but probably ones meeting some reputation threshold). - Karl Knechtel 11 hours ago * 1 Given how divergent those interpretations are, does the post actually tell us anything substantial? - user3840170 11 hours ago * @KarlKnechtel Not sure if that's scalable. The review queues are already quite full. I just responded to a complete beginner question (badly written too) from somebody with 7000 reputation, so we can't guard by reputation either (unless you're prepared to let a lot slip through). - Andreas detests censorship 10 hours ago * 3 @user3840170 see my post above, that is EXACTLY my point. The original post tell us nothing, yet actually ask us for feedback and questions about that very same "nothing". That is why tripleee had to write down three totally divergent scenarios only to be able to post something significant. - SPArcheon 9 hours ago * 2 I presume the third one is entirely sarcastic. It will never happen given the current KPIs of the company (quantity) (or be amended with "...or the number of users helped by the Stack Overflow Generative AI Help Wizard (SOGAHW)"). - This_is_NOT_a_forum 9 hours ago * I mean, they did try some ML with their suggested questions experiment in questions. It went about as well as you'd expect. - Makoto 16 mins ago Add a comment | 9 It is precisely this symbiotic relationship between humans and AI that ensures the ongoing relevance of community-driven platforms like Stack Overflow. This is obviously not correct! How can somebody who understands anything about Stack Overflow think this is correct? The relevance of Stack Overflow has nothing to do with AI generated content. AI generated content is not even allowed there, and the community overwhelmingly supports that ban. Did anyone check this announcement before it was published? Is there anybody at the company who can tell the CEO he is totally wrong about this? Share Improve this answer Follow answered 5 hours ago kaya3's user avatar kaya3kaya3 85122 silver badges88 bronze badges Add a comment | 4 Things I look forward to (then a question) 1. Finding citeable answers faster with ML guidance With a ML-driven chatbot, or similar interface, I can imagine it will be easier to find human-written answers without actually posting a question. The image below is a screenshot from an ML-driven search engine (not shown are the citations below the answer, probably the most valuable part): An ML-driven search engine answers the question "What is a src layout in Python?" 2. Writing better questions with ML guidance I can easily imagine the Ask Wizard evolving to help newcomers and old-timers alike improve the quality of their questions. Including: * answering them without posting a question (see above) * helping them find resources that might help them answer their own question (Obviously I could use ML tools to do these today, no integration necessary.) 3. Generate hypothetical answers I can test, approve and then publish I'm most skeptical of this one, but with the right gamification it might be possible. (Harsh penalties for anyone publishing nonsense.) 4. Continue to have the Stack Overflow data dump publicly available under BY-SA 4.0 (I assume this still exists, I sometimes miss the news.) This public data dump of the community's hard work is a social asset. I would love to see SO release it as an LLM model compatible with Dolly 2.0 or whatever the open format ends up being. Failing that, keep the data openly and publicly available so that others can try is a must. (Obviously this doesn't apply to Stack Overflow for Teams data.) My question Am I being overly optimistic to think that these are the kinds of ideas Stack Overflow is considering, rather than generating work for volunteer moderators? Share Improve this answer Follow edited 5 hours ago TylerH's user avatar TylerH 11.3k33 gold badges3333 silver badges6262 bronze badges answered 5 hours ago lofidevops's user avatar lofidevopslofidevops 1,33077 silver badges1818 bronze badges 3 * 1 "Am I being overly optimistic to think that these are the kinds of ideas Stack Overflow is considering, rather than generating work for volunteer moderators?" -- Yes. - user3840170 4 hours ago * 1 I mean... i wouldn't be surprised if these kinds of things are what they're prophesizing, I would however be surprised if they were able to deliver. - Kevin B 4 hours ago * If you see my answer - I think Phillipe covers some of that in his edit - Journeyman Geek Mod 9 mins ago Add a comment | -16 People worrying that AI code generation is moving us towards some existential risk ridiculously misinterpret what ChatGPT does. The software simply recycles online content, filters it, and formats it in a readable format using a language model. If we called this software, Search 3.0, there would be a chorus of approving voices just as there were when Google came along and made the internet more consumable. Fear of AI becoming AGI is as ridiculously overblown as the prophecy that the Y2K bug would cause social collapse. This software is simply a data recycling tool, and if you get behind it, it makes online resources like SO more useful. AI cannot replace human developers. It just streamlines many laborious repetitive coding tasks that have been explained to death online, because so many people repeat those tasks. If AI could ever write new code for us going forward, it would imply there is a sufficient corpus of existing code from which to write remaining software. This is an astronomical miscalculation that fails to comprehend the vast complexity of program variations. How sufficient is the existing body of code, compared to the code we might possibly choose to write? We can enumerate possible programs as sets of input-output pairs. So an example program might only accept a 0 input to be converted into a 1 output, and so be defined as ((0,1)). Another might be defined as ((0,1),(123,456)) and so on. How many possible trivial programs are there that have a single ASCII character input and output? A lower bound on the number of variations is the powerset, so 2**128 variations for trivial single character input-output processes. How many possible programs involve character pairs? 2**16384. Typical programs have complex inputs and outputs. These are numbers that make all the programs written to date look infinitesimal. 'AI' cannot write our code for us. What we call AI today is nothing like 'AGI'. AI is a system that recycles our existing ridiculously tiny body of software. It cannot possibly extrapolate what we might want to write. It is not at all in the realm of possibility. When people claim 'AI wrote my app' then either they are using components that have been written 1000 times before, in which case it is a somewhat trivial invention, or it's fake click bait. Share Improve this answer Follow edited 23 hours ago This_is_NOT_a_forum's user avatar This_is_NOT_a_forum 6,68244 gold badges3434 silver badges5555 bronze badges answered yesterday Riaz Rizvi's user avatar Riaz RizviRiaz Rizvi 9322 bronze badges New contributor Riaz Rizvi is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct. 17 * 22 I mean... i don't see anyone here arguing "AI BAD IT'S GONNA REPLACE US" - Kevin B yesterday * -38 for OP's comment supporting AI? Why? There's fear of AI and of AGI on the internet and I want to address that. - Riaz Rizvi yesterday * 11 there's certainly more reasons to not want this change, as expressed in the several other answers, that aren't related to some ridiculous "existential risk" - Kevin B yesterday * 1 There is also fear of snakes. I don't see you addressing that. Is it because nobody has actually expressed such fear? - VLAZ yesterday * I've provided links to these widespread fears in my edited comment. - Riaz Rizvi yesterday * 5 Y2K failed to cause the collapse of society because people recognized that it was an issue and put in a lot of hard work leading up to 2000-01-01 to fix their software. Similarly, ChatGPT and related models could cause major societal issues if left unchecked indefinitely. Right now, we should have the discussion of what we need to do to avoid that. - The Guy with The Hat yesterday * The cottage industry of Y2K consultants helped society by paying a few bills for some people. No social collapse crisis was averted. - Riaz Rizvi yesterday * 1 You know, it would kinda be nice if AI could replace us. It would be very useful, and we'd get a lot more time for fun stuff. Yet I posted a very unhappy answer to this statement from the CEO. I'm not afraid of going out of business. - Andreas detests censorship yesterday * 1 Re "many laborious repetitive coding tasks": What are some examples of those tasks? - This_is_NOT_a_forum 23 hours ago * 2 @RiazRizvi Teach them how to use a search engine, not how to eat bullshit from ChatGPT. - Andreas detests censorship 21 hours ago * 1 @Andreasdetestscensorship correct me if I'm wrong, but I think your concern is a feedback cycle where AI is recycling content posted by humans on SO and then humans are further posting output from that AI because they can do it prolifically and boost their own status on the sites, but in so doing, they degrade the status of people like me who spent years manually submitting content? That to me is a problem of content moderation in the face of a new technology that provides an astroturfing capability. I do support SO enhancements to prevent astroturfing and content degradation. - Riaz Rizvi 21 hours ago * 5 The first two paragraphs of this answer miss the mark completely. SO is not some sort of tin hat haven, and does not harbor any sort of conspiracy views on AI. The transformer algorithm, specifically the popular one which is pre trained and aimed at generative language, is horrific in its falsehoods. While it may be useful for looking up existing knowledge (#plagairism), it is absolutely worthless when it comes to creating new works (aka generating). This is the reason for the pushback, regardless of what the name is or what public talking points seem to be around the subject or concept. - Travis J 20 hours ago * 1 Huh?? A system that efficiently looks up existing knowledge can obviously be used to create new works, that are composed of elements from those lookups. Think Lego. So no it is not 'absolutely worthless'. The whole premise of SO is to provide existing knowledge for others. 'plagiarism' is loaded language and implies you're in conflict with yourself as a user/ contributor of SO. - Riaz Rizvi 19 hours ago * 5 I didn't say it cannot be used to create new works, I said it cannot create new works, there is a very strong difference there whereby in your out of context version the person is creating versus my in context version where the algorithm is creating. SO is premised on providing knowledge... non hallucinated, factual, helpful, knowledge in the form of adaptation for use. Plagiarism is not loaded language, the algorithm will directly reproduce existing works. I am not in conflict with myself. Please let me know if you have any other thoughts, aside from Lego's.... - Travis J 19 hours ago * 1 'Please let me know if you have any other thoughts, aside from Lego's..' that was good, it gave me a chuckle. I think we can agree AI doesn't create. - Riaz Rizvi 19 hours ago | Show 2 more comments -21 The announcement is vague, but integrating an LLM to help write SE answers would make sense (instead of banning LLMs). However, we may want to first analyze how often LLMs give an incorrect answer to an SE question to see whether it is good enough to be used more systematically. Otherwise, one could simply let users decide when to use LLMs, since the LLM accuracy depends a lot on the type of question. SE may want to add some mechanism to mark an answer as incorrect that would complement the current voting system (which could also be beneficial for non-LLMs answers too), at least for the answers mostly generated by LLMs. Lastly, it's still a bit unclear how a language model can keep track of the provenance of the main knowledge/sources used to generate a given output, and in some cases LLMs may go beyond the fuzzy boundaries of fair use: aside from the legal aspect, unreferenced answers sometimes aren't that satisfying (e.g., hard to check the sources and can't read the additional information that the sources may have provided). But it's rather rare, esp. for non-expert questions, and humans are known to plagiarize too anyway. If that's an issue, SE could also integrate some plagiarism detection tool. Share Improve this answer Follow answered yesterday Franck Dernoncourt's user avatar Franck DernoncourtFranck Dernoncourt 28.8k44 gold badges4747 silver badges121121 bronze badges 6 * 2 Are you spamming your own content now? That's novel. I don't see how this answers the post. - Mast 6 hours ago * 1 @Mast this answer support the idea of integrating an LLM to help write SE answers. - Franck Dernoncourt 6 hours ago * 1 The community here tends to vote down overt self-promotion and flag it as spam. Post good, relevant answers, and if some (but not all) happen to be about your product or website, that's okay. However, if you mention your product, website, etc. in your question or answer (or any other contribution to the site), you must disclose your affiliation in your post. - How not to be a spammer - Mast 6 hours ago * 1 @Mast this answer support the idea of integrating an LLM to help write SE answers. Therefore, it is relevant. I'm only linking to other SE posts. - Franck Dernoncourt 6 hours ago * 1 Other SE posts you wrote. All of them. You neglected to disclose that. People would get the wrong idea. - Mast 6 hours ago * 1 @Mast Post authors don't matter. "People would get the wrong idea" What idea? - Franck Dernoncourt 5 hours ago Add a comment | You must log in to answer this question. Not the answer you're looking for? Browse other questions tagged * discussion * featured * blog * announcements * company . Welcome! Welcome! Meta Stack Exchange is intended for bugs, features, and discussions that affect the whole Stack Exchange family of Q&A sites. About Help * Featured * Improving the copy in the close modal and post notices - 2023 edition * New blog post from our CEO Prashanth: Community is the future of AI Visit chat Linked 191 Ban ChatGPT network-wide 53 What motivates people to answer questions in Stack Overflow? 56 Is there a list of ChatGPT or other AI-related discussions and policies for our sites? -19 Could ChatGPT be a viable way to answer people's questions? -18 What quality threshold should be established for language models/ question-answering systems to be allowed to write answers on Stack Exchange? -22 How often does ChatGPT give an incorrect answer to an SE question? 26 We're moving Traducir to our internal infrastructure -16 The future role of Stack Exchange vs. emerging AIs Related 1514 Fastest Gun in the West Problem 2806 A Terms of Service update restricting companies that scrape your profile information without your permission 2534 Firing mods and forced relicensing: is Stack Exchange still interested in cooperating with the community? 2303 Stack Overflow is doing me ongoing harm; it's time to fix it! 490 To reach out: on Monica, the Lavender community, and the future of the Stack Exchange network 1575 Thank you, Shog9 1003 Firing Community Managers: Stack Exchange is not interested in cooperating with the community, is it? 556 The company's commitment to rebuilding the relationship with you, our community 764 New Feature: Table Support 725 Stack Exchange Public Q&A access will not be restricted in Russia Hot Network Questions * A thought is considered a separate sin? * Understanding an argument from How to Prove It * Mathematicians who wrote fiction * Home Electric Rework * Why many classes in the Bitcoin Core prefix with a `C` while it is prohibited in the developer notes? * A little number theoretic game * Use Raster Layer as a Mask over a polygon in QGIS * Why is a "TeX point" slightly larger than an "American point"? * Natural way of saying "I don't think X" * Limit how much GPU an app gets to use * Short story, the last human and his dog servants * Did predators evolve eyes first? * "Excusez-moi" as a way to ask for time? * Does Chain Lightning deal damage to its original target first? * Alternating factorial * Mike Sipser and Wikipedia seem to disagree on Chomsky's normal form * Changing active definition query for multiple layers in group layer using ArcPy in ArcGIS Pro * Can someone please tell me what is written on this score? * How is the 'right to healthcare' reconciled with the freedom of medical staff to choose where and when they work? * Choosing waterproof or protected GPIO connectors * Do EU or UK consumers enjoy consumer rights protections from traders that serve them from abroad? * Ridiculously low current gain in 2N3904-based current mirror * basic RAII spinlock class * Godel encoding - Part II (decoding) more hot questions Question feed Subscribe to RSS Question feed To subscribe to this RSS feed, copy and paste this URL into your RSS reader. [https://meta.stackex] * Meta Stack Exchange * Tour * Help * Chat * Contact * Feedback Company * Stack Overflow * Teams * Advertising * Collectives * Talent * About * Press * Legal * Privacy Policy * Terms of Service * Cookie Settings * Cookie Policy Stack Exchange Network * Technology * Culture & recreation * Life & arts * Science * Professional * Business * API * Data * Blog * Facebook * Twitter * LinkedIn * Instagram Site design / logo (c) 2023 Stack Exchange Inc; user contributions licensed under CC BY-SA. rev 2023.4.18.43394 Your privacy By clicking "Accept all cookies", you agree Stack Exchange can store cookies on your device and disclose information in accordance with our Cookie Policy. Accept all cookies Necessary cookies only Customize settings