[HN Gopher] Terence Tao on O1
___________________________________________________________________
Terence Tao on O1
Author : dselsam
Score : 383 points
Date : 2024-09-14 16:41 UTC (6 hours ago)
(HTM) web link (mathstodon.xyz)
(TXT) w3m dump (mathstodon.xyz)
| jarbus wrote:
| I wonder how long it took for each of the responses it gave
| diggan wrote:
| It varies a lot. If it's a simple question, it just does 3-4
| sections of "thinking & reflection" but for more complicated
| ones I think I've seen something like 10 or more. Maybe 3-4
| seconds per section on average I'd guess.
| zamadatix wrote:
| It's unclear if Terence is referring to "GPT-o1... a
| prototype version of the model that I was granted access to"
| as in "he was given access to GPT-o1 by the research team" or
| as in "he is using o1-preview". The differences in scale and
| quality between his shared output and the answer I get trying
| the same prompt from o1-preview suggest perhaps the former
| (otherwise luck). I haven't actually seen any examples of how
| long o1 "full" will think about this kind of question, though
| I expect it's somewhere in the same ballpark given the
| thought expansion still only has one real concept in it.
| d0mine wrote:
| "with even the latest tools the effort put in to get the model to
| produce useful output is still some multiple (but not an enormous
| multiple now, say 2x to 5x) of the effort needed to properly
| prompt and verify the output. However, I see no reason to prevent
| this ratio from falling below 1x in a few years, which I think
| could be a tipping point for broader adoption of these tools in
| my field"
|
| Given the log scale on compute to improve performance, it is not
| a guarantee that the ratio can be improved so much in a few years
| aoeusnth1 wrote:
| The y axis is also log scale (log likelihood). It's a power
| law, not an exponential law.
| d0mine wrote:
| I was referring to the o1 AIME accuracy figure (x log scale
| compute, y is % (not log)) and similar
| https://openai.com/index/learning-to-reason-with-llms/
| bitexploder wrote:
| The novelty to me is that the "The experience seemed roughly on
| par with trying to advise a mediocre, but not completely
| incompetent, graduate student." in so many subject areas! I have
| found great value in using LLMs to sort things out. In areas
| where I am very experienced it can be really helpful at tons of
| small chores. Like Terrence was pointing out in his third
| experiment -- if you break the problem down it does solid work
| filling in smaller blanks. You need the conceptual understanding.
| Part of this is prompting skill. If you go into an area you don't
| know you have to try and build the prompts up. Dive into
| something small and specific and work outward if the answer is
| known. Start specific and focused if starting from the outside
| in. I have used this to cut through conceptual layers of very
| complex topics I have zero knowledge in and then verify my
| concepts via experts on YT/research papers/trusted sources. It is
| an amazing tool.
| wenc wrote:
| This has been my experience as well. I treat LLMs like an
| intern or junior who can do the legwork that I have no
| bandwidth to do myself. I have to supervise it and help it
| along, checking for mistakes, but I do get useful results in
| the end.
|
| Attitudinally, I suspect people who have had experience
| supervising interns or mentoring juniors are probably those who
| are able to get value out of LLMs (paid ones - free ones are no
| good) rather than grizzled lone individual contributors -- I
| myself have been in this camp for most of my early career --
| who don't know how to coax value out of people.
| wslh wrote:
| > ... that I have no bandwidth to do myself.
|
| One of the most interesting aspects of this thread is how it
| brings us back to the fundamentals of attention in machine
| learning [1]. This is a key point: while humans have
| intelligence, our attention is inherently limited. This is
| why the concept behind Attention Is All You Need [2] is so
| relevant to what we're discussing.
|
| My 2 cents: our human intelligence is the glue that binds
| everything together.
|
| [1]
| https://en.wikipedia.org/wiki/Attention_(machine_learning)
|
| [2] https://en.wikipedia.org/wiki/Attention_Is_All_You_Need
| wenc wrote:
| Once GPT is tuned more heavily on Lean (proof assistant) -- the
| way it is on Python -- I expect its usefulness for research level
| math to increase.
|
| I work in a field related to operations research (OR), and
| ChatGPT 4o has ingested enough of the OR literature that it's
| able to spit out very useful Mixed Integer Programming (MIP)
| formulations for many "problem shapes". For instance, I can give
| it a logic problem like "i need to put i items in n buckets based
| on a score, but I want to fill each bucket sequentially" and it
| actually spits out a very usable math formulation. I usually just
| need to tweak it a bit. It also warns against weak formulations
| where the logic might fail, which is tremendously useful for
| avoiding pitfalls. Compare this to the old way, which is to rack
| my brain over a weekend to figure out a water-tight formulation
| of MIP optimization problem (which is often not straightforward
| for non-intuitive problems). GPT has saved me so much time in
| this corner of my world.
|
| Yes, you probably wouldn't be able to use ChatGPT well for this
| purpose unless you understood MIP optimization in the first place
| -- and you do need to break down the problem into smaller chunks
| so GPT can reason in steps -- but for someone who can and does,
| the $20/month I pay for ChatGPT more than pays for itself.
|
| side: a lot of people who complain on HN that (paid/good - only
| Sonnet 3.5 and GPT4o are in this category) LLMs are useless to
| them probably (1) do not know how to use LLMs in way that
| maximizes their strengths; (2) have expectations that are too
| high based on the hype, expecting one-shot magic bullets. (3)
| LLMs are really not good for their domain. But many of the low-
| effort comments seem to mostly fall into (1) and (2) -- cynicism
| rather than cautious optimism.
|
| Many of us who have discovered how to exploit LLMs in their areas
| of strength -- and know how to check for their mistakes -- often
| find them providing significant leverage in our work.
| threeseed wrote:
| > side
|
| Or (4) LLMs simply do not work properly for many use cases in
| particular where large volumes of trained data doesn't exist in
| its corpus.
|
| And in these scenarios rather than say "I don't know" it will
| over and over again gaslight you with incoherent answers.
|
| But sure condescendingly blame on the user for their ignorance
| and inability to understand or use the tool properly. Or call
| their criticism low-effort.
| wenc wrote:
| That's category (3).
| zamadatix wrote:
| What's the difference between (3) and (4), shouldn't the
| former contain the latter?
| WhatIsDukkha wrote:
| I entirely agree about their utility.
|
| HN, and the internet in general, have become just an ocean of
| reactionary sandbagging and blather about how "useless" LLMs
| are.
|
| Meanwhile, in the real world, I've found that I haven't written
| a line of code in weeks. Just paragraphs of text that specify
| what I want and then guidance through and around pitfalls in a
| simple iterative loop of useful working code.
|
| It's entirely a learned skill, the models (and very importantly
| the tooling around them) have arrived at the base line they
| needed.
|
| Much Much more productive world by just knuckling down and
| learning how to do the work.
|
| edit: https://aider.chat/ + paid 3.5 sonnet
| benterix wrote:
| > I've found that I haven't written a line of code in weeks
|
| Which is great until your next job interview. Really, it's
| tempting in the short run but I made a conscious decision to
| do certain tasks manually only so that I don't lose my basic
| skills.
| calmworm wrote:
| Job interview? You might be surprised at the number of us
| who don't code for a job.
| __loam wrote:
| I'd bet most people on this forum program professionally.
| calmworm wrote:
| I would take that bet.
| idiotsecant wrote:
| Me too.
| whamlastxmas wrote:
| I've made the decision to embrace being bad at coding but
| getting a ton of work done using an LLM and if my future
| employer doesn't want massive productivity and would prefer
| being able to leetcode really well then I unironically
| respect that and that's ok.
|
| I'm not doing ground breaking software stuff, it's just web
| dev at non massive scales.
| __loam wrote:
| You future employer might expect you to bring some value
| through your expertise that doesn't come from her LLM. If
| you want to insist on degrading your own employability
| like this, I guess it's your choice.
| fragmede wrote:
| For the most part, businesses don't care how you deliver
| value, just that you do. If programmer A does a ticket in
| 3 days with an LLM, and programmer B takes a week to do
| the same ticket, but doesn't use an LLM, with programmer
| B choosing not to out of some notion of purity, who's
| more employable?
| ed wrote:
| This. I've been using elixir for ~6 months (guided by
| Claude) and probably couldn't solve fizz buzz at a
| whiteboard without making a syntax error. Eek.
| stavros wrote:
| Who cares? If I'm hiring you to make a product, I care
| that the higher order logic is correct, that the
| requirements are all catered for, and that the code does
| reasonable things in all cases. Things I don't care about
| are FizzBuzz, programming on whiteboards, and not making
| syntax errors.
| Der_Einzige wrote:
| See, if you work in AI, say, as an AI researcher, asking
| them not to be allowed to use AI models in the interview is
| basically not an option.
|
| Also, often folks in this space are better at cheating than
| you will be at detecting them. Don't believe me?
| https://bigvu.tv/captions-video-maker/ai-eye-contact-fix
| vasco wrote:
| ChatGPT voice interface plugged into the audio stream, with
| the prompt:
|
| - I need you to assist me during a programming interview,
| you will be listening to two people, the interviewer and
| me. When the interviewer asks a question, I'd like you to
| feed me lines that seem realistic for an interview where
| I'm nervous, don't give me a full blown answer right away.
| Be very succinct. If I think you misunderstood something, I
| will mention the key phrase "I'm nervous today and had too
| much coffee". In this situation, remember I'm the one that
| will say the phrase, and it might be because you've
| mistaken me by the interviewer and I want you to "reset".
| If I want you to dig deeper than what you've provided me
| with, I'll say the key phrase "Let's dig deeper now". If I
| think you've hallucinated and want you to try again, I'll
| say "This might be wrong, let me think for just a minute
| please". Remember, other than these key phrases, I'll only
| be talking to the interviewer, not you.
|
| On a second screen of some sort. Other than that,
| interviewers will just have to accept that nobody will be
| doing the job without these sort of assistants from now on
| anyway. As an interviewer I let candidates consult online
| docs for specific things already because they'll have
| access to Google during the job, this is just an extension
| of that.
| apsurd wrote:
| This made me laugh. I can't deny it isn't already
| happening. But wow people work so hard to avoid working
| hard.
| vasco wrote:
| And I wrote this as a knee-jerk reaction after reading
| the parent, I imagine people will be putting way more
| effort if it can get them a great job. And to be honest,
| if they can fool you, they can most likely do the job as
| well. Most of the industry tests at a higher skill level
| than what they actually require on the day to day anyway.
| bessbd wrote:
| It's almost inspiring, isn't it?
| gcanyon wrote:
| I recently interviewed a number of people about their SQL
| skills. The format I used was to share two queries with
| them a couple days ahead of time in a google doc, and
| tell them I will ask them questions about those queries
| during the interview.
|
| Out of maybe twenty people I interviewed this way, only
| three of them pointed out that one of the queries had a
| failing error in it. It was something any LLM would
| immediately point out.
|
| Beyond that: the first question I asked was: "What does
| this query do, what does it return?" I got responses
| ranging from people who literally read the query back to
| me word by word, giving the most shallow and direct
| explanation of what each bit did step-by-step, to people
| who clearly summarized what the query did in high-level,
| abstract terms, as you might describe what you want to
| accomplish before you write the query.
|
| I don't think anyone did something with ChatGPT live, but
| _maybe_?
| jamesmotherway wrote:
| Not everyone is doing coding interviews. Some might
| struggle with a particular language due to lack of muscle
| memory, but can dictate the logic in pseudocode and can
| avoid pitfalls inferred from past experience. This sort of
| workflow is compatible with LLMs, assuming a sufficient
| background (otherwise one can't recognize when the output
| diverges from your intent).
|
| I personally treat the LLM as a rubber duck. Often I reject
| its output. In other cases, I can accept it and refactor it
| into something even better. The name of the game is
| augmentation.
| LouisSayers wrote:
| You need to prep for job interviews anyway. I'd rather
| spend the majority of my time being productive.
| dmd wrote:
| I sometimes get the idea from statements like this - and
| HN's focus on interviewing in general - that people are
| switching jobs a dozen times a year or something. How often
| are most people switching jobs? I've had 5 jobs in the last
| 20 years.
| macintux wrote:
| I'm old, and well-paid for my geographic region (but for
| various mostly stupid reasons utterly broke). No amount
| of value created (at least, for my skill level) will
| protect me from ageism and/or budget cuts.
| apsurd wrote:
| LLMs are certainly not useless.
|
| But "lines of code written" is a hollow metric to prove
| utility. Code literacy is more effective than code
| illiteracy.
|
| Lines of natural language vs discrete code is a kind of
| preference. Code is exact which makes it harder to recall and
| master. But it provides information density.
|
| > by just knuckling down and learning how to do the work?
|
| This is the key for me. What work? If it's the years of
| learning and practice toward proficiency to "know it when you
| see it" then I agree.
| riku_iki wrote:
| > Meanwhile, in the real world, I've found that I haven't
| written a line of code in weeks. Just paragraphs of text that
| specify what I want and then guidance through and around
| pitfalls in a simple iterative loop of useful working code.
|
| could it be that you are mostly engaged in "boilerplate
| coding", where LLMs are indeed good?
| threeseed wrote:
| > HN, and the internet in general, have become just an ocean
| of reactionary sandbagging and blather about how "useless"
| LLMs are.
|
| This is cult like behaviour that reminds me so much of the
| crypto space.
|
| I don't understand why people are not allowed to be critical
| of a technology or not find it useful.
|
| And if they are they are somehow ignorant, over-reacting or
| deficient in some way.
| wenc wrote:
| I think it's perfectly ok to be critical of technology as
| long as one is thoughtful rather than dismissive. There is
| a lot of hype right now and pushing back against it is the
| right thing to do.
|
| I'm more reacting against simplistic and categorical
| pronouncements of straight up "uselessness," which to me
| seems un-curious and deeply cynical, especially since it is
| evidentially untrue in many domains (though it is true for
| some domains). I just find this kind of emotional cynicism
| (not a healthy skepticism, but cynicism) to be contrary to
| the spirit of innovation and openness, and indeed contrary
| to evidence. It's also an overgeneralization -- "I don't
| find it useful, so it's useless" -- rather than "Why don't
| I find it useful, and why do others do? Let me learn more."
|
| As future-looking HNers, I'd expect we would understand the
| world through a lens of "trajectories" rather than "current
| state". Just because LLMs hallucinate and make mistakes
| with a tone of confidence today -- a deep weakness --
| doesn't mean they are altogether useless. We've witnessed
| that despite their weaknesses, we are getting a lot of
| value from them in many domains today and they are getting
| better over time.
|
| Take neural networks themselves for instance. For most of
| the 90s-2000s, people thought they were a dead end. My own
| professor had great vitriol against Neural Networks. Most
| of the initial promises in the 80s truly didn't pan out.
| Turns out what was missing was (lots of) data, which the
| Internet provided. And look where we are today.
|
| Another area of cynicism is self-driving cars (Level 5).
| Lots of hype and overpromise, and lots of people saying it
| will never happen because it requires a cognitive model of
| the world, which is too complicated, and there are too many
| exceptional cases for there to ever be Level 5 autonomy.
| Possibly true, but I think "never" is a very strong
| sentiment that is unworthy of a curious person.
| threeseed wrote:
| This entire comment can be summarised as: everyone who
| doesn't think like me is wrong.
|
| Not everyone is interested in seeing the world through
| the hopes and dreams of e/acc types and would prefer to
| see it as it is today.
|
| LLMs are a technology. Nothing more. It can be as amazing
| or useless as anyone likes.
| fragmede wrote:
| And this comment can be summarized as "Nuh uh, I'm
| right". When summarizing longer bits of text down to a
| single sentence, nuance and meaning gets lost, making the
| summarization ultimatele useless, contributing nothing to
| the discussion.
| rainsford wrote:
| I generally agree, although an important aspect of
| thinking in terms of "trajectories" is recognizing when a
| particular trajectory might end up at a dead end. One
| perspective on the weaknesses of current LLMs is that
| it's just where the things are today and they can still
| provide value even while the technology improves. But
| another perspective is that the persistence of these
| weaknesses indicates something more fundamentally broken
| with the whole approach that means it's not really the
| path towards "real" AI, even if you can finesse it into
| doing useful things in certain applications.
|
| There's also an important nuance differentiating
| rejection of a general technological endpoint (e.g. AGI
| or Level 5 self-driving cars) with a particular
| technological approach to achieving those goals (e.g.
| current LLM design or Tesla's autopilot). As you said,
| "never" is a long time and it takes a lot of unwarranted
| confidence to say we will never be able to achieve goals
| like AGI or Level 5 self-driving. But it seems a lot more
| reasonable to argue Tesla or OpenAI (and everyone else
| doing essentially the same thing as OpenAI) are
| fundamentally on the wrong track to achieving those goals
| without significantly changing their approach.
|
| I agree that none of that really warrants dismissive
| cynicism of new technology, but being curious and future-
| looking also requires being willing to say when you think
| something is a bad approach even if it's not totally
| useless. Among other reasons, our ability to explore new
| technology is not limitless, and hype for a flawed
| technology isn't just annoying but may be sucking all the
| oxygen out of the room not leaving any for a potentially
| better alternative. Part of me wants to be optimistic
| about LLMs, but another part of me thinks about how much
| energy (human and compute) has gone into this thing that
| does not seem to be providing a corresponding amount of
| value.
| wenc wrote:
| I appreciate this thoughtful comment.
|
| You are absolutely right that the trajectories, if taken
| linearly, might hit a dead end. I should clarify that
| when I mentioned "trajectories" I don't mean unpunctuated
| ones.
|
| I am myself not convinced that LLMs -- despite their
| value to me today -- will eventually lead to AGI as a
| matter of course, nor the type of techniques used in
| autopilot will lead to L5 autonomy. And you're right that
| they are consuming a lot of our resources, which could
| well be better invested in a possibly better alternative.
|
| I subscribe to Thomas Kuhn's [1] idea of scientific
| progress happening in "paradigms" rather than through a
| linear accumulation of knowledge. For instance, the path
| to LLMs itself was not linear, but through a series of
| new paradigms disrupting older ones. Early natural
| language processing was more rule-based (paradigm), then
| it became more statistical (paradigm), and then LLMs
| supplanted the old paradigms through transformers
| (paradigm) which made it scale to large swaths of data. I
| believe there is still significant runway left for LLMs,
| but I expect another paradigm must supplant it to get
| closer to AGI. (Yann Lecun said that he doesn't believe
| LLMs will lead to AGI).
|
| Does that mean the current exuberant high investments in
| LLMs are misplaced? Possibly, but in Kuhn's philosophy,
| typically what happens is a paradigm will be milked for
| as much as it can be, until it reaches a crisis/anomaly
| when it doesn't work anymore, at which point another
| paradigm will supplant it.
|
| At present, we are seeing how far we can push LLMs, and
| LLMs as they are have value even today, so it's not a bad
| approach per se even though it will hit its limits at
| some point. Perhaps what is more important are the
| second-order effects: the investments we are seeing in
| GPUs (essentially we are betting on linear algebra) might
| unlock the kind of commodity computational power the next
| paradigm needs to disrupt the current one. I see
| parallels between this and investments in NASA resulting
| in many technologies that we take for granted today, and
| military spend in California producing the technology
| base that enabled Silicon Valley today. Of course, these
| are just speculations and I have no more evidence that
| this is happening with LLMs than anyone else.
|
| I appreciate your point however and it is always good to
| step back and ask, non-cynically, whether we are headed
| down a good path.
|
| [1] https://en.wikipedia.org/wiki/The_Structure_of_Scient
| ific_Re...
| wenc wrote:
| Also, I'm deeply skeptical of crypto too due to its present
| scamminess, but I am keeping an open mind that there is a
| future in which crypto -- once it gets over this phase of
| get-rich-quick schemers -- will be seen as just another
| asset class.
|
| I read somewhere that historically bonds in their early
| days were also associated with scamminess but today they're
| just a vanilla asset.
| evilfred wrote:
| bonds have utility, crypto does not
| rainsford wrote:
| I'm honestly more optimistic about cryptocurrency as a
| mechanism of exchange rather than an asset. As a
| mechanism of exchange, cryptocurrency has some actually
| novel properties like distributed consensus that could be
| useful in certain cases. But an asset class which has
| zero backing value seems unworkable except for wild
| speculation and scams. Unfortunately the incentives
| around most cryptocurrencies (and maybe fundamental to
| cryptocurrency as an idea) greatly emphasize the asset
| aspects, and it's getting to be long enough since it
| became a thing that I'm starting to become skeptical
| cryptocurrency will be a real medium of exchange outside
| of illegal activities and maybe a few other niche cases.
| evilfred wrote:
| just like with crypto and NFTs and the metaverse, they are
| always focused on what is suppsoedly coming down the pipe
| in the future and not what is actually possible today
| ben_w wrote:
| Crypro and AI have similarities and differences.
|
| The similarities include intense "true believer" pitches
| and governments taking them seriously.
|
| The differences include that the most famous cryptocurrency
| can't function as a direct payment mechanism for just lunch
| purchases in just Berlin (IIRC nor is it enough for all
| interbank transactions so it can't even be a behind-the-
| scenes system by itself), while GenAI output keeps ending
| up in places people would rather not find it like homework
| and that person on Twitter who's telling you Russia Did
| Nothing Wrong (and also giving you a nice cheesecake recipe
| because they don't do any input sanitation).
| delusional wrote:
| What sort of problems do you solve? I tried to use it. I
| really did. I've been working on a tree edit distance
| implementation base on a paper from 95. Not novel stuff. I
| just can't get it to output anything coherent. The code
| rarely runs, it's written in absolutely terrible style, it
| doesn't follow any good practices for performant code. I've
| struggled with getting it to even implement the algorithm
| correctly, even though it's in the literature I'm sure it was
| trained on.
|
| Even test cases have brought me no luck. The code was poorly
| written, being too complicated and dynamic for test code in
| the best case and just wrong on average. It constantly
| generated test cases that would be fine for other definitions
| of "tree edit distance" but were nonsense for my version of a
| "tree edit distance".
|
| What are you doing where any of this actually works? I'm not
| some jaded angry internet person, but I'm honestly so
| flabbergasted about why I just can't get anything good out of
| this machine.
| thesz wrote:
| I think that contemporary models are trained for
| engagement, not for actual help.
|
| My experience is the same as yours, but I noticed that
| while LLMs circa two years ago tried to come up with the
| answer, current generation of LLMs tries to make me come
| with the answer. And that not helping at all.
| bongodongobob wrote:
| Did you tell it that? Are you trying to converse and
| discuss or are you trying to one shot stuff? If it gets
| something wrong, tell it. Don't just stop and try another
| prompt. You have to think of it as another person. You can
| talk to it, question it, guide it.
|
| Try starting from ground zero and guiding it to the
| solution rather than trying to one shot your entire
| solution in one go.
|
| I want you to implement this kind of tree in language x.
|
| Ok good, now I want you to modify it to do Y.
|
| Etc.
| delusional wrote:
| I've tried both. One time I actually tried so hard that I
| ran out of context, and aider just dumped me back to the
| main prompt. I don't think It's possible to guide it any
| more than that.
|
| My problem is that the solution is right there in the
| paper. I just have to understand it. Without first
| understanding that paper, I can't possibly guide the AI
| towards a reasonable implementation. The process of
| finding the implementation is exactly the understanding
| of the paper, and the AI just doesn't help me with that.
| In fact, all too often I would ask it to make some minor
| change, and it would start making random changes all over
| the file, completely destroying my mental model of how
| the program worked. Making it change that back completely
| pulls me out of the problem.
|
| When it's a junior at my job, at least I can feel like
| I'm developing a person. They retain the conversation and
| culture I impart as part of the problem solving process.
| When I struggle against the computer, it's just a waste
| of my time. It's not learning anything.
|
| I'm still really curious what you're doing with it.
| macrolime wrote:
| This kind of problems is really not where LLMs shine.
|
| Where you save loads of time is when you need to write lots
| of code using unfamiliar APIs. Especially when it's APIs
| you won't work with a lot and spending loads of time
| learning then would just be a waste of time. In these cases
| LLMs call tell you the correct API cells and it's easy to
| verify. The LLM isn't really solving some difficult
| technical problem, but saves lots of work.
| anujsjpatel wrote:
| For someone who didn't study a STEM subject or CS in school,
| I've gone from 0 to publishing a production modern looking
| app in a matter of a few weeks (link to it on my profile).
|
| Sure, it's not the best (most maintainable, non-redundant
| styling) code that's powering the app but it's more than
| enough to put an MVP out to the world and see if there's
| value/interest in the product.
| holoduke wrote:
| People in general don't like change and are naturally
| defending against it. And the older people get the greater
| the percentage of people fighting against it. A very useful
| and powerful skill is to be flexible and adaptable. You
| positioned yourself in the happy few.
| skydhash wrote:
| > _Much Much more productive world by just knuckling down and
| learning how to do the work._
|
| The fact everyone that say they've become more productive
| with LLMs won't say how exactly. I can talk about how VIM
| have make it more enjoyable to edit code (keybinding and
| motions), how Emacs is a good environment around text tooling
| (lisp machine), how I use technical books to further my
| learning (so many great books out here). But no one really
| show how they're actually solving problems with LLMs and how
| the alternatives were worse for them. It's all claims that
| it's great with no further elaboration on the workflows.
|
| > _I haven 't written a line of code in weeks. Just
| paragraphs of text that specify what I want and then guidance
| through and around pitfalls in a simple iterative loop of
| useful working code._
|
| Code is intent described in terms of machinery actions. Those
| actions can be masked by abstracting them in more
| understandable units, so we don't have to write opcodes, but
| we can use python instead. Programming is basically make the
| intent clear enough so that we know what units we can use.
| Software engineering is mostly selecting the units in a way
| to do minimal work once the intent changes or the
| foundational actions do.
|
| Chatting with a LLM look to me like your intent is either
| vague or you don't know the units to use. If it's the former,
| then I guess you're assuming it is the expert and will guide
| you to the solution you seek, which means you believe it
| understands the problem more than you do. The second is more
| strange as it looks like playing around with car parts, while
| ignoring the manuals it comes with.
|
| What about boilerplate and common scenarios? I agree that
| LLMs helps a great deal with that, but the fact is that there
| are perfectly good tools that helped with that like snippets,
| templates, and code generators.
| Nadya wrote:
| Ever seen someone try and search something on Google and
| they are just AWFUL at it? They can never find what they're
| looking for and then you try and can pull it up in a single
| search? That's what it is like watching some people try to
| use LLM's. Learning how to prompt an LLM is as much a
| learned skill as much as learning how to phrase internet
| searches is a learned skill. And as much as people decried
| that "searching Google isn't a real skill" tech-savvy
| people knew better.
|
| Same thing except now it's also many tech-savvy people
| joining in with the tech-unsavvy in saying that prompting
| isn't a real skill...but people who know better know that
| it is.
|
| On average, people are awfully bad at describing exactly
| what it is they want. Ever speak with a client? And you
| have to go back and forward for a few hours to finally
| figure out what it is they wanted? In that scenario you're
| the LLM. Except the LLM won't keep asking probing questions
| and clarifications - it will simply give them what they
| originally asked for (which isn't what they want). Then
| they think the LLM is stupid and stop trying to ask it for
| things.
|
| Utilizing an LLM to its full potential is a lot of
| iterative work and, at least for the time being, requires
| having some understanding of how it works underneath the
| hood (eg. would you get better results by starting a new
| session or asking it to forget previous, poorly worded
| instructions?).
| ben_w wrote:
| > On average, people are awfully bad at describing
| exactly what it is they want. Ever speak with a client?
| And you have to go back and forward for a few hours to
| finally figure out what it is they wanted?
|
| One of them it was the entire duration of me working for
| them.
|
| They didn't understand why it was taking so long despite
| constantly changing what they asked for.
| codr7 wrote:
| Building the software is usually like 10% of the actual
| job, we could do a better job of teaching that.
|
| The other 90% is mostly mushy human stuff, fleshing out
| the problem, setting expectations etc. Helping a group of
| people reach a solution everyone is happy with has little
| to do with technology.
| skydhash wrote:
| I'm not arguing that you can't get result with LLMs, I'm
| just asking is it worth the actual effort especially when
| there's better way to get that result you're seeking (or
| if the result is really something that you want).
|
| An LLM is a word (token?) generator which can be
| amazingly consistent according to its model. But rarely
| is my end goal to generate text. It's either to do
| something, to understand something, or to communicate.
| For the first, there are guides (books, manuals, ...),
| for the second, there are explanations (again books,
| manuals,...), and the third is just using language to
| communicate what's on my mind.
|
| That's the same thing with search engines. I use them to
| look for something. What I need first is a description of
| that something, not how to do the "looking for". Then
| once you know what you want to find, it's easier to use
| the tool to find it.
|
| If your end goal can be achieved with LLMs, be my guest
| to use them. But, I'm wary of people taking them at face
| value and then pushing the workload unto everyone else
| (like developers using electron).
| rvnx wrote:
| It works quite nicely if you consider LLMs as a
| translator (and that's actually why Transformers were
| created).
|
| Enter technical specifications in English as input
| language, get code as destination language.
| smallnamespace wrote:
| > asking is it worth the actual effort
|
| If prompting ability varies then this is not some
| objective question, it depends on each person.
|
| For me I've found more or less every interaction with an
| LLM to be useful. The only reason I'm not using it
| continually for 8 hours a day is because my brain is not
| able to usefully manage that torrent of new information
| and I need downtime.
| pbrowne011 wrote:
| > But no one really show how they're actually solving
| problems with LLMs and how the alternatives were worse for
| them. It's all claims that it's great with no further
| elaboration on the workflows.
|
| To give an example, one person (a researcher at DeepMind)
| recently wrote about specific instances of his uses of
| LLMs, with anecdotes about alternatives to each example.
| [1] People on HN had different responses with similar
| claims with elaborations on how it has changed some of
| their workflows. [2]
|
| While it would be interesting to see randomized controlled
| trials on LLM usage, hearing people's anecdotes brings to
| mind the (often misquoted) phrase: "The plural of anecdote
| is data". [3] [4]
|
| [1] https://nicholas.carlini.com/writing/2024/how-i-use-
| ai.html
|
| [2] https://news.ycombinator.com/item?id=41150317
|
| [3] http://blog.danwin.com/don-t-forget-the-plural-of-
| anecdote-i...
|
| [4] originally misquoted as "Anecdote is the plural of
| data."
| kristianp wrote:
| > (often misquoted) phrase
|
| You misquoted it there! It should be: The plural of
| anecdote is data.
| stavros wrote:
| It's actually "the plural of 'anecdote' is not 'data'".
| kristianp wrote:
| Apparently what you've said is the most common
| misquotation. See [3] above.
| stavros wrote:
| Oh interesting, thanks! I much prefer that formulation.
| pbrowne011 wrote:
| Thank you! Another instance of a variant of Muphry's Law.
|
| https://en.wikipedia.org/wiki/Muphry's_law
| kristianp wrote:
| In the CUDA example [1] from carlini's "how I Use AI", I
| would guess that o1 would need less handholding to do
| what he wanted.
|
| [1] https://chatgpt.com/share/1ead532d-3bd5-47c2-897c-2d7
| 7a38964...
| ben_w wrote:
| > But no one really show how they're actually solving
| problems with LLMs and how the alternatives were worse for
| them.
|
| I'm an iOS dev, my knowledge of JS and CSS is circa 2004.
| I've used ChatGPT to convert some of my circa 2009 Java
| games into browser games.
|
| > Chatting with a LLM look to me like your intent is either
| vague or you don't know the units to use
|
| Or that you're moving up the management track.
|
| Managers don't write code either. Some prefer it that way.
| joseluis wrote:
| The key difference is that this is a multidisciplinary
| conversational interface, and a tool in itself for
| interrelating structured meaning and reshaping it
| coherently enough so that it can be of great value both in
| the specific domain of the dialog, and in the potential to
| take it on any tangent in any way that can be expressed.
|
| Of course it has limitations and you can't be sleep at the
| wheel, but that's true of any tool or task.
| DiogenesKynikos wrote:
| For one, I spend less time on Stackoverflow. LLMs can
| usually give you the answer to little questions about
| programming or command-line utilities right away.
| fragmede wrote:
| Here's one from simonw
|
| https://gist.github.com/simonw/97e29b86540fcc627da4984daf5b
| 7...
|
| There are more to be found on his blog on the ai-assisted-
| programming tag. https://simonwillison.net/tags/ai-
| assisted-programming/
| sweeter wrote:
| Or people say "I've been pumping out thousands of lines of
| perfectly good code by writing paragraphs and paragraphs of
| text explaining what I want!" its like what are you
| programming dog? and they will never tell you, and then you
| look at their github and its like a dead simple starter
| project.
|
| I recently built a Brainfuck compiler and TUI debugger and
| I tested out a few LLM's just to see if I could get some
| useful output regarding a few niche and complicated issues,
| and it just gave me garbage that looked mildly correct.
| Then I'm told its because I'm not prompting hard enough...
| I'd rather just learn how to do it at that point. Once I
| solve that problem, I can solve it again in the future in
| .25x the time.
| ijustlovemath wrote:
| How much do you typically pay in a month of tokens?
| evilfred wrote:
| writing code is the easy part, designing is hard and not
| LLMable
| fragmede wrote:
| Given how hard we thought programming was a year or two
| ago, I wouldn't bank my future on design being too hard for
| an LLM. They're already quite good at helping writing
| design docs.
| bongodongobob wrote:
| Lol nope. When I'm trying to get it do make something
| big/complicated I start by telling it it's a software
| project manager and have me build a spec sheet on the
| design. Then I hand that off to an architect to flesh out
| the languages, libraries, files needed etc. Then from that
| list you can have it work on individual files and
| functions.
| cjbgkagh wrote:
| In my view these models produce above average code which is
| good enough for most jobs. But the hacker news sampling could
| be biased towards the top tier of coders - so their personal
| account of it not being good enough can also be true. For me
| the quality isn't anywhere close to good enough for my
| purposes, all of my easy code is already done so I'm only
| left working on gnarly niche stuff which the LLMs are not yet
| helpful with.
|
| For the effect on the industry, I generally make the point
| that even if AI only replaces the below average coder it will
| cause a downward pressure on above average coders
| compensation expectation.
|
| Personally, humans appear to be getting dumber at the same
| time that AI is getting smarter and while, for now, the
| crossover point is at a low threshold that threshold will of
| course increase over time. I used to try to teach ontologies,
| stats, SMT solvers to humans before giving up and switching
| to AI technologies where success is not predicated on human
| understanding. I used to think that the inability for most
| humans to understand these topics was a matter of motivation,
| but have rather recently come to understand that these
| limitations are generally innate.
| bcoates wrote:
| Re: Compensation expectations, I figured out a long time
| ago that bad programmers create bad code, and bad code
| creates work for good programmers.
|
| If the amount of bad code is no longer limited by the
| availability of workers who can be trained up to "just
| below average" and instead anyone who knows how to work a
| touchscreen can make AI slop, this opens up a big economic
| opportunity.
| cjbgkagh wrote:
| One could hope, but in my view perception precedes
| reality and even if that is the reality the perception is
| that AI will lower compensation demands and those doing
| the layoffs/hiring will act accordingly.
|
| You could also make the same claims about outsourcing,
| and while it appears that in most cases the outsourcing
| doesn't pay off, the perception that it would has really
| damaged CS as a career.
| rvnx wrote:
| It is also a problem of ego.
|
| It is difficult if you have been told all your life that
| you are the best, to accept the fact that a computer or
| even other people might be better than you.
|
| It requires lot of self-reflection.
|
| Real top-tiers programmers actually don't feel threatened
| by LLMs. For them it is just one more tool in the toolbox
| like syntax highlighting or code completion.
|
| They choose to use these tools based on productivity gains
| or losses, depending on the situation.
| minkles wrote:
| That's fine until your code makes its way to production, an
| unconsidered side effect occurs and then you have to face me.
|
| You are still responsible for what you do regardless of the
| means you used to do it. And a lot of people use this not
| because it's more productive but because it requires less
| effort and less thought because those are the hard bits.
|
| I'm collecting stats at the moment but the general trend in
| quality as in producing functional defects is declining when
| an LLM is involved in the process.
|
| So far it's not a magic bullet but a push for mediocrity in
| an industry with a rather bad reputation. Never a good story.
| bongodongobob wrote:
| Ugh, dude, I used to push bad code into production without
| ChatGPT. It is such a stupid argument. Do you really think
| people are just blindly pushing code they can't make heads
| or tails of? That they haven't tested? Do you seriously
| think people are just one shotting code and blasting it
| into prod? I am completely baffled by people in this
| industry that _just don 't get it_. Learn to prompt. Write
| tests. Wtf.
| minkles wrote:
| Yes that's exactly what they are doing.
|
| I literally had someone with the balls to tell me that it
| was ChatGPT's fault.
|
| Due diligence and intelligence has shit the fucking bed
| quite frankly.
| scubbo wrote:
| > Do you really think people are just blindly pushing
| code they can't make heads or tails of? That they haven't
| tested? Do you seriously think people are just one
| shotting code and blasting it into prod?
|
| Yes, and I see proof of it _literally every day_ in Code
| Reviews where I ask juniors to describe or justify their
| choices and they shrug and say "That's what Copilot told
| me to put".
| hughesjj wrote:
| My problem is that, for a surprising number of
| applications, it's taken me longer to have the
| conversation with chatgpt to get the code I want than
| just doing it myself.
|
| Copilot and the likes are legit for boilerplate, some
| test code, and posix/power shell scripting. Anything
| that's very common it's great.
|
| Anything novel though and it suffers. Did AWS just
| release some new functionality and only like 4 people
| have touched it so far on GitHub? Are you getting source
| docs incomplete or spread out amongst multiple pages with
| some implicit/betwen-the-lines spec? Eh, good luck,
| you're probably better off just reading the docs yourself
| or guess and checking.
|
| Same goes for versioning, sometimes it'll fall back into
| an older version of the system (ex Kafka with kraft vs
| zookeeper)
|
| Personally, the best general use case of LLMs for me is
| focus. I know how to break down a task, but sometimes I
| have an issue staying focused on doing it and having a
| reasonably competent partner to rubber duck with is super
| useful. It helps that the chat log then becomes an easy
| artifact to more or less copy paste, and chatgpt doesn't
| do a terrible job reformatting either. Like for 90% of
| the stuff it's easier than using vim commands.
| hobs wrote:
| Do you think ChatGPT has changed any of those answers
| from Yes to No? Because it hasn't.
|
| People blindly copied stack overflow code, they blindly
| copied every example off of MSDN, they blindly copy from
| ChatGPT - your holier than thou statements are funny, and
| frankly most LLMs cannot leave a local maxima, so anyone
| who says they dont write any code anymore I frankly think
| they are not capable of telling the mistakes, both
| architecturally and specifically that they are making.
|
| More and different prompting will not dig you out of the
| hole.
| blargey wrote:
| Wasn't there a recent post about many research papers
| getting published with conclusions derived from
| buggy/incorrect code?
|
| I'd put more hope in improving LLMs/derivatives than
| improving the level of effort and thought in code across
| the entire population of "people who code", especially the
| subset who would rather be doing something else with their
| time and effort / see it as a distraction from the "real"
| work that leverages their actual area of expertise.
| a_wild_dandan wrote:
| > You are still responsible for what you do regardless of
| the means you used to do it. And a lot of people use this
| not because it's more productive but because it requires
| less effort and less thought because those are the hard
| bits.
|
| Yeah, that's...the whole point of tools. They reduce
| effort. And they don't shift your responsibility. For many
| of us, LLMs are overwhelmingly worth the tradeoffs. If your
| experience differs, then it's unfortunate, and I hate that
| for you. Don't use 'em!
| amrrs wrote:
| Curious why Aider? Why not Cursor ?
| rafaelmn wrote:
| I use sonet 3.5 and while it's actually usable for codegen
| (compared to gpt/copilot) it's still really not that great.
| It does well at tasks like "here's a stinky collection of
| tests that accrued over time - clean this up in style of x"
| but actually writing code still shows fundamental lack of
| understanding of underlying API and problem (the most banal
| example being constantly generating `x || Array.isArray(x)`
| test)
| po76 wrote:
| Give it a few months. ChatGPT will be recommending GPTs to use
| or do it automatically.
|
| Nothing is static in the way things are moving.
| airstrike wrote:
| It also doesn't help that Lean has had so many breaking changes
| in such little time. When I tried using GPT-4 for it, it mostly
| rendered old code that would fail to run unless you already
| knew the answer and how to fix it, which basically made it
| entirely unhelpful.
| benterix wrote:
| > people who complain on HN that (paid/good - only Sonnet 3.5
| and GPT4o are in this category)
|
| Correction: I complain that the only decent model in "Open"AI's
| arsenal, that is GPT-4, has been replaced by a cheaper GPT-4o,
| which gives subpar answers to most of my question (I don't care
| it does it faster). As they moved it to "old, legacy" models, I
| expect they will phase it out, at which point I'll cancel my
| OpenAI subscriptions and Sonnet 3.5 will become the clear
| leader for my daily tasks.
|
| Kudos to Anthropic for their great work, you guys are going in
| the right direction.
| bongodongobob wrote:
| Nah, o1 is fucking impressive. It's really fucking good. I'm
| guessing you haven't used it yet.
| eab- wrote:
| Why do you expect GPT being tuned on Lean will help it for
| research-level math?
| CamperBob2 wrote:
| _But many of the low-effort comments seem to mostly fall into
| (1) and (2) -- cynicism rather than cautious optimism._
|
| One good riposte to reflexive LLM-bashing is, "Isn't _that_
| just what a stochastic parrot would say? " Some HN'ers would
| dismiss a talking dog because the C code it wrote has a buffer
| overflow error.
| riffraff wrote:
| _can_ GPT be tuned more heavily on Lean? It looks like the
| amount of python code in the corpus would outnumber Lean
| something like 1000:1. Although I guess OpenAI could generate
| more and train on that.
| EvgeniyZh wrote:
| There is ~3 order of magnitude more Python code in the internet
| than Lean code (200GB vs 200MB in the stack v2). You can't tune
| it "the same way"
| agumonkey wrote:
| Fair point but a lot of python code is redundant and low
| quality.
| agumonkey wrote:
| side question, are there good OR websites / platforms (reddit,
| mastodon) to get involved in the field ?
| rabf wrote:
| Most people are on X.
| l33t7332273 wrote:
| I an also working in OR and I have had the complete opposite
| experience with respect to MILP optimization(and the research
| actually agrees; there was a big survey paper published earlier
| this year showing LLMs were mostly correct on textbook problems
| but got more and more useless as complexity and novelty
| increased.)
|
| The results are boiler plate at best, but misleading and
| insidious at worst, especially when you get into detailed
| tasks. Ever try to ask a LLM what a specific constraint does or
| worse ask it to explain the mathematical model of some
| proprietary CPLEX syntactic sugar? It hallucinates the math,
| the syntax, the explanation, everything.
| wenc wrote:
| Can you point me to that paper? What version of the model are
| they using?
|
| Have you tried again with the latest LLMs? ChatGPT4 actually
| (correctly) explains what each constraint does in English --
| it doesn't just provide the constraint when you ask it for
| the formulation. Also, not sure if CPLEX should be involved
| at all -- I usually just ask it for mathematical
| formulations, not CPLEX calling code (I don't use CPLEX). The
| OR literature primarily contains math formulations and that's
| where LLMs can best do pattern matching to problem shape.
|
| Many of the standard formulations are in here:
|
| https://msi-jp.com/xpress/learning/square/10-mipformref.pdf
|
| All the LLM is doing is fitting the problem description to a
| combination of these formulations (and others).
| Davidzheng wrote:
| I'm not sure the lean coverage of pure math research is that
| much (maybe like 1% is represented on mathlib). But I think a
| system like alpha proof could even today be useful for
| mathematicians--I mostly dislike systems like o1 where they
| confidently say nonsense with such high frequency. But i think
| value is already there.
| thelastparadise wrote:
| > but for someone who can and does, the $20/month I pay for
| ChatGPT more than pays for itself.
|
| Would you be willing to pay even more, if it meant you were
| getting proportionally more valuable answers?
|
| E.g. $200/month or $2,000/month (assuming the $2,000/month gets
| into employee/intern/contractor level of results.)
|
| This might drive a positive feedback loop.
| ninetyninenine wrote:
| A specialized LLM could possibly meet his criteria already.
| 317070 wrote:
| Probably. The missing factor is the dataset and the fact that
| so far OAI seems to be the only one who has figured out how to
| train this thing for reasoning.
|
| But yeah, given o1 exists, it looks very doable. It's hard to
| imagine a reason for why something matching his criteria would
| be more than a decade out.
| nyc111 wrote:
| I checked the links and I think it's amazing and it answers with
| Latex formatted notation.
|
| But I was curious and I asked something very simple, Euclid's
| first postulate and I got this answer:
|
| Euclid's Postulate 1: "Through any two points, there is exactly
| one straight line."
|
| In fact Euclid's Postulate 1 is "To draw a straight line from any
| point to any point."
| http://aleph0.clarku.edu/~djoyce/java/elements/bookI/bookI.h...
|
| I think AI answer is not correct, it may be some textbook
| interpretation but I was expecting Euclid's exact wording.
|
| Edit: Google's Gemini gives the exact wording of the postulate
| and then comments that this means that you can draw one line
| bitween two points. I think this is better
| supermatt wrote:
| > I think AI answer is not correct, it may be some textbook
| interpretation but I was expecting Euclid's exact wording
|
| It was written before English even existed. That said, the
| original never implied "exactly one", so I agree its a bad
| translation.
| slavboj wrote:
| Euclid wrote in ancient Greek, so the "exact wording" in
| English does not exist.
| pama wrote:
| The original text is: Eitestho apo pantos semeiou epi pan
| semeion eutheian grammen agagein. Roughly: let it be required
| that from any point to any point it is possible to draw a
| straight line.
|
| Both gpt4o and o1 roughly know the correct original text, so
| prompting, the model's background memory, or random chance may
| influence your outcomes, though hopefully (in an improved
| model) you should never get you incorrect info.
|
| https://farside.ph.utexas.edu/Books/Euclid/Elements.pdf
|
| Edit: in case it isnt clear, I could not reproduce this error
| on my end with o1-mini
| roywiggins wrote:
| it's definitely wrong though, "exactly one" straight line
| between two points is a different postulate and a stronger
| one.
|
| Euclid has been translated, restated, and re-presented in
| enough books and textbooks that I'd expect a big-enough LLM
| to have actually memorized this correctly tbh
| pama wrote:
| Agreed. That is what the original poster said. I didnt
| manage to reproduce the error on my end, but I dont know
| the full context or maybe the memory on my end changes the
| output.
| wging wrote:
| "Exact wording" would be Ancient Greek. Euclid did not even
| write in English. You're checking whether the model matches a
| specific translation, which is not valuable. If you search
| around you'll find many sources that choose a more intelligible
| phrasing.
| layer8 wrote:
| Regardless of the wording being exact or not, ChatGPT's answer
| is incorrect in its contents. The statement "exactly one"
| requires the parallel postulate, since otherwise it's not
| necessarily true. Specifically in spherical geometry, which is
| considered to be consistent with Euclid's first four postulates
| (i.e. without the parallel postulate).
|
| The bottom line is, you can't take any single LLM statement at
| face value, even in seemingly easy to answer cases like this.
| roywiggins wrote:
| it's the point-line postulate, you _can_ use it as part of a
| set of axioms equivalent to Euclid but it definitely not one
| of Euclid 's
|
| https://en.wikipedia.org/wiki/Point%E2%80%93line%E2%80%93pla.
| ..
| nyrikki wrote:
| Euclid's Elements is less pervasive on the Internet then
| content produced for Liberal Arts math courses. As those
| courses tend to emphasize critical thinking and problem-solving
| math over pure theory and advanced concepts, they tend to be
| far more common and tend to win compared to more domain
| specific meanings.
|
| Examples:
|
| https://en.wiktionary.org/wiki/Euclidean_geometry
|
| https://www.cerritos.edu/dford/SitePages/Math_70_F13/Postula...
|
| Problems with polysemy across divergent, more advanced theories
| has been one of my biggest challenges in probing some of my
| areas of intrest.
|
| Funny enough, one of my pet areas of obscure interest, riddled
| basins, is constantly muddied not by math, but LSAT questions,
| specifically non-math content directed at a reading
| comprehension test: "September 2006 LSAT Section 1 Question 26"
|
| IMHO a lot of the prompt engineering you have to do with these
| highly domain specific problems is avoiding the most common
| responses in the corpus.
|
| LLM responses will tend to reflect common usage, not academic
| terminology unless someone cares enough to change that for a
| specific case.
| kzz102 wrote:
| It's interesting that humans would also benefit from the "chain
| of thought" type reasoning. In fact, I would argue all students
| studying math will greatly increase their competence if they are
| required to recall all relevant definition and information before
| using it. We don't do this in practice (including teachers and
| mathematicians!) because recall is effortful, and we don't like
| to spent more effort than necessary to solve a problem. If recall
| fails, then we have to look up information which takes even more
| effort. This is why in practice, there is a tremendous incentive
| to just "wing it".
|
| AI has no emotional barrier to wasted effort, which make them
| better reasoners than their innate ability would suggest.
| Satam wrote:
| Wow! I love this take. Somehow with all this evidence of COT
| helping out LLMs, I never thought about using it more myself.
| Sure, we kind of do it already but definitely not to the degree
| of LLMs, at least not usually. Maybe that's why writing is so
| often admired as a way to do great thinking - it enables longer
| chains of thoughts with less effort.
| schappim wrote:
| Showing your work in tests is kind of like "chain of thought"
| reasoning, but there's a slight difference. Both force you to
| break down your process step by step, making sure the logic
| holds and you aren't skipping crucial steps. But while showing
| your work is more about demonstrating the correct procedure,
| "chain of thought" reasoning pushes you to recall relevant
| definitions and concepts as you go, ensuring a deeper
| understanding. In both cases, the goal is to avoid just
| "winging it," but "chain of thought" really digs into the
| recall aspect, which humans tend to avoid because it's
| effortful.
| perihelions wrote:
| I'm so excited in anticipation of my near-term return to studying
| math, as an independent curiosity hobby. It's going to be
| epically fun this time around with LLM's to lean on.
| Coincidentally like Terence Tao, I've also been asking complex
| analysis queries* of LLM's, things I was trying to understand
| better in my working through textbooks. Their ability to
| interpret open-ended math questions, and quickly find _distant
| conceptual links_ that are helpful and relevant, astonishes me.
| Fields laureate Professor Tao (naturally) looks _down_ on the
| current crop of mathematics LLM-- "not completely incompetent
| graduate student..."--but at my current ability level that just
| means looking _up_.
|
| *(I remember a specific impressive example from 6 months ago: I
| asked if certain definitions could be relaxed to allow complex
| analysis on a non-orientable manifold, like a Klein bottle,
| something I spent a lot of time puzzling over, and an LLM
| instantly figured out it would make the Cauchy-Riemann equations
| globally inconsistent. (In a sense the arbitrary sign convention
| in CR _defines_ an orientation on a manifold: reversing manifold
| orientation is the same as swapping i with -i. I understand this
| now, solely because an LLM suggested looking at it). Of course, I
| 'm sure this isn't original LLM thinking--the math's certainly
| written down somewhere in its training material, in some highly
| specific postgraduate textbook I have no knowledge of. That's not
| relevant to me. For me, it's _absolutely impossible_ to answer
| this type of question, where I have very little idea where to
| start, without either an LLM or a PhD-level domain specialist.
| There is _no other tool_ that can make this kind of _semantic-
| level_ search accessible to me. I 'm very carefully thinking how
| best to make use of such an, incredibly powerful but alien,
| tool...)
| WanderPanda wrote:
| How will we even measure this? Benchmarks are gamed/trained on
| and there is no way that there is much signal in the chatbot
| arena for these types of queries?
|
| I think in just a few month the average user will not be able
| to tell the difference in performance between the major models
| nybsjytm wrote:
| How will you know if its answers are correct or not?
| perihelions wrote:
| Because I'm verifying everything by hand, as is the whole
| point of studying pure mathematics.
| SOTGO wrote:
| How can you verify a proof though? Pure math isn't really
| about computations, and it can be very hard to spot subtle
| errors in a proof that an LLM might introduce, especially
| since they seem better at sounding convincing rather than
| being right.
| fragmede wrote:
| By using Lean, a proof assistant and a functional
| programming language.
|
| Here's @tao on mathstodon saying he's learning it.
|
| https://mathstodon.xyz/@tao/111206761117553482
| nybsjytm wrote:
| To code proofs in lean, you have to understand the proof
| very well. It doesn't seem to be very reasonable for
| someone learning material for the first time.
| nybsjytm wrote:
| Good luck! That can be pretty hard to do when you're at the
| learning stage, and I would think doubly so given the LLM
| style where everything 'looks' very convincing.
| artninja1988 wrote:
| >could not generate conceptual ideas of their own
|
| Is the most important part imo. A big goal should be some ai
| system coming up with its own discovery and ideas. Really unclear
| how we can get from the current paradigm to it coming up with
| something like general relativity, like Einstein. Does it require
| embodiment?
| sfink wrote:
| Why should that be a big goal? It's difficult, it's not what
| they are good at, and they can get a lot better at assisting in
| other ways through incremental improvements. I'm happy to leave
| this part to the humans, at least for now, especially when
| there's so much more improvement still possible in other
| directions.
|
| It also seems like one of those things where we ought to ask
| whether we should, before asking whether we could. Why not
| focus on areas that are easier, more beneficial, and less
| problematic from a "should" perspective?
| roywiggins wrote:
| we don't know how to reliably produce _humans_ who produce GR-
| level ideas, this might be biting off a lot more than we can
| chew
| sgt101 wrote:
| Is there a list of discoveries or siginficant works/constructions
| made by people collaborating with LLM's? I mean as opposed to
| specific deep networks like Alphafold or Graphcast?
| adt wrote:
| I'd like to see that, too.
|
| I have a related list of GPT accomplishments here:
| https://docs.google.com/spreadsheets/d/1kc262HZSMAWI6FVsh0zJ...
| rvnx wrote:
| It may cause a reputation or legal issue, so it is not in their
| interest to admit it. In the real world, is there PhD students
| or researchers using ChatGPT to move forward and help them
| think their ideas ?
|
| Obviously yes, but admitting it may not be the right move.
| abstractbill wrote:
| My experience with O1 has been very different. I wouldn't even
| say it's performing at a "good undergrad" level for me.
|
| For example, I asked a pretty simple question here and it got
| completely confused:
|
| https://moorier.com/math-chat-1.png https://moorier.com/math-
| chat-2.png https://moorier.com/math-chat-3.png
|
| (Full chat should be here:
| https://chatgpt.com/share/66e5d2dd-0b08-8011-89c8-f6895f3217...)
| jghn wrote:
| Anecdata, but I've been finding O1 to be worse than 4o & Claude
| 3.5 Sonnet. To add insult to injury, it's slower & chattier.
| anujsjpatel wrote:
| And sometimes it just bugs out and doesn't give any response?
| Faced that twice now, it "thought" for like 10-30s then no
| answer and I had to click regenerate and wait for it again.
| jghn wrote:
| I've seen it take over a couple of minutes, at which point
| I switched to Claude. And have seen reports of it taking
| even longer. So it may be that you didn't wait long enough.
| abdullahkhalids wrote:
| Thinking about training LLMs on geometry. A lot of information
| in the sources would be contained in the diagrams accompanying
| the text. This model is not multi-modal, so maybe it wasn't
| trained on the accompanying diagrams at all.
|
| I would really like if people check on a set of geometry and a
| set of analysis questions and compare the difference.
| jazzyjackson wrote:
| It will be trash. I'll have to dig up a chat I had the
| weekend GPT4 was released, I was musing about dodecahedron
| packing problems and GPT4 started with an assertion that a
| line through a sphere intersects the surface 3 times.
|
| Maybe if you fine tuned it on Euclid's Elements and then
| allowed it to run experiments with Mathematica snippets it
| could check its assumptions before spouting nonsense
| almostgotcaught wrote:
| Why would they do this - make it speak like a customer service
| agent. The ideal experience here is short and succinct, not
| verbose and obsequious.
| ljlolel wrote:
| Performs better on chat bot arena head to head
| svdr wrote:
| Did you find out what the error was in computing the volume of
| the truncated icosidodecahedron?
| gary_0 wrote:
| Tao mentions grad students; I wonder how they feel reading this?
|
| As LLMs continue to improve I feel like anyone making a living
| doing the "99% perspiration" part of intellectual labor is about
| to enter a world of hurt.
| fragmede wrote:
| > The experience seemed roughly on par with trying to advise a
| mediocre, but not completely incompetent, graduate student.
|
| And you thought you had imposter syndrome before!
| teaearlgraycold wrote:
| Or can everyone now lead research projects and build
| businesses?
| asdasjhG wrote:
| No, almost everyone who gets funding for a business already
| belongs to the monied royalty and gets it either directly
| from his family, via friends of the family or laundered
| through a VC.
|
| There are exceptions of course, but that's how the bulk of
| businesses, especially those with stupid ideas are funded. In
| the latter category success does not even matter, the trust
| fund baby just has to have the appearance of a leader
| position.
| eigenvalue wrote:
| The o1 model is really remarkable. I was able to get very
| significant speedups to my already highly optimized Rust code in
| my fast vector similarity project, all verified with careful
| benchmarking and validation of correctness.
|
| Not only that, it also helped me reimagine and conceptualize a
| new measure of statistical dependency based on Jensen-Shannon
| divergence that works very well. And it came up with a super fast
| implementation of normalized mutual information, something I
| tried to include in the library originally but struggled to find
| something fast enough when dealing with large vectors (say,
| 15,000 dimensions and up).
|
| While it wasn't able to give perfect Rust code that compiled on
| the very first try, it was able to fix all the bugs in one more
| try after pasting in all the compiler warning problems from
| VScode. In contrast, gpt-4o usually would take dozens of tries to
| fix all the many rust type errors, lifetime/borrowing errors, and
| so on that it would inevitably introduce. And Claude3.5 sonnet is
| just plain stupid when it comes to Rust for some reason.
|
| I really have to say, this feels like a true game changer,
| especially when you have really challenging tasks that you would
| be hard pressed to find many humans capable of helping with (at
| least without shelling out $500k+/year in compensation for).
|
| And it's not just the performance optimization and relatively bug
| free code-- it's the creative problem solving and synthesis of
| huge amounts of core mathematical and algorithmic knowledge plus
| contemporary research results, combined with a strong ability to
| understand what you're trying to accomplish and making it happen.
|
| Here is the diff to the code file showing the changes:
|
| https://github.com/Dicklesworthstone/fast_vector_similarity/...
| aprilthird2021 wrote:
| But a lot of what you pay humans $500k a year for is to work
| with enormous existing systems that an LLM cannot understand
| just yet. Optimizing small libraries and implementing fast
| functions though is a huge improvement in any programmer's
| toolbox.
| eigenvalue wrote:
| Yes, that's certainly true, and that's why I selected that
| library in particular to try with it. The fact that it's
| mathematical-- so not many lines of code, but each line packs
| a lot of punch and requires careful thought to optimize--
| makes it a perfect test bed for this model in particular. For
| larger projects that are simpler, you're probably better off
| with Claude3.5 sonnet, since it has double the context
| window.
| dyauspitr wrote:
| Can't Gemini work with a million+ input tokens?
| aprilthird2021 wrote:
| It doesn't work well though. You can't just stuff your
| entire codebase into it and get good results. I work
| somewhere that tries to do this internally
| eigenvalue wrote:
| Yes, but its reasoning ability is extremely poor in my
| experience with real world programming tasks. I'm talking
| about stuff that Claude3.5 Sonnet handles easily, and GPT4o
| can also handle if it can fit in its smaller context
| window, where Gemini 1.5 pro just completely fails.
|
| Bigger context is definitely helpful, but not if it comes
| at the expense of reasoning/analytical ability. I'm always
| a bit puzzled why people stress the importance of these
| "needle in a haystack" tests where the model has to find
| one specific thing in a huge document. That seems far less
| relevant to me in terms of usefulness in the real world.
| derefr wrote:
| > I'm always a bit puzzled why people stress the
| importance of these "needle in a haystack" tests where
| the model has to find one specific thing in a huge
| document. That seems far less relevant to me in terms of
| usefulness in the real world.
|
| How do you mean?
|
| Half of writing code within a codebase, is knowing what
| functions already exist in the codebase for you to call
| in your own code; and/or, what code you'll have to change
| upstream and downstream of the code you're modifying
| within the same codebase -- or even by forking your
| dependencies and changing _them_ -- to get what you want
| to happen, to happen.
|
| And half of, say, writing a longform novel, is knowing
| all the promises you've made to the reader, the active
| Chekov's guns, and all the other constraints you've
| placed on yourself by hundreds of pages or even several
| books ago, that just became relevant again as of this
| very sentence. Or, moreover, which of those details it's
| the proper time to _make_ relevant again for maximum
| impact and proper first-in-last-out narrative bridging
| structure.
|
| In both cases, these aren't really literal "needle in a
| haystack" stress-tests; they should _properly_ be tests
| of the model 's ability to perform some kind of
| "associational priority indexing" on the context,
| allowing it to build concepts into associational sub-
| networks and then make long-distance associations where
| the nodes are entire subnetworks. (Which isn't something
| we really see yet, in any model.)
| jdiez17 wrote:
| > 1,337 additions
|
| _cough_
| Ylpertnodi wrote:
| >you would be hard pressed to find many humans capable of
| helping with (at least without shelling out $500k+/year in
| compensation for).
|
| And now we have a $number we can relate, and refer, to.
| kldnav wrote:
| Tao and Aaronson are optimistic about LLMs. What are they telling
| their students? That math and science degrees will soon have the
| same value as a degree in medieval dance theory?
|
| If they are overly optimistic, perhaps it would be good to hear
| the opinions of Wiles and Perelman.
| ljlolel wrote:
| If you look at a lot of people's PHDs, we now teach these
| things to 1st years. PhDs today do incredible deep work and the
| edge of science will just go further.
| raincole wrote:
| Tao isn't _that_ optimistic. His opinion on LLMs is rather
| conservative.
|
| https://www.scientificamerican.com/article/ai-will-become-ma...
|
| > If you want to prove an unsolved conjecture, one of the first
| things you need to do is to break it up into smaller pieces,
| each of which has a better chance of being proven. But you will
| often break up a problem into harder problems. It's very easy
| to transform a problem into one that's harder than into one
| that's simpler. And AI has not demonstrated any ability to be
| any better than humans in this regard.
|
| Not sure if O1 changed his mind tho.
| Davidzheng wrote:
| What does this mean? Of course math AI will take over top
| research in next ten years but usefulness to society has never
| been a goal of pure mathematics. I don't know if you understand
| the motivation for studying pure math. Personally I think it
| will be mostly good for research math
| asdasjhG wrote:
| The "value of a degree" means the employment prospects for
| the degree holder.
|
| Which is going to zero if the optimistic predictions are
| correct, so the optimistic professors should warn their
| students.
|
| I understand the motivation for pure math quite well. It is
| about beauty, understanding things and discovering things for
| oneself. If computers do the work, the discovery part is gone
| and pure math is ruined.
|
| For the non-research part, the AI zealots will want to
| replace all human labor with software.
| olalonde wrote:
| Why are you saying this as if it was a bad thing? Just
| because software becomes better at us at something doesn't
| mean we can't do it out of fun (e.g. see chess community
| for example).
| ein0p wrote:
| Idk I think the fact that it needs "hints" and "prodding" is a
| good thing, myself. Otherwise we wouldn't need humans to get
| those answers, would we. I want it to augment humans, not replace
| them.
| MrFots wrote:
| Incompetent Graduate Students is the name of my new sketch group.
| itissid wrote:
| One thing it's certainly doing better is exploring the search
| space better e.g.:
| https://x.com/sg3487/status/1835040593703010714
|
| If you know the contours of the answer and can describe what you
| are looking for it can quickly find it for you.
| nmca wrote:
| Note the selection effect in "a mediocre graduate student" (that
| got to work with Terry Tao)
| reverseblade2 wrote:
| Here's a little test I try on LLMs. So far only O1 and Microsoft
| Copilot (bing chat) was able to solve it:
|
| Find a, b, c distinct positive integers satisfying a^3 + b^3 =
| c^4. Hint: try dividing all sides by c^3, then giving values to
| (a/c) and (b/c).
| nybsjytm wrote:
| Daniel Litt, an algebraic geometer on twitter, said "Pretty
| impressed by o1-preview! Still not having much luck asking it to
| do any interesting math but it seems much more reliable with
| simple things; I can actually imagine it being a net time-saver
| at this point with some non-mathematical tasks."
|
| Any other takes by mathematicians out there?
| fsndz wrote:
| Completely agree with Terence Tao. this is a real advancement.
| I've always believed that with the right data allowing the LLM to
| be trained to imitate reasoning, it's possible to improve its
| performance. However, this is still pattern matching, and I
| suspect that this approach may not be very effective for creating
| true generalization. As a result, once o1 becomes generally
| available, we will likely notice the persistent hallucinations
| and faulty reasoning, especially when the problem is sufficiently
| new or complex, beyond the "reasoning programs" or "reasoning
| patterns" the model learned during the reinforcement learning
| phase. https://www.lycee.ai/blog/openai-o1-release-agi-reasoning
| gcanyon wrote:
| > The experience seemed roughly on par with trying to advise a
| mediocre, but not completely incompetent, graduate student.
|
| Coming from Terence Tao that seems pretty remarkable to me?
| fnordpiglet wrote:
| Rewind your mind to 2019 and imagine reading a post that said
|
| "The experience seemed roughly on par with trying to advise a
| mediocre, but not completely incompetent, graduate student."
|
| With regard to interacting with the equivalent of Alexa. That's a
| remarkable difference in 5 years.
| talldayo wrote:
| To be honest, I have gotten 100x more useful answers out of
| Siri's WolframAlpha integration than I ever have out of
| ChatGPT. People don't want a "not completely incompetent
| graduate student" responding to their prompts, they want NLP
| that reliably processes information. Last-generation voice
| assistants could at least do their job consistently, ChatGPT
| couldn't be trusted to flick a light switch on a regular basis.
| meowface wrote:
| I use both for different things. WolframAlpha is great for
| well-defined questions with well-defined answers. LLMs are
| often great for anything that doesn't fall into that.
| thelastparadise wrote:
| Wait til you generate WolframAlpha queries from natural
| language using Claude 3.5 and use it to interpret results as
| well.
| talldayo wrote:
| I've tried the ChatGPT integration and it was kinda just
| useless. On smaller datasets it told me nothing that wasn't
| obviously apparent from the charts and tables; on larger
| datasets it couldn't do much besides basic key/value
| retrieval. Asking it to analyze a large time-series table
| was an exercise in futility, I remain pretty unimpressed
| with current offerings.
| Karrot_Kream wrote:
| How does this square up with literally what Terence Tao (TFA)
| writes about O1? Is this meant to say there's a class of
| problems that O1 is still really bad at (or worse than
| intuition says it should be, at least)? Or is this "he says,
| she says" time for hot topics again on HN?
| segmondy wrote:
| Then you have a skill issue. 10 million paying are for GPT
| monthly because a large of them are getting useful value out
| of it. WolframAlpha has been out for a while and didn't take
| off for a reason. "GPT couldn't be trusted to flick a light
| switch on a regular basis" pretty much implies you are not
| serious or your knowledge about the capabilities of LLM is
| pretty much dated or derived from things you have read.
| codr7 wrote:
| FACT: The technology is inherently unreliable in its
| current form. And the weakness is built in, its not going
| to go away anytime soon.
| jonahx wrote:
| The same is true of search engines, yet they are still
| incredibly useful.
| jazzyjackson wrote:
| Wolframalpha is a free service
|
| Wolfram Research is a profitable company btw
| noch wrote:
| The important point is, I feel, that most people are not even
| at the level of intelligence of a _" a mediocre, but not
| completely incompetent, graduate student."_ A mediocre graduate
| science student, especially of the sort who graduates and
| doesn't quit, is a very impressive individual compared to the
| rest of us.
|
| For "us", having such a level of intelligence available as an
| assistant throughout the day is a massive life upgrade, if we
| can just afford more tokens.
| thewanderer1983 wrote:
| >A mediocre graduate science student, especially of the sort
| who graduates and doesn't quit, is a very impressive
| individual compared to the rest of us.
|
| Incorrect. University graduates shows a good work ethic, a
| certain character and a ability to manage time. It's not a
| measure of being better than the rest of humanity. Also, it's
| not a good measure of intelligence. If you only want to view
| the world through credentials. Academics don't consider your
| intelligence until you have a Ph.D and X years of work in
| your field. Industry only uses graduates as a entry
| requirement for junior roles and then favors and cares only
| about your years of experience after that. Given that
| statement I can only assume you haven't been to University.
| You are mistaken to think, especially in time we are in now
| that the elite class are any more knowledgeable then you are.
| kiba wrote:
| We are more limited by our emotions, and then our skills in
| learning and acquiring knowledge.
|
| Intelligence is probably a distant third.
| atleastoptimal wrote:
| Nah. Dogs are far emotionally better than most humans.
| Their intelligence is their limitation. Also "skills in
| learning and acquiring knowledge" is basically intelligence
| benreesman wrote:
| Reading anything Terrence Tao writes is thought provoking and I
| doubt I'm seeing anything others haven't.
|
| There's at least a "complexity" if not a "problem" in terms of
| judging models that to a first approximation have been trained on
| "everything".
|
| Have people tried putting these things up against serious
| mathematical problems that are well studied? With or with Lean
| hinting has anyone gotten like, the Shimura-Taniyama
| conjecture/proof out?
| ak_111 wrote:
| He mentions that he posed to O1 the same challenge he posed to a
| previous GPT (which he also previously blogged about), so I am
| wondering how much O1 benefited from potentially "seeing" this
| discussion in its training set (which probably contains a very
| well recent snapshot of the world wide web).
| 2muchcoffeeman wrote:
| What a burn
|
| _"The experience seemed roughly on par with trying to advise a
| mediocre, but not completely incompetent, graduate student."_
| busyant wrote:
| Well, one thing is clear.
|
| Math grad students everywhere now have a benchmark to determine
| if Terry Tao considers them to be mediocre or incompetent.
___________________________________________________________________
(page generated 2024-09-14 23:00 UTC)