Post ASRwQec1r6LEgz6VLE by russss@chaos.social
(DIR) More posts by russss@chaos.social
(DIR) Post #ASRmqLfXuSJPvcMWg4 by simon@fedi.simonwillison.net
2023-02-07T20:24:04Z
1 likes, 1 repeats
Ouch. Isabel Angelo points out that Google's tweet announcing Bard, their new large language model search tool (ChatGPT competitor) includes a screenshot confidently stating that "JWST took the very first pictures of a planet outside of our own solar system"... which isn't true!Not a great start.https://twitter.com/IsabelNAngelo/status/1623013720011194368
(DIR) Post #ASRn34VZ1c8aT8lC9w by simon@fedi.simonwillison.net
2023-02-07T20:25:58Z
0 likes, 0 repeats
A regular Google Search for that returns stories like this one: https://www.axios.com/2022/09/01/jwst-image-planet-outside-solar-system - which states "JWST captures its first direct image of a planet outside our solar system"There's a pretty important difference between "its first direct image of" and "the very first pictures of"
(DIR) Post #ASRnoVY4J6dDwZYDsO by russss@chaos.social
2023-02-07T20:34:13Z
0 likes, 0 repeats
@simon Also even that is wrong! (I think it's fine to say "first directly imaged" = first photo though) https://exoplanets.nasa.gov/resources/300/2m1207b-first-image-of-an-exoplanet/
(DIR) Post #ASRoKx85IVLGkcrsi8 by giancampo@masto.measure.chat
2023-02-07T20:39:46Z
0 likes, 0 repeats
@simon doh 😣
(DIR) Post #ASRpOo4N1XPzlTX4Hg by beholdnec@mas.to
2023-02-07T20:51:18Z
0 likes, 0 repeats
@simon And you know Google is only showing the best screenshots they have.
(DIR) Post #ASRpc7xsMh1BEWqXq4 by nevali@troet.cafe
2023-02-07T20:54:24Z
0 likes, 0 repeats
@simon considering Google search pages' summary/info panels are NOTORIOUSLY frequently inaccurate, framing their content as confident answers to questions doesn't seem especially prudent but what do i know
(DIR) Post #ASRrAswuHhkAkjWyhM by arfon@fosstodon.org
2023-02-07T21:12:25Z
0 likes, 0 repeats
@simon the green peas is also wrong (different telescope - SDSS) : https://arxiv.org/abs/0907.4155
(DIR) Post #ASRrRIudWMnzuUIXUe by deebloo@indieweb.social
2023-02-07T21:14:27Z
0 likes, 0 repeats
@simon because LLMs have no concept of truth.
(DIR) Post #ASRrgQmEYzQYe3a4ci by simon@fedi.simonwillison.net
2023-02-07T21:15:54Z
0 likes, 0 repeats
@arfon that one looks justifiable to me - JWST did spot three green peas (I'd argue that "a number of" sounds like it means more than just three though) - the summary here didn't imply that it was the very first time green peas had been discovered, unlike the "very first pictures of a planet" claimhttps://www.nature.com/articles/d41586-023-00064-7
(DIR) Post #ASRrtWiPnOqgpM2UPA by kevinrns@mstdn.social
2023-02-07T21:19:12Z
0 likes, 0 repeats
@simon https://blogs.nasa.gov/webb/2022/09/01/nasas-webb-takes-its-first-ever-direct-image-of-distant-world/
(DIR) Post #ASRs6HM6DNzqxCgKwK by simon@fedi.simonwillison.net
2023-02-07T21:21:53Z
0 likes, 0 repeats
@kevinrns the headline there is "NASA’s Webb Takes Its First-Ever Direct Image of Distant World"The Bard summary said "the very first pictures of a planet outside of our own solar system"That's a very important distinction!
(DIR) Post #ASRsKQPdL0vD6852hs by kiwired@mastodon.nz
2023-02-07T21:24:25Z
0 likes, 0 repeats
@simon You're not buying chat ai tech with any expectation of accuracy or correct facts.If you're not aware of that going in, your investigators might have used chat ai to create the reports you based your decision to buy on.
(DIR) Post #ASRtKG4xrCU50Htfma by kevinrns@mstdn.social
2023-02-07T21:35:30Z
0 likes, 0 repeats
@simon I am sure I'm missing something, or a joke is being played on me. This is the first sentence of the linked article, if the headline isnt clear enough."For the first time, astronomers have used NASA’s James Webb Space Telescope to take a direct image of a planet outside our solar system."Was there a photo of a planet earlier? Kepler doesnt take photos of exo planets, Kepler infers stellar brightness differences to posit planets.
(DIR) Post #ASRtwHViyzNURpXpE8 by simon@fedi.simonwillison.net
2023-02-07T21:40:22Z
0 likes, 0 repeats
@kevinrns Huh... OK, yeah maybe there's more subtlety to this than I first thoughthttps://exoplanets.nasa.gov/resources/300/2m1207b-first-image-of-an-exoplanet talks about the first "image" of a planet by the VLT in 2004, but is that necessarily the same thing as "the very first pictures"? What's the difference between a photo, an image and a picture?
(DIR) Post #ASRuNwvv5CM2jhzSeu by simon@fedi.simonwillison.net
2023-02-07T21:45:10Z
0 likes, 0 repeats
Important: I may myself be guilty of spreading misinformation herehttps://blogs.nasa.gov/webb/2022/09/01/nasas-webb-takes-its-first-ever-direct-image-of-distant-world/ states:"For the first time, astronomers have used NASA’s James Webb Space Telescope to take a direct image of a planet outside our solar system."So are these "the very first pictures of a planet outside of our own solar system"?https://exoplanets.nasa.gov/resources/300/2m1207b-first-image-of-an-exoplanet/ credits "the first exoplanet directly imaged" to VLT in 2004"Directly imaged" v.s. "very first pictures of"?
(DIR) Post #ASRuO1NMZ6gQVyrQzQ by simon@fedi.simonwillison.net
2023-02-07T21:46:23Z
0 likes, 0 repeats
So the statement "JWST tok the very first pictures of a planet outside of our own solar system" may be defensible after all - it's not a slam dunk case of a LLM authoring a clearly incorrect summary
(DIR) Post #ASRuxnxBwhEvJOgMls by simon@fedi.simonwillison.net
2023-02-07T21:52:30Z
0 likes, 0 repeats
... or maybe it is a slam dunk mistake? This NASA press release includes the following: https://blogs.nasa.gov/webb/2022/09/01/nasas-webb-takes-its-first-ever-direct-image-of-distant-world/> While this is not the first direct image of an exoplanet taken from space – the Hubble Space Telescope has captured direct exoplanet images previously – HIP 65426 b points the way forward for Webb’s exoplanet exploration.
(DIR) Post #ASRvB6bltrJ2vs0PQG by gabrielesvelto@fosstodon.org
2023-02-07T21:56:54Z
0 likes, 0 repeats
@simon it seems that these bots have passes that alter the output of their NNs to hide the fact that they cannot generate things that are not in their training sets. The alterations often appear to be rephrasing things and using synonyms, this could be one of those cases where the change is big enough to alter the meaning of the phrase.
(DIR) Post #ASRvQO3rIBSpPDQiB6 by ian@social.modest.com
2023-02-07T21:57:44Z
0 likes, 0 repeats
@simon I read that as this is the first time JWST has taken an image of an exoplanet. Other sensors have taken images before, but this is the first one from JWST.
(DIR) Post #ASRvbfbMPyb0zXTpwW by simon@fedi.simonwillison.net
2023-02-07T21:58:26Z
0 likes, 0 repeats
@gabrielesvelto they absolutely can answer questions based on things that aren't in their training sets though: the trick they're using here is to run a search, then paste the search results into the start of a prompt and answer the question based on that additional informationHere's how one of the other "AI search engines" does that: https://simonwillison.net/2023/Jan/22/perplexityai/
(DIR) Post #ASRw1Lg17YG8izu2E4 by simon@fedi.simonwillison.net
2023-02-07T21:59:53Z
0 likes, 0 repeats
@ian that was my interpretation of that initial headline, "JWST captures its first direct image of a planet outside our solar system" - which I then blamed the AI for incorrectly summarizing as "the very first pictures of a planet outside of our own solar system"
(DIR) Post #ASRwEk0xwcBhyK5U6y by misc@mastodon.social
2023-02-07T22:00:41Z
0 likes, 0 repeats
@simon Kudos to you for setting a great example of how to handle this kind of situation. I appreciated getting notified of the edit to the original post, I appreciate you writing a new post that can also get shared, and I appreciate your transparency in posting through it. :)
(DIR) Post #ASRwQec1r6LEgz6VLE by russss@chaos.social
2023-02-07T22:01:30Z
0 likes, 0 repeats
@simon No, I think you were right. But the wording there is so ambiguous that it's hard for a human to parse. I'm pretty sure that 2004 VLT image is the first picture of an exoplanet.(We knew of the existence of exoplanets earlier due to them transiting in front of stars.)
(DIR) Post #ASRwQg8gBC3LQXO5s8 by russss@chaos.social
2023-02-07T22:04:21Z
0 likes, 0 repeats
@simon The painfully specific wording here ("directly imaged", "for the first time, astronomers have used the JWST to") is a really interesting test of LLMs' (and humans'!) ability to parse this
(DIR) Post #ASRweTzO9JF1nbclEW by gabrielesvelto@fosstodon.org
2023-02-07T22:04:00Z
0 likes, 0 repeats
@simon ah, I see, thanks for that
(DIR) Post #ASRwprb0ihEaCReGFk by SpaceLifeForm@infosec.exchange
2023-02-07T22:06:34Z
0 likes, 0 repeats
@simon @kevinrns Yes. As JWST only sees in infrared, there is no way it can be a picture.AI/ML sucks.
(DIR) Post #ASRx6I0kHEhIl3WBhQ by naught101@mastodon.social
2023-02-07T22:14:44Z
0 likes, 0 repeats
@simon given all that confusion, this is perhaps less of an untruth and more of an uncertainty.. A human writer would definitely be excused for this description, arguable as it is.. Probably not the best example of misinformation.Edit: also, thank you for editing your clarification in the the original post!
(DIR) Post #ASS1N6KxKk9VI7F9ai by tk@social.apreslanu.it
2023-02-07T23:06:50Z
0 likes, 0 repeats
@simon in your first link :> While this is not the first direct image of an exoplanet taken from space – the Hubble Space Telescope has captured direct exoplanet images previously – HIP 65426 b points the way forward for Webb’s exoplanet exploration.this is not very ambivalent. the important part of speech here is "**its** First-Ever Direct Image of Distant World" in the title
(DIR) Post #ASSPoOI95IA5hATTSi by tasket@infosec.exchange
2023-02-08T03:40:54Z
0 likes, 0 repeats
@simon As opposed to "JWST just took its first picture of a planet outside our solar system".It lands its ideas in the same ballpark as true statements, but that park includes falsehoods as well.
(DIR) Post #ASSXOAX6PqHl7xTgTg by dogfox@mastodon.social
2023-02-08T05:05:41Z
0 likes, 0 repeats
@simon @ian If it was just the firs time JWST imaged an exoplanet, which had been done before, what's the *discovery*? Images of that particular exoplanet? Because that is not at all what GPT said.