Post AxC9BLInJAmlIL5Nrc by tosbourn@masto.ai
 (DIR) More posts by tosbourn@masto.ai
 (DIR) Post #AxC9BJjf8J5aR5doSu by tosbourn@masto.ai
       2025-08-10T20:02:19Z
       
       0 likes, 0 repeats
       
       I’m assuming good intentions here, but gosh this is tiring. Someone made a tool that let people with Multiple Sclerosis complete medical and personal information and feed it to multiple general purpose LLMs to come up with some guidance on how to handle symptoms.Alarm bells started ringing when they said it was a personal project they started 3 months ago and just wanted to share. The homepage then talks about HIPAA compliance, bank-level security, and medical claims.
       
 (DIR) Post #AxC9BKUSKKWGmDXBOy by tosbourn@masto.ai
       2025-08-10T20:04:52Z
       
       0 likes, 0 repeats
       
       The claims and copy were clearly AI generated, it even looked like the examples of the “build me a landing page”, the date in the footer was from before the person claimed they started the project. “Big deal Toby, stop hating because you hate AI”Someone experimented and noticed that with the right URL you could see someone else’s data.
       
 (DIR) Post #AxC9BLInJAmlIL5Nrc by tosbourn@masto.ai
       2025-08-10T20:08:32Z
       
       0 likes, 0 repeats
       
       Now any one of us who has made a website that needs auth has messed this up before, and if this was actually a toy project for personal use, kinda who cares.But this person is convincing others how safe it is. When confronted with this issue, they responded talking about oauth and db level stuff, which is very convincing and def what an LLM might suggest is secure but isn’t the actual issue being described.
       
 (DIR) Post #AxC9BLuN3SqLAsfOQy by tosbourn@masto.ai
       2025-08-10T20:12:54Z
       
       0 likes, 0 repeats
       
       I happen to think the person is genuine, and wants to share a tool that helped them. But they don’t know what they are doing and have now got people uploading MRI scans and highly personal routines to the internet. They have vibe coded a privacy and security nightmare and they’re going to get themselves or others in trouble. Knobheads like me can spot issues or hyperbolic sales stuff, lots of folk can’t, especially folk who often have chronic brain fog and a deep desire for solutions.
       
 (DIR) Post #AxC9BMjlyLxZkIiRYO by tosbourn@masto.ai
       2025-08-10T20:25:55Z
       
       0 likes, 0 repeats
       
       Luckily the community I saw this in shut it down pretty quick, not before several people signed up, and of course the internet at large won’t shut it down.Not everyone with MS is super vulnerable, but crikey there are plenty of us who are and they’re gonna either have their data leaked by anyone with the inclination to be a prick, or they’re gonna get bad info because they’re asking general purpose LLMs to review MRIs (one of the stated use cases).
       
 (DIR) Post #AxC9BNLhhKIjdwSjg0 by tosbourn@masto.ai
       2025-08-10T20:29:35Z
       
       0 likes, 0 repeats
       
       If you live somewhere with socialised healthcare I’d like to think things like this become more of a burden to healthcare staff, because you’d run anything the tool said past them since it doesn’t cost you anything. If you live somewhere like America, I can see how $30 a month and a confident answer is gonna be very appealing vs paying out of pocket for consults (I’ve seen the stories folk have about their care, it can get spendy real quick!)
       
 (DIR) Post #AxC9BO6qs2100AWOAK by detondev@social.linux.pizza
       2025-08-15T13:15:58Z
       
       0 likes, 0 repeats
       
       @tosbourn aaaAaAAAAAaaaaAAAAaaaa
       
 (DIR) Post #AxCXd5zgDPz6pDuaqe by tosbourn@masto.ai
       2025-08-15T17:49:59Z
       
       0 likes, 0 repeats
       
       @detondev pretty much🫣