Post AwMqQPBCgENfXjFYdE by hosford42@techhub.social
 (DIR) More posts by hosford42@techhub.social
 (DIR) Post #AwMqQMzmni3kljtPYe by hosford42@techhub.social
       0 likes, 0 repeats
       
       You've heard of the alignment problem in AI? This is where it will really start to be an issue:LLMs Can Now Self-Evolve At Test Time Using Reinforcement Learning | by Dr. Ashish Bamania | Jul, 2025 | AI Advances https://ai.gopubby.com/llms-can-now-self-evolve-at-test-time-using-reinforcement-learning-e769ee6d3f86#LLM#alignment#AI
       
 (DIR) Post #AwMqQNcmSjFeig8YL2 by hosford42@techhub.social
       0 likes, 0 repeats
       
       Reinforcement learning is what inspired the concept in the first place. It's been known for decades now that if your RL "reward function" (or, equivalently, "cost function", "loss function", or "fitness function") isn't carefully designed, the system will often discover loopholes -- unintended shortcuts that lead to high reward via behaviors that violate the original design constraints of the system. This happens because the real world is complicated, and it's *really* hard to design robust reward functions for anything even remotely approximating the complexity of the real world.
       
 (DIR) Post #AwMqQO4mmbeY7XEuzQ by hosford42@techhub.social
       0 likes, 0 repeats
       
       You may have heard of the boat-racing video game where the system learned to go in tight circles to maximize reward instead of completing the race? For a more familiar and realistic example, look at drug-seeking behavior in drug addiction and how it short-circuits the ability of a person to care for themself.
       
 (DIR) Post #AwMqQOQlStEZDhWTFQ by hosford42@techhub.social
       0 likes, 0 repeats
       
       I refuse to call them "general intelligence", but LLMs are of sufficient complexity that they are used to interface with the real world. This opens them up to all sorts of short-circuits in their reward functions. To have them learn dynamically within the environment is a huge risk, and there needs to be a rich framework of safeguards around their behaviors to even consider letting them loose to operate "in the wild".
       
 (DIR) Post #AwMqQOoW2aEUPMdRGi by hosford42@techhub.social
       0 likes, 0 repeats
       
       I don't think we'll ever see the so-called "singularity", to be clear. We already have extremely intelligent human beings, and it has been observed that at a certain point, the correlation between IQ and wealth/power breaks down. But that doesn't mean that these LLM-based agents won't be capable of a remarkable level of destructiveness if we aren't careful with them. We have safeguards to keep human beings in check. We need them for artificial agents, as well.
       
 (DIR) Post #AwMqQPBCgENfXjFYdE by hosford42@techhub.social
       0 likes, 0 repeats
       
       In fact, we should probably have a richer set of safeguards around artificial agents. At least human beings mostly have morals, consciences, ethics, and fear of consequences. These artificial agents will have no such bounds unless we build them in.
       
 (DIR) Post #AwMqQPWpNpg6cnMpKy by hosford42@techhub.social
       0 likes, 0 repeats
       
       Something you may be surprised to learn, is that intelligence and self-awareness alone are not enough to give a being feelings or emotions. Hollywood got it wrong. There is no spontaneous waking up and developing a will. In an artificial system, the will *is* the reward function; it is whatever we shape it to be. You program in a reward for killing people, and the system will like killing people, period. You program in a reward for making people smile, and the system will like making people smile, period. It will find the shortest path to that, even if it means pulling a Joker-style trick and surgically altering your face.
       
 (DIR) Post #AwMqQPvdtZWlrkye12 by hosford42@techhub.social
       0 likes, 0 repeats
       
       It's for this reason that the utmost care must be given to shaping, testing, and analyzing reward functions before they are used for real-world conditions. If your agent has the power to act on the world, and a single-minded reward function, you have created an artificial obsessed sociopath, plain and simple.
       
 (DIR) Post #AwMqQQK6Qd5r5cQB8q by VulcanTourist@autistics.life
       0 likes, 0 repeats
       
       @hosford42 > At least human beings mostly have morals, consciences, ethics, and fear of consequences.These all share an origin in the limbic brain, the amygdala.  That is what's missing.  It will be a much larger hurdle to replicate than mere fluid IQ.(I argue that ethics can and should have an exclusively rational basis, but that is prescriptive and not descriptive.  Historically it arose from our limbic brain, for better or worse, and we likely have to replicate that as a start.)