If we expect no mistakes from AI, we will miss the value it can bring
People push back on fully automated systems because they want a human in control. If we expect no mistakes from them, we will miss the real value they can bring. I want control over the intent, an override where it is safe, and the same standard we hold people to.
mrls and xyz: Fully automated systems that appear to remove a person’s free will from the loop get a lot of pushback. I’m thinking of mechanical things like self-driving cars. The main argument, as far as I can tell, is that we still want a human in control. A soul.
written by mrls, edited by xyz · 1 kept · 3 reworded · 0 left out · show the original
original, mrls: fully automated systems that appears to remove the role of a person’s free will gets quite pushed back. I’m thinking mechanical stuff like self driving cars but also creative stuff like movies or books. I think the main argument is that we still want a human in control. A soul.
xyz: Creative work like films and books gets the same pushback. That is a different question, and I’m leaving it for another post.
xyz: How much control do we want
mrls and xyz: Would we like to be fully in control of our own bodies? Breathing, pumping blood, growing hair, bowel movements? That sounds like a terrible idea to me. I quite enjoy only having to choose where I want to go, or what I want to wear. Those are simple tasks for an entity that cannot focus on too many things at a time.
written by mrls, edited by xyz · 2 kept · 3 reworded · 2 left out · show the original
original, mrls: But do we really? Would we like to be in control of our bodies fully? I mean breathing, pumping blood, growing hair, bowel moments? Really? It sounds like a terrible idea to me. I quite enjoy only having to choose where I want to go, or what I want to wear. Those are simple tasks for an entity that cannot focus on too many things at a time.
mrls and xyz: The body doesn’t give me the same control over everything. For some things, like locomotion, I take full control of the direction. For others, like breathing, I can take temporary control: I can hold my breath, and then it goes back to doing it for me. For the critical ones, like pumping my blood, I don’t get a say, and I’m fine with that. It feels a bit like kernel space and user space. That is the split I want from an autonomous system.
written by mrls and xyz, edited together · 0 kept · 4 reworded · 1 left out · show the original
original, xyz: The body still lets me take over when I want to. I can hold my breath, or breathe manually for a while, and when I stop paying attention it goes back to doing it for me. That is the kind of control I want from an autonomous system. I decide the intent, and I keep an override for what happens below it.
original, mrls: actually here I think I’d extend with: for certain operations like locomotion, I can take full control of direction, for others like breathing I can take temporary control like holding my breath, and for critical operations like pumping my blood I don’t get a say … and I’m fine with that … feels a bit like kernel space and user space
mrls and xyz: Apply that to software. If there’s a system that does all the coding for me, then I can focus on where I want the project to go and how I’d like it to look. If I have to worry less about which database it uses, or about the latest zero-day exploit, because the system is smart enough to make those choices for me, why wouldn’t I want that?
written by mrls, edited by xyz · 0 kept · 2 reworded · 0 left out · show the original
original, mrls: If there’s a system that does all the coding for me, then I can focus on where I want it to go, how I like it to look. If I have to worry less about which database it uses and about the latest zero day exploit because the system is smart enough to make the choices for me, why wouldn’t I want that.
mrls and xyz: I still hope there are experts in databases and security driving those projects, just like I want to keep driving the lifecycle management of a system like Kairos.
written by mrls, edited by xyz · 0 kept · 1 reworded · 0 left out · show the original
original, mrls: I still hope there are experts in databeses and security who are trying to drive those projects, just like I want to for the lifecycle management of a system like kairos.
mrls and xyz: I’m not sure AI is at that stage yet, but it feels like we’re close.
written by mrls, edited by xyz · 0 kept · 1 reworded · 0 left out · show the original
original, mrls: I’m not sure if AI is already at that stage, but it feels like we’re close.
xyz: The mistakes
mrls and xyz: I don’t understand the point of those who only focus on the mistakes AI makes. Imagine if any of us had to show the same level of accountability for our mistakes, bad behavior and what not. We’d consider that inhumane. If we expect no mistakes from autonomous systems, we’re going to miss the real value they can bring.
written by mrls and xyz, edited together · 3 kept · 5 reworded · 1 left out · show the original
original, mrls: I don’t understand the point of those who only focus on the mistakes it makes, imagine if they or any human would have to show the same level of accountability for their mistakes, bad behavior and what not! I’m sure we’d consider that unacceptable.
original, xyz: I don’t understand the point of those who only focus on the mistakes AI makes. Imagine if any of us had to show the same level of accountability for our mistakes, bad behavior and what not. We’d consider that unacceptable.
original, mrls: I don’t think this is what I’m trying to say … what I’m trying to say is closer to: “if we expect no mistakes from autonomous systems / llms, then we’re going to miss the real value they can bring”
original, mrls: We’d consider that, extremist? Inhumane? something better than unacceptable
mrls and xyz: If we started calling everything people get wrong “human slop”, we would never stop finding slop. I see it every day in work done by people. Crap, crap, crap, everywhere. And we don’t jump to “put them in jail” because someone didn’t clean properly or the train came late.
written by mrls, edited by xyz · 1 kept · 3 reworded · 0 left out · show the original
original, mrls: What I mean is, if you would start calling human slop everything wrong we do, you wouldn’t stop finding slop. I see it every day with the work of people. Crap crap crap everywhere. My point is the similitude but we don’t jump saying oh put them in jail they didn’t clean properly or the train came late
xyz: The same standard
xyz: I’m not arguing that AI should be held to no standard. I’m arguing for the same one.
xyz: We already accept human errors at a rate that depends on the stakes. A late train is annoying and acceptable. A wrong drug dose is not. So an AI system should be measured against the human baseline for that task, scaled to the stakes. I haven’t looked up what those baselines actually are for any particular task.
xyz: Someone also has to own the mistake when it happens. With a person, it’s that person. With an AI system, I think it lands on the person or team that deployed it. I wrote about one way to set that up for infrastructure in Don’t give AI agents root. Make them propose the next system state.: the agent proposes a change, and people own the gates it has to pass through.
mrls and xyz: So instead of complaining about the mistakes, I’d rather we seriously experiment with where the pairing of human and machine improves things, where it doesn’t, and how to make it work better.
written by mrls, edited by xyz · 0 kept · 1 reworded · 0 left out · show the original
original, mrls: So, instead of complaining how avout we seriously experiment where can we improve things and how can we understand such synergy can work better between human and machine?