BlogJuly 10, 2026
What a Trustworthy AI System Actually Looks Like
I've been talking about the same thing in different ways in the last few articles. Prompt injection looks like a security issue. Knowledge bases that are poisoned without anyone knowing about it look like a data problem. It seems like a problem with the design of the product when the assistants keep saying yes. It looks like a software architecture problem when behaviour is brittle and monolithic. The lessons about trust you learn from riding in a Waymo can feel like they belong to a different conversation. I don't think they are different things at all.
The only thing these examples have in common is that AI systems can make mistakes. A better question is why they make these specific mistakes and what that says about the systems that are built around them.
A lot of the trust issues people have with AI aren't really failures of intelligence. Better models can think more clearly, follow directions more closely, and make fewer mistakes, but this doesn't always mean that people will trust them. When AI is used for real work in messy places with missing information, competing sources, and users who are tired of second-guessing every answer, the most important question is: does the system stay bounded, understandable, and reliable when things aren't so neat?
A good comparison is a car. In AI, we often talk about the LLM as an engine as if it were the whole machine. Model capability is important, just like engine performance is, but no one decides whether to trust a car just by looking at the engine. Steering, braking, stability, instrumentation, and control are all very important too. Putting a V8 engine in a hatchback might be fun, but you wouldn't want to use it as your daily driver to go shopping. In real life, what you trust is the whole car and whether it was built to work well in the conditions it will actually face.
I believe that is where the true distinction in AI will manifest. It won't just be a matter of stronger and weaker models; it will also be a matter of systems that have AI in them and systems that have been built correctly around it.
The same flaw in different forms
One reason that conversations about AI get a little confusing is that we talk about failures as if they were separate groups. A security flaw is talked about as a prompt injection attack. A false retrieval result is talked about as a grounding problem. A too confident answer is seen as a problem with calibration. A system that keeps extending the user's framing long after it should have slowed down is considered a product decision.
Each of those descriptions is true, but they also hide the common structure. Prompt injection shows that a system hasn't done a good enough job of separating instruction from content. Quiet poisoning shows that the system hasn't been strict enough about who has the authority to speak. A system with weak boundaries around scope shows too much compliance. A monolithic, prompt-heavy design shows that the system's behaviour is too complicated to test and improve with a lot of accuracy.
These failures look different on the outside, but they are all very similar on the inside. In each case, the system has not set clear limits on what can affect its behaviour, how that influence is ranked, and where the limits of operation should be. In other words, trust is a part of the system.
The Five Boundaries
In my opinion, the best way to explain this is as the Five Boundaries Framework. There need to be clear lines around control, authority, scope, structure, and visibility for a system to be trustworthy.
- Control asks: what can be used as an instruction?
- Authority asks: Once information is accepted, what should be given the most weight?
- Scope asks, "What can the system actually try to do?"
- Structure asks: How is behaviour organised, tested, and made better?
- Visibility asks: what can the user look at, question, and follow?
None of these limits makes a system perfect, and none of them takes away uncertainty. What they do is make it easier to understand and control behaviour.
Why systems that you can trust feel different
In practice, a reliable system feels different because it draws a clearer line between help and drift. It doesn't let untrusted content act like instruction, and it doesn't give every source the same power. It doesn't slide smoothly past the task it can handle, and it's not put together as one big conversational blob that makes testing hard. Also, it doesn't hide uncertainty behind smooth output.
Trustworthy systems might seem a little more strict, narrow, and willing to slow down or show uncertainty. That is not a flaw; in fact, it's the opposite. A system that seems to help without any effort but quietly goes beyond what it can handle responsibly is not one that can be trusted.
You don't get trust by writing about it, you earn it by acting
This is also why the comparison to Waymo has stuck with me. Most people don't trust a system like that because they know everything about it. They gradually trust it because its behaviour seems limited. It slows down when conditions are awkward and hesitates when things are uncertain. It doesn't gain trust by sounding convincing. It builds trust by acting in a way that is clear and consistent even when things aren't perfect.
AI will have to earn trust in a similar way, not because it is always nice to the user or it can quickly write well. People will trust it when they can see that it has limits, that failure doesn't go away when it becomes fluent, and most importantly, that the human user hasn't been quietly pushed out of the loop.
I don't think the question is whether AI will get better in the end. The more difficult and important question is whether we can trust the systems that are built around that ability. That isn't as exciting as latest big news about AI, but it matters more in the real world of work.
Based on what I’ve learnt over the years the future of AI will depend on whether their behaviour is designed to stay within limits, be clear, and be reliable when it counts.
Why this is important for us
This way of thinking is what Cyclone Sage is based on. Sage is an AI that works with boundaries instead of using fluency to hide weak ones. The goal is not to hide complexity behind fluent output. It is important to take the trust issue seriously and build an AI that can be controlled, reviewed, and used for serious engineering work.
That doesn't mean the problem is solved. It will still need scrutiny, testing and carefully use. Better boundaries don't get rid of doubt but they can help people control, understand, and improve their behaviour. To me, that's the difference between an AI feature and an AI system that is really relied upon.
This article concludes our Building Trustworthy AI Systems series exploring how reliability, control, and system design are important for engineering AI systems.
-
View the full series: Building Trustworthy AI Systems
These ideas inform the development of Cyclone Sage, our AI assistant for MCNP being built with a focus on structured, reliable system design.
If you’ve enjoyed the series, if you're working on similar challenges or exploring how AI can be applied reliably in technical environments, feel free to get in touch at support@orthrussoftware.com
