From an AI idea to a governed solution
How a request for an AI assistant becomes a scoped, governed and testable solution before any model is chosen.
TakeawayDecide what the assistant must refuse, and who owns each failure, before deciding what it can answer.
Illustrative reference design. It shows how I approach this kind of problem and is not a record of a delivered project.
Problem and constraints
Problem
The request is a capability ("ask our data questions"), not a requirement. Scope drifts, nobody owns defects, and no one can say what "good" looks like.
Constraints
- Never widen a user's entitlements.
- Numbers come from governed data, not model memory.
- Every request, including refusals, is auditable.
- Latency targets differ by type of question.
Options considered
- Not chosen
Open-ended assistant over all data
Fastest demo, but entitlement, accuracy and ownership problems appear after launch.
- Not chosen
Fixed dashboards and reports only
Predictable and governable, but new questions still go unanswered.
- Chosen
Gated assistant over a curated, entitlement-aware data layer
Narrower at first, with every failure named and testable.
Architecture
Treat the assistant as a governed workflow, not a chat window. Each request is classified, checked against entitlements and data availability, answered only from a curated layer, and logged.
Select any component to see what it does and how it connects. The diagram is illustrative: it shows the shape of a solution, not a specific system.
Illustrative reference architecture
Request
Decide
Fetch
Answer
Observe
Select a component
Choose any block to read what it does and how it connects to the rest.
- User request
- A natural-language question from a signed-in user with a known role.
- Decision gates
- Checks intent, ambiguity, entitlement and data availability. A failed gate returns a specific, explainable message.
- Governed data layer
- Curated, read-only data with row- and column-level access rules applied before any result is returned.
- Orchestration and model
- Builds the query, calls the model for wording or reasoning, and assembles the answer with its sources.
- Audit and evaluation
- Every request, refusal and answer is logged; a fixed evaluation set measures quality over time.
Decisions and trade-offs
Pass every request through explicit gates before any model call.
Trade-offGates add design effort and some latency, but turn vague failures into named, testable ones.
Give the assistant read-only, entitlement-aware access to curated data.
Trade-offSome questions become unanswerable by design. Access control stays in the data layer, not in prompts.
Split ownership: the interface owns experience, the platform owns model behaviour.
Trade-offTwo owners need a shared error vocabulary, and defects stop bouncing between teams.
How it would be delivered
Frame
Describe each user group's decision and write plain-language acceptance criteria.
Budget
Set latency targets per type of question, not one global number.
Gate
Specify each decision gate and what the user sees when it stops a request.
Choose
Pick the orchestration approach by control, maintainability and team skills.
Test
Agree the evaluation set and the audit fields before the first build.
Lessons and next steps
Current statusIllustrative reference design. It has not been implemented as described.
Lesson: features that sound small, such as "what if" questions, often hide the biggest scope decisions. Name them, then accept or decline them on purpose.
Next steps
- Write a first evaluation set from real user questions and agree the pass threshold.
- Prototype the decision gates with a few representative requests.
- Define the audit fields and the retention rule with the people who will review them.
Related work
Talk it through
If this kind of problem looks familiar, I would like to hear how it shows up for you.