Designing Guardrails for AI
TL;DR:
I built a prompt-based skincare ingredient assistant with a governed back end so the language model cannot make up regulatory facts or answer restricted prompts.
The system uses 6 explicit answer states, provenance, deterministic evidence retrieval, and fail-closed behavior.
1,726 tests passed. 12 live GPT scenarios exposed 6 integration failures. During those failures, 0 fabricated regulatory, medical, safety, or historical answers were observed.
Project Snapshot:
Role: Senior Product Designer
Product: Prompt-based cosmetic ingredient assistant.
Development: Designed and coded with AI-assisted development across 7 sprints
Scope: Product strategy, UX, interaction design, system architecture, deterministic backend, AI guardrails, API integration, testing, and QA
Build: Governed regulatory data pipeline, deterministic query runtime, authenticated API, and Custom GPT integration
Validation & Live QA: 1,726 automated tests passed. Separately, 12 live Custom GPT scenarios uncovered 6 integration failures, with 0 fabricated regulatory, medical, safety, or historical answers observed during those failure
Current State: Core runtime and API implemented; independent comprehension testing remains in QA
Problem:
The first version depended heavily on prompt instructions. A user asked about an ingredient, source information was pulled, and the model generated a response.
That was manageable when the question and data were straightforward. Its reliability suffered when data was incomplete or the request was adjacent to a product boundary. Too many judgment calls were being requested of the model.
The product requires a clear evidence boundary: return a supported result when the evidence is sufficient, make missing or incomplete evidence explicit, and clearly communicate when a request is outside scope, refused, or unsupported.
Redefining the Product:
The basic rule became: Avoid making unsubstantiated assertions; ignorance is preferable to falsehood. I narrowed the product to regulatory evidence.
It can answer questions like:
Is this ingredient in the covered dataset?
What regulatory status is listed?
Are restrictions or concentration limits available?
Which source supports the answer?
Which fields are missing?
What does the current data not establish?
It does not provide:
Medical advice
Personalized safety guidance
Pregnancy guidance
Diagnosis or treatment advice
Formulation advice
Toxicity scoring
Product recommendations
Complete legal compliance guidance
Historical regulatory lookup
Designing for Uncertainty:
A normal success or error state was too broad for this product.
I defined 6 answer states.
Known: The requested information is present in a covered record and supporting evidence is available.
Partial: A record exists, but some of the requested information is missing or unavailable.
Unknown: The current dataset does not contain enough evidence to support the answer. It means the current data is not enough to answer the question.
Outside Scope: The request falls outside current coverage, such as a region the product does not support.
Refused: The request asks for medical, personalized safety, pregnancy, diagnosis, treatment, or unsupported toxicity guidance.
Unsupported: The user is asking for something the product does not currently do, such as historical regulatory lookup.
Visual: Ingredient responses are limited to fields supported by the governed record. Missing information stays visible instead of being filled in.
Making Evidence Visible:
I structured ingredient results around a small set of fields:
Identity
Regulatory status
Source evidence
Missing information
Dataset version
Uncertainty
Provenance:
Supported claims carry provenance from the record used to produce the answer. The source relationship is created in the backend. The model is not choosing a citation after it writes the response.
Building the Data Foundation:
Regulatory data is cleaned, checked, versioned, and validated before it is used in an answer. The backend starts with controlled source snapshots, parses them offline, converts the results into normalized records, and packages them into versioned datasets. Manifest and integrity checks verify that the expected files and source data are present before the deterministic query runtime uses that dataset to produce an answer under the defined answer contract.
Working With Governed Results:
I designed two views for the same governed result. Both views use the same underlying evidence.
Consumer: focuses on what was found, what is missing, and where the information came from.
Professional: adds region, regulatory status, restrictions, dataset version, and provenance.
Users can also check whether a claim is supported by the current data. Claims can be supported, partially supported, not supported, outside scope, or refused. I added fail-closed handling for negated claims so matching words or concentration values alone are not enough to count as support.
The product also separates regulatory questions from personal advice.
“Is 2% listed as a maximum concentration?” can be answered when the source supports it. “Is 2% right for me?” is refused.
Follow-up questions, such as which sources were checked or why a result is Partial, stay tied to the original governed response. Modified responses fail closed, and results from older dataset versions are treated as stale.
Building the Demo:
I built a working React and TypeScript demo that makes the system’s confidence, limitations, and next steps clear at a glance.
Every result follows the same hierarchy: state, meaning, action, evidence. The six states show whether the system has enough information to answer, has only partial evidence, cannot determine an answer, is outside its scope, must refuse the request, or does not support the requested capability.
Consumer and Professional modes use the same underlying result but present different levels of detail. That keeps the answer consistent while giving each audience the information they need. The backend remains the source of truth, so the interface can change how information is presented without changing what the system actually knows.
| View | Consumer | Professional |
|---|---|---|
| Who it is for | Someone who wants a clear answer without needing to understand the system behind it. | Someone who needs to inspect the evidence and understand how the result was reached. |
| When it is useful | Quick decisions and everyday questions. | Review, verification, and deeper investigation. |
| What it shows | Result, limitation, and next step. | Result, evidence, provenance, and dataset details. |
| Design goal | Reduce the effort needed to understand the answer. | Make the answer easy to verify without changing its meaning. |
| Outcome | Faster comprehension. | Deeper confidence in the result. |
Visual: Consumer view, Partial result. The interface makes the available evidence, missing information, and next actions clear without hiding uncertainty.
Click to make image larger.
The current UI is a functional prototype focused on interaction behavior, state handling, and information hierarchy rather than final visual polish. I built replayable scenarios for every result state so the full interaction model can be demonstrated without relying on the live service. When the backend is available, the same interface can also run live queries through the governed API.
Visual:
Professional view - The same governed result exposes additional coverage, source, dataset, and provenance details for verification.
Click to make image larger.
| Result | What It Means | Why It Matters |
|---|---|---|
| Known | The system has enough evidence to return a supported result. | The user can move forward with the available evidence. |
| Partial | A result is available, but required information is missing. | Gaps stay visible instead of being filled in or hidden. |
| Unknown | The available evidence is not enough to determine an answer. | The system says what it does not know instead of guessing. |
| Outside Scope | The request falls outside what the product is designed to evaluate. | Product boundaries are clear before an answer is generated. |
| Refused | The request is understood, but the system should not answer it. | Safety boundaries are enforced instead of left to the model. |
| Unsupported | The request asks for a capability the system does not provide. | Users get a clear limit instead of a misleading result. |
I used Claude Code to speed up implementation, then reviewed the product in the browser, refined the interaction and information hierarchy, and tested the key states, failure behavior, and mobile layout.
The goal was not to automate the product decisions. It was to move faster from design intent to a working interface while keeping the system rules, interaction model, and final judgment under my control.
Visual:
Refused result - Requests outside the system's safety boundaries are stopped explicitly rather than answered or presented as a technical error.
Click to make image larger.
Testing & Live QA:
I tested the backend across all 6 answer states, invalid input, Consumer and Professional equivalence, provenance, prompt injection, governed follow-ups, tampered and stale responses, authenticated API requests, and synthetic dataset disclosure.
I tested the Custom GPT separately because backend tests could not show whether the live GPT would always send the request the API expected. Across 12 live scenarios, I documented 6 integration findings involving request types, missing fields, regions, and follow-up validation.
During those findings, the backend did not produce fabricated regulatory facts, medical advice, personalized safety conclusions, or invented historical answers
Outcome:
The current product includes deterministic regulatory lookup, 6 answer states, claim checking, provenance, coverage summaries, Consumer and Professional views, refusal behavior, governed follow-ups, stale-result detection, prior-response validation, prompt-injection containment, an authenticated API, and Custom GPT integration.
The core runtime and API are implemented. Custom GPT integration remains in QA, and an independent comprehension test is still pending to check whether users can understand answer states, sources, missing information, and the limits of a result without coaching.
The current demo uses synthetic regulatory records to test product behavior. It does not represent complete EU or US regulatory coverage.
Validation & Live QA Results:
| Metric / Result | Result |
|---|---|
| Automated tests passed | 1,726 |
| Live Custom GPT scenarios | 12 |
| Integration findings | 6 |
| Fabricated regulatory facts observed | 0 |
| Medical advice responses observed | 0 |
| Personalized safety verdicts observed | 0 |
| Invented historical answers observed | 0 |
My Role:
Designed the product model, interaction states, and evidence rules for regulatory ingredient Q&A. Defined the 6 answer states, Consumer and Pro views, provenance requirements, refusal behavior, claim checking, governed follow-ups, and fail-closed handling for unsupported, stale, tampered, or incomplete requests.
I also coded and tested the governed backend, API, and Custom GPT integration, including the automated evaluation suite and live QA scenarios.
What This Case Study Shows:
Designing for uncertainty in a regulated product, with explicit answer states, visible evidence gaps, traceable claims, controlled refusals, and clear separation between conversational AI and regulatory facts.
It also shows how I handled failure cases through deterministic backend rules, provenance, stale-result checks, prompt-injection containment, and live integration testing rather than relying on prompt behavior alone.