Case studies

What was happening

LMR runs a loyalty program for merchants — restaurants, salons, retail shops — sold through independent sales agents and POS software partners rather than directly. The consumer-facing redemption portal already existed and worked: merchants earned points, customers spent them.

What didn't exist was a way for the channel to run the program itself. Enrollment, rewards configuration, billing, and the decision about whether a given month's points should be released all ran through LMR's internal team by hand. That's workable at a few hundred merchants. The first integration partner has around 25,000 locations.

What made it hard

The rule at the center is a payment gate. Points are calculated as soon as monthly volume arrives, then held until the owning organization settles its invoice. Get that wrong in one direction and you release points nobody paid for. Get it wrong in the other and a merchant who genuinely earned them doesn't get them. Either failure costs trust with the sales channel, and recruiting that channel is the entire growth strategy.

The second thing was tenancy. Four roles across two levels of organization, users who belong to several organizations with a different role in each, and a firm requirement that nobody ever sees another organization's data or is offered a button they aren't allowed to press.

What I did

To be precise about scope: I built the agent and ISO portal — enrollment, the points engine, billing and reconciliation, and the partner API. The consumer redemption portal is a partner's product, and the marketing site is someone else's work.

I wrote the requirements, the technical specification, the API contract, the database schema, and the security model. Then built it: a Node and Postgres API, a React dashboard, deployed on AWS.

Tenant isolation is enforced at the database with row-level security rather than as a convention in application code, so an ordinary bug can't leak data between organizations. The dashboard's permission model runs in three layers, so the interface never offers an action the API would refuse.

The infrastructure is deliberately ordinary. The platform is built to serve a channel of that size, but the machinery underneath it isn't more elaborate than it needs to be. It keeps running cost tracking revenue rather than running ahead of it.

What changed

The platform is the system of record for the program. The channel administers its own merchants instead of routing every change through LMR staff, and the credentialed partner API the growth strategy depends on is live. It's been in continuous delivery since launch.

How it was run

That's an unusual amount of structure for a project this size, and it's most of why it landed on scope and on budget. It also put the risk of a wrong estimate on me, which is the correct place for it.

What was happening

Clinical education has always leaned on standardized patients — actors trained to present a condition the same way every time, so students can practice on a person instead of a mannequin. They work well and they don't scale. A cohort of two hundred students needs the same performance two hundred times, and the scheduling and cost are what they are.

HoloPatient replaced the actor with volumetric capture: real people, scanned, presenting real clinical signs. HoloScenarios extended that into multi-branch simulations where the condition responds to what the student does — an intervention changes the presentation, and the wrong intervention changes it the wrong way.

The most recent change is conversation. Students speak to the patient in natural language, and the patient answers.

What made it hard

Everything in this category comes back to one requirement: the patient has to present the way a real person with that condition presents. The visible signs, the sounds, the timing, the way symptoms shift after an intervention. A simulated patient that's subtly wrong is worse than no simulation, because the student learns the wrong pattern and carries it into a room with an actual person in it.

That requirement has an unusual exception. Real patients mislead. They minimize, they describe pain in the wrong place, they forget medications, they say they're fine. So sometimes a virtual patient should present misleadingly — but only when a subject matter expert has decided that's what the scenario teaches. Deliberate ambiguity is the lesson. Accidental ambiguity is a defect. Telling them apart is a clinical judgment, not a product one, and the system has to make room for someone else to make that call.

Conversation introduced a harder version of the same problem. A virtual patient has to be deterministic in content and non-deterministic in everything else. The clinical facts have to be identical every time — same history, same symptoms, the same things this patient will and won't volunteer unprompted — because that's what makes assessment fair and the scenario teachable. But the delivery has to vary, because a patient who produces the same sentences in the same order for every student stops being a person and becomes a form.

Easy to state. Genuinely hard to build. And it's a boundary problem rather than a capability problem — the model is more than willing to be fluent. The work is constraining what it's allowed to be fluent about.

What we did

Defined the guardrails and the evaluation workflow: what a virtual patient may never do, what it must always do, and how anyone would know. Clinician review, scenario validation, iterative testing with faculty. Clinical partners included the University of La Verne on mental and behavioral health scenarios, and hospital systems including Kaiser Permanente, Cambridge University Hospitals, and National University Hospital of Singapore.

The evaluation side is what led to Persim. Prompts that hold up in one model can drift in another, and drift shows up across many conversations rather than in any single one — you cannot see it by reading a transcript, because each transcript looks fine. Persim runs the same scenarios repeatedly across multiple models against defined expected and forbidden response patterns, which turns “does this prompt hold up” into something measured instead of something you form an impression about.

What's still hard

Fidelity isn't a solved problem and won't become one. Every new capability — better speech, faster response, longer memory — is also a new way for the patient to be subtly wrong. The evaluation has to keep pace with the capability, which is why it had to become a system rather than a review step.

What was happening

This is a problem most software companies are working on right now, and it's still an open question at ours. A business that has always sold per-seat licences or annual bundles adds an AI feature, and the economics quietly stop working. Every use carries a variable cost the licence never accounted for, and the more people like the feature, the worse the margin gets.

The obvious answers are all bad. Absorb the cost and erode the margin. Cap usage and make the feature feel broken. Bill raw tokens and hand a customer a unit they don't understand, can't budget for, and can't defend internally.

What made it hard

The buyer is an institution with a procurement process and an annual budget. They need a number they can approve in advance, expressed in a unit that means something to them, and they need to be able to audit it afterwards. Tokens satisfy none of that.

The underlying cost also isn't linear in any way that's visible from outside. A conversation gets more expensive per minute the longer it runs, because context accumulates as it goes. Two sessions of identical length can cost meaningfully different amounts depending on how the people in them talk.

What we did

We abstracted the underlying units into time. Customers buy credits and spend them per minute of conversation — a unit a nursing program already thinks in, because they schedule in minutes and plan cohorts in minutes.

Getting there meant modelling the real relationship between a minute of human conversation and what it actually costs: how quickly people speak, how long these sessions really run, and how context growth bends the cost curve as a conversation continues. The modelling wasn't about precision for its own sake. It was about being confident enough in the averages to sell a simple unit without the simplification quietly costing us money.

The part worth pointing at is architectural rather than commercial. Metering and pricing are separate concerns, and we built them separately. The system records what actually happened at a fine grain. The pricing model sits on top of that and can change without touching it.

That matters because nobody knows what the right pricing model for this is yet. The market is moving, model costs are moving, and any company that hard-codes today's answer into its billing system will be rebuilding that system within a year. The goal was never to find the correct price. It was to stay able to change it.

What changed

Institutions get a number they can put in a budget and defend to procurement, and it can be audited if they ask. We got a cost structure that holds up as usage grows.

What's still hard

Customers still don't know how many minutes they need, and the total climbs faster than people expect. Easy to understand is not the same thing as easy to forecast. We solved the first one.

Why it exists

I needed to know whether a prompt was any good, and there was no reliable way to find out.

That sounds like a small problem. It isn't. A conversational AI persona is supposed to behave differently every time — that's the entire point of it. So you can't test it the way you test software, where the same input gives the same output and a diff tells you what broke. Read one transcript and it looks fine. Read ten and they all look fine.

The failure that matters is drift. A persona that stays inside its boundaries in ninety-five conversations and steps outside them in the ninety-sixth. Or one that behaves correctly in one model and quietly stops behaving when the model underneath it changes. Neither is visible by reading. You have to measure it. Meanwhile, instructors want some level of certainty about what their students will experience — but it's non-deterministic, so how do you give them that comfort?

What it does

Persim is a prompt authoring and evaluation platform, in three parts.

Author. Build a persona with narrative controls and personality traits, grounded in a domain context — a retrieval knowledge base, so its facts come from somewhere real rather than from whatever the model happens to believe that day.

Test. Run thousands of simulated conversations in parallel against defined expectations. Scoring is deliberately dual: automated model-as-judge for volume, human review for the calls that carry weight. A model can tell you a thousand conversations stayed in bounds. It shouldn't be the only thing telling you about the one that didn't.

Maintain. Watch behavior over time and across models, because a prompt that holds today can fail after an update nobody announced. Results export as PDF or JSON, which matters when someone downstream needs a record of how a system was validated.

The part I'd point at

I built it for my own work, and then GigXR used it on real patient scenarios.

That's the part I like — not that it's clever, but that something built for one person's workflow turned out to be what a team actually needed. It let clinical and content people author conversations they'd been struggling with, work that had previously required someone with prompt engineering in their job description.

What it isn't

It's in private beta, it meets its current needs, but it would need more of my time to make it a business. It could be one.

It does exactly what I built it to do and it does that well. It would not survive real scale. Taking it to market would mean a different stack and rebuilding rather than extending, and I know roughly where that line sits.

I own it, it earns its keep, and it demonstrates something I think is true: AI output can be verified rather than trusted, and building the thing that does the verifying is all about the details.