Simulated Human Conversations

Purpose

Create a powerful, testable chat application powered by LLMs that simulates realistic, structured conversations for training, education, or enterprise use cases.
Demonstrate ability to create a functional application utilizing the Cursor IDE and agentic application development.
Gather feedback from SMEs in multiple verticals (e.g., healthcare, therapy, and others).

Problem to Solve

Lack of reliable test harnesses for LLM-generated conversations.
Difficulty maintaining guardrails, branching logic, and narrative expectations in multi-turn simulations.
No reusable prompt + context system with admin tooling for non-engineers.
Need for scalable, secure, role-based environments that support prompt iteration, validation, and real-time evaluation.

Solution

The LLM Chat Application is a full-stack web platform that enables users to:

Build and manage reusable prompts with structured contexts and roles.
Simulate conversations with OpenAI LLMs using different personas and settings.
Create and evaluate test cases using keyword or LLM-based evaluation logic.
Store, version, and export test runs for audits, iteration, or training.
Provide admins and testers with a clean UI to build, test, and debug multi-turn scenarios.

Tech Stack

Frontend

Next.js 15+ (App Router) – React-based SSR/ISR app with nested layouts
Tailwind CSS – Utility-first styling, responsive layout
TypeScript – Type-safe components and logic
ShadCN/UI – Polished UI primitives (cards, tabs, buttons)

Backend & Infra

Supabase – Postgres DB, Auth, Edge Functions, RBAC
OpenAI API – GPT-4o (or 4-turbo), streaming support
Vercel – Hosting, edge rendering, CI/CD
Vitest – Unit tests (100+), coverage tracking
Playwright (soon) – E2E and interaction testing

Features

Role-based access (admin/member/tester)
JSON-based test suite format with branching logic
Import/export support (CSV, JSON)
Prompt and context management per conversation
Realtime LLM evaluation (keyword or LLM narrative match)
Voice I/O and multimedia support (planned)
Grading using instructor provided rubrics (planned)

Application Overview

This application allows admins or content designers to create structured simulation conversations with LLMs—perfect for medical training, virtual patients, peer counseling scenarios, or compliance test cases. Core features include:

Prompt Builder: Create reusable prompts tied to roles and context.
Conversation Runner: Simulate live chat interactions with streaming support.
Test Case Engine: Define expected and forbidden response patterns or narratives.
Evaluation System: Score AI outputs using keyword logic or full LLM-based reasoning.
Export Tools: Download results and test cases for training, compliance, or analysis.
Multi-User Access: Admin, member, and tester roles with secure login and permissions.

Built for scale, the platform supports branching dialogue trees, modular prompt reuse, and rich metadata tagging to simulate a wide variety of real-world use cases.