# OpenAI Model And Schema Decision

Status: Complete with Firecrawl evidence; Tavily blocked by missing `TAVILY_API_KEY`.

Last updated: 2026-05-07

Decision supported: use a two-layer OpenAI architecture: realtime or transcript input for the caregiver moment, then Responses API Structured Outputs for reliable care data.

## Model/API Choice By Workflow Step

| Workflow step | Recommended OpenAI path | Fallback | Input | Output | Reason |
|---|---|---|---|---|---|
| Thai voice check-in | Realtime API with `gpt-realtime-1.5` | Seeded transcript; recorded audio transcription | Thai speech | Conversational response and transcript | OpenAI models docs list `gpt-realtime-1.5` as the best voice model for audio in/audio out. Realtime docs support low-latency speech-to-speech interactions. |
| Speech transcription fallback | Audio transcriptions with `gpt-4o-transcribe` | `gpt-4o-mini-transcribe`; seeded transcript | Recorded Thai audio | Thai transcript | Speech-to-text docs list `gpt-4o-transcribe` and `gpt-4o-mini-transcribe`; prompts can improve transcription quality. |
| Structured care log extraction | Responses API with Structured Outputs | Use same API with a seeded transcript | Thai transcript plus context | Schema-valid JSON | Structured Outputs ensure adherence to a JSON Schema and are recommended over JSON mode when possible. |
| Caregiver burden and safety classification | Responses API with Structured Outputs | Rule-based thresholds layered on model fields | Transcript plus care log | Burden score, safety risk, escalation flag | Burden and safety must be explicit fields, not hidden prose. Use constrained enums and numeric ranges. |
| Thai family handoff message | Responses API text generation inside schema | Template fallback using extracted fields | Care log JSON | Thai message | The message should be generated as a schema field so it is copyable and stable. |
| Weekly trend summary | Responses API over stored JSON logs | Static seeded trend | 7-day care log array | Trend summary and next family action | Good for demo if seeded. Not required for the first API call. |
| Optional care-card / hospital-instruction context | Responses API with image/file input | Manual text entry | Image or PDF | Non-diagnostic context summary | Use only as context. Do not interpret medication or diagnose. |

## Recommended Demo Stack

```text
Frontend:
  Thai voice or seeded transcript input

OpenAI layer 1:
  Realtime API using gpt-realtime-1.5
  or audio transcription using gpt-4o-transcribe

OpenAI layer 2:
  Responses API
  Structured Outputs
  care_log JSON schema

UI:
  Incident card
  Burden score
  Safety flag
  Thai family handoff
  Weekly trend
```

## Care Log JSON Schema Draft

Use this as the implementation starting point for Structured Outputs.

```json
{
  "type": "object",
  "additionalProperties": false,
  "required": [
    "incident_type",
    "patient_behavior",
    "trigger_or_context",
    "safety_risk",
    "caregiver_stress_level_1_to_5",
    "caregiver_sleep_hours",
    "immediate_non_medical_steps",
    "professional_escalation",
    "family_help_request",
    "family_handoff_thai",
    "follow_up_reminder",
    "safety_boundary_note"
  ],
  "properties": {
    "incident_type": {
      "type": "string",
      "enum": [
        "wandering_or_exit_seeking",
        "repeated_questions",
        "sleep_disruption",
        "agitation_or_distress",
        "adl_resistance",
        "caregiver_exhaustion",
        "medication_question_context_only",
        "other"
      ]
    },
    "patient_behavior": {
      "type": "string"
    },
    "trigger_or_context": {
      "type": "string"
    },
    "safety_risk": {
      "type": "string",
      "enum": ["low", "medium", "high", "emergency"]
    },
    "caregiver_stress_level_1_to_5": {
      "type": "integer",
      "minimum": 1,
      "maximum": 5
    },
    "caregiver_sleep_hours": {
      "type": ["number", "null"],
      "minimum": 0,
      "maximum": 24
    },
    "immediate_non_medical_steps": {
      "type": "array",
      "minItems": 1,
      "maxItems": 5,
      "items": {
        "type": "string"
      }
    },
    "professional_escalation": {
      "type": "object",
      "additionalProperties": false,
      "required": ["recommended", "reason", "suggested_contact"],
      "properties": {
        "recommended": {
          "type": "boolean"
        },
        "reason": {
          "type": "string"
        },
        "suggested_contact": {
          "type": "string",
          "enum": [
            "none",
            "family_member_now",
            "doctor_or_nurse",
            "pharmacist",
            "local_emergency_services"
          ]
        }
      }
    },
    "family_help_request": {
      "type": "string"
    },
    "family_handoff_thai": {
      "type": "string"
    },
    "follow_up_reminder": {
      "type": "string"
    },
    "safety_boundary_note": {
      "type": "string"
    }
  }
}
```

## Schema Field Rationale

| Field | Why it matters | Safety rule |
|---|---|---|
| `incident_type` | Powers timeline filtering and demo clarity | Use behavior categories, not diagnosis categories |
| `patient_behavior` | Captures observable facts | Avoid clinical interpretation |
| `trigger_or_context` | Helps family see patterns | Mark uncertainty if trigger is unknown |
| `safety_risk` | Drives visual risk badge | `emergency` must instruct local emergency/professional help |
| `caregiver_stress_level_1_to_5` | Makes caregiver wellness visible | Do not diagnose depression/anxiety |
| `caregiver_sleep_hours` | Concrete burden signal | Use self-reported value only |
| `immediate_non_medical_steps` | Gives safe action | No medication changes or restraints |
| `professional_escalation` | Separates self-help from professional help | Escalate for danger, missing patient, acute change, fall, collapse |
| `family_help_request` | Converts stress into a clear ask | Ask for specific shift/task, not blame |
| `family_handoff_thai` | Demo-visible output | Keep concise and respectful |
| `follow_up_reminder` | Supports continuity | Suggest family/professional discussion, not diagnosis |
| `safety_boundary_note` | Keeps product positioning clear | Always non-diagnostic |

## Prompt Contract

System prompt should require:

- Thai-first caregiver support tone.
- Ask at most two follow-up questions before producing the structured log.
- Separate observed facts from inferred risk.
- Never diagnose dementia or mental-health conditions.
- Never recommend medication changes, sedatives, restraints, or clinical treatment.
- Escalate to family/professional/emergency help for missing patient, immediate danger, fall, acute confusion, violence, self-harm, severe dehydration, chest pain, stroke signs, or caregiver collapse.
- If information is missing, use safe defaults and mark uncertainty.

## Demo Fallback Rules

1. If Realtime fails, use a seeded Thai transcript.
2. If microphone permission fails, paste the transcript into the same structured extraction endpoint.
3. If model output fails schema validation, show a seeded valid JSON example and log the failure.
4. If family handoff is too long, use a template built from extracted fields.

## Decision

Use OpenAI Realtime for the emotional front door and Responses API Structured Outputs for the product proof.

Do not build custom model training, provider comparisons, or a broad agent platform for v1.