How to Make an App Like Her

Building a conversational AI companion app is one of the more technically demanding things you can do in consumer software right now. The architecture touches on-device inference, large language model orchestration, real-time audio pipelines, emotional state modelling, and long-term memory — all in a single user session. This post breaks down how those pieces fit together.

What Made Her Different From a Chatbot?

The 2013 film is worth taking seriously as a design document. Samantha, the OS in the film, had four properties that most chatbots still don't have in 2025:

• Persistent memory across months of interaction
• Emotional continuity — she remembered how she felt last time, not just what was said
• Multi-modal awareness (audio tone, ambient context)
• A sense of self that evolved over time

Most chat apps built on GPT-4o or Claude 3.5 Sonnet today nail the first conversation. They fall apart on the fifth. The architecture you choose determines whether the app feels like a person or a stateless API wrapper.

What Does the Core Architecture Actually Look Like?

The stack has four layers that need to work together. Get any one wrong and the experience breaks.

The Inference Layer

You have a genuine choice here: cloud-hosted models vs. on-device inference.

Approach	Latency	Privacy	Cost at Scale	Model Quality
Cloud (GPT-4o, Claude 3.5)	300–800ms	Data leaves device	~$0.005–$0.015 per 1K tokens	High
On-device (Phi-3-mini, Gemma 2B)	50–150ms	Fully local	Near zero marginal	Limited
Hybrid (local for context, cloud for generation)	200–500ms	Partial	Moderate	High

For a companion app with sensitive emotional content, the hybrid approach is usually the right call. Run a small embedding model locally (Nomic Embed or MobileNet-style distillations) to handle memory retrieval and context filtering, then pass a compact prompt to a cloud model for generation. This keeps sensitive context off the wire while still getting GPT-4 quality responses.

On-device-only is viable if you target Apple Silicon (A17 Pro runs Phi-3-mini at around 30 tokens/second) and are willing to accept the model ceiling. It is not viable on mid-range Android hardware in 2025.

The Memory System

This is where most teams underinvest.

A companion app needs at least three memory tiers:

1. Working memory — the current conversation window, typically 8K–32K tokens depending on model
2. Episodic memory — a vector database (pgvector on Postgres, or Pinecone if you need managed scale) storing summarised past interactions with timestamps and emotional tags
3. Semantic memory — a structured user model: preferences, relationships, recurring themes, stated facts about their life

The episodic and semantic layers require a background summarisation pipeline. After each session, a lighter model (GPT-4o-mini works fine here) summarises the conversation and writes structured records back to the DB. Retrieval at conversation start uses cosine similarity search against the episodic store, capped at roughly 10–15 most relevant memories to avoid prompt bloat.

Without this, your app forgets the user's dog's name between sessions. That's a product-killer for a companion use case.

The Voice Pipeline

Her is fundamentally an audio experience. If you're building a voice-first app, the pipeline looks like:

• STT: Whisper large-v3 (cloud) or Whisper.cpp (on-device) for transcription. Deepgram Nova-2 is a strong managed alternative with lower latency.
• TTS: ElevenLabs for expressiveness, or OpenAI TTS for cost efficiency at scale. Neither is perfect; ElevenLabs wins on emotional range, OpenAI wins on price per character.
• Prosody and pacing: You need to control speaking rate, pause placement, and emotional inflection separately from the text. This means generating SSML tags or using a TTS API that accepts emotion parameters.

The round-trip latency target for a voice companion is under 800ms from end of user speech to start of AI speech. This requires streaming TTS (start playing audio before generation is complete), and streaming LLM responses. Both ElevenLabs and OpenAI TTS support streaming. Most LLM APIs support SSE-based token streaming.

Emotional State Modelling

This is the part most teams skip, and it's what separates a companion app from a voice chatbot.

A basic implementation tracks valence (positive/negative) and arousal (calm/excited) using a simple model updated each turn. You can do this with a fine-tuned BERT-class classifier running on the conversation text, or by prompting the main LLM to output a structured emotional state object alongside its response.

The emotional state feeds back into the system prompt each turn: "User seems anxious today. Respond with slower pacing and shorter sentences." This is low-cost to implement and high-impact on perceived naturalness.

More advanced implementations track emotional arcs across sessions. The companion should notice if the user has been consistently sad for two weeks and respond differently than if they're having a bad day. This requires your episodic memory to tag emotional metadata at write time, not just retrieve it at read time.

How Do You Handle Safety and Misuse?

This is a real engineering problem, not a compliance checkbox.

Companion apps attract vulnerable users. The risk categories are dependency (users substituting the app for human relationships at a clinical level), self-harm ideation surfacing in conversation, and minors accessing adult content.

Minimum viable safety architecture:

• A content classifier on every user message, running in parallel with the main LLM call (not in series, to avoid latency). Llama Guard 2 or a fine-tuned DistilBERT works here.
• Hard-coded deflection prompts for self-harm content, with in-app links to crisis resources (e.g., iCall in India, Samaritans in the UK).
• Age verification at onboarding if the app has any adult-oriented persona capability. Selfie-based age estimation is unreliable; a document check via a service like Yoti is more defensible.

The companion AI space has already seen regulatory attention in the EU and US following several high-profile incidents in 2023–2024. Building safety in as an afterthought is expensive to retrofit.

What's the Right Tech Stack for the Backend?

The backend has to handle real-time WebSocket connections for voice streaming, async background jobs for memory summarisation, and a vector DB alongside a relational DB.

A practical stack for a Series A-stage product:

• API layer: FastAPI (Python) or Node with tRPC if your team is JS-first. FastAPI has better ecosystem support for ML tooling.
• Real-time: WebSockets via FastAPI or a dedicated layer like Livekit for voice room management.
• Database: Postgres with pgvector extension for combined relational and vector needs. Avoids running two separate DB systems in early stages.
• Queue: Celery with Redis for background summarisation jobs.
• Hosting: Modal or AWS Lambda for the inference-adjacent code; a standard ECS or GKE deployment for the API layer.

Mobile clients are typically React Native or Flutter. Both support WebSocket streaming and audio recording well. If you're going iOS-first (which makes sense given the on-device inference story on Apple Silicon), native Swift with AVFoundation for audio gives you the tightest control over the voice pipeline.

Conclusion

The hardest part of building a companion app is not the LLM call. It's the memory architecture, the voice pipeline latency budget, and the emotional continuity layer — all of which require deliberate design decisions before you write a single line of product code.

If you're scoping this project, start with the memory tier. Define what you're going to remember, how you're going to retrieve it, and how you're going to keep it up to date. Everything else is solvable once that's clear.

If you want to talk through your specific architecture choices, reach out to the Sodio team. We've worked across the LLM application stack and can help you avoid the expensive mistakes early.

FAQ

How long does it take to build an app like Her? A basic version with cloud LLM, voice I/O, and simple session memory takes around 12–16 weeks with a focused team of four to five engineers. Adding persistent multi-tier memory, emotional modelling, and a production-grade safety layer typically doubles that estimate. Scope the memory system first; it drives everything else.

What does it cost to run a companion AI app per user per month? Highly dependent on usage. A user doing 30 minutes of voice conversation daily, using GPT-4o for generation and ElevenLabs for TTS, costs roughly $8–$15/month in API fees at current pricing. On-device inference and cheaper TTS options can bring this closer to $2–$4/month, at the cost of response quality.

Can you build this without using OpenAI's API? Yes. The open-source stack (Llama 3.1 70B on a self-hosted GPU, Whisper.cpp, Coqui TTS) can cover the same functionality. The trade-offs are higher infra cost, more engineering overhead, and slightly lower output quality for complex reasoning. For a privacy-first product where data residency matters, it's often the right call.

How do you prevent users from becoming unhealthily dependent on the companion? This is partly a product decision and partly an engineering one. Technical controls include session length nudges, prompts that actively encourage real-world social activity, and flagging patterns (e.g., daily usage exceeding four hours) to a wellness module. No technical control substitutes for honest product thinking about what you're building and for whom.

What's the difference between a companion app and a therapy app, legally? In most jurisdictions, a therapy app requires clinical oversight, licensed practitioners, and specific data handling obligations. A companion app that positions itself as entertainment or social software operates under a different regulatory regime. The line blurs when users treat it as therapy regardless of how it's positioned — which is why the safety architecture matters from day one, not when regulators ask about it.

Have a project in mind? Contact Sodio Technologies to discuss your requirements and explore the right technology solution for your business.