Nadir/backend/PLAN.md
ZTatman 0c0ba74c09 Phase 1: restructure into backend/ and fix module import side effects
- Move all Python source files into backend/ directory
- Fix db.py: use absolute path (Path(__file__).parent) for chroma_db
  so the DB is always found regardless of where the server is launched from
- Fix store.py: guard test entry under if __name__ == "__main__" to
  prevent duplicate inserts on every import
- Fix query.py: extract query_entries(question) -> str function and
  move interactive loop under __main__ guard so it can be imported
  cleanly by FastAPI without blocking on input()
- Add .claude/launch.json with FastAPI dev server configuration

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-18 16:30:34 -04:00

15 KiB
Raw Blame History

🎙️ Voice Mood Diary — Complete Build Plan

AI-Powered Voice Journal with Mood Tracking & Pattern Discovery
Python + Streamlit + RAG | Beginner Friendly


How to Use This Plan

This plan takes you from zero to a fully working voice diary app you can use every day. Each phase builds on the last. You never need to know everything upfront — just follow one step at a time.


Project Overview

You are building a voice-first personal diary that:

  • Records your voice and transcribes it to text automatically
  • Uses AI to extract your mood, energy level, and triggers from what you said
  • Stores everything in a local vector database on your computer
  • Lets you ask deep questions about your emotional patterns over time
  • Helps you understand what triggers your mood swings

🔒 Privacy First: Your data never leaves your computer unless you choose. No cloud, no subscriptions, no one reading your private thoughts.

The Final App — Two Modes

Mode 1 — Record Entry: Press a button, speak freely, AI auto-tags your mood and saves it.

Mode 2 — Ask Questions: Type a question like "What triggers my low moods?" and get pattern-based answers drawn from all your past entries.


Your Tech Stack

Tool What It Does Why You Need It
Python Programming language The foundation everything is built on
Streamlit Web UI framework Turns Python scripts into a real app with buttons and chat
OpenAI Whisper Speech-to-text Converts your voice recordings into text with high accuracy
OpenAI API / Claude AI language model Extracts mood/triggers and answers pattern questions
LangChain RAG framework Connects your entries to the AI for question answering
ChromaDB Vector database Stores your entries so they can be searched by meaning
sounddevice / PyAudio Audio recording Captures your microphone input in Python

Project Folder Structure

Create this folder structure on your computer before you start coding:

voice_diary/
├── app.py                  ← Main Streamlit app (you run this)
├── record.py               ← Microphone recording
├── transcribe.py           ← Whisper: audio → text
├── extract_metadata.py     ← AI: pulls mood/triggers from text
├── store.py                ← Saves entries to ChromaDB
├── query.py                ← RAG: answers your pattern questions
├── requirements.txt        ← List of packages to install
├── .env                    ← Your API keys (never share this file)
├── /journal_entries        ← Raw text backups of every entry
└── /chroma_db              ← Your local vector database

⚠️ Important: The .env file holds your secret API keys. Never upload this to GitHub or share it with anyone.


Phase 1 — Environment Setup

Estimated time: 2–3 hours | Difficulty: Beginner

Step 1: Install Python Tools

Open your terminal (Mac: Terminal app, Windows: Command Prompt or PowerShell) and run:

# Install UV (package manager)
curl -LsSf https://astral.sh/uv/install.sh | sh

# Create your project
uv init voice_diary
cd voice_diary

# Add all dependencies
uv add streamlit langchain langchain-community chromadb
uv add openai pydantic openai-whisper sounddevice scipy python-dotenv

Step 2: Get Your API Key

  • Go to platform.openai.com and create an account
  • Add $5 of credit (will last weeks of testing)
  • Create an API key and copy it
  • Create a file called .env in your project folder
  • Add this line to it:
OPENAI_API_KEY=your_key_here

Step 3: Test Your Setup

Create a file called test.py and run it to confirm everything works:

import openai, chromadb, streamlit
print("All imports successful!")
uv run python test.py

✅ Phase 1 Complete when: You can run test.py with no errors and see the success message.


Phase 2 — Text Entry + RAG (No Voice Yet)

Estimated time: 1 day | Difficulty: Beginner–Intermediate

Build the brain of the app first. Skip voice for now — just type entries to test the RAG layer.

What You Build in This Phase

Task What You Do Outcome
store.py Save a text entry + metadata to ChromaDB Entries persist to disk
query.py Ask a question, retrieve relevant entries, get AI answer RAG is working
Basic app.py Streamlit UI with text input and question box Usable in browser

The Key Code Concepts

Saving an entry (store.py):

import chromadb

client = chromadb.PersistentClient(path="./chroma_db")
collection = client.get_or_create_collection("diary")

collection.add(
    documents=[entry_text],
    metadatas=[{"date": "2026-03-10", "mood": "anxious"}],
    ids=[unique_id]
)

Asking a question (query.py):

results = collection.query(
    query_texts=["What triggers my low moods?"],
    n_results=5   # retrieve 5 most relevant entries
)
# Then send results + question to OpenAI to get an answer

Basic Streamlit UI (app.py):

import streamlit as st

st.title("🎙️ Voice Mood Diary")

mode = st.sidebar.radio("Mode", ["Record Entry", "Ask Questions"])

if mode == "Record Entry":
    entry = st.text_area("Type your journal entry:")
    if st.button("Save Entry"):
        # call store.py here
        st.success("Entry saved!")

if mode == "Ask Questions":
    question = st.text_input("Ask about your patterns:")
    if question:
        # call query.py here
        st.write(answer)

✅ Phase 2 Complete when: You can type a journal entry, save it, ask a question, and get a meaningful AI answer back.


Phase 3 — Add Voice Recording + Whisper

Estimated time: 1–2 days | Difficulty: Intermediate

Now you add the voice layer on top of your working RAG system.

What You Build in This Phase

Task What You Do Outcome
record.py Record audio from microphone using sounddevice You can record your voice
transcribe.py Send audio file to Whisper, get back text Voice becomes text automatically
Update app.py Add a Record button that triggers record → transcribe → save Full voice entry flow works

The Key Code Concepts

Recording audio (record.py):

import sounddevice as sd
import scipy.io.wavfile as wav

def record(seconds=60, filename="recording.wav"):
    audio = sd.rec(int(seconds * 44100), samplerate=44100, channels=1)
    sd.wait()   # wait until done
    wav.write(filename, 44100, audio)

Transcribing with Whisper (transcribe.py):

import whisper

model = whisper.load_model("base")   # downloads once, ~150MB

def transcribe(filename: str) -> str:
    result = model.transcribe(filename)
    return result["text"]

✅ Phase 3 Complete when: You can press Record, speak for 30 seconds, and see your words appear as text on screen.


Phase 4 — AI Metadata Extraction

Estimated time: 1 day | Difficulty: Intermediate

After transcription, you send the text to an AI and ask it to extract structured information about your emotional state. You never fill this out manually — the AI does it automatically.

Your Extraction Prompt

Send this to the AI with your transcript:

Read this journal entry and extract the following.
Respond in JSON only, no other text:

- mood: one word describing the dominant mood
- energy: low, medium, or high
- triggers: list of things that caused the mood
- themes: list of life areas mentioned (work, relationships, health, etc.)
- intensity: number from 1-10 (1=very low, 10=very high)
- swing_direction: high, low, or neutral

Entry: {transcript}

What You Get Back

{
  "mood": "anxious",
  "energy": "low",
  "triggers": ["work deadline", "poor sleep"],
  "themes": ["work", "health"],
  "intensity": 7,
  "swing_direction": "low"
}

Validate It with Pydantic (extract_metadata.py)

from pydantic import BaseModel

class EntryMetadata(BaseModel):
    mood: str
    energy: str
    triggers: list[str]
    themes: list[str]
    intensity: int
    swing_direction: str

# Pydantic catches bad AI output immediately
metadata = EntryMetadata(**json_response)

This JSON gets stored as metadata alongside your entry in ChromaDB. Over time it builds a rich, queryable picture of your emotional life.

✅ Phase 4 Complete when: Every saved entry automatically has mood, energy, triggers, and intensity attached to it.


Phase 5 — Full Streamlit UI

Estimated time: 1–2 days | Difficulty: Intermediate

Polish everything into a proper two-mode interface you will actually enjoy using.

Mode 1: Record New Entry

  • Big red Record button
  • Shows recording timer while you speak
  • Displays transcript after recording
  • Shows auto-detected mood, energy, and triggers
  • Save button to store to ChromaDB

Mode 2: Reflect / Ask Questions

  • Chat input box
  • Suggested questions to get you started
  • AI response with references to which entries it used
  • Entry count and date range shown in sidebar

Suggested Starter Questions to Build In

Pre-load these as clickable buttons so you can use the app immediately:

  • "What triggers my low moods most often?"
  • "What do my highest energy days have in common?"
  • "How long do my low periods typically last?"
  • "What activities or events improve my mood?"
  • "What was I worried about 3 months ago?"
  • "When was the last time I felt truly at peace?"

✅ Phase 5 Complete when: The app looks and feels like a real product you want to open every day.


Phase 6 — Weekly Reflection Feature

Estimated time: Half a day | Difficulty: Beginner (by this point)

Add one special feature: every week, the app automatically generates a gentle summary of your emotional week.

How It Works

  • On Sunday evenings (or whenever you open the app), it fetches your last 7 entries
  • Sends them to the AI with a reflection prompt
  • Generates a warm, non-judgmental summary of your week
  • Notes patterns, improvements, and things to watch

Example Weekly Summary

"This week your energy was generally low Monday through Wednesday, with intensity scores averaging 3/10. Thursday showed a notable shift after you mentioned going for a walk — your entries after outdoor time consistently show higher mood scores. Work deadlines appeared as a trigger in 4 of 7 entries. Friday and Saturday were your highest energy days this week."

✅ Phase 6 Complete when: You get an automatic weekly digest that surfaces patterns you wouldn't have noticed yourself.


Full Build Roadmap at a Glance

Phase Name What You Build Done When...
1 Setup Install tools, get API key, test imports test.py runs with no errors
2 RAG Core Text entries + ChromaDB + question answering Can ask questions about typed entries
3 Voice Microphone recording + Whisper transcription Voice becomes text on screen
4 Metadata AI extracts mood, energy, triggers from transcript Every entry has auto-tags
5 UI Polish Full Streamlit two-mode interface App feels real and usable
6 Weekly Auto weekly reflection summaries You get a Sunday digest

Toolchain Summary

Tool Purpose
UV Package manager — faster than pip, simpler than Poetry
Pydantic Data validation — defines shapes for entries and AI responses
Pyright / Pylance Type checker — catches bugs as you type in VS Code
Ruff Linter + formatter — keeps your code clean
VS Code Editor — ties everything together

Tips for Success

When You Get Stuck

  • Google the exact error message — almost every Python error has a Stack Overflow answer
  • Ask Claude to explain what an error means — paste the full error
  • Don't skip phases — each one builds on the last
  • It's normal for Phase 2 to take longer than expected. That's where the real learning happens.

Common Beginner Mistakes to Avoid

  • Trying to build everything at once — do one phase at a time
  • Not testing after each small change — test often
  • Ignoring error messages — read them carefully, they tell you exactly what's wrong
  • Skipping the .env file — hardcoding API keys in code is a security risk

How to Run the App

Once you have app.py built, run the whole app with one command:

uv run streamlit run app.py

Your browser will open automatically at localhost:8501 — that's your app running locally.


What Comes After This

Once you finish all 6 phases, here is where you can go next:

  • Deploy the backend to a server (Railway or Render — both have free tiers)
  • Learn FastAPI to turn your Python code into a proper API
  • Build a React Native or Flutter phone app that connects to your backend
  • Add Ollama for a fully private, fully local version with no API costs
  • Add charts and visualizations of your mood over time using Plotly

What You Will Have Learned

By the time you finish this project, you will have hands-on experience with:

  • RAG — retrieval-augmented generation
  • LangChain — the industry standard RAG framework
  • ChromaDB — local vector database
  • Whisper — speech-to-text used in real products
  • Structured LLM outputs — extracting JSON from AI, a hugely valuable skill
  • Pydantic — data validation used across the Python AI ecosystem
  • Streamlit — building real browser UIs in pure Python
  • UV — modern Python package management

That is a genuinely strong AI engineering foundation — built by doing, not just reading.


Good luck. Build something you are proud of. 🐦