Tea Leaves

A self-hosted research tool for organizing, connecting, and publishing observations about themes and motifs across media. Notes scattered across Twitter, Bluesky, phone notes, and conversations come together into one structured, searchable, connected workspace.

---

Table of Contents

• Overview
• Architecture
• Prerequisites
• Setup
• Configuration
• Running the App
• Development
• Authentication
• AI Integration
• Data & Backups
• Import
• Timeline
• Motifs & Connections
• Image Annotation
• Tags
• Public Profiles & Publishing
• Comments
• ActivityPub / Fediverse
• Collaborative Access
• Admin Panel
• Keyboard Shortcuts
• Quick Capture
• Integrations
• Build Phases · Roadmap · Changelog

---

Overview

Tea Leaves is a mobile-first PWA designed for a single researcher. It is entirely self-hosted with no cloud dependencies — all storage, AI processing, and serving happens on your own infrastructure.

Key principles:

• Your data is irreplaceable. Every decision protects years of research.
• Mobile-first, but fully usable on desktop.
• No cloud. Everything runs on your homelab.
• Settings are live-editable in the UI — no restarts required.
• Export everything, at any time, in standard formats.

---

Architecture

[Browser / Phone]
      │
      ▼
[Reverse Proxy]          ← Tailscale tunnel from remote server (external, not in this stack)
      │
      ▼
   app:80                ← nginx (React frontend + reverse proxy)
      │
      ├── /api/*  ──────► api:3000   ← Node.js / Express (internal only)
      ├── /auth/* ──────► api:3000
      └── /*      ──────► React app (static files)
                              │
                    ┌─────────┼─────────┐
                    ▼         ▼         ▼
                 db:5432  redis:6379  [AI Server]
           (PostgreSQL    (sessions,  (external,
           + pgvector)    job queues)  HTTP calls)

Docker Compose services:

Service	Image	Purpose
app	Built from ./app	React frontend, served by nginx
api	Built from ./api	Node.js + Express backend
db	pgvector/pgvector:pg16	PostgreSQL with vector search
redis	redis:7-alpine	Sessions, rate limiting, background jobs

The AI server and reverse proxy (Tailscale + whatever proxy you use) are external — this stack makes HTTP calls out to AI and receives traffic in from the proxy.

---

Prerequisites

• Docker and Docker Compose (v2)
• An Authentik instance with an OAuth2/OIDC provider configured for Tea Leaves
• (Optional) A self-hosted AI server — Ollama, any OpenAI-compatible server (llama.cpp, LM Studio, vLLM), or an Anthropic API key. AI features are disabled until configured but the app runs fully without it.

---

Setup

1. Clone the repository

git clone <your-repo-url> tea-leaves
cd tea-leaves

2. Create your environment file

cp .env.example .env

Then edit .env and fill in every value. See Configuration for details.

3. Configure Authentik

In your Authentik instance, create a new OAuth2/OpenID Connect Provider:

• Name: Tea Leaves
• Client type: Confidential
• Redirect URIs: https://yourdomain.com/auth/callback
• Scopes: openid, email, profile

Then create an Application backed by that provider. Copy the Client ID and Client Secret into your .env.

Set AUTHENTIK_ISSUER to the provider's issuer URL — typically:

https://auth.yourdomain.com/application/o/tea-leaves/

4. Build and start

docker compose up --build -d

Optionally stamp the build with the deployed commit so the admin version panel can show exactly which build is running — see Rebuild after code changes.

Database migrations run automatically on API startup. On first boot, logs will show each migration being applied:

Applying migration: 001_users.sql
  ✓ 001_users.sql
Tea Leaves API listening on port 3000 (production)

5. Verify

curl http://localhost:3000/health
# {"status":"ok"}

Navigate to http://localhost (or whatever APP_PORT you set) and you should see the Tea Leaves login screen.

---

Configuration

All configuration lives in .env. Copy .env.example as your starting point.

Database

Variable	Description
POSTGRES_DB	Database name (default: tealeaves)
POSTGRES_USER	Database user (default: tealeaves)
POSTGRES_PASSWORD	Required. Database password — use a strong random value
DATABASE_URL	Full connection string — must match the three vars above

DATABASE_URL uses the internal Docker hostname db, not localhost.

Redis

Variable	Description
REDIS_URL	Redis connection string (default: redis://redis:6379)

Authentik OIDC

Variable	Description
AUTHENTIK_ISSUER	OIDC issuer URL from your Authentik provider
AUTHENTIK_CLIENT_ID	OAuth2 client ID
AUTHENTIK_CLIENT_SECRET	OAuth2 client secret
AUTHENTIK_REDIRECT_URI	Must exactly match the redirect URI registered in Authentik

Session

Variable	Description
SESSION_SECRET	Required. Long random string (32+ characters) used to sign session cookies

Generate a suitable value with:

openssl rand -base64 48

AI Server

All AI settings can be changed live in the Settings UI without a restart. The env vars below serve as boot-time defaults that seed the settings table on first run.

Variable	Default	Description
AI_PROVIDER	ollama	ollama · openai · anthropic
AI_BASE_URL	http://localhost:11434	Base URL without path suffix
AI_API_KEY	(blank)	Required for OpenAI and Anthropic; leave blank for Ollama
AI_VISION_MODEL	llava	Model used for image description and tag suggestions
AI_EMBEDDING_MODEL	nomic-embed-text	Model used for semantic search and duplicate detection
AI_TIMEOUT_MS	60000	Per-request timeout in milliseconds

All variables are optional — the app starts and runs without them. AI-dependent features (image descriptions, tag suggestions, semantic search, duplicate detection) are simply unavailable until configured.

Media Storage

Variable	Description
MEDIA_DIR	Path inside the api container where uploads are stored (default: /data/media)

Uploaded files are stored here, outside the web root, and served only through the authenticated API.

App & Ports

Variable	Default	Description
APP_URL	—	Public-facing base URL, used to construct the OIDC redirect URI
NODE_ENV	production	Set to development for verbose logging
APP_PORT	80	Host port the app is served on — all traffic (UI, API, auth) goes through here
PORT	3000	Internal port the API listens on inside its container (rarely needs changing)

The API is not exposed directly to the host. All requests go through nginx on APP_PORT, which proxies /api/* and /auth/* to the API container internally.

Change APP_PORT if port 80 is already in use on your host:

APP_PORT=8080

---

Running the App

Start

docker compose up -d

Stop

docker compose down

Stop and remove all data (destructive)

docker compose down -v

View logs

# All services
docker compose logs -f

# API only
docker compose logs -f api

Rebuild after code changes

docker compose up --build -d

Recommended: stamp the build with the deployed commit so Admin → Version & updates can show exactly which build is running:

GIT_SHA=$(git rev-parse HEAD) GIT_DESCRIBE=$(git describe --tags --always) docker compose up --build -d

This is optional and only affects the diagnostic commit/git describe shown in the admin version panel — the plain command above works fine without it (the version/update check still functions). Run it from inside the git checkout; if you deploy from a source tarball without git history, just use the plain command.

---

Development

Use the dev override file, which mounts source directories into the containers for hot-reload:

docker compose -f docker-compose.yml -f docker-compose.dev.yml up --build

The frontend dev server (Vite) runs on port 5173 and proxies /api and /auth to the API container. Open http://localhost:5173 during development.

Running services individually

If you prefer to run the API and frontend outside Docker during development:

# Start only the infrastructure
docker compose up -d db redis

# API (from /api)
cd api
npm install
npm run dev

# Frontend (from /app)
cd app
npm install
npm run dev

Ensure your .env uses localhost hostnames for DATABASE_URL and REDIS_URL when running outside Docker.

---

Authentication

Tea Leaves uses Authentik as its sole authentication provider via OIDC. There are no local usernames or passwords — all login is handled through the SSO flow.

• /auth/login — redirects to Authentik
• /auth/callback — handles the OIDC return, creates or updates the user record, establishes a session
• /auth/logout — destroys the session
• /auth/me — returns the current user (used by the frontend to check auth state)

All API routes except /auth/* and /health require a valid session. Unauthenticated requests receive a 401.

---

AI Integration

Tea Leaves calls an external AI server over HTTP. No AI services run inside the Docker Compose stack. AI is optional — the app runs fully without it; AI-dependent features (image descriptions, tag suggestions, semantic search) are simply unavailable until configured.

Supported providers

Provider	Value	Notes
Ollama	ollama	Default. Uses /api/chat and /api/embed native endpoints.
OpenAI-compatible	openai	Covers OpenAI, llama.cpp, LM Studio, vLLM, Jan, Kobold, and any server with /v1/chat/completions + /v1/embeddings.
Anthropic	anthropic	Uses /v1/messages. No embedding API — configure a separate Ollama or OpenAI embedding provider in the Settings UI (ai.embeddingProvider, ai.embeddingBaseUrl, ai.embeddingApiKey).

Set AI_PROVIDER in .env to select the provider, or change it any time in the Settings UI. See the Configuration section for all AI variables.

Model types

Type	Purpose
Vision model (AI_VISION_MODEL)	Describe uploaded images, suggest tags from images
Embedding model (AI_EMBEDDING_MODEL)	Semantic search, duplicate detection; not used by Anthropic

Example models by provider:

Provider	Vision	Embedding
Ollama	llava, llava:13b, llama3.2-vision, qwen2-vl:7b	nomic-embed-text (768-dim), mxbai-embed-large
OpenAI	gpt-4o, gpt-4o-mini	text-embedding-3-small (1536-dim)
Anthropic	claude-sonnet-4-6, claude-opus-4-6	(none — use separate provider)

Embedding dimensions

The database column defaults to 768 dimensions (matching nomic-embed-text). If you use a model with different dimensions, adjust the column before first boot:

ALTER TABLE entries ALTER COLUMN text_embedding TYPE vector(1536);

Replace 1536 with the actual output dimension of your chosen model.

Settings panel

All AI settings (provider, base URL, API key, model names, timeout, and a separate embedding provider for Anthropic users) are live-editable from the Settings UI with no restart required. The env vars serve as boot-time defaults that seed the settings table on first run.

All AI suggestions require your approval — nothing is applied to your data automatically.

Bulk re-analysis

Settings → AI tools lets you re-run AI processing across all existing content — useful when switching providers or models:

• Re-generate all embeddings — processes every entry where text_embedding is null; enables semantic search on imported content
• Re-describe all images — runs the vision model over every media file without an AI description

Both jobs run in the background via BullMQ and show a live progress bar in Settings. Individual failures are skipped so one bad file doesn't abort the whole run.

---

Data & Backups

Volumes

Volume	Contents
pg_data	All PostgreSQL data
redis_data	Redis persistence
media_files	Uploaded images and screenshots

Export

Export any time from the Settings → Export section:

Format	Contents
entries.json	All entries with tags and media refs
entries.csv	Flat CSV, suitable for spreadsheets
motifs.json	All motifs with entries and connections
motif/<id>.json	Single motif (entries + connections)
motif/<id>.md	Single motif as readable Markdown essay
full.json	Everything in one file

Backups

Restic-based automated backups cover the PostgreSQL database and media files. Configure from Settings → Backup.

Multiple destinations — add as many backup destinations as you need. Each has its own:

• Repository — any restic backend: local path, Backblaze B2, S3-compatible, SFTP, rclone
• Schedule — independent cron expression (e.g. 0 3 * * * for 3am daily); leave blank to disable
• Retention — number of snapshots to keep (older ones pruned automatically)
• Failure alerting — per-destination webhook URL; Tea Leaves POSTs { event, error, service } on failure; compatible with Slack, Discord, n8n, or any HTTP endpoint

Verification — restic check runs after every backup; result shown in job history. The integrity trend sparkline shows the last 12 weeks of pass/fail at a glance.

Snapshot browser — list snapshots per destination and browse what changed between any two snapshots via the diff viewer.

Granular restore — recover specific entries from any snapshot without a full database wipe. Entries are staged for review before being applied; conflicts are flagged so you can decide per-entry.

---

Import

Tea Leaves can import your existing posts from Twitter and Bluesky. Imports run as background jobs (BullMQ, Redis-backed) so large archives don't time out. Progress is shown live on the Import page while the job runs.

Twitter

Export your data from Twitter/X (Settings → Your account → Download an archive of your data). You will receive a .zip file. Upload it directly — no unpacking needed.

The importer reads data/tweet.js inside the archive. That file uses a JavaScript assignment format (window.YTD.tweet.part0 = [...]); the importer strips this prefix automatically. Each tweet's full_text (or text fallback), id_str, created_at, and first expanded URL are imported.

Bluesky

Bluesky does not currently offer a first-party data export. You can use a community tool such as bsky-export or similar to produce a JSON export.

The importer accepts a .json file containing an array of post objects. Two formats are supported:

• Flat: [{ "text": "…", "createdAt": "…" }, …]
• AT Protocol: [{ "uri": "at://did:plc:…/app.bsky.feed.post/…", "value": { "text": "…", "createdAt": "…" } }, …]

Duplicate detection

Pass 1 (source ID): Any post whose source_id (e.g. twitter:1234567890) already exists in your entries is silently skipped — no duplicate is created.

Pass 2 (semantic similarity): When AI is configured, posts that are semantically similar to existing entries above the configured threshold are flagged for review rather than imported automatically.

Flagged duplicates appear in the Duplicate review queue on the Import page. For each pair you can see the existing entry alongside the incoming post and choose to Skip (discard the incoming post) or Import anyway (create a new entry regardless).

---

Timeline

The /timeline view lays all entries out chronologically with grouping, filtering, and a density histogram.

• Zoom levels: month (default), week, or day — switch with one click
• Density histogram — a row of proportional bars at the top, one per period; clicking a bar scrolls to that section; only shown when 2+ groups exist
• Filters: source, certainty, motif (dropdown), date-from/to — all synced to URL params so you can share or bookmark a filtered view
• Entry cards show time label (precision adapts to zoom), source badge, certainty dot, attributed source, truncated content, and tags
• A "capped at 500 entries" notice appears if your filtered result hits the limit

---

Motifs & Connections

Motifs are thematic groupings that connect entries across sources and time. Manage them from the /motifs route.

Creating motifs

The motif list page has a quick-create form — title only, no navigation required. The form stays open after each creation so you can batch-name a queue. Newly created motifs get a green border and land in the "Needs details" filter tab until they have a description and at least one entry.

Assigning entries to motifs

Every entry detail page has an "Add to motif" panel. You can:

• Search existing motifs and assign with one click
• Type a name that doesn't exist to get a "Create & add" option — creates a skeleton motif and links the entry in one step

Connections

Within a motif, entries can be linked with typed connections: related · contradicts · precedes · exemplifies · extends · responds to. Connections support optional timestamps — record 1:23, 45:00, or raw seconds to mark the exact moment within a video where a connection appears. Timestamps are rendered as chips on each connection card and on the motif graph.

Graph view

Each motif has a List / Graph toggle. Graph view renders entries as nodes and connections as labeled edges using an interactive force-directed layout.

---

Image Annotation

Draw and label regions directly on screenshots and media files.

• Draw a region — click and drag on any image in the media viewer to create a rectangle; touch-start/move/end works on mobile
• Label it — a dialog appears after drawing; type a label and save
• Edit or delete — annotations are listed below the image with edit and delete controls
• Annotation mode temporarily disables pinch-to-zoom while active to avoid gesture conflicts

---

Tags

Settings → Tags lets you manage all tags across your entries.

• Rename — edit a tag name in place; all entry associations are preserved automatically
• Merge — absorb one tag into another; all associations transfer to the target tag and the source is deleted
• Delete — removes the tag and all its associations; a confirmation dialog shows how many entries will be affected

---

Public Profiles & Publishing

Tea Leaves has a full public publishing surface for sharing research.

Profile

Set up your public identity at /profile: display name, username (becomes your public slug), bio, avatar, Bluesky handle, and website. Your username appears in all public URLs — /u/:slug — and changes with optional redirect preservation (forward old links, or break them for a clean break).

Visibility

Entries and motifs each have a three-tier visibility setting:

Level	Behaviour
private	Only visible to you (default)
unlisted	Accessible via direct link, not listed publicly
public	Listed on your public profile and discoverable

Public pages

• /u/:slug — your public researcher profile: avatar, bio, social links, list of public motifs
• /u/:slug/motifs/:motif-slug — a public motif page: title, description, entries, read-only connection graph
• /u/:slug/motifs/:motif-slug/embed — embeddable graph view for <iframe> use (no nav chrome)

Public motif pages include:

• Open Graph / Twitter Card meta tags for social preview cards on Bluesky, Mastodon, etc.
• APA-style citation copy-to-clipboard
• Print / Save PDF via the browser's native print dialog
• Comments (see below)

A /sitemap.xml lists all public profiles and motifs. robots.txt is admin-configurable from Settings (defaults to allowing all crawlers).

---

Comments

Threaded comments (one level of replies) are available on motifs. Who can comment depends on context:

Viewer	Can comment?
Logged-in user	Yes — as themselves
Commenter-role share-link holder	Yes — with a required display name
Viewer-role share-link holder	No — read-only
Unauthenticated visitor on a public motif page	Yes — as a logged-in user after signing in

• Motif owners can delete any comment; commenters can delete their own
• Comments are threaded one level deep (reply to a top-level comment; replying to a reply is not supported)
• All content is sanitized through the same pipeline as entries

---

ActivityPub / Fediverse

Tea Leaves supports outgoing ActivityPub so you can publish research to the fediverse (Mastodon, Pixelfed, etc.).

Setup: Set a username in /profile. Your fediverse address will be @username@yourdomain.

Publishing: On any public motif, click "Publish to fediverse" to deliver an Article activity to all your followers.

Following: Anyone on the fediverse can follow you by searching for @username@yourdomain in their client.

Controls:

• Each user can opt out of federation entirely from /profile → Federation section
• Admins can disable ActivityPub instance-wide from Settings → Federation

Each actor has a stable UUID-based URL (/ap/actors/:id) that never changes even if you rename, ensuring followers on other instances aren't broken by username changes. Private keys are AES-256-GCM encrypted at rest and never returned by any API.

Incoming federation (processing Likes, Announces, and remote replies) is planned for v4.

---

Collaborative Access

Motifs can be shared with others via invite links — no Tea Leaves account required.

Creating a share link:

1. Open any motif detail page
2. Click "Share links" to expand the share panel
3. Choose a role: Viewer (read-only) or Commenter (can post comments with a display name)
4. Enter an optional label and click "New link"
5. Copy the generated URL and send it to anyone

What the recipient sees: A standalone page (/share/:token) showing the motif title, description, entries, connections, and (if the role allows) a comment form. No edit controls, no login prompt.

Revoking access: Click "Revoke" next to any share link. The link stops working immediately.

Share links use a UUID invite token — anyone with the link gets access at the assigned role level.

---

Admin Panel

The first user to log in is automatically seeded as admin. Admin users see an Admin link in the navigation bar.

User management (/admin):

• View all users with entry count, media storage, and join date
• Promote or demote users between admin and member roles
• Enable or disable user accounts — disabled users are rejected at login and have their sessions terminated immediately

Audit log viewer — paginated, filterable table of all write operations across the instance (by user, entity type, action, and date range). Rows expand to show before/after JSON diffs.

---

Keyboard Shortcuts

Key	Action
n	New entry (on Entries page)
s	Focus search (on Search page)
c	Open Quick Capture
?	Show all shortcuts

Quick Capture

The + floating button (bottom-right on mobile, bottom-right corner on desktop) opens a quick-capture sheet for fast note entry. Paste a URL, type a note, or drop an image hint. Select certainty and save with ⌘↵.

If you're offline, captures are saved to an IndexedDB queue and synced automatically when the network returns. A badge on the button shows how many items are queued.

---

Integrations

API Tokens

API tokens authenticate webhook requests and any future programmatic access. Generate and revoke tokens from Settings → Integrations.

• Tokens are displayed only once on creation — copy the value before closing the page.
• Each token is stored as a SHA-256 hash; the raw value is never persisted.
• Use a token as a Authorization: Bearer <token> header on requests to webhook endpoints.

Stash Webhook

Tea Leaves can receive scene events directly from Stash via its webhook plugin.

Setup:

1. In Settings → Integrations, generate a token (e.g. named "Stash") and copy both the token and the webhook URL shown on that page.
2. In Stash, go to Settings → Plugins and install or configure the Webhook plugin.
3. Set the webhook URL to the value copied from Tea Leaves.
4. Add a request header: Authorization: Bearer <your token>.
5. Configure the plugin to fire on scene events (create, update).

Received scenes are imported through the same duplicate-detection pipeline as other sources. Each scene gets source_id = stash:<scene_id>, so re-sending the same scene is silently skipped.

Browser Extension / Share Sheet

POST /api/webhooks/browser accepts a page capture from any HTTP client — a browser extension, iOS share sheet, or script:

POST /api/webhooks/browser
Authorization: Bearer <token>
Content-Type: application/json

{
  "url": "https://example.com/article",
  "title": "Article title",
  "selected_text": "The passage you highlighted",
  "tags": ["theme", "motif"]
}

• url is required and must be http or https
• selected_text is used as the entry content; if omitted, title is used instead
• tags is optional — tags are created and linked automatically
• Returns { "id": "<entry-uuid>" } on success

The entry is created with source = 'web' and the URL stored as source_url. Use any active API token in the Authorization: Bearer header.

---

Build Phases

v1.0.0 through v6.0.0 are all complete — see CHANGELOG.md for the full history. Planned work is tracked as issues and milestones; ROADMAP.md summarizes the milestone ladder.