rbrooks/TeaLeaves
1
0
Fork 0
Code Issues 17 Pull requests Projects Releases 10 Packages Wiki Activity Actions

v9 P1: Public-data watch + one-click capture (polling MVP) #22

New issue
Open
opened 2026-06-22 23:54:53 +00:00 by rbrooks · 0 comments
rbrooks commented 2026-06-22 23:54:53 +00:00
Owner
Copy link

Baseline Watch feature: no account linking, public data only, works immediately. P1 alone is a complete, useful feature.

Platform reality check (sets expectations, not a blocker):

  • Bluesky has a public AppView API and effectively one global index, so both account and keyword/hashtag watching have real reach (app.bsky.feed.getAuthorFeed, app.bsky.feed.searchPosts).
  • Mastodon / the fediverse is decentralized - there is no global hashtag index. Hashtag watching only sees what the queried instance has already federated in, so it is inherently a partial view, not a bug. Account watching is reliable (query the account's home instance directly). The UI must say this explicitly next to Mastodon tag watches.

Schema:

  • watches table: id uuid PK, project_id uuid NOT NULL REFERENCES projects ON DELETE CASCADE, platform text NOT NULL CHECK (platform IN ('bluesky','mastodon')), target_type text NOT NULL CHECK (target_type IN ('account','tag')), target_value text NOT NULL (handle/hashtag without @/#), instance_url text (Mastodon only; validated via isSafeUrl()), enabled boolean NOT NULL DEFAULT true, cursor text (pagination/since-token; reused as the P2 stream resume-point), last_seen_at timestamptz, last_error text, created_at timestamptz NOT NULL DEFAULT now(). UNIQUE (project_id, platform, target_type, target_value, instance_url).
  • discovered_posts table: id uuid PK, watch_id uuid NOT NULL REFERENCES watches ON DELETE CASCADE, source_id text NOT NULL (bluesky:<at-uri> / mastodon:<instance_url>:<status_id> - same dedup shape as import source_id), author_handle text, author_display_name text, content text NOT NULL (sanitized on write), post_url text NOT NULL, media_urls jsonb, posted_at timestamptz NOT NULL, status text NOT NULL DEFAULT 'new' CHECK (status IN ('new','added','dismissed')), created_entry_id uuid REFERENCES entries ON DELETE SET NULL, created_at timestamptz NOT NULL DEFAULT now(). UNIQUE (source_id) - doubles as the ingestion dedup guard across all three phases.

Polling worker (services/watchWorker.ts):

  • BullMQ repeatable job, same pattern as federationSyncWorker; interval in Settings (watch.pollIntervalMinutes, default 10)
  • Platform adapters behind a small interface (lib/watchAdapters/bluesky.ts, mastodon.ts), mirroring how importWorker separates parsers
  • Mastodon adapter: resolve target_value via /api/v1/accounts/lookup, then poll /api/v1/accounts/:id/statuses?since_id=; tag watches poll /api/v1/timelines/tag/:hashtag?since_id=
  • Bluesky adapter: public unauthenticated AppView host (public.api.bsky.app) for both account feeds and search

One-click "Add as entry":

  • Shared addSocialPostAsEntry(post, projectId, userId) helper (used by both the queue and direct search) calls the existing checkDedup() BEFORE insert
  • On a dedup hit, surface "already in this project, view existing entry" rather than silently creating a duplicate
  • POST /api/discovered-posts/:id/add (sets status='added', created_entry_id) and POST /api/discovered-posts/:id/dismiss
  • Bulk add-all / dismiss-all deferred unless the per-post queue proves too slow in practice

Direct search (3rd tab on the Watch page, alongside Watches + Queue):

  • GET /api/watch/search?platform=&type=account|tag&q=&instanceUrl= - Bluesky via app.bsky.actor.searchActors / app.bsky.feed.searchPosts; Mastodon via that instance's /api/v2/search + /api/v1/timelines/tag/:hashtag preview. Results rendered live, not persisted.
  • Account/tag result -> "Add to watchlist" pre-fills the watch-creation form (only asks for the project)
  • Individual post -> "Add as entry" calls addSocialPostAsEntry() directly (no discovered_posts row), same dedup-hit handling
  • Mastodon caveat carries over: unauthenticated resolve=true cross-instance lookup typically needs a session, so account search only reliably finds accounts the queried instance already knows. One-line UI note; improves once P3 ships.

Nav + security:

  • New top-level Watch nav item with an unread / pending-review count badge
  • instance_url (user-supplied, watches + direct search) validated via isSafeUrl() with a private-IP-range / SSRF check before the first outbound request to a new instance. All P1 requests are unauthenticated GETs to public endpoints - no credentials in play yet.

Originated in ROADMAP.md (now retired; this tracker is the single source of truth).

Baseline Watch feature: no account linking, public data only, works immediately. P1 alone is a complete, useful feature. **Platform reality check (sets expectations, not a blocker):** - Bluesky has a public AppView API and effectively one global index, so both account *and* keyword/hashtag watching have real reach (`app.bsky.feed.getAuthorFeed`, `app.bsky.feed.searchPosts`). - Mastodon / the fediverse is decentralized - there is no global hashtag index. Hashtag watching only sees what the queried instance has already federated in, so it is inherently a partial view, not a bug. Account watching is reliable (query the account's home instance directly). **The UI must say this explicitly next to Mastodon tag watches.** **Schema:** - [ ] `watches` table: `id uuid PK`, `project_id uuid NOT NULL REFERENCES projects ON DELETE CASCADE`, `platform text NOT NULL CHECK (platform IN ('bluesky','mastodon'))`, `target_type text NOT NULL CHECK (target_type IN ('account','tag'))`, `target_value text NOT NULL` (handle/hashtag without `@`/`#`), `instance_url text` (Mastodon only; validated via `isSafeUrl()`), `enabled boolean NOT NULL DEFAULT true`, `cursor text` (pagination/since-token; reused as the P2 stream resume-point), `last_seen_at timestamptz`, `last_error text`, `created_at timestamptz NOT NULL DEFAULT now()`. `UNIQUE (project_id, platform, target_type, target_value, instance_url)`. - [ ] `discovered_posts` table: `id uuid PK`, `watch_id uuid NOT NULL REFERENCES watches ON DELETE CASCADE`, `source_id text NOT NULL` (`bluesky:<at-uri>` / `mastodon:<instance_url>:<status_id>` - same dedup shape as import `source_id`), `author_handle text`, `author_display_name text`, `content text NOT NULL` (sanitized on write), `post_url text NOT NULL`, `media_urls jsonb`, `posted_at timestamptz NOT NULL`, `status text NOT NULL DEFAULT 'new' CHECK (status IN ('new','added','dismissed'))`, `created_entry_id uuid REFERENCES entries ON DELETE SET NULL`, `created_at timestamptz NOT NULL DEFAULT now()`. `UNIQUE (source_id)` - doubles as the ingestion dedup guard across all three phases. **Polling worker (`services/watchWorker.ts`):** - [ ] BullMQ repeatable job, same pattern as `federationSyncWorker`; interval in Settings (`watch.pollIntervalMinutes`, default 10) - [ ] Platform adapters behind a small interface (`lib/watchAdapters/bluesky.ts`, `mastodon.ts`), mirroring how `importWorker` separates parsers - [ ] Mastodon adapter: resolve `target_value` via `/api/v1/accounts/lookup`, then poll `/api/v1/accounts/:id/statuses?since_id=`; tag watches poll `/api/v1/timelines/tag/:hashtag?since_id=` - [ ] Bluesky adapter: public unauthenticated AppView host (`public.api.bsky.app`) for both account feeds and search **One-click "Add as entry":** - [ ] Shared `addSocialPostAsEntry(post, projectId, userId)` helper (used by both the queue and direct search) calls the existing `checkDedup()` BEFORE insert - [ ] On a dedup hit, surface "already in this project, view existing entry" rather than silently creating a duplicate - [ ] `POST /api/discovered-posts/:id/add` (sets `status='added'`, `created_entry_id`) and `POST /api/discovered-posts/:id/dismiss` - [ ] Bulk add-all / dismiss-all deferred unless the per-post queue proves too slow in practice **Direct search (3rd tab on the Watch page, alongside Watches + Queue):** - [ ] `GET /api/watch/search?platform=&type=account|tag&q=&instanceUrl=` - Bluesky via `app.bsky.actor.searchActors` / `app.bsky.feed.searchPosts`; Mastodon via that instance's `/api/v2/search` + `/api/v1/timelines/tag/:hashtag` preview. Results rendered live, not persisted. - [ ] Account/tag result -> "Add to watchlist" pre-fills the watch-creation form (only asks for the project) - [ ] Individual post -> "Add as entry" calls `addSocialPostAsEntry()` directly (no `discovered_posts` row), same dedup-hit handling - [ ] Mastodon caveat carries over: unauthenticated `resolve=true` cross-instance lookup typically needs a session, so account search only reliably finds accounts the queried instance already knows. One-line UI note; improves once P3 ships. **Nav + security:** - [ ] New top-level **Watch** nav item with an unread / pending-review count badge - [ ] `instance_url` (user-supplied, watches + direct search) validated via `isSafeUrl()` with a private-IP-range / SSRF check before the first outbound request to a new instance. All P1 requests are unauthenticated GETs to public endpoints - no credentials in play yet. --- _Originated in `ROADMAP.md` (now retired; this tracker is the single source of truth)._
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
deps/wave-1-low-risk
v7.0.0
v6.2.0
v6.1.0
v6.0.0
v5.0.1
v5.0.0
v4.0.0
v3.0.0
v2.0
v1.0.0
Labels
Clear labels   
bug
Something is not working

  
duplicate
This issue or pull request already exists

  
enhancement
New feature

  
future
Unscoped / not yet prioritized — no milestone

  
help wanted
Need some help

  
invalid
Something is wrong

  
question
More information is needed

  
wontfix
This won't be fixed

No labels
bug
duplicate
enhancement
future
help wanted
invalid
question
wontfix
Milestone
Clear milestone
No items
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
rbrooks/TeaLeaves#22
No description provided.