feat(desktop): paginate stream particles via windowed Firestore subscription #296

2026-06-21T02:15:02Z

talksik commented

2026-06-21 02:15:02 +00:00

(Migrated from github.com)

Streams previously prefetched every particle through an unbounded Firestore
subscription. This adds a windowed source that anchors a created_at desc
limit query at the newest particle and grows it backward on demand, so the
already-seen history before a viewer's playback marker is no longer loaded.

useWindowedStreamParticles: tail-anchored live window that grows backward
to cover the resume marker and to service loadOlder() (list scroll-up).
Because the window always includes the newest particle, new arrivals stream
in and forward playback never needs a fetch. Includes anti-eviction growth
so a new tail particle never pushes loaded particles out of the window.
useStreamPlayback: consumes the windowed source instead of loading all
children. New-tail and removal handling are derived from the children array
(Firestore change events can't tell a genuine arrival from pagination
backfill). Resume-from-marker waits for backward growth to reach an older
marker; prev at the window edge pulls in older history.

Playback resume, forward-only marker persistence, auto-advance-on-new, and
removal fallback are all preserved. List view and progress indicator continue
to render off the (now windowed) children; their pagination UX is a follow-up.

Co-Authored-By: Claude Opus 4.8 (1M context) noreply@anthropic.com
Claude-Session: https://claude.ai/code/session_01V8gsmdVd7R8PJtn4UFnC4J

Summary by CodeRabbit

New Features
- Added windowed, live-updating stream playback with seamless auto-resume when new particles arrive.
- Enabled incremental “load older” pagination across the stream view, sidebar, and playback indicators.
Bug Fixes
- Improved fallback behavior when the current particle is no longer available in the loaded window.
- Enhanced navigation behavior when switching between streams and resuming from older markers.
UI / Accessibility
- Added support for forwarding a viewport ref in the ScrollArea component.

Streams previously prefetched every particle through an unbounded Firestore subscription. This adds a windowed source that anchors a `created_at desc` limit query at the newest particle and grows it backward on demand, so the already-seen history before a viewer's playback marker is no longer loaded. - `useWindowedStreamParticles`: tail-anchored live window that grows backward to cover the resume marker and to service `loadOlder()` (list scroll-up). Because the window always includes the newest particle, new arrivals stream in and forward playback never needs a fetch. Includes anti-eviction growth so a new tail particle never pushes loaded particles out of the window. - `useStreamPlayback`: consumes the windowed source instead of loading all children. New-tail and removal handling are derived from the children array (Firestore change events can't tell a genuine arrival from pagination backfill). Resume-from-marker waits for backward growth to reach an older marker; `prev` at the window edge pulls in older history. Playback resume, forward-only marker persistence, auto-advance-on-new, and removal fallback are all preserved. List view and progress indicator continue to render off the (now windowed) children; their pagination UX is a follow-up. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01V8gsmdVd7R8PJtn4UFnC4J  ## Summary by CodeRabbit * **New Features** * Added windowed, live-updating stream playback with seamless auto-resume when new particles arrive. * Enabled incremental “load older” pagination across the stream view, sidebar, and playback indicators. * **Bug Fixes** * Improved fallback behavior when the current particle is no longer available in the loaded window. * Enhanced navigation behavior when switching between streams and resuming from older markers. * **UI / Accessibility** * Added support for forwarding a viewport ref in the ScrollArea component.

coderabbitai[bot] commented

2026-06-21 02:15:11 +00:00

(Migrated from github.com)

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Free

Run ID: a7dcaead-cd97-4c6c-92e3-d67fcc34d181

📥 Commits

Reviewing files that changed from the base of the PR and between af69f98583 and 362a22bbc6.

📒 Files selected for processing (5)

js/desktop/src/components/ui/scroll-area.tsx
js/desktop/src/features/particles/playback-page-indicator.tsx
js/desktop/src/features/particles/stream-bottom-bar.tsx
js/desktop/src/features/particles/stream-list-sidebar.tsx
js/desktop/src/features/particles/stream-view.tsx

📝 Walkthrough

Walkthrough

Introduces useWindowedStreamParticles, a tail-anchored Firestore hook with dynamic anti-eviction and backward-coverage growth that maintains a windowed view of particles. Rewires useStreamPlayback to use this hook instead of live Firestore subscriptions, extends its return value with backward pagination controls, and integrates those controls throughout the playback UI (page indicator, bottom bar, list sidebar) with scroll preservation and intersection-observer-driven older-item loading.

Changes

Windowed particle stream and playback rewiring

Layer / File(s)	Summary
`useWindowedStreamParticles` hook `js/desktop/src/hooks/use-windowed-stream-particles.ts`	Defines public result contract and internal window tracking state, implements tail-anchored Firestore subscription ordered by `created_at` descending with dynamic limit. On new snapshots, freezes backward-coverage target from marker, tracks newest/oldest timestamps, grows limit to prevent anti-eviction when new tail items saturate the query, and grows limit by page size to cover the marker when saturated. Exposes reversed (ascending) children, `loadOlder()` callback, and loading/error flags. Returns inert result when path is undefined.
`useStreamPlayback` core rewiring `js/desktop/src/hooks/use-stream-playback.ts`	Replaces `useLiveParticleChildren` with `useWindowedStreamParticles`, extends return type with `hasMoreOlder`, `loadOlder`, `isLoadingOlder`. Derives `currentIndex` and `currentParticle` from the loaded window. Replaces Firestore add/remove callbacks with effects that track new tail IDs (dispatch `PARTICLE_ADDED` for auto-advance) and detect removal of current particle from window (dispatch `PARTICLE_REMOVED`). Updates stream-reset block.
Initialization backoff, `prev` navigation, and return `js/desktop/src/hooks/use-stream-playback.ts`	Modifies initialization: if marker is older than loaded window and `hasMoreOlder` is true, waits via `initFallback` timeout instead of initializing to oldest; if history is exhausted, catches up to last loaded particle. Updates `prev` callback to call `loadOlder()` when at window start and older history exists, otherwise navigates backward within window. Preserves forward-progress guard on persisted marker update. Returns pagination fields.
`ScrollArea` viewport ref support `js/desktop/src/components/ui/scroll-area.tsx`	Adds optional `viewportRef` prop and forwards it to the underlying viewport primitive, enabling components to access and control scroll position.
`PlaybackPageIndicator` windowed display `js/desktop/src/features/particles/playback-page-indicator.tsx`	Refactors from fixed-size pagination model (`total`, `PAGE_SIZE`) to sliding-window view using `loadedCount` and `VISIBLE_SEGMENTS`. Computes clamped visible slice from current index. Adds optional `hasMoreOlder` and `onLoadOlder` props. Rewires stub navigation into `goOlder`/`goNewer` handlers, with `goOlder` invoking `onLoadOlder` when scrolling to oldest segment. Updates rendered segments to use slice-derived indices.
`BottomBar` pagination props and wiring `js/desktop/src/features/particles/stream-bottom-bar.tsx`	Updates props to remove `total` and add `loadedCount` with `hasMoreOlder` and `onLoadOlder`. Rewires both `PlaybackPageIndicator` render sites to pass new pagination props.
`StreamListSidebar` scroll preservation and older-history loader `js/desktop/src/features/particles/stream-list-sidebar.tsx`	Extends props with `hasMoreOlder`, `onLoadOlder`, `isLoadingOlder`. Adds scroll position preservation via anchor-before-load and restore-after-items pattern using `useLayoutEffect`. Implements `IntersectionObserver` on top sentinel to trigger `onLoadOlder` when user scrolls near top and older items exist. Switches from `currentIndex`-driven scrolling to `selectedId`-driven scrolling (scrolls only on true selection change). Passes `viewportRef` to `ScrollArea` and conditionally renders loader indicator.
`StreamViewInner` pagination wiring `js/desktop/src/features/particles/stream-view.tsx`	Updates `useStreamPlayback` destructuring to include `hasMoreOlder`, `loadOlder`, `isLoadingOlder`. Threads these values and `loadedCount` into `BottomBar` (replacing `total`), and threads pagination state into `StreamListSidebar` in list mode.

Sequence Diagram(s)

sequenceDiagram
  participant StreamViewInner
  participant useStreamPlayback
  participant useWindowedStreamParticles
  participant Firestore

  StreamViewInner->>useStreamPlayback: stream path, playback marker
  useStreamPlayback->>useWindowedStreamParticles: path, marker, pageSize
  useWindowedStreamParticles->>Firestore: subscribe(created_at DESC, limit)
  Firestore-->>useWindowedStreamParticles: snapshot (reversed to ascending)
  useWindowedStreamParticles-->>useStreamPlayback: children[], hasMoreOlder, loadOlder, isLoadingOlder

  alt marker found in loaded window
    useStreamPlayback->>useStreamPlayback: initialize to first particle after marker
  else marker older than window and hasMoreOlder
    useStreamPlayback->>useStreamPlayback: wait via initFallback timeout
    useStreamPlayback->>useWindowedStreamParticles: loadOlder() increases limit
    useWindowedStreamParticles->>Firestore: re-subscribe with larger limit
    Firestore-->>useWindowedStreamParticles: wider snapshot with older particles
  else history exhausted
    useStreamPlayback->>useStreamPlayback: catch up to last loaded particle
  end

  StreamViewInner->>BottomBar: loadedCount, hasMoreOlder, onLoadOlder
  StreamViewInner->>PlaybackPageIndicator: loadedCount, hasMoreOlder, onLoadOlder
  StreamViewInner->>StreamListSidebar: hasMoreOlder, onLoadOlder, isLoadingOlder

  StreamListSidebar->>StreamListSidebar: IntersectionObserver on top sentinel
  alt user scrolls near top and hasMoreOlder
    StreamListSidebar->>StreamListSidebar: preserve scroll position (anchor)
    StreamListSidebar->>useStreamPlayback: onLoadOlder → loadOlder()
    useStreamPlayback->>useWindowedStreamParticles: loadOlder()
    useWindowedStreamParticles->>Firestore: re-subscribe with extended limit
    Firestore-->>StreamListSidebar: older particles prepended, scroll restored
  end

  StreamViewInner->>useStreamPlayback: prev()
  alt currentIndex == 0 and hasMoreOlder
    useStreamPlayback->>useWindowedStreamParticles: loadOlder()
  else currentIndex > 0
    useStreamPlayback->>useStreamPlayback: navigate to prior index
  end

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~75 minutes

Poem

🐰 A windowed view of the stream flows true,
With anti-eviction and old stories new!
When the marker calls and the past is long,
The rabbit loads older, bold and strong.
The sidebar scrolls smooth as history grows,
Backward pagination—that's how it goes! 📜✨

Note

🎁 Summarized by CodeRabbit Free

Your organization is on the Free plan. CodeRabbit will generate a high-level summary and a walkthrough for each pull request. For a comprehensive line-by-line review, please upgrade your subscription to CodeRabbit Pro by visiting https://app.coderabbit.ai/login.

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[![Review Change Stack](https://storage.googleapis.com/coderabbit_public_assets/review-stack-in-coderabbit-ui.svg)](https://app.coderabbit.ai/change-stack/flowy-live/llink/pull/296?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)  <details> <summary>ℹ️ Recent review info</summary> <details> <summary>⚙️ Run configuration</summary> **Configuration used**: defaults **Review profile**: CHILL **Plan**: Free **Run ID**: `a7dcaead-cd97-4c6c-92e3-d67fcc34d181` </details> <details> <summary>📥 Commits</summary> Reviewing files that changed from the base of the PR and between af69f985833a75d712930b744419f9bf73b37927 and 362a22bbc66bcc6c2431db4fb1443a1269c9443d. </details> <details> <summary>📒 Files selected for processing (5)</summary> * `js/desktop/src/components/ui/scroll-area.tsx` * `js/desktop/src/features/particles/playback-page-indicator.tsx` * `js/desktop/src/features/particles/stream-bottom-bar.tsx` * `js/desktop/src/features/particles/stream-list-sidebar.tsx` * `js/desktop/src/features/particles/stream-view.tsx` </details> </details> ---  <details> <summary>📝 Walkthrough</summary> ## Walkthrough Introduces `useWindowedStreamParticles`, a tail-anchored Firestore hook with dynamic anti-eviction and backward-coverage growth that maintains a windowed view of particles. Rewires `useStreamPlayback` to use this hook instead of live Firestore subscriptions, extends its return value with backward pagination controls, and integrates those controls throughout the playback UI (page indicator, bottom bar, list sidebar) with scroll preservation and intersection-observer-driven older-item loading. ## Changes **Windowed particle stream and playback rewiring** | Layer / File(s) | Summary | |---|---| | **`useWindowedStreamParticles` hook** `js/desktop/src/hooks/use-windowed-stream-particles.ts` | Defines public result contract and internal window tracking state, implements tail-anchored Firestore subscription ordered by `created_at` descending with dynamic limit. On new snapshots, freezes backward-coverage target from marker, tracks newest/oldest timestamps, grows limit to prevent anti-eviction when new tail items saturate the query, and grows limit by page size to cover the marker when saturated. Exposes reversed (ascending) children, `loadOlder()` callback, and loading/error flags. Returns inert result when path is undefined. | | **`useStreamPlayback` core rewiring** `js/desktop/src/hooks/use-stream-playback.ts` | Replaces `useLiveParticleChildren` with `useWindowedStreamParticles`, extends return type with `hasMoreOlder`, `loadOlder`, `isLoadingOlder`. Derives `currentIndex` and `currentParticle` from the loaded window. Replaces Firestore add/remove callbacks with effects that track new tail IDs (dispatch `PARTICLE_ADDED` for auto-advance) and detect removal of current particle from window (dispatch `PARTICLE_REMOVED`). Updates stream-reset block. | | **Initialization backoff, `prev` navigation, and return** `js/desktop/src/hooks/use-stream-playback.ts` | Modifies initialization: if marker is older than loaded window and `hasMoreOlder` is true, waits via `initFallback` timeout instead of initializing to oldest; if history is exhausted, catches up to last loaded particle. Updates `prev` callback to call `loadOlder()` when at window start and older history exists, otherwise navigates backward within window. Preserves forward-progress guard on persisted marker update. Returns pagination fields. | | **`ScrollArea` viewport ref support** `js/desktop/src/components/ui/scroll-area.tsx` | Adds optional `viewportRef` prop and forwards it to the underlying viewport primitive, enabling components to access and control scroll position. | | **`PlaybackPageIndicator` windowed display** `js/desktop/src/features/particles/playback-page-indicator.tsx` | Refactors from fixed-size pagination model (`total`, `PAGE_SIZE`) to sliding-window view using `loadedCount` and `VISIBLE_SEGMENTS`. Computes clamped visible slice from current index. Adds optional `hasMoreOlder` and `onLoadOlder` props. Rewires stub navigation into `goOlder`/`goNewer` handlers, with `goOlder` invoking `onLoadOlder` when scrolling to oldest segment. Updates rendered segments to use slice-derived indices. | | **`BottomBar` pagination props and wiring** `js/desktop/src/features/particles/stream-bottom-bar.tsx` | Updates props to remove `total` and add `loadedCount` with `hasMoreOlder` and `onLoadOlder`. Rewires both `PlaybackPageIndicator` render sites to pass new pagination props. | | **`StreamListSidebar` scroll preservation and older-history loader** `js/desktop/src/features/particles/stream-list-sidebar.tsx` | Extends props with `hasMoreOlder`, `onLoadOlder`, `isLoadingOlder`. Adds scroll position preservation via anchor-before-load and restore-after-items pattern using `useLayoutEffect`. Implements `IntersectionObserver` on top sentinel to trigger `onLoadOlder` when user scrolls near top and older items exist. Switches from `currentIndex`-driven scrolling to `selectedId`-driven scrolling (scrolls only on true selection change). Passes `viewportRef` to `ScrollArea` and conditionally renders loader indicator. | | **`StreamViewInner` pagination wiring** `js/desktop/src/features/particles/stream-view.tsx` | Updates `useStreamPlayback` destructuring to include `hasMoreOlder`, `loadOlder`, `isLoadingOlder`. Threads these values and `loadedCount` into `BottomBar` (replacing `total`), and threads pagination state into `StreamListSidebar` in list mode. | ## Sequence Diagram(s) ```mermaid sequenceDiagram participant StreamViewInner participant useStreamPlayback participant useWindowedStreamParticles participant Firestore StreamViewInner->>useStreamPlayback: stream path, playback marker useStreamPlayback->>useWindowedStreamParticles: path, marker, pageSize useWindowedStreamParticles->>Firestore: subscribe(created_at DESC, limit) Firestore-->>useWindowedStreamParticles: snapshot (reversed to ascending) useWindowedStreamParticles-->>useStreamPlayback: children[], hasMoreOlder, loadOlder, isLoadingOlder alt marker found in loaded window useStreamPlayback->>useStreamPlayback: initialize to first particle after marker else marker older than window and hasMoreOlder useStreamPlayback->>useStreamPlayback: wait via initFallback timeout useStreamPlayback->>useWindowedStreamParticles: loadOlder() increases limit useWindowedStreamParticles->>Firestore: re-subscribe with larger limit Firestore-->>useWindowedStreamParticles: wider snapshot with older particles else history exhausted useStreamPlayback->>useStreamPlayback: catch up to last loaded particle end StreamViewInner->>BottomBar: loadedCount, hasMoreOlder, onLoadOlder StreamViewInner->>PlaybackPageIndicator: loadedCount, hasMoreOlder, onLoadOlder StreamViewInner->>StreamListSidebar: hasMoreOlder, onLoadOlder, isLoadingOlder StreamListSidebar->>StreamListSidebar: IntersectionObserver on top sentinel alt user scrolls near top and hasMoreOlder StreamListSidebar->>StreamListSidebar: preserve scroll position (anchor) StreamListSidebar->>useStreamPlayback: onLoadOlder → loadOlder() useStreamPlayback->>useWindowedStreamParticles: loadOlder() useWindowedStreamParticles->>Firestore: re-subscribe with extended limit Firestore-->>StreamListSidebar: older particles prepended, scroll restored end StreamViewInner->>useStreamPlayback: prev() alt currentIndex == 0 and hasMoreOlder useStreamPlayback->>useWindowedStreamParticles: loadOlder() else currentIndex > 0 useStreamPlayback->>useStreamPlayback: navigate to prior index end ``` ## Estimated code review effort 🎯 4 (Complex) | ⏱️ ~75 minutes ## Poem > 🐰 A windowed view of the stream flows true, > With anti-eviction and old stories new! > When the marker calls and the past is long, > The rabbit loads older, bold and strong. > The sidebar scrolls smooth as history grows, > Backward pagination—that's how it goes! 📜✨ </details>   --- > [!NOTE] > <details> > <summary>🎁 Summarized by CodeRabbit Free</summary> > > Your organization is on the Free plan. CodeRabbit will generate a high-level summary and a walkthrough for each pull request. For a comprehensive line-by-line review, please upgrade your subscription to CodeRabbit Pro by visiting <https://app.coderabbit.ai/login>. > > </details> Comment `@coderabbitai help` to get the list of available commands and usage tips.

This pull request has changes conflicting with the target branch.

js/desktop/src/features/particles/playback-page-indicator.tsx
js/desktop/src/features/particles/stream-list-sidebar.tsx

View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.

git fetch -u origin claude/paginated-firestorm-particles-omb5bs:claude/paginated-firestorm-particles-omb5bs

git checkout claude/paginated-firestorm-particles-omb5bs

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: FlowyLabs/llink#296