migrate orion repo into monorepo structure
This commit is contained in:
talksik
+617
@@ -0,0 +1,617 @@
|
||||
# API Documentation
|
||||
|
||||
All protected endpoints require a `Bearer` token in the `Authorization` header.
|
||||
|
||||
## Authentication
|
||||
|
||||
### Request Sign-In Code
|
||||
`POST /auth/request-code`
|
||||
|
||||
Sends a 4-digit sign-in code to the provided email. Creates user if not exists.
|
||||
|
||||
**Request Body:**
|
||||
```json
|
||||
{
|
||||
"email": "[email protected]"
|
||||
}
|
||||
```
|
||||
|
||||
**Response:** `204 No Content`
|
||||
|
||||
### Sign In
|
||||
`POST /auth/sign-in`
|
||||
|
||||
Verifies the code and returns a session token.
|
||||
|
||||
**Request Body:**
|
||||
```json
|
||||
{
|
||||
"email": "[email protected]",
|
||||
"code": "1234"
|
||||
}
|
||||
```
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"human": { "id": "...", "email": "...", "email_prefix": "...", "created_at": "..." },
|
||||
"token": "session_token"
|
||||
}
|
||||
```
|
||||
|
||||
### Sign Out
|
||||
`POST /auth/sign-out` (Protected)
|
||||
|
||||
Invalidates the current session. Returns `204 No Content`.
|
||||
|
||||
### Get Current User
|
||||
`GET /auth/me` (Protected)
|
||||
|
||||
Returns the authenticated user.
|
||||
|
||||
---
|
||||
|
||||
## Startup
|
||||
|
||||
### Get Startup Data
|
||||
`GET /startup` (Protected)
|
||||
|
||||
Bootstrap endpoint for initial app load. Returns all networks the user belongs to, with all streams and their particles fully enriched.
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"networks": [
|
||||
{
|
||||
"id": "net-456",
|
||||
"name": "My Team",
|
||||
"admin_human": { "id": "...", "email": "...", "email_prefix": "...", "created_at": "..." },
|
||||
"humans": [{ "id": "...", "email": "...", "email_prefix": "...", "created_at": "..." }],
|
||||
"open_stream_count": 2,
|
||||
"open_stream_capacity": 5,
|
||||
"created_at": "2025-01-15T10:30:00Z",
|
||||
"streams": [
|
||||
{
|
||||
"id": "p-001",
|
||||
"name": "Sprint Planning",
|
||||
"description": "Weekly sync",
|
||||
"status": "open",
|
||||
"members": ["[email protected]"],
|
||||
"particles": [
|
||||
{
|
||||
"id": "p-002",
|
||||
"type": "text",
|
||||
"data": { "content": "Hello" },
|
||||
"created_by_email": "[email protected]",
|
||||
"seen": true,
|
||||
"acks": [],
|
||||
"updated_at": "...",
|
||||
"created_at": "..."
|
||||
}
|
||||
],
|
||||
"unseen_count": 0
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Networks
|
||||
|
||||
### Create Network
|
||||
`POST /networks` (Protected)
|
||||
|
||||
**Request Body:**
|
||||
```json
|
||||
{
|
||||
"name": "My Network"
|
||||
}
|
||||
```
|
||||
|
||||
### List Networks
|
||||
`GET /networks` (Protected)
|
||||
|
||||
Returns all networks the user is a member of.
|
||||
|
||||
### Get Network
|
||||
`GET /networks/{id}` (Protected)
|
||||
|
||||
Returns a specific network by ID.
|
||||
|
||||
### Add Members to Network
|
||||
`POST /networks/{id}/members` (Protected)
|
||||
|
||||
**Request Body:**
|
||||
```json
|
||||
{
|
||||
"email_addresses": ["[email protected]", "[email protected]"]
|
||||
}
|
||||
```
|
||||
|
||||
### Remove Member from Network
|
||||
`DELETE /networks/{id}/members/{email}` (Protected)
|
||||
|
||||
Removes a member by email from the network.
|
||||
|
||||
### Set Open Stream Capacity
|
||||
`PUT /networks/{id}/capacity` (Protected, Admin only)
|
||||
|
||||
**Request Body:**
|
||||
```json
|
||||
{
|
||||
"capacity": 10
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Streams
|
||||
|
||||
Streams are top-level particles of type `stream`. They have dedicated endpoints for creation and management, and contain child particles.
|
||||
|
||||
### Create Stream
|
||||
`POST /networks/{network_id}/streams` (Protected)
|
||||
|
||||
**Request Body:**
|
||||
```json
|
||||
{
|
||||
"name": "Sprint Planning",
|
||||
"description": "Weekly sync",
|
||||
"visibility": "custom",
|
||||
"members": ["[email protected]"]
|
||||
}
|
||||
```
|
||||
|
||||
- `visibility`: `network_all` (default) or `custom`
|
||||
- `members` is required when visibility is `custom`
|
||||
|
||||
**Response:** `201 Created` — returns a [Stream](#stream-1) object.
|
||||
|
||||
### Get Stream
|
||||
`GET /streams/{id}` (Protected)
|
||||
|
||||
Returns a stream with all its child particles, enriched with seen/ack state.
|
||||
|
||||
**Response:** returns a [Stream](#stream-1) object.
|
||||
|
||||
### Update Stream
|
||||
`PATCH /streams/{id}` (Protected)
|
||||
|
||||
Updates a stream's name and/or description. Status is not affected (use the open/close endpoints instead). Only provided fields are updated.
|
||||
|
||||
**Request Body:**
|
||||
```json
|
||||
{
|
||||
"name": "New Name",
|
||||
"description": "New description"
|
||||
}
|
||||
```
|
||||
|
||||
- Both fields are optional — omit a field to leave it unchanged
|
||||
- `name` cannot be empty if provided
|
||||
|
||||
**Response:** returns the updated [Stream](#stream-1) object.
|
||||
|
||||
### Create Stream Particle
|
||||
`POST /streams/{id}/particles` (Protected)
|
||||
|
||||
Creates a child particle inside a stream. Child particles inherit visibility from the stream.
|
||||
|
||||
**Request Body:**
|
||||
```json
|
||||
{
|
||||
"type": "text|media|file|quest|paper",
|
||||
"data": {}
|
||||
}
|
||||
```
|
||||
|
||||
- Cannot create `stream` or `folder` types as children
|
||||
- For `media` and `file` types, `data` must include a valid `object_id` from depot
|
||||
|
||||
**Response:** `201 Created` — returns a [StreamParticle](#streamparticle) object.
|
||||
|
||||
### Open Stream
|
||||
`POST /streams/{id}/open` (Protected)
|
||||
|
||||
Opens a closed stream. Fails with `409` if capacity would be exceeded.
|
||||
|
||||
### Close Stream
|
||||
`POST /streams/{id}/close` (Protected)
|
||||
|
||||
Closes an open stream.
|
||||
|
||||
### Add Members to Stream
|
||||
`POST /streams/{id}/members` (Protected)
|
||||
|
||||
**Request Body:**
|
||||
```json
|
||||
{
|
||||
"emails": ["[email protected]"]
|
||||
}
|
||||
```
|
||||
|
||||
### Remove Members from Stream
|
||||
`DELETE /streams/{id}/members` (Protected)
|
||||
|
||||
**Request Body:**
|
||||
```json
|
||||
{
|
||||
"emails": ["[email protected]"]
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Particles
|
||||
|
||||
### List Particles
|
||||
`GET /networks/{network_id}/particles` (Protected)
|
||||
|
||||
**Query Parameters:**
|
||||
- `parent_id` (optional): Filter by parent particle
|
||||
- `cursor` (optional): Pagination cursor
|
||||
- `direction` (optional): `after` or `before` (default: `after`)
|
||||
- `type` (optional, repeatable): Filter by particle type
|
||||
|
||||
**Response enrichment:**
|
||||
Each particle in the response includes:
|
||||
- `seen` (boolean): Whether the requester has marked this particle as seen
|
||||
- `acks` (array): List of acknowledgments `[{email, acked_at}]`
|
||||
- `unseen_count` (integer, streams only): Count of unseen child particles
|
||||
|
||||
### Get Particle
|
||||
`GET /particles/{id}` (Protected)
|
||||
|
||||
### Update Particle
|
||||
`PATCH /particles/{id}` (Protected)
|
||||
|
||||
**Request Body:**
|
||||
```json
|
||||
{
|
||||
"data": {}
|
||||
}
|
||||
```
|
||||
|
||||
### Delete Particle
|
||||
`DELETE /particles/{id}` (Protected)
|
||||
|
||||
Deletes the particle and all children. If it references a depot object, that is also deleted.
|
||||
|
||||
### Download Particle
|
||||
`GET /particles/{id}/download` (Protected)
|
||||
|
||||
Returns a `302` redirect to a signed download URL. Only works for `media` and `file` particles.
|
||||
|
||||
### Mark Seen
|
||||
`POST /particles/{id}/seen` (Protected)
|
||||
|
||||
Marks a particle as seen by the requester. This is private state, only visible to the requester.
|
||||
|
||||
Returns `204 No Content` on success.
|
||||
|
||||
### Mark Seen (Batch)
|
||||
`POST /particles/seen` (Protected)
|
||||
|
||||
Marks multiple particles as seen by the requester.
|
||||
|
||||
**Request Body:**
|
||||
```json
|
||||
{
|
||||
"particle_ids": ["particle_uuid1", "particle_uuid2"]
|
||||
}
|
||||
```
|
||||
|
||||
Returns `204 No Content` on success.
|
||||
|
||||
### Acknowledge Particle
|
||||
`POST /particles/{id}/ack` (Protected)
|
||||
|
||||
Acknowledges a particle. Acknowledgments are public and permanent, visible to all users with access. Also marks the particle as seen.
|
||||
|
||||
Returns `204 No Content` on success.
|
||||
|
||||
---
|
||||
|
||||
## Depot (File Storage)
|
||||
|
||||
### Prepare Upload
|
||||
`POST /depot/upload` (Protected)
|
||||
|
||||
Prepares a signed URL for direct upload to GCS.
|
||||
|
||||
**Request Body:**
|
||||
```json
|
||||
{
|
||||
"network_id": "network_uuid",
|
||||
"name": "filename.png",
|
||||
"content_type": "image/png",
|
||||
"content_length": 12345
|
||||
}
|
||||
```
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"object_id": "uuid",
|
||||
"upload_url": "https://storage.googleapis.com/...",
|
||||
"upload_headers": { "Content-Type": "image/png" }
|
||||
}
|
||||
```
|
||||
|
||||
### Confirm Upload
|
||||
`POST /depot/objects/{id}/confirm` (Protected)
|
||||
|
||||
Confirms that an upload has been completed.
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"id": "uuid",
|
||||
"name": "filename.png",
|
||||
"content_type": "image/png",
|
||||
"content_length": 12345,
|
||||
"contains_content": true,
|
||||
"created_at": "..."
|
||||
}
|
||||
```
|
||||
|
||||
### Upload Flow
|
||||
|
||||
1. `POST /depot/upload` — get signed URL and `object_id`
|
||||
2. Upload file directly to GCS using the signed URL
|
||||
3. `POST /depot/objects/{id}/confirm` — mark upload complete
|
||||
4. Create a `media` or `file` particle with the `object_id` in its data
|
||||
|
||||
---
|
||||
|
||||
## Object Reference
|
||||
|
||||
### Human
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `id` | `string \| null` | Unique identifier. Null if not yet registered. |
|
||||
| `email` | `string` | Email address (stable identifier). |
|
||||
| `email_prefix` | `string` | The local part of the email (before `@`). |
|
||||
| `created_at` | `string \| null` | ISO 8601 timestamp. Null if not registered. |
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "abc-123",
|
||||
"email": "[email protected]",
|
||||
"email_prefix": "alice",
|
||||
"created_at": "2025-01-15T10:30:00Z"
|
||||
}
|
||||
```
|
||||
|
||||
### Network
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `id` | `string` | Unique identifier. |
|
||||
| `name` | `string` | Display name of the network. |
|
||||
| `admin_human` | `Human` | The network administrator. |
|
||||
| `humans` | `Human[]` | All members of the network (including admin). |
|
||||
| `open_stream_count` | `integer` | Number of currently open streams. |
|
||||
| `open_stream_capacity` | `integer` | Maximum number of concurrent open streams (default: 5). |
|
||||
| `created_at` | `string` | ISO 8601 timestamp. |
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "net-456",
|
||||
"name": "My Team",
|
||||
"admin_human": { "id": "abc-123", "email": "[email protected]", "email_prefix": "alice", "created_at": "..." },
|
||||
"humans": [
|
||||
{ "id": "abc-123", "email": "[email protected]", "email_prefix": "alice", "created_at": "..." }
|
||||
],
|
||||
"open_stream_count": 2,
|
||||
"open_stream_capacity": 5,
|
||||
"created_at": "2025-01-15T10:30:00Z"
|
||||
}
|
||||
```
|
||||
|
||||
### Stream
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `id` | `string` | Unique identifier (this is a particle ID). |
|
||||
| `name` | `string` | Stream name. |
|
||||
| `description` | `string` | Stream description. |
|
||||
| `status` | `string` | `"open"`, `"closed"`, or `"unspecified"`. |
|
||||
| `members` | `string[]` | Emails of stream members. Omitted for `network_all` visibility. |
|
||||
| `particles` | `StreamParticle[]` | Child particles in the stream. |
|
||||
| `unseen_count` | `integer` | Number of unseen child particles for the requester. |
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "p-001",
|
||||
"name": "Sprint Planning",
|
||||
"description": "Weekly sync",
|
||||
"status": "open",
|
||||
"members": ["[email protected]"],
|
||||
"particles": [],
|
||||
"unseen_count": 0
|
||||
}
|
||||
```
|
||||
|
||||
### StreamParticle
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `id` | `string` | Unique identifier. |
|
||||
| `type` | `string` | One of: `media`, `file`, `text`, `quest`, `paper`. |
|
||||
| `data` | `object` | Type-specific payload (see [Particle Data by Type](#particle-data-by-type)). |
|
||||
| `created_by_email` | `string` | Email of the creator. |
|
||||
| `seen` | `boolean` | Whether the requester has seen this particle. |
|
||||
| `acks` | `AckInfo[]` | Acknowledgments from users. |
|
||||
| `updated_at` | `string` | ISO 8601 timestamp. |
|
||||
| `created_at` | `string` | ISO 8601 timestamp. |
|
||||
|
||||
### Particle
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `id` | `string` | Unique identifier. |
|
||||
| `type` | `string` | One of: `stream`, `folder`, `media`, `file`, `text`, `quest`, `paper`. |
|
||||
| `network_id` | `string` | The network this particle belongs to. |
|
||||
| `parent_id` | `string \| null` | Parent particle ID, if nested. |
|
||||
| `created_by_email` | `string` | Email of the creator. |
|
||||
| `visibility` | `string` | `"network_all"`, `"custom"`, or `"inherited"`. |
|
||||
| `stream_status` | `string \| null` | Only on `stream` type: `"open"` or `"closed"`. |
|
||||
| `data` | `object` | Type-specific payload (see [Particle Data by Type](#particle-data-by-type)). |
|
||||
| `download_url` | `string \| null` | Signed download URL. Only on `media`/`file` particles. |
|
||||
| `seen` | `boolean \| null` | Whether the requester has seen this particle. Only in list responses. |
|
||||
| `acks` | `AckInfo[]` | Acknowledgments. Only in list responses. |
|
||||
| `unseen_count` | `integer \| null` | Unseen child count. Only on `stream` particles in list responses. |
|
||||
| `updated_at` | `string` | ISO 8601 timestamp. |
|
||||
| `created_at` | `string` | ISO 8601 timestamp. |
|
||||
|
||||
### AckInfo
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `email` | `string` | Email of the user who acknowledged. |
|
||||
| `acked_at` | `string` | ISO 8601 timestamp of the acknowledgment. |
|
||||
|
||||
### ParticleList
|
||||
|
||||
Returned by `GET /networks/{network_id}/particles`.
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `particles` | `Particle[]` | Array of enriched particle objects. |
|
||||
| `has_more` | `boolean` | Whether more results exist beyond this page. |
|
||||
| `next_cursor` | `string \| null` | Cursor to fetch the next page. |
|
||||
| `prev_cursor` | `string \| null` | Cursor to fetch the previous page. |
|
||||
|
||||
### DepotObject
|
||||
|
||||
Returned by `POST /depot/objects/{id}/confirm`.
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `id` | `string` | Unique object identifier. |
|
||||
| `name` | `string` | Original filename. |
|
||||
| `content_type` | `string` | MIME type (e.g. `image/png`). |
|
||||
| `content_length` | `integer` | Size in bytes. |
|
||||
| `contains_content` | `boolean` | Whether the object has been uploaded successfully. |
|
||||
| `created_at` | `string` | ISO 8601 timestamp. |
|
||||
|
||||
---
|
||||
|
||||
## Particle Data by Type
|
||||
|
||||
The `data` field on a Particle is a JSON object whose schema depends on the particle's `type`.
|
||||
|
||||
### `stream`
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `name` | `string` | Stream name (required). |
|
||||
| `status` | `string` | `"open"` or `"closed"` (required). |
|
||||
| `description` | `string \| null` | Stream description (optional). |
|
||||
|
||||
```json
|
||||
{ "name": "Sprint Planning", "status": "open", "description": "Weekly sync" }
|
||||
```
|
||||
|
||||
### `folder`
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `name` | `string` | Folder name (required). |
|
||||
| `color` | `string \| null` | Display color (optional). |
|
||||
|
||||
```json
|
||||
{ "name": "Design Assets", "color": "#FF5733" }
|
||||
```
|
||||
|
||||
### `media`
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `object_id` | `string` | Reference to depot storage object (required). |
|
||||
| `mime_type` | `string` | MIME type of the media (required). |
|
||||
| `duration_ms` | `integer` | Duration in milliseconds, must be > 0 (required). |
|
||||
|
||||
```json
|
||||
{ "object_id": "obj-123", "mime_type": "image/jpeg", "duration_ms": 5000 }
|
||||
```
|
||||
|
||||
### `file`
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `object_id` | `string` | Reference to depot storage object (required). |
|
||||
| `filename` | `string` | Original filename (required). |
|
||||
| `mime_type` | `string` | MIME type (required). |
|
||||
| `size` | `integer` | File size in bytes, must be > 0 (required). |
|
||||
|
||||
```json
|
||||
{ "object_id": "obj-456", "filename": "report.pdf", "mime_type": "application/pdf", "size": 204800 }
|
||||
```
|
||||
|
||||
### `text`
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `content` | `string` | Text content (required). |
|
||||
|
||||
```json
|
||||
{ "content": "Hello world" }
|
||||
```
|
||||
|
||||
### `quest`
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `title` | `string` | Quest/task title (required). |
|
||||
| `description` | `string` | Details about the quest (required). |
|
||||
| `status` | `string \| null` | Current status (optional, e.g. `"todo"`, `"in_progress"`, `"done"`). |
|
||||
| `assigned_to` | `string \| null` | Email of the assigned user (optional). |
|
||||
| `due_date` | `string \| null` | ISO date string (optional, e.g. `"2025-03-01"`). |
|
||||
|
||||
```json
|
||||
{ "title": "Fix login bug", "description": "Login fails on Safari", "status": "todo", "assigned_to": "[email protected]" }
|
||||
```
|
||||
|
||||
### `paper`
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `title` | `string` | Document title (required). |
|
||||
| `content` | `string` | Document body in markdown (required). |
|
||||
|
||||
```json
|
||||
{ "title": "Architecture RFC", "content": "## Overview\n..." }
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Visibility
|
||||
|
||||
Particles support three visibility modes:
|
||||
|
||||
| Mode | Description |
|
||||
|------|-------------|
|
||||
| `network_all` | Visible to all network members. |
|
||||
| `custom` | Visible only to specified members (requires `members` list). |
|
||||
| `inherited` | Inherits visibility from parent particle. Used for child particles in streams. |
|
||||
|
||||
Root-level particles (streams, folders) use `network_all` or `custom`. Child particles created via `POST /streams/{id}/particles` automatically use `inherited`.
|
||||
|
||||
---
|
||||
|
||||
## Error Responses
|
||||
|
||||
All endpoints return standard HTTP status codes with plain text error bodies:
|
||||
|
||||
- **400** Bad Request — invalid input or missing required fields
|
||||
- **401** Unauthorized — missing or invalid auth token
|
||||
- **403** Forbidden — user lacks permission (not a member, not creator, not admin)
|
||||
- **404** Not Found — resource doesn't exist
|
||||
- **409** Conflict — state conflict (stream already open/closed, capacity exceeded)
|
||||
- **500** Internal Server Error
|
||||
@@ -0,0 +1,39 @@
|
||||
# Design Decisions
|
||||
|
||||
## Email as Identifier
|
||||
|
||||
Email is used as the stable identifier across services (auth, human, network).
|
||||
|
||||
**Trade-off:** Email changes are not supported. Users who want a different email create a new account.
|
||||
|
||||
**Rationale:**
|
||||
- Keeps services decoupled (no foreign keys between domains)
|
||||
- Simpler queries - no joins needed
|
||||
- Consistent across all services
|
||||
|
||||
This is a product decision, not a technical limitation.
|
||||
|
||||
## Domain Packages
|
||||
|
||||
Each domain (`internal/human/`, `internal/network/`) is isolated. Implementation details (data access, internal errors) are private to the package. Only the service interface, domain types, and exported errors are public.
|
||||
|
||||
## Handler Orchestration for Cross-Domain Concerns
|
||||
|
||||
The particle service stores `object_id` references for media/file types, not URLs. Object storage (upload URLs, signed download URLs) is handled by a separate depot service.
|
||||
|
||||
**Pattern:** Handlers orchestrate multiple services; domain services stay pure.
|
||||
|
||||
```
|
||||
Client -> Handler -> [Depot Service, Particle Service]
|
||||
```
|
||||
|
||||
- **Create media/file:** Handler calls depot for upload URL, client uploads directly to storage, then handler creates particle with `object_id`
|
||||
- **Fetch media/file:** Handler gets particle, then enriches response with signed URL from depot
|
||||
|
||||
**Rationale:**
|
||||
- Particle service focuses on hierarchy, access control, metadata
|
||||
- Depot service focuses on blob storage, signing, lifecycle
|
||||
- Handlers are already the coordination layer
|
||||
- Services remain independently testable and evolvable
|
||||
|
||||
**Same pattern applies to:** Drafts (auto-save state), real-time presence, or other concerns that don't belong in the core particle model.
|
||||
@@ -0,0 +1,110 @@
|
||||
# Particle Hierarchy & Storage Organization
|
||||
|
||||
## The Question
|
||||
|
||||
Particles form a recursive file-system-like hierarchy (streams contain folders, folders contain files, etc.). Should we align the logical hierarchy with physical storage paths?
|
||||
|
||||
```
|
||||
Logical: stream_123 -> folder_456 -> file_789
|
||||
Storage: gs://bucket/stream_123/folder_456/file_789/document.pdf
|
||||
```
|
||||
|
||||
## Two Approaches
|
||||
|
||||
### Adjacency List (Current)
|
||||
|
||||
Each particle stores a reference to its parent:
|
||||
|
||||
```sql
|
||||
CREATE TABLE particles (
|
||||
id TEXT PRIMARY KEY,
|
||||
parent_id TEXT REFERENCES particles(id),
|
||||
...
|
||||
);
|
||||
```
|
||||
|
||||
**To get ancestors:** Recursive CTE query
|
||||
**To move a particle:** Update one `parent_id`
|
||||
**Storage path:** Independent, based on `object_id`
|
||||
|
||||
### Materialized Path
|
||||
|
||||
Each particle stores its full path:
|
||||
|
||||
```sql
|
||||
CREATE TABLE particles (
|
||||
id TEXT PRIMARY KEY,
|
||||
path TEXT UNIQUE, -- '/net_abc/stream_123/folder_456'
|
||||
...
|
||||
);
|
||||
```
|
||||
|
||||
**To get ancestors:** Parse the path string
|
||||
**To get descendants:** `WHERE path LIKE '/net_abc/stream_123/%'`
|
||||
**To move a particle:** Update paths of particle AND all descendants
|
||||
|
||||
## Why Materialized Path is Tempting
|
||||
|
||||
If `path` doubles as the storage URI:
|
||||
|
||||
```
|
||||
Particle path: /net_abc/stream_123/folder_456/file_789
|
||||
Storage URI: gs://bucket/net_abc/stream_123/folder_456/file_789/video.mp4
|
||||
```
|
||||
|
||||
- Single source of truth for "where things live"
|
||||
- No recursive queries for hierarchy
|
||||
- Elegant alignment between logical and physical structure
|
||||
|
||||
## Why It Breaks Down
|
||||
|
||||
**Moves are expensive:**
|
||||
|
||||
Moving `folder_456` under a different stream requires:
|
||||
1. Update `folder_456.path`
|
||||
2. Update paths of ALL descendants (could be thousands)
|
||||
3. Move ALL storage objects to new GCS paths
|
||||
|
||||
GCS "moves" are copy + delete operations:
|
||||
- Slow and costs money
|
||||
- Links/references break during move
|
||||
- Concurrent access during move is undefined
|
||||
- Failure mid-move leaves inconsistent state
|
||||
|
||||
**Storage should be immutable:**
|
||||
|
||||
Once a file is uploaded to `gs://bucket/obj_abc123`, that path should never change. This enables:
|
||||
- Stable URLs (even if signed)
|
||||
- CDN caching
|
||||
- No coordination during particle reorganization
|
||||
|
||||
## Decision: Keep Them Decoupled
|
||||
|
||||
```
|
||||
Logical hierarchy: parent_id references (mutable, cheap to change)
|
||||
Physical storage: object_id (immutable, never moves)
|
||||
```
|
||||
|
||||
| Operation | Adjacency List | Materialized Path |
|
||||
|-----------|---------------|-------------------|
|
||||
| Move particle | O(1) - update parent_id | O(n) - update all descendant paths + move storage |
|
||||
| Get ancestors | O(depth) recursive query | O(1) parse path |
|
||||
| Get descendants | O(n) recursive query | O(1) prefix match |
|
||||
| Storage move | Not needed | Required on every move |
|
||||
|
||||
The read optimization of materialized paths doesn't justify the write complexity, especially when writes involve physical storage operations.
|
||||
|
||||
## Storage Organization
|
||||
|
||||
Objects are stored with stable IDs, optionally prefixed by network for operational convenience:
|
||||
|
||||
```
|
||||
gs://bucket/{network_id}/{object_id}/{original_filename}
|
||||
```
|
||||
|
||||
This enables:
|
||||
- Bulk operations per network (audit, delete, lifecycle policies)
|
||||
- Preserved original filename for downloads
|
||||
- No coupling to particle hierarchy
|
||||
|
||||
The particle stores `object_id` in its data. The depot service handles signed URL generation. The particle's logical position in the hierarchy is independent of where its assets physically live.
|
||||
@@ -0,0 +1,694 @@
|
||||
# Particle System & Network Capacity Implementation Plan
|
||||
|
||||
## Summary
|
||||
|
||||
Implement a unified particle system where all content types (streams, folders, media, files, text, quests, papers, AI chats) are particles with a common structure but type-specific data. Add network capacity management for billing/limiting open streams.
|
||||
|
||||
---
|
||||
|
||||
## Key Design Decisions
|
||||
|
||||
| Decision | Choice |
|
||||
|----------|--------|
|
||||
| Data model | Unified particle with `type` field + JSONB `data` |
|
||||
| Hierarchy | Arbitrary nesting (parent_id references another particle) |
|
||||
| Access control | Split: Handler checks network membership, Particle service checks particle visibility |
|
||||
| Stream membership | Auto-visible to all network members OR custom member list |
|
||||
| Open streams | Only "open" streams count against capacity |
|
||||
| Substream counting | All stream particles count (including nested) |
|
||||
| Service coupling | **Loose** - no FK constraints, particle service is independent |
|
||||
| Listing API | Unified `ListParticles(networkID, parentID, ...)` - file explorer style |
|
||||
| Sorting | Streams: `updated_at DESC` (activity), Folders: `created_at` (client sorts by type) |
|
||||
| Pagination | Bidirectional cursor for streams (chat-like); folders return all |
|
||||
| Network capacity | Stored on networks table (`open_stream_capacity`), updated via admin API |
|
||||
| Data validation | Start simple with required field validation in service code |
|
||||
|
||||
---
|
||||
|
||||
## Architecture: Access Control Split
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────┐
|
||||
│ HANDLER LAYER │
|
||||
│ 1. Extract user email from auth context │
|
||||
│ 2. Check network membership (via network service) │
|
||||
│ 3. Call particle service with (networkID, email) │
|
||||
│ 4. Transform Particle structs → Response DTOs │
|
||||
└─────────────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────────┐
|
||||
│ PARTICLE SERVICE │
|
||||
│ - Trusts that handler verified network membership │
|
||||
│ - Checks particle-level visibility (network_all vs custom) │
|
||||
│ - Filters results to only particles user can see │
|
||||
│ - Returns Particle domain structs │
|
||||
│ - NO dependency on network service │
|
||||
└─────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
**Why this split?**
|
||||
- Particle service stays decoupled from network service
|
||||
- Network membership is a cross-cutting concern (handler already knows user context)
|
||||
- Particle visibility is domain-specific (belongs in particle service)
|
||||
|
||||
---
|
||||
|
||||
## UI Data Flow Examples
|
||||
|
||||
### Mental Model: File Explorer
|
||||
|
||||
The API follows a file explorer pattern:
|
||||
- `parentID = nil` → root items of network
|
||||
- `parentID = "p_123"` → children of that particle
|
||||
- Same method works at every level
|
||||
- Response includes parent for breadcrumbs/context
|
||||
|
||||
### Example 1: User Opens App → Network Sidebar
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────┐
|
||||
│ Streams in Acme Corp │
|
||||
│ ├─ 📂 Projects (folder) │
|
||||
│ │ ├─ 💬 Website Redesign (stream, open) │
|
||||
│ │ └─ 💬 Mobile App (stream, closed) │
|
||||
│ ├─ 💬 General Chat (stream, open) │
|
||||
│ └─ 💬 Support Tickets (stream, open) │
|
||||
└─────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
**API Calls:**
|
||||
|
||||
```go
|
||||
// 1. Get root particles for selected network
|
||||
resp := particleService.ListParticles(ctx, "net_abc", nil, email, ListFilter{}, nil)
|
||||
// Returns:
|
||||
// {
|
||||
// Parent: nil, // No parent at root
|
||||
// Particles: [
|
||||
// {ID: "p_1", Type: "folder", Data: {"name": "Projects"}, ...},
|
||||
// {ID: "p_2", Type: "stream", Data: {"title": "General Chat"}, StreamStatus: "open"},
|
||||
// {ID: "p_3", Type: "stream", Data: {"title": "Support Tickets"}, StreamStatus: "open"},
|
||||
// ]
|
||||
// }
|
||||
|
||||
// 2. User clicks "Projects" folder → fetch children
|
||||
resp := particleService.ListParticles(ctx, "net_abc", ptr("p_1"), email, ListFilter{}, nil)
|
||||
// Returns:
|
||||
// {
|
||||
// Parent: {ID: "p_1", Type: "folder", Data: {"name": "Projects"}}, // For breadcrumbs
|
||||
// Particles: [
|
||||
// {ID: "p_4", Type: "stream", Data: {"title": "Website Redesign"}, StreamStatus: "open"},
|
||||
// {ID: "p_5", Type: "stream", Data: {"title": "Mobile App"}, StreamStatus: "closed"},
|
||||
// ]
|
||||
// }
|
||||
```
|
||||
|
||||
### Example 2: User Opens Stream → Chat View
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────┐
|
||||
│ 💬 Website Redesign [Members] │
|
||||
├─────────────────────────────────────────────────────┤
|
||||
│ ↑ Load older │
|
||||
│ ───────────────────────────────────────────────── │
|
||||
│ [Alice] Here's the new mockup │
|
||||
│ 📎 mockup-v2.png (media particle) │
|
||||
│ ───────────────────────────────────────────────── │
|
||||
│ [Bob] Looks great! Question about nav │
|
||||
│ ───────────────────────────────────────────────── │
|
||||
│ 📋 Update navigation colors (quest) │
|
||||
│ ───────────────────────────────────────────────── │
|
||||
│ [Type a message...] [Send] │
|
||||
└─────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
**API Calls:**
|
||||
|
||||
```go
|
||||
// 1. Initial load - most recent particles (sorted by updated_at DESC)
|
||||
resp := particleService.ListParticles(ctx, networkID, ptr("stream_123"), email, ListFilter{}, nil)
|
||||
// Returns:
|
||||
// {
|
||||
// Parent: {ID: "stream_123", Type: "stream", Data: {"title": "Website Redesign"}},
|
||||
// Particles: [newest...oldest], // Sorted by updated_at DESC
|
||||
// HasMore: true,
|
||||
// PrevCursor: {Position: "p_oldest_in_batch", Direction: "before"},
|
||||
// NextCursor: nil // At newest
|
||||
// }
|
||||
|
||||
// 2. User scrolls UP → load older messages
|
||||
resp := particleService.ListParticles(ctx, networkID, ptr("stream_123"), email, ListFilter{},
|
||||
&Cursor{Position: "p_oldest_in_batch", Direction: "before"})
|
||||
|
||||
// 3. User scrolls DOWN → load newer (after scrolling up)
|
||||
resp := particleService.ListParticles(ctx, networkID, ptr("stream_123"), email, ListFilter{},
|
||||
&Cursor{Position: "some_id", Direction: "after"})
|
||||
|
||||
// 4. User sends message
|
||||
newParticle := particleService.Create(ctx, CreateInput{
|
||||
Type: TypeText,
|
||||
NetworkID: networkID,
|
||||
ParentID: ptr("stream_123"),
|
||||
Data: json.RawMessage(`{"content": "My message"}`),
|
||||
}, email)
|
||||
// UI inserts at bottom
|
||||
```
|
||||
|
||||
### Handler Implementation
|
||||
|
||||
```go
|
||||
// Handler pseudocode
|
||||
func (h *Handler) ListParticles(w http.ResponseWriter, r *http.Request) {
|
||||
email := getAuthEmail(r.Context())
|
||||
networkID := r.URL.Query().Get("network_id")
|
||||
parentID := r.URL.Query().Get("parent_id") // Optional
|
||||
|
||||
// 1. Check network membership (handler responsibility)
|
||||
network, err := h.networkService.GetByID(ctx, networkID)
|
||||
if err != nil { return NotFound }
|
||||
|
||||
if !network.HasMember(email) && network.AdminEmail != email {
|
||||
return Forbidden("not a network member")
|
||||
}
|
||||
|
||||
// 2. Parse cursor if provided
|
||||
var cursor *particle.Cursor
|
||||
if r.URL.Query().Has("cursor") {
|
||||
cursor = parseCursor(r.URL.Query().Get("cursor"))
|
||||
}
|
||||
|
||||
// 3. Call particle service
|
||||
var parentPtr *string
|
||||
if parentID != "" {
|
||||
parentPtr = &parentID
|
||||
}
|
||||
|
||||
result, err := h.particleService.ListParticles(ctx, networkID, parentPtr, email, filter, cursor)
|
||||
if err != nil { return err }
|
||||
|
||||
// 4. Transform to response
|
||||
json.NewEncoder(w).Encode(toParticleListResponse(result))
|
||||
}
|
||||
```
|
||||
|
||||
### Response DTO structure
|
||||
|
||||
```go
|
||||
// Handler layer DTOs (in handler.go)
|
||||
type ParticleResponse struct {
|
||||
ID string `json:"id"`
|
||||
Type string `json:"type"`
|
||||
NetworkID string `json:"network_id"`
|
||||
ParentID *string `json:"parent_id,omitempty"`
|
||||
Visibility string `json:"visibility"`
|
||||
StreamStatus *string `json:"stream_status,omitempty"` // Only for streams
|
||||
Data json.RawMessage `json:"data"`
|
||||
CreatedBy string `json:"created_by"`
|
||||
UpdatedAt string `json:"updated_at"`
|
||||
CreatedAt string `json:"created_at"`
|
||||
}
|
||||
|
||||
// Transform function
|
||||
func toParticleResponse(p *particle.Particle) ParticleResponse {
|
||||
return ParticleResponse{
|
||||
ID: p.ID,
|
||||
Type: string(p.Type),
|
||||
NetworkID: p.NetworkID,
|
||||
ParentID: p.ParentID,
|
||||
Visibility: string(p.Visibility),
|
||||
StreamStatus: (*string)(p.StreamStatus),
|
||||
Data: p.Data,
|
||||
CreatedBy: p.CreatedByEmail,
|
||||
UpdatedAt: p.UpdatedAt.Format(time.RFC3339),
|
||||
CreatedAt: p.CreatedAt.Format(time.RFC3339),
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Particle Types
|
||||
|
||||
| Type | Purpose | Required Data Fields |
|
||||
|------|---------|---------------------|
|
||||
| `stream` | Temporal container (chat-like) | `title` |
|
||||
| `folder` | Structural container | `name` |
|
||||
| `media` | Images, video, audio, clips | `url`, `mime_type` |
|
||||
| `file` | Documents, PDFs, attachments | `url`, `filename` |
|
||||
| `text` | Quick text messages | `content` |
|
||||
| `quest` | Tasks/requests | `title` |
|
||||
| `paper` | Rich documents | `title` |
|
||||
| `think` | AI chat container | `title` |
|
||||
|
||||
**Note:** Quest with `assigned_to = current_user` is presented as a "request" in UI.
|
||||
|
||||
---
|
||||
|
||||
## Database Schema
|
||||
|
||||
### Migration 1: Network Capacity
|
||||
|
||||
```sql
|
||||
-- migrations/000003_network_capacity.up.sql
|
||||
ALTER TABLE networks
|
||||
ADD COLUMN open_stream_capacity INTEGER NOT NULL DEFAULT 5,
|
||||
ADD COLUMN open_stream_count INTEGER NOT NULL DEFAULT 0;
|
||||
```
|
||||
|
||||
```sql
|
||||
-- migrations/000003_network_capacity.down.sql
|
||||
ALTER TABLE networks
|
||||
DROP COLUMN open_stream_capacity,
|
||||
DROP COLUMN open_stream_count;
|
||||
```
|
||||
|
||||
### Migration 2: Particles
|
||||
|
||||
```sql
|
||||
-- migrations/000004_particles.up.sql
|
||||
|
||||
CREATE TYPE particle_type AS ENUM (
|
||||
'stream', 'folder', 'media', 'file', 'text', 'quest', 'paper', 'think'
|
||||
);
|
||||
|
||||
CREATE TYPE visibility_mode AS ENUM ('network_all', 'custom');
|
||||
|
||||
CREATE TABLE particles (
|
||||
id TEXT PRIMARY KEY,
|
||||
type particle_type NOT NULL,
|
||||
network_id TEXT NOT NULL, -- NO FK constraint, loose coupling
|
||||
parent_id TEXT, -- NO FK constraint, loose coupling
|
||||
created_by_email VARCHAR(255) NOT NULL,
|
||||
visibility visibility_mode NOT NULL DEFAULT 'network_all',
|
||||
|
||||
-- Stream-specific (NULL for non-streams)
|
||||
stream_status VARCHAR(20) CHECK (stream_status IN ('open', 'closed')),
|
||||
|
||||
-- Type-specific data
|
||||
data JSONB NOT NULL DEFAULT '{}',
|
||||
|
||||
-- Timestamps
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
|
||||
CONSTRAINT stream_status_check CHECK (
|
||||
(type = 'stream' AND stream_status IS NOT NULL) OR
|
||||
(type != 'stream' AND stream_status IS NULL)
|
||||
)
|
||||
);
|
||||
|
||||
CREATE TABLE particle_members (
|
||||
particle_id TEXT NOT NULL, -- NO FK constraint
|
||||
email VARCHAR(255) NOT NULL,
|
||||
added_at TIMESTAMPTZ DEFAULT NOW(),
|
||||
PRIMARY KEY (particle_id, email)
|
||||
);
|
||||
|
||||
-- Indexes for query performance
|
||||
CREATE INDEX idx_particles_parent_updated ON particles(parent_id, updated_at DESC);
|
||||
CREATE INDEX idx_particles_network_root ON particles(network_id, updated_at DESC) WHERE parent_id IS NULL;
|
||||
CREATE INDEX idx_particles_open_streams ON particles(network_id) WHERE type = 'stream' AND stream_status = 'open';
|
||||
CREATE INDEX idx_particle_members_email ON particle_members(email, particle_id);
|
||||
CREATE INDEX idx_particles_network_id ON particles(network_id);
|
||||
```
|
||||
|
||||
```sql
|
||||
-- migrations/000004_particles.down.sql
|
||||
DROP TABLE IF EXISTS particle_members;
|
||||
DROP TABLE IF EXISTS particles;
|
||||
DROP TYPE IF EXISTS visibility_mode;
|
||||
DROP TYPE IF EXISTS particle_type;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Service Layer Design
|
||||
|
||||
### Package Structure
|
||||
|
||||
```
|
||||
internal/particle/
|
||||
├── models.go # Particle struct, type constants, data structs
|
||||
├── errors.go # ErrNotFound, ErrCapacityExceeded, ErrAccessDenied
|
||||
├── repository.go # Database operations (internal)
|
||||
├── service.go # Public Service interface + implementation
|
||||
├── validation.go # Type-specific validation (simple required fields)
|
||||
└── service_test.go # Integration tests
|
||||
```
|
||||
|
||||
### Domain Models
|
||||
|
||||
```go
|
||||
// internal/particle/models.go
|
||||
|
||||
type ParticleType string
|
||||
|
||||
const (
|
||||
TypeStream ParticleType = "stream"
|
||||
TypeFolder ParticleType = "folder"
|
||||
TypeMedia ParticleType = "media"
|
||||
TypeFile ParticleType = "file"
|
||||
TypeText ParticleType = "text"
|
||||
TypeQuest ParticleType = "quest"
|
||||
TypePaper ParticleType = "paper"
|
||||
TypeThink ParticleType = "think"
|
||||
)
|
||||
|
||||
type VisibilityMode string
|
||||
|
||||
const (
|
||||
VisibilityNetworkAll VisibilityMode = "network_all"
|
||||
VisibilityCustom VisibilityMode = "custom"
|
||||
)
|
||||
|
||||
type StreamStatus string
|
||||
|
||||
const (
|
||||
StreamOpen StreamStatus = "open"
|
||||
StreamClosed StreamStatus = "closed"
|
||||
)
|
||||
|
||||
type Particle struct {
|
||||
ID string
|
||||
Type ParticleType
|
||||
NetworkID string
|
||||
ParentID *string
|
||||
CreatedByEmail string
|
||||
Visibility VisibilityMode
|
||||
StreamStatus *StreamStatus // Only for type=stream
|
||||
Data json.RawMessage
|
||||
UpdatedAt time.Time
|
||||
CreatedAt time.Time
|
||||
}
|
||||
|
||||
type CreateInput struct {
|
||||
Type ParticleType
|
||||
NetworkID string
|
||||
ParentID *string
|
||||
Visibility VisibilityMode
|
||||
Data json.RawMessage
|
||||
MemberEmails []string // For custom visibility
|
||||
}
|
||||
|
||||
type ListFilter struct {
|
||||
Types []ParticleType
|
||||
StreamStatus *StreamStatus
|
||||
}
|
||||
|
||||
type Cursor struct {
|
||||
Position string // particle ID or timestamp
|
||||
Direction string // "before" | "after"
|
||||
}
|
||||
|
||||
type ParticleList struct {
|
||||
Parent *Particle // The parent particle (nil if root level)
|
||||
Particles []*Particle
|
||||
HasMore bool
|
||||
NextCursor *Cursor // For loading more in same direction
|
||||
PrevCursor *Cursor // For bidirectional (streams)
|
||||
}
|
||||
```
|
||||
|
||||
### Service Interface
|
||||
|
||||
```go
|
||||
// internal/particle/service.go
|
||||
|
||||
type Service interface {
|
||||
// Core CRUD
|
||||
// Note: Caller (handler) is responsible for verifying network membership
|
||||
// Service handles particle-level visibility filtering
|
||||
|
||||
Create(ctx context.Context, input CreateInput, creatorEmail string) (*Particle, error)
|
||||
GetByID(ctx context.Context, id, requesterEmail string) (*Particle, error)
|
||||
Update(ctx context.Context, id string, data json.RawMessage, requesterEmail string) (*Particle, error)
|
||||
Delete(ctx context.Context, id, requesterEmail string) error
|
||||
|
||||
// Unified listing - file explorer style
|
||||
// - parentID = nil → root particles of network
|
||||
// - parentID = "p_123" → children of that particle
|
||||
// - Automatically filters by visibility
|
||||
// - Streams: sorted by updated_at DESC (activity-based)
|
||||
// - Folders: sorted by created_at (client can re-sort by type)
|
||||
ListParticles(ctx context.Context, networkID string, parentID *string, requesterEmail string, filter ListFilter, cursor *Cursor) (*ParticleList, error)
|
||||
|
||||
// Stream lifecycle
|
||||
OpenStream(ctx context.Context, id, requesterEmail string) error
|
||||
CloseStream(ctx context.Context, id, requesterEmail string) error
|
||||
|
||||
// Returns current open stream count for a network (for capacity check)
|
||||
GetOpenStreamCount(ctx context.Context, networkID string) (int, error)
|
||||
|
||||
// Membership (for custom visibility)
|
||||
SetVisibility(ctx context.Context, id string, mode VisibilityMode, requesterEmail string) error
|
||||
AddMembers(ctx context.Context, id string, emails []string, requesterEmail string) error
|
||||
RemoveMembers(ctx context.Context, id string, emails []string, requesterEmail string) error
|
||||
GetMembers(ctx context.Context, id string) ([]string, error)
|
||||
}
|
||||
```
|
||||
|
||||
### ListParticles Implementation Logic
|
||||
|
||||
```go
|
||||
func (s *service) ListParticles(ctx context.Context, networkID string, parentID *string, email string, filter ListFilter, cursor *Cursor) (*ParticleList, error) {
|
||||
var parent *Particle
|
||||
var sortBy string
|
||||
|
||||
// 1. Determine parent and sort strategy
|
||||
if parentID != nil {
|
||||
var err error
|
||||
parent, err = s.repo.getByID(ctx, *parentID)
|
||||
if err != nil {
|
||||
return nil, ErrNotFound
|
||||
}
|
||||
|
||||
// Check visibility access to parent
|
||||
if !s.canAccess(ctx, *parentID, email) {
|
||||
return nil, ErrAccessDenied
|
||||
}
|
||||
|
||||
// Sort based on parent type
|
||||
if parent.Type == TypeStream {
|
||||
sortBy = "updated_at DESC" // Activity-based for streams
|
||||
} else {
|
||||
sortBy = "created_at DESC" // Chronological for folders
|
||||
}
|
||||
} else {
|
||||
sortBy = "updated_at DESC" // Root level: activity-based
|
||||
}
|
||||
|
||||
// 2. Fetch particles with visibility filtering
|
||||
particles, hasMore, nextCursor, prevCursor := s.repo.listChildren(ctx, networkID, parentID, email, sortBy, filter, cursor)
|
||||
|
||||
return &ParticleList{
|
||||
Parent: parent,
|
||||
Particles: particles,
|
||||
HasMore: hasMore,
|
||||
NextCursor: nextCursor,
|
||||
PrevCursor: prevCursor,
|
||||
}, nil
|
||||
}
|
||||
```
|
||||
|
||||
### Simple Validation (Start Simple)
|
||||
|
||||
```go
|
||||
// internal/particle/validation.go
|
||||
|
||||
func validateData(t ParticleType, data json.RawMessage) error {
|
||||
switch t {
|
||||
case TypeStream, TypeQuest, TypePaper, TypeThink:
|
||||
return requireField(data, "title")
|
||||
case TypeFolder:
|
||||
return requireField(data, "name")
|
||||
case TypeMedia, TypeFile:
|
||||
return requireFields(data, "url", "mime_type")
|
||||
case TypeText:
|
||||
return requireField(data, "content")
|
||||
default:
|
||||
return ErrInvalidParticleType
|
||||
}
|
||||
}
|
||||
|
||||
func requireField(data json.RawMessage, field string) error {
|
||||
var m map[string]interface{}
|
||||
if err := json.Unmarshal(data, &m); err != nil {
|
||||
return fmt.Errorf("invalid data JSON: %w", err)
|
||||
}
|
||||
if _, ok := m[field]; !ok {
|
||||
return fmt.Errorf("missing required field: %s", field)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Particle Visibility Logic
|
||||
|
||||
The particle service handles visibility filtering. It does **not** check network membership (handler does that).
|
||||
|
||||
### For `visibility = 'network_all'`
|
||||
- All network members can see it
|
||||
- Since handler already verified network membership, service returns it
|
||||
|
||||
### For `visibility = 'custom'`
|
||||
- Only users in `particle_members` table can see it
|
||||
- Service checks `particle_members` table
|
||||
|
||||
### Inheritance Rule
|
||||
- Children can only **restrict** access, not expand
|
||||
- If parent has `custom` visibility, child must also be `custom` (or more restrictive)
|
||||
- Creating a child with `network_all` under a `custom` parent → error
|
||||
|
||||
### Access Check Query
|
||||
|
||||
```sql
|
||||
-- Check if user can access a specific particle
|
||||
-- Walks up the ancestor chain, verifies access at each level
|
||||
WITH RECURSIVE ancestors AS (
|
||||
SELECT id, parent_id, visibility
|
||||
FROM particles
|
||||
WHERE id = $1
|
||||
|
||||
UNION ALL
|
||||
|
||||
SELECT p.id, p.parent_id, p.visibility
|
||||
FROM particles p
|
||||
JOIN ancestors a ON p.id = a.parent_id
|
||||
)
|
||||
SELECT bool_and(
|
||||
CASE
|
||||
WHEN visibility = 'network_all' THEN true -- Handler already verified network membership
|
||||
WHEN visibility = 'custom' THEN (
|
||||
EXISTS (SELECT 1 FROM particle_members pm
|
||||
WHERE pm.particle_id = ancestors.id AND pm.email = $2)
|
||||
)
|
||||
END
|
||||
) AS has_access
|
||||
FROM ancestors;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Network Service Updates
|
||||
|
||||
### New Methods (no changes to coupling)
|
||||
|
||||
```go
|
||||
// In internal/network/service.go
|
||||
type Service interface {
|
||||
// ... existing methods ...
|
||||
|
||||
// Admin capacity management
|
||||
SetOpenStreamCapacity(ctx context.Context, networkID string, capacity int) error
|
||||
GetCapacityInfo(ctx context.Context, networkID string) (capacity int, current int, err error)
|
||||
|
||||
// Stream count updates (called by handler, not particle service)
|
||||
IncrementOpenStreamCount(ctx context.Context, networkID string) error
|
||||
DecrementOpenStreamCount(ctx context.Context, networkID string) error
|
||||
}
|
||||
```
|
||||
|
||||
### Capacity Enforcement (in Handler)
|
||||
|
||||
```go
|
||||
// Handler: Opening a stream
|
||||
func (h *Handler) OpenStream(w http.ResponseWriter, r *http.Request) {
|
||||
// ... auth and validation ...
|
||||
|
||||
// 1. Get particle to find its network
|
||||
particle, err := h.particleService.GetByID(ctx, particleID, email)
|
||||
|
||||
// 2. Check capacity
|
||||
capacity, current, err := h.networkService.GetCapacityInfo(ctx, particle.NetworkID)
|
||||
if current >= capacity {
|
||||
return Error("stream capacity exceeded")
|
||||
}
|
||||
|
||||
// 3. Open the stream
|
||||
err = h.particleService.OpenStream(ctx, particleID, email)
|
||||
|
||||
// 4. Increment counter
|
||||
err = h.networkService.IncrementOpenStreamCount(ctx, particle.NetworkID)
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Files to Modify
|
||||
|
||||
| File | Changes |
|
||||
|------|---------|
|
||||
| `internal/network/models.go` | Add `OpenStreamCapacity`, `OpenStreamCount` fields |
|
||||
| `internal/network/repository.go` | Add capacity CRUD methods |
|
||||
| `internal/network/service.go` | Add `SetOpenStreamCapacity`, `GetCapacityInfo`, counter methods |
|
||||
| `internal/handler/handler.go` | Wire up particle endpoints, add admin capacity endpoint |
|
||||
| `migrations/` | Add 000003 and 000004 migration files |
|
||||
|
||||
## Files to Create
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `internal/particle/models.go` | Particle struct, type constants |
|
||||
| `internal/particle/errors.go` | Domain errors |
|
||||
| `internal/particle/repository.go` | Database operations |
|
||||
| `internal/particle/service.go` | Business logic + visibility filtering |
|
||||
| `internal/particle/validation.go` | Simple required field validation |
|
||||
| `internal/particle/service_test.go` | Integration tests |
|
||||
|
||||
---
|
||||
|
||||
## Implementation Phases
|
||||
|
||||
### Phase 1: Network Capacity
|
||||
1. Create migration `000003_network_capacity`
|
||||
2. Update network models with capacity fields
|
||||
3. Update network repository with capacity methods
|
||||
4. Update network service with `SetOpenStreamCapacity`, `GetCapacityInfo`
|
||||
5. Add admin endpoint in handler
|
||||
|
||||
### Phase 2: Core Particle CRUD
|
||||
1. Create migration `000004_particles`
|
||||
2. Create particle package with models, errors
|
||||
3. Implement repository with basic CRUD
|
||||
4. Implement simple validation
|
||||
5. Write integration tests
|
||||
|
||||
### Phase 3: Particle Visibility
|
||||
1. Implement visibility filtering in list queries
|
||||
2. Implement access check for GetByID
|
||||
3. Implement membership management (AddMembers, RemoveMembers)
|
||||
4. Test visibility scenarios
|
||||
|
||||
### Phase 4: Stream Lifecycle
|
||||
1. Implement `OpenStream`/`CloseStream` in particle service
|
||||
2. Wire up capacity checks in handler
|
||||
3. Test capacity enforcement
|
||||
|
||||
### Phase 5: Unified ListParticles
|
||||
1. Implement unified `ListParticles` method
|
||||
2. Add parent-type-aware sorting (streams: updated_at, folders: created_at)
|
||||
3. Implement bidirectional cursor pagination
|
||||
4. Include parent in response for breadcrumbs
|
||||
|
||||
### Phase 6: Handler Integration
|
||||
1. Wire particle service to handler
|
||||
2. Implement all particle endpoints
|
||||
3. Add DTO transformations
|
||||
4. End-to-end testing
|
||||
|
||||
---
|
||||
|
||||
## Verification Plan
|
||||
|
||||
1. **Integration tests:** Full flow with test database (following existing pattern in `network/service_test.go`)
|
||||
2. **Manual testing:**
|
||||
- Create network with capacity 2
|
||||
- Open 2 streams → succeeds
|
||||
- Open 3rd stream → fails with capacity error
|
||||
- Close a stream → can open new one
|
||||
- Create nested particles, verify visibility inheritance
|
||||
- Add custom members, verify restricted access
|
||||
- Test handler data flow end-to-end
|
||||
@@ -0,0 +1,117 @@
|
||||
# Real-Time Architecture
|
||||
|
||||
## Overview
|
||||
|
||||
The system layers real-time delivery on top of the REST API without coupling domain logic to UI concerns. The REST API remains the source of truth; real-time events are hints that trigger client-side cache invalidation or refetches.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
┌─────────────────┐
|
||||
│ Service Layer │ Domain logic, publishes events
|
||||
└────────┬────────┘
|
||||
│ publishes
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ Event Bus │ In-process or external queue
|
||||
└────────┬────────┘
|
||||
│ subscribes
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ RealtimeService │ Routes events to connected clients
|
||||
└────────┬────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ Pusher/Ably │ Delivery to clients
|
||||
└─────────────────┘
|
||||
```
|
||||
|
||||
**Key principle**: Services emit *what happened* in the domain. Consumers decide what to do with it.
|
||||
|
||||
**Why decouple?** Domain events have multiple consumers beyond real-time delivery—analytics, audit logs, webhooks, background workers. The service layer doesn't know or care who's listening.
|
||||
|
||||
**Naming**: We call it `RealtimeService` (not "NotificationService") because it handles real-time message delivery to connected clients, not human-facing notifications like alerts or emails.
|
||||
|
||||
**Why not broadcast from handlers?** If handlers called Pusher directly after calling domain services, every handler that mutates state must remember to broadcast. This leads to duplication, inconsistency, and missed broadcasts when mutations happen outside handlers (background jobs, CLI). Events from the service layer ensure broadcasts happen automatically for any code path.
|
||||
|
||||
## Channel Strategy
|
||||
|
||||
Channels map to visibility boundaries:
|
||||
|
||||
| Channel | Audience | Use Case |
|
||||
|---------|----------|----------|
|
||||
| `network:{id}` | All network members | Particles with `network_all` visibility |
|
||||
| `particle:{id}` | Particle members | Particles with `custom` visibility |
|
||||
| `human:{email}` | Single human | Private state (seen counts, mentions) |
|
||||
|
||||
## Event Payloads
|
||||
|
||||
Keep websocket messages minimal—just enough to identify what changed:
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "particle.created",
|
||||
"particle_id": "particle_abc",
|
||||
"parent_id": "stream_xyz",
|
||||
"network_id": "network_123"
|
||||
}
|
||||
```
|
||||
|
||||
The client decides relevance ("Am I viewing this stream?") and fetches full data via REST if needed. This avoids payload duplication and staleness.
|
||||
|
||||
## Client Behavior
|
||||
|
||||
On receiving an event:
|
||||
|
||||
1. **Viewing affected context** → Fetch the new/updated particle, merge into local state
|
||||
2. **Not viewing** → Update counts (e.g., increment unseen), no fetch needed
|
||||
|
||||
This is simpler than full client-side sync and keeps REST as the authority.
|
||||
|
||||
## Event Types
|
||||
|
||||
| Event | Trigger | Broadcast To |
|
||||
|-------|---------|--------------|
|
||||
| `particle.created` | Particle created | Network or particle members |
|
||||
| `particle.updated` | Data changed | Network or particle members |
|
||||
| `particle.deleted` | Particle removed | Network or particle members |
|
||||
| `particle.acked` | Human acknowledged | Network or particle members |
|
||||
| `stream.opened` | Stream reopened | Network or particle members |
|
||||
| `stream.closed` | Stream closed | Network or particle members |
|
||||
|
||||
Note: `particle.seen` is private—no broadcast needed, client updates local state on API success.
|
||||
|
||||
## Decoupling Benefits
|
||||
|
||||
- **Service layer** stays focused on domain logic, testable without Pusher mocks
|
||||
- **Realtime service** owns routing policy, testable in isolation
|
||||
- **Adding events** doesn't require core domain changes if the action already exists
|
||||
- **Additional listeners** (analytics, audit, webhooks) attach without modifying services
|
||||
|
||||
## Adding New Real-Time Experiences
|
||||
|
||||
When UI needs a new real-time message:
|
||||
|
||||
1. Check if the domain event already exists
|
||||
2. If yes → update realtime routing only
|
||||
3. If no → add the domain event (often reveals a missing abstraction)
|
||||
|
||||
The UI informs what events matter, but implementation remains decoupled.
|
||||
|
||||
## Migration Path
|
||||
|
||||
Start in-process with a simple event bus:
|
||||
|
||||
```go
|
||||
func (s *serviceImpl) Create(...) (*Particle, error) {
|
||||
created, err := s.repo.create(ctx, p)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
s.events.Publish(ctx, ParticleCreated{Particle: created})
|
||||
return created, nil
|
||||
}
|
||||
```
|
||||
|
||||
The realtime service subscribes to `ParticleCreated` events. Other consumers (analytics, audit) subscribe independently. Later, swap the in-process event bus for a durable queue—subscribers don't change.
|
||||
Reference in New Issue
Block a user