Product shows realtime data for a multiplayer experience
Optimistic updates make product interactions feel instantaneous
If the client goes offline, it must either re-bootstrap, or ask for all events since x change
We should keep all data synced real-time on the client, not only particles. For example, billing, networks, members.
Not doing
No offline mode. Client cannot write data without being connected, and there will be no local cache to load data offline on boot.
However, client should receive replay of all transactions from the sync server on reconnect.
Components
There are a few core components for building the sync engine:
Backend services (e.g. particle service) appends changes to the transaction log. No need for pubsub to another worker which then would add to transaction log.
Sync server (web socket) processes new transaction log events and forwards to the appropriate subscribers / clients
Client side object pool
Client side object graph to make it easier for the view layer
Flow
Client fetches snapshot data on load and receives a sequence number of the snapshot
Client connects to sync server via web sockets and passes in the sequence number to receive all missed events during snapshot
Client keeps in-memory data up to date with snapshot data
For writes:
On user interaction, the local store is updated as an optimistic update
API request is sent to orion server
On failure, revert + toast
On success, do nothing
Client state management
On the client side, we want to either use MobX or Zustand. All data should reside in a client-side object pool.
And then particular views can construct derived state based on the object pool.
## Requirements
- Product shows realtime data for a multiplayer experience
- Optimistic updates make product interactions feel instantaneous
- If the client goes offline, it must either re-bootstrap, or ask for all events since x change
- We should keep all data synced real-time on the client, not only particles. For example, billing, networks, members.
## Not doing
No offline mode. Client cannot write data without being connected, and there will be no local cache to load data offline on boot.
However, client should receive replay of all transactions from the sync server on reconnect.
## Components
There are a few core components for building the sync engine:
- Backend services (e.g. particle service) appends changes to the transaction log. No need for pubsub to another worker which then would add to transaction log.
- Sync server (web socket) processes new transaction log events and forwards to the appropriate subscribers / clients
- Client side object pool
- Client side object graph to make it easier for the view layer
## Flow
- Client fetches snapshot data on load and receives a sequence number of the snapshot
- Client connects to sync server via web sockets and passes in the sequence number to receive all missed events during snapshot
- Client keeps in-memory data up to date with snapshot data
For writes:
- On user interaction, the local store is updated as an optimistic update
- API request is sent to orion server
- On failure, revert + toast
- On success, do nothing
## Client state management
On the client side, we want to either use MobX or Zustand. All data should reside in a client-side object pool.
And then particular views can construct derived state based on the object pool.
Note to self: many of these sync engine abstractions end up being overly complex and create leaky abstractions.
It's easier to build our own system that we own and understand. Start simple and expand over time.
Note to self: many of these sync engine abstractions end up being overly complex and create leaky abstractions.
It's easier to build our own system that we own and understand. Start simple and expand over time.
As advised here, there are various loopholes with local-first.
We will not be taking on this work for now:
Local first is equivalent to dealing with distributed systems
Migrations are harder because the client and server have to be updated
More complexity pushed onto the client, and we rather write more complexity in our backend
We just don't need offline-first at this stage in the game. We can still be more performant than M365 with a client/server model. Having a lean client and performant server accomplishes the same in terms of performance
We can still do optimistic updates
We barely have that many entities so this abstraction is not warranted
We barely have many mutation endpoints, and they can be maintained easily with a few functions
We can always build it later after we have built a business.
Instead, for now, we will focus on creating a real-time experience that is more conventional similar to the following:
Individual API calls depending on the view and data required for rendering (e.g. structured data and caching with react query)
User takes action
Optimistic update
Mutation API call
Invalid cache OR refetch relevant data to keep state up to date
Websocket connection to hear about multi-player updates and update state
On network offline, the application will be unusable, and on reconnect, we can invalidate all caches to trigger refetches
To keep it simpler, we can even group data together in react query; for example, one react query hook can fetch 3 endpoints but represent one key, and invalidations would refetch entire sets of data. Particular views can reuse this cached data and filter appropriately.
As advised [here](https://rxdb.info/articles/local-first-future.html#challenges-and-limitations-of-local-first), there are various loopholes with local-first.
We will not be taking on this work for now:
- Local first is equivalent to dealing with distributed systems
- Migrations are harder because the client and server have to be updated
- More complexity pushed onto the client, and we rather write more complexity in our backend
- We just don't need offline-first at this stage in the game. We can still be more performant than M365 with a client/server model. Having a lean client and performant server accomplishes the same in terms of performance
- We can still do optimistic updates
- We barely have that many entities so this abstraction is not warranted
- We barely have many mutation endpoints, and they can be maintained easily with a few functions
- _We can always build it later after we have built a business._
Instead, for now, we will focus on creating a real-time experience that is more conventional similar to the following:
- Individual API calls depending on the view and data required for rendering (e.g. structured data and caching with react query)
- User takes action
- Optimistic update
- Mutation API call
- Invalid cache OR refetch relevant data to keep state up to date
- Websocket connection to hear about multi-player updates and update state
- On network offline, the application will be unusable, and on reconnect, we can invalidate all caches to trigger refetches
To keep it simpler, we can even group data together in react query; for example, one react query hook can fetch 3 endpoints but represent one key, and invalidations would refetch entire sets of data. Particular views can reuse this cached data and filter appropriately.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Requirements
Not doing
No offline mode. Client cannot write data without being connected, and there will be no local cache to load data offline on boot.
However, client should receive replay of all transactions from the sync server on reconnect.
Components
There are a few core components for building the sync engine:
Flow
For writes:
Client state management
On the client side, we want to either use MobX or Zustand. All data should reside in a client-side object pool.
And then particular views can construct derived state based on the object pool.
Note to self: many of these sync engine abstractions end up being overly complex and create leaky abstractions.
It's easier to build our own system that we own and understand. Start simple and expand over time.
As advised here, there are various loopholes with local-first.
We will not be taking on this work for now:
Instead, for now, we will focus on creating a real-time experience that is more conventional similar to the following:
To keep it simpler, we can even group data together in react query; for example, one react query hook can fetch 3 endpoints but represent one key, and invalidations would refetch entire sets of data. Particular views can reuse this cached data and filter appropriately.
We will setup firestore with our gcp project to launch a real-time multiplayer app sooner.