A local-first, offline-capable web sync engine implementation with CRDT-based synchronization. The project provides a complete solution for data synchronization between multiple devices while maintaining data consistency and offline capabilities.
The project is structured as a monorepo with three main components:
-
Client
- Web-based UI implementation
- Local-first storage using IndexedDB (dexie)
- Background sync using Web Workers
- Client-side schema migrations
- Offline-first data management
-
Server
- REST API for data synchronization
- JWT-based authentication and authorization
- Byte-level data storage (client-agnostic)
- Workspace management and backup
- Token-based access control
-
Shared Packages
- Schema definitions
- Type definitions
- Shared utilities
- Migration utilities
- Local-First Architecture: Data is primarily stored and managed locally, with server acting as sync + backup
- CRDT-Based Sync: Uses loro-crdt for conflict-free data synchronization
- Offline Support: Full offline capability with background sync
- Multi-Device Access: Share workspaces across devices using access tokens
- Type-Safe Migrations: Version-controlled schema migrations
- Secure Authentication: JWT-based auth with master/access token system
- Client stores and syncs data using CRDT (Conflict-free Replicated Data Type)
- Server is client-agnostic, storing only byte-level data
- Master client controls workspace access through token generation
- Schema migrations are performed client-side
- Background syncing handled by Web Workers
- No server-side querying - all data operations happen client-side
- Client creates local workspace
- Server stores workspace and designates client as "master"
- Master client receives master token
- Master client generates access tokens for other devices
- Access tokens can be shared via links
- Server handles token verification and authorization
Client (Local Storage) <-> Web Worker (Background Sync) <-> Server (Byte Storage)
- UI reads directly from local storage
- Sync operations happen in background
- Server stores encoded CRDT data as bytes
- Migrations happen during client initialization
- CRDT: loro-crdt for data synchronization
- Storage: IndexedDB (client), Database (server)
- Authentication: JWT tokens
- Background Processing: Web Workers
- UI & Routing
- react - UI framework
- @tanstack/react-router - Type-safe routing
- Storage & Sync
- loro-crdt - CRDT implementation
- dexie - IndexedDB wrapper
- dexie-react-hooks - React hooks for Dexie
- Core
- effect - Functional programming toolkit
- @effect/platform-browser - Browser-specific Effect utilities
- Database
- drizzle-orm - TypeScript ORM
- @effect/sql - SQL integration for Effect
- @effect/sql-pg - PostgreSQL driver
- pg - PostgreSQL client
- Authentication
- jsonwebtoken - JWT implementation
- Core
- effect - Functional programming toolkit
- @effect/platform-node - Node.js-specific Effect utilities
- turbo - Monorepo build system
- vite - Frontend build tool
- typescript - Type system
This is a functional implementation of a web sync engine, optimized for single-user scenarios (not real-time collaboration). The focus is on providing reliable data synchronization while maintaining offline capabilities.