# Arrowstack Integration Plan This document describes the planned path from the public, local-first FieldLens Core to a production FieldLens application running on Arrowstack. The core repository stays reusable and infrastructure-neutral. Arrowstack-specific authentication, remote persistence, tenancy, integrations, deployment configuration, and operational tooling should live in the host application or deployment layer rather than being coupled into capture logic. ## Target architecture ```text Arrowstack │ Keycloak OIDC │ ▼ ┌──────────────────────────────────────────────┐ │ FieldLens PWA │ │ │ │ React / Vite │ │ barcode scanner │ │ EAN pallet verification │ │ carton-label OCR / conformity │ │ IndexedDB │ │ │ │ local records ───────► durable sync outbox │ └──────────────────────────┬───────────────────┘ │ HTTPS ▼ ┌──────────────────────────────────────────────┐ │ FieldLens API │ │ │ │ JWT validation / RBAC │ │ sync ingestion │ │ pallet sessions │ │ stations / devices │ │ reference-data provider boundary │ │ integration adapters │ └──────────────────────────┬───────────────────┘ │ ┌─────────┴─────────┐ ▼ ▼ PostgreSQL integrations │ ┌─────────┼─────────┐ ▼ ▼ ▼ ERP reporting MCP ``` ## Design principles 1. **Offline capture remains authoritative during operation.** Scanner, EAN, and OCR workflows must continue functioning without network access. 2. **Remote persistence is additive.** A failed login refresh, unavailable API, database outage, or integration outage must not stop local capture. 3. **No infrastructure secrets reach the browser.** Database, ERP, MCP, deployment, and provider credentials remain server-side. 4. **Sync is idempotent.** Replaying an outbox item must never create duplicate server records. 5. **The core stays portable.** Arrowstack integration should wrap or extend FieldLens Core rather than fork scanner logic. 6. **Human and machine authentication stay separate.** Browser sessions use OIDC Authorization Code + PKCE; service-to-service integrations use dedicated machine identities. 7. **Production deployment stays portable.** Container images should work under Arrowstack's current application platform and a later Kubernetes deployment without application redesign. ## Repository boundary FieldLens Core should continue to contain: - barcode decoding and validation - camera/device handling - EAN pallet logic - label detection and OCR - conformity calculations - IndexedDB persistence - offline UX - provider interfaces - pure tests The Arrowstack host application should own: - Keycloak configuration - API implementation - PostgreSQL persistence - tenancy and RBAC - sync/outbox transport - station/device registration - server-side integrations - Docker/runtime configuration - deployment manifests - secrets - monitoring and audit data A likely production repository layout is: ```text fieldlens/ ├── apps/ │ ├── web/ │ │ └── consumes FieldLens Core │ └── api/ │ ├── auth/ │ ├── sync/ │ ├── scans/ │ ├── pallets/ │ ├── stations/ │ └── integrations/ ├── packages/ │ ├── contracts/ │ └── fieldlens-core/ ├── migrations/ ├── Dockerfile └── README.md ``` The core can initially be consumed from Git and later published as a package if that becomes useful. ## Phase 1 — Stable core boundary Goal: make the current browser application explicitly integration-neutral. ### Work - Keep capture records written locally first. - Preserve generated UUIDs as stable record identifiers. - Remove direct service-specific sync from scanner code. - Keep external reference lookups behind the provider registry. - Define shared serializable contracts for scans, pallet sessions, OCR observations, and conformity results. - Add schema/version information to records that will cross the network boundary. ### Acceptance criteria - All existing capture workflows work with no backend configured. - Tests, lint, and build pass. - No remote provider is required to start or complete a scan. - Public core contains no deployment credentials or account-specific integration secrets. ## Phase 2 — Durable sync outbox Goal: make browser-to-server synchronization reliable and retry-safe. The local model should evolve from a simple local/synced flag into an explicit outbox. ```text capture │ ▼ IndexedDB transaction ├── domain record └── outbox event │ ▼ connectivity available │ ▼ batch sync request │ ▼ idempotent API write │ ▼ acknowledge event IDs │ ▼ mark local events sent ``` ### Suggested outbox record ```ts type OutboxRecord = { id: string aggregateId: string eventType: string schemaVersion: number payload: unknown createdAt: string attempts: number nextAttemptAt?: string syncedAt?: string } ``` ### API shape ```http POST /api/v1/sync Authorization: Bearer Content-Type: application/json ``` Example request: ```json { "deviceId": "device-uuid", "events": [ { "id": "event-uuid", "eventType": "scan.captured", "schemaVersion": 1, "payload": {} } ] } ``` The server should use the event ID or domain record UUID as an idempotency key. ### Acceptance criteria - Offline scans queue without error. - Reconnecting automatically resumes sync. - Replaying the same batch is safe. - Partial batch failure does not lose successful acknowledgements. - Operators can see pending/synced/error state without being blocked by it. ## Phase 3 — Arrowstack authentication Goal: authenticate humans and machines without changing offline capture behavior. ### Browser authentication Use Keycloak: - Authorization Code flow - PKCE - short-lived access tokens - refresh/session handling outside scanner logic - API validates issuer, signature, expiry, and intended audience/client The PWA should be able to launch and operate locally even when an existing authenticated session cannot immediately refresh. Remote sync waits until authentication is available. ### Machine authentication Server-side integrations should use dedicated Keycloak service accounts and `client_credentials`. Examples: - FieldLens API to another Arrowstack service - Arrowstack MCP to FieldLens API - background integration workers Machine credentials must never be exposed to the PWA. ### Initial roles A minimal role model could be: ```text fieldlens-operator fieldlens-supervisor fieldlens-admin fieldlens-integration ``` Role enforcement belongs at the API boundary. ## Phase 4 — PostgreSQL data model Goal: make synchronized capture data centrally queryable and auditable. ### Core hierarchy ```text tenant └── site └── station └── device └── scan events ``` ### Suggested entities #### tenants - id - name - created_at #### sites - id - tenant_id - name - timezone - created_at #### stations - id - site_id - name - station_type - active - created_at #### devices - id - station_id - installation_id - label - last_seen_at - app_version #### scans - id - tenant_id - site_id - station_id - device_id - operator_subject - raw_value - format - duplicate - captured_at - received_at - schema_version #### pallet_sessions - id - tenant_id - site_id - station_id - device_id - operator_subject - workflow_type - target_count - started_at - completed_at - status - conforms - result_status #### pallet_observations - id - pallet_session_id - carton_index - ean - fields_json - raw_text - confidence - captured_at #### pallet_exceptions - id - pallet_session_id - observation_id - field - expected_value - actual_value - exception_type ### Data ownership Tenant/site/station identity should be derived from trusted server-side registration and token context where possible, rather than accepting arbitrary tenant identifiers from the browser. ## Phase 5 — Device and station registration Goal: make a PWA installation identifiable without turning device identity into a secret. On first run: 1. Generate a persistent installation UUID. 2. Store it in IndexedDB. 3. After login, register it with the API. 4. An authorized user assigns or confirms its station. 5. The API returns non-secret station metadata. 6. Every synced event includes the installation UUID. A reset browser/device can register as a new installation without affecting historical records. ## Phase 6 — Reference-data providers Goal: move environment-specific expected-label lookups behind server-side adapters. FieldLens Core already exposes a provider boundary. The Arrowstack host can supply a provider backed by: - PostgreSQL - ERP API - sorter/packhouse database - Google Sheets - another Arrowstack service - a cached reference dataset Preferred production flow: ```text FieldLens Core │ ▼ host provider │ ▼ FieldLens API │ ├── cached reference data ├── ERP adapter └── other server-side provider ``` The browser should not need provider credentials. Reference lookup failure must degrade to local OCR/conformity rather than block the workflow. ## Phase 7 — API surface Initial endpoints: ```http GET /api/health/live GET /api/health/ready POST /api/v1/sync GET /api/v1/me GET /api/v1/stations POST /api/v1/devices/register GET /api/v1/scans GET /api/v1/pallet-sessions/:id GET /api/v1/reference/labels/:ean ``` Administrative write endpoints should be added only when needed. ## Phase 8 — Containerisation and Arrowstack deployment Goal: make the application deployable as a normal Arrowstack workload. ### Web Build the Vite application into static assets and serve them from a small web image or through the API image if a single-container deployment is preferred initially. ### API Container requirements: - non-root runtime user - health endpoints - graceful shutdown - configuration through environment variables - no baked-in secrets - structured logs - database migrations as an explicit deployment step ### Database Provision PostgreSQL through Arrowstack's tenant/database tooling. The runtime database role should have only the permissions needed by the FieldLens database. Schema migrations should use a migration role rather than requiring broad privileges from the normal API runtime identity. ### Deployment stages 1. Build image. 2. Run tests. 3. Run migrations. 4. Deploy API. 5. Verify readiness. 6. Deploy web. 7. Run a smoke sync from a test device. 8. Promote/retain the healthy revision. The first production target can use Arrowstack's current application deployment platform. The container contract should stay compatible with a later Kubernetes/K3s deployment. ## Phase 9 — Observability and audit Capture at least: - API request/error rate - sync batch success/failure - pending client outbox count where available - database health - active installations - last-seen station/device timestamps - application version - reference-provider latency/failures Do not log: - access tokens - authorization headers - Keycloak client secrets - database URLs with passwords - provider credentials - complete environment dumps Audit-sensitive actions such as station reassignment, administrative corrections, and integration changes should have actor, timestamp, and before/after metadata. ## Phase 10 — Arrowstack MCP integration FieldLens can later expose a narrow machine-facing API or MCP surface for operational tooling. Useful read operations: - list active stations - summarize recent scans - show unsynced/error conditions - fetch pallet conformity results - list recent exceptions Possible controlled write operations: - register/disable a station - update reference data - acknowledge or annotate an exception Machine access should use a dedicated Keycloak client and explicit FieldLens roles. MCP access must not imply unrestricted database access. ## Delivery sequence Recommended implementation order: ```text 1. shared contracts 2. IndexedDB outbox 3. sync client 4. API skeleton + health 5. PostgreSQL migrations 6. idempotent /sync endpoint 7. Keycloak browser auth 8. device/station registration 9. reference provider API 10. Arrowstack container deployment 11. admin/operations views 12. MCP integration ``` This sequence gets durable central data working before adding secondary integrations or dashboards. ## First production milestone The first Arrowstack-backed release is complete when: - the PWA installs and works offline - scans and pallet sessions persist locally first - users authenticate through Keycloak when online - queued data syncs idempotently to PostgreSQL - a device is associated with a station - the API enforces tenant/station access - reference lookups can be supplied by the API but are optional - no server credentials are present in browser bundles - application images deploy through Arrowstack - health checks, logs, and database migrations are operational - CI runs tests, lint, and build before deployment At that point FieldLens is no longer a standalone scanner with optional integrations; it is a local-first Arrowstack application whose capture path remains resilient when the platform or network is unavailable.