fieldlens-deploy/packages/fieldlens-core/ARROWSTACK_PLAN.md

Arrowstack Integration Plan

This document describes the planned path from the public, local-first FieldLens Core to a production FieldLens application running on Arrowstack.

The core repository stays reusable and infrastructure-neutral. Arrowstack-specific authentication, remote persistence, tenancy, integrations, deployment configuration, and operational tooling should live in the host application or deployment layer rather than being coupled into capture logic.

Target architecture

                         Arrowstack
                            │
                       Keycloak OIDC
                            │
                            ▼
┌──────────────────────────────────────────────┐
│                FieldLens PWA                 │
│                                              │
│ React / Vite                                 │
│ barcode scanner                              │
│ EAN pallet verification                      │
│ carton-label OCR / conformity                │
│ IndexedDB                                    │
│                                              │
│ local records ───────► durable sync outbox   │
└──────────────────────────┬───────────────────┘
                           │ HTTPS
                           ▼
┌──────────────────────────────────────────────┐
│               FieldLens API                  │
│                                              │
│ JWT validation / RBAC                        │
│ sync ingestion                               │
│ pallet sessions                              │
│ stations / devices                           │
│ reference-data provider boundary             │
│ integration adapters                         │
└──────────────────────────┬───────────────────┘
                           │
                 ┌─────────┴─────────┐
                 ▼                   ▼
             PostgreSQL          integrations
                                    │
                          ┌─────────┼─────────┐
                          ▼         ▼         ▼
                         ERP     reporting    MCP

Design principles

1. Offline capture remains authoritative during operation. Scanner, EAN, and OCR workflows must continue functioning without network access.
2. Remote persistence is additive. A failed login refresh, unavailable API, database outage, or integration outage must not stop local capture.
3. No infrastructure secrets reach the browser. Database, ERP, MCP, deployment, and provider credentials remain server-side.
4. Sync is idempotent. Replaying an outbox item must never create duplicate server records.
5. The core stays portable. Arrowstack integration should wrap or extend FieldLens Core rather than fork scanner logic.
6. Human and machine authentication stay separate. Browser sessions use OIDC Authorization Code + PKCE; service-to-service integrations use dedicated machine identities.
7. Production deployment stays portable. Container images should work under Arrowstack's current application platform and a later Kubernetes deployment without application redesign.

Repository boundary

FieldLens Core should continue to contain:

• barcode decoding and validation
• camera/device handling
• EAN pallet logic
• label detection and OCR
• conformity calculations
• IndexedDB persistence
• offline UX
• provider interfaces
• pure tests

The Arrowstack host application should own:

• Keycloak configuration
• API implementation
• PostgreSQL persistence
• tenancy and RBAC
• sync/outbox transport
• station/device registration
• server-side integrations
• Docker/runtime configuration
• deployment manifests
• secrets
• monitoring and audit data

A likely production repository layout is:

fieldlens/
├── apps/
│   ├── web/
│   │   └── consumes FieldLens Core
│   └── api/
│       ├── auth/
│       ├── sync/
│       ├── scans/
│       ├── pallets/
│       ├── stations/
│       └── integrations/
├── packages/
│   ├── contracts/
│   └── fieldlens-core/
├── migrations/
├── Dockerfile
└── README.md

The core can initially be consumed from Git and later published as a package if that becomes useful.

Phase 1 — Stable core boundary

Goal: make the current browser application explicitly integration-neutral.

Work

• Keep capture records written locally first.
• Preserve generated UUIDs as stable record identifiers.
• Remove direct service-specific sync from scanner code.
• Keep external reference lookups behind the provider registry.
• Define shared serializable contracts for scans, pallet sessions, OCR observations, and conformity results.
• Add schema/version information to records that will cross the network boundary.

Acceptance criteria

• All existing capture workflows work with no backend configured.
• Tests, lint, and build pass.
• No remote provider is required to start or complete a scan.
• Public core contains no deployment credentials or account-specific integration secrets.

Phase 2 — Durable sync outbox

Goal: make browser-to-server synchronization reliable and retry-safe.

The local model should evolve from a simple local/synced flag into an explicit outbox.

capture
   │
   ▼
IndexedDB transaction
   ├── domain record
   └── outbox event
            │
            ▼
      connectivity available
            │
            ▼
       batch sync request
            │
            ▼
      idempotent API write
            │
            ▼
     acknowledge event IDs
            │
            ▼
      mark local events sent

Suggested outbox record

type OutboxRecord = {
  id: string
  aggregateId: string
  eventType: string
  schemaVersion: number
  payload: unknown
  createdAt: string
  attempts: number
  nextAttemptAt?: string
  syncedAt?: string
}

API shape

POST /api/v1/sync
Authorization: Bearer <access-token>
Content-Type: application/json

Example request:

{
  "deviceId": "device-uuid",
  "events": [
    {
      "id": "event-uuid",
      "eventType": "scan.captured",
      "schemaVersion": 1,
      "payload": {}
    }
  ]
}

The server should use the event ID or domain record UUID as an idempotency key.

Acceptance criteria

• Offline scans queue without error.
• Reconnecting automatically resumes sync.
• Replaying the same batch is safe.
• Partial batch failure does not lose successful acknowledgements.
• Operators can see pending/synced/error state without being blocked by it.

Phase 3 — Arrowstack authentication

Goal: authenticate humans and machines without changing offline capture behavior.

Browser authentication

Use Keycloak:

• Authorization Code flow
• PKCE
• short-lived access tokens
• refresh/session handling outside scanner logic
• API validates issuer, signature, expiry, and intended audience/client

The PWA should be able to launch and operate locally even when an existing authenticated session cannot immediately refresh. Remote sync waits until authentication is available.

Machine authentication

Server-side integrations should use dedicated Keycloak service accounts and client_credentials.

Examples:

• FieldLens API to another Arrowstack service
• Arrowstack MCP to FieldLens API
• background integration workers

Machine credentials must never be exposed to the PWA.

Initial roles

A minimal role model could be:

fieldlens-operator
fieldlens-supervisor
fieldlens-admin
fieldlens-integration

Role enforcement belongs at the API boundary.

Phase 4 — PostgreSQL data model

Goal: make synchronized capture data centrally queryable and auditable.

Core hierarchy

tenant
 └── site
      └── station
           └── device
                └── scan events

Suggested entities

tenants

• id
• name
• created_at

sites

• id
• tenant_id
• name
• timezone
• created_at

stations

• id
• site_id
• name
• station_type
• active
• created_at

devices

• id
• station_id
• installation_id
• label
• last_seen_at
• app_version

scans

• id
• tenant_id
• site_id
• station_id
• device_id
• operator_subject
• raw_value
• format
• duplicate
• captured_at
• received_at
• schema_version

pallet_sessions

• id
• tenant_id
• site_id
• station_id
• device_id
• operator_subject
• workflow_type
• target_count
• started_at
• completed_at
• status
• conforms
• result_status

pallet_observations

• id
• pallet_session_id
• carton_index
• ean
• fields_json
• raw_text
• confidence
• captured_at

pallet_exceptions

• id
• pallet_session_id
• observation_id
• field
• expected_value
• actual_value
• exception_type

Data ownership

Tenant/site/station identity should be derived from trusted server-side registration and token context where possible, rather than accepting arbitrary tenant identifiers from the browser.

Phase 5 — Device and station registration

Goal: make a PWA installation identifiable without turning device identity into a secret.

On first run:

1. Generate a persistent installation UUID.
2. Store it in IndexedDB.
3. After login, register it with the API.
4. An authorized user assigns or confirms its station.
5. The API returns non-secret station metadata.
6. Every synced event includes the installation UUID.

A reset browser/device can register as a new installation without affecting historical records.

Phase 6 — Reference-data providers

Goal: move environment-specific expected-label lookups behind server-side adapters.

FieldLens Core already exposes a provider boundary. The Arrowstack host can supply a provider backed by:

• PostgreSQL
• ERP API
• sorter/packhouse database
• Google Sheets
• another Arrowstack service
• a cached reference dataset

Preferred production flow:

FieldLens Core
     │
     ▼
host provider
     │
     ▼
FieldLens API
     │
     ├── cached reference data
     ├── ERP adapter
     └── other server-side provider

The browser should not need provider credentials.

Reference lookup failure must degrade to local OCR/conformity rather than block the workflow.

Phase 7 — API surface

Initial endpoints:

GET  /api/health/live
GET  /api/health/ready

POST /api/v1/sync

GET  /api/v1/me
GET  /api/v1/stations
POST /api/v1/devices/register

GET  /api/v1/scans
GET  /api/v1/pallet-sessions/:id

GET  /api/v1/reference/labels/:ean

Administrative write endpoints should be added only when needed.

Phase 8 — Containerisation and Arrowstack deployment

Goal: make the application deployable as a normal Arrowstack workload.

Web

Build the Vite application into static assets and serve them from a small web image or through the API image if a single-container deployment is preferred initially.

API

Container requirements:

• non-root runtime user
• health endpoints
• graceful shutdown
• configuration through environment variables
• no baked-in secrets
• structured logs
• database migrations as an explicit deployment step

Database

Provision PostgreSQL through Arrowstack's tenant/database tooling.

The runtime database role should have only the permissions needed by the FieldLens database. Schema migrations should use a migration role rather than requiring broad privileges from the normal API runtime identity.

Deployment stages

1. Build image.
2. Run tests.
3. Run migrations.
4. Deploy API.
5. Verify readiness.
6. Deploy web.
7. Run a smoke sync from a test device.
8. Promote/retain the healthy revision.

The first production target can use Arrowstack's current application deployment platform. The container contract should stay compatible with a later Kubernetes/K3s deployment.

Phase 9 — Observability and audit

Capture at least:

• API request/error rate
• sync batch success/failure
• pending client outbox count where available
• database health
• active installations
• last-seen station/device timestamps
• application version
• reference-provider latency/failures

Do not log:

• access tokens
• authorization headers
• Keycloak client secrets
• database URLs with passwords
• provider credentials
• complete environment dumps

Audit-sensitive actions such as station reassignment, administrative corrections, and integration changes should have actor, timestamp, and before/after metadata.

Phase 10 — Arrowstack MCP integration

FieldLens can later expose a narrow machine-facing API or MCP surface for operational tooling.

Useful read operations:

• list active stations
• summarize recent scans
• show unsynced/error conditions
• fetch pallet conformity results
• list recent exceptions

Possible controlled write operations:

• register/disable a station
• update reference data
• acknowledge or annotate an exception

Machine access should use a dedicated Keycloak client and explicit FieldLens roles. MCP access must not imply unrestricted database access.

Delivery sequence

Recommended implementation order:

1. shared contracts
2. IndexedDB outbox
3. sync client
4. API skeleton + health
5. PostgreSQL migrations
6. idempotent /sync endpoint
7. Keycloak browser auth
8. device/station registration
9. reference provider API
10. Arrowstack container deployment
11. admin/operations views
12. MCP integration

This sequence gets durable central data working before adding secondary integrations or dashboards.

First production milestone

The first Arrowstack-backed release is complete when:

• the PWA installs and works offline
• scans and pallet sessions persist locally first
• users authenticate through Keycloak when online
• queued data syncs idempotently to PostgreSQL
• a device is associated with a station
• the API enforces tenant/station access
• reference lookups can be supplied by the API but are optional
• no server credentials are present in browser bundles
• application images deploy through Arrowstack
• health checks, logs, and database migrations are operational
• CI runs tests, lint, and build before deployment

At that point FieldLens is no longer a standalone scanner with optional integrations; it is a local-first Arrowstack application whose capture path remains resilient when the platform or network is unavailable.