Making portfolio browsing fast and available offline

Users should be able to move between stocks, dates and accounts without waiting for a server request each time. They should also be able to open the app and keep working without an internet connection.

The proposal is to save the data needed for every stock the user can access on their device, then use that saved data to build the pages. The server continues to serve pages during the first download and whenever newer data is being saved.

How it works today

The ingestion and nightly engine prepares financial records in the database. When a stock page needs information, the app asks the server. The server reads the relevant records, calculates the page values and sends them back to the browser.

Current architecture — the production stock pages

Ingestion + nightly engine

Imports transactions and prices. Builds holdings, lot matches and daily NAV.

Postgres database

Stores the prepared financial records and their history.

App server

Reads records for the selected stock, accounts and dates. Builds the page results.

Browser asks for page data ↑ ↓ Server sends results over the network

Browser: stock page + TanStack Query

Shows charts and tables. TanStack Query keeps recent results in memory and manages requests. A selection without a usable cached result goes back to the server.

A stock page has several requests, such as its header, price chart and transaction history. The diagram groups them together.

Keeping recently viewed results helps when someone revisits the same selection. It does not make every unvisited stock or date available locally, and it does not by itself make the app open offline.

How it will work

Keep the existing server path. Add a complete saved copy of the data needed for supported pages. Once that copy is ready, stock, date and account changes read from the device instead of making another network round trip.

Proposed architecture — download the data, then browse locally

Engine + database

Continue preparing the financial records. Make one completed data version available to read.

↓ The same completed data version supplies both paths ↓

New download publisher

Packages the user's permitted data into downloadable files.

↓ Background download to the data worker, which saves it on disk.

Existing page APIs

Build page results when local data is missing or being updated.

↓ Results sent directly to the page when it uses the server.

Both paths reach the browser. Once saving is complete, normal browsing uses the local path below.

On the user's device

IndexedDB

Browser storage on disk. Holds all downloaded stock data, history and shared information.

Data worker

Reads the selected data and runs page calculations away from the screen's main processing thread.

Existing stock page

Shows the same charts and tables. A small adapter chooses local results or server results.

The data worker also downloads updates and writes them to IndexedDB.

Service worker + saved app files

Saves the HTML, JavaScript, styles and other files needed to open the app without internet. This is separate from saving financial data.

Green boxes are new or expanded responsibilities. The local flow shows how a result is produced; requests travel from the page to the data worker.

Save everything needed for browsing

Download the full authorized stock universe, not just stocks already visited. Include historical prices, transactions, holdings, lot results, account information, exchange rates and any other inputs required by the supported views.

Keep only active work in memory

Saving the whole dataset on disk does not require loading it all into RAM. The worker reads the selected stock and shared inputs, keeps useful recent data in memory, and releases older data as needed.

The two workers have different jobs. The data worker saves and reads financial data and calculates page results. The service worker saves and serves the app's files so it can start offline. Financial calculations belong in the data worker.

What changes, and what stays

Most of the visible app can stay. The main work is changing where its data comes from and making downloads, saved data and updates reliable.

Change map — from the current code to the proposed code

Green means a responsibility changes or is added.

Charts, tables and controlsRender stock details.

KeepUse the same page results and interactions. Add download and freshness status.

Query hooksCall server functions.

Change the call destinationAsk a shared adapter. It chooses the saved data worker or the server.

Server servicesFetch records and calculate page values together.

Separate reading from calculationKeep database reads on the server. Put reusable calculations in modules both server and worker can call.

SQL queriesSelect and sometimes summarize database records.

Add an equivalent local readSupply the required prepared records, then select and summarize saved data in the worker.

Ingestion + nightly calculationsPrepare the financial records.

Keep, with an export handoffMake a completed data version available for downloading.

No complete production offline setupRecent query results are not a saved app and dataset.

Add storage and offline startupDownload manager, IndexedDB, update handling, saved app files and offline route startup.

Moving a function into a worker is straightforward when it only takes data and returns a result. A service that also queries Postgres must first be split. SQL cannot simply be moved into the browser: we must either download its prepared results or implement the equivalent selection over saved records.

The intended structure is one set of shared page calculations, with two ways to provide their inputs: server database reads and local saved-data reads. The production pages should retain their existing account, date, currency and cost-policy options.

What users see

Situation	Page behavior	What happens in the background
First visit	Pages load from the server. The user can work immediately.	Save the full dataset and app files. Show download progress.
Saved data is current	Stock, date and account changes use local data.	Check whether the server has a newer data version.
New data is available	Keep the current page visible. Use the server for fresh results while the new version downloads.	Keep the old saved version until the replacement is complete. Then switch subsequent queries to the new local version.
No internet	Open the saved app and browse the last complete dataset. Display its date.	Retry the update when the connection returns.
Storage unavailable or cleared	Use the server while online. Explain that offline use is not ready.	Offer to save again when storage is available.

A saved page can appear while the app checks for updates. Once the app learns that the saved data is out of date, it uses the server for current results. If the device is offline, the last saved version remains useful and is clearly dated.

“Ready offline” means both the app files and the complete required dataset are saved. A Home Screen shortcut alone does not do either job.

What we need to build

Define the downloadable data.List every input the stock pages need for their supported dates, accounts, currencies and cost policies. Include shared data such as the stock list and account list so the app can start without server requests.
Publish a completed version from the server.Produce a small index file listing the data files, their sizes and checksums. Give each dataset a version and a date. The files must belong to the same completed engine output. Keep access checks on the server.
Share the page calculations.Separate database reads from header, chart and transaction assembly. Call the same pure calculation functions from both the server and the data worker.
Build the saved-data layer.Download in manageable pieces, verify and save each piece in IndexedDB, and resume interrupted work. Make a new version active only when all required pieces are present. Keep memory use bounded.
Connect the existing pages.Let TanStack Query ask the local/server adapter for results. Include the data version in its cache keys. Add update status, retry and clear-saved-data controls. Prevent late replies from replacing a newer selection.
Make the real app start offline.Use the service worker to save the application files, including files for routes not yet visited. Update startup and route loaders to use saved account and stock lists when offline.

What is needed in gpm_new_engine?

The engine stays responsible for ingestion, holdings, lot matching and its nightly NAV calculations. For this proposal, the added requirement is a reliable handoff: “this complete data version is ready to read.”

The download publisher must be able to read that version without some tables changing partway through the export. This can be provided by a separate immutable export made at the end of a build, or by versioned tables that remain readable while the next build runs. The exact storage implementation needs to be chosen with the engine's build process. An ordinary “build succeeded” timestamp alone is not enough to pin later reads.

The app server then turns that completed output into downloads limited to the user's permitted accounts. Rewriting ingestion or financial formulas is not part of this performance proposal.

Where the work goes in this repository

Existing location	Change
Stock detail UI`src/views/positionDetail/`	Retain charts and tables. Add save/update status and connect the new data adapter.
Page query hooks`src/views/positionDetail/queries/fetch/`	Change query functions such as `positionHeaderQueryOptions` and `priceTradeHistoryQueryOptions` to use the local/server adapter. Keep TanStack Query.
Page services`src/server/services/positionDetail/`	Split `getPositionHeader`, `getPriceTradeHistory`, `getPositionLegHistory` and `getPositionTransactions` into server reads and shared result assembly.
Pure calculations`src/server/compute/positionDetail/`	Move `computeTradeMarkers`, `indexBenchmarkSeries`, `groupLotMatchesByPurchase` and `groupLotMatchesBySale` into shared modules. Also extract the pure totals, account grouping and formula helpers they need.
Database fetches`src/server/fetch/positionDetail/`	Keep database access on the server. Add equivalent saved-data reads or exported prepared results for the worker.
Existing local-data code`src/experiments/stockLab/{cache,cache.worker,compute,types}.ts`	Use this as a starting point for the production data worker and IndexedDB store. Replace duplicated page calculations with shared functions. Add storage recovery, updates, memory limits and coordination between tabs.
Existing export code`src/server/services/stockLab/{snapshotData,snapshot,live}.ts`	Use `captureStockSnapshot` and related code as a starting point for authorized downloads of one completed version. Add production delivery and new-version discovery.
App startup + service worker`src/routes/__root.tsx public/nav-lab-sw.js`	Make production startup work from saved data. Replace experiment-only asset handling with a build-generated list of the app files needed offline.

Suggested new locations are src/shared/portfolio/ for reusable calculations and src/data/portfolio/ for the adapter, worker and storage code. These are proposed locations, not completed changes.

Practical limits to design for

The first download takes time. Keep page requests responsive while it runs. Compress the files and limit simultaneous downloads. Do not promise a fixed download time across connections.
Browser storage can be removed. IndexedDB normally survives refreshes and browser restarts, but users or the browser can clear it. Detect that and return to the online path.
A phone may pause background work. Save progress and resume when the app becomes active again. Do not depend on downloads continuing after the app closes.
Updates need temporary space. Keep the working version while saving its replacement. Reuse unchanged files and delete old unneeded files after the switch.
Permissions still matter. Save only allowed accounts, separate different signed-in users and clear their data on logout. Define how long previously authorized data can be used offline; disconnected devices cannot receive immediate permission changes.

Test the complete flow on desktop and actual iPhones: open an unvisited stock offline, restart the browser offline, interrupt a download, install a newer data version, and recover when storage is full or has been cleared.

Initial scope: detailed stock browsing with its supported date and account selections. Category, composite and arbitrary-portfolio NAV can use the same saved-data approach later, once their required inputs and shared calculations are included.