AsTeRICS

# Data storage

This chapter is about the data storage concept used in AsTeRICS Grid, meaning where and how user configuration and application data is stored.

  1. Introduction
  2. Abstraction layers
  3. One database per user
  4. Authentication layer
  5. Inspecting locally saved data

Back to Overview

# Introduction

Since AsTeRICS Grid is designed to work offline, all configuration and user data have to be accessible without internet connection. There are the following possibilities for providing a data storage for an offline web-application:

  1. Any external locally running storage service, e.g. any locally running database like MySQL or MariaDB. This possibility has the downside that users would have to install additional Software in order to use AsTeRICS Grid.
  2. LocalStorage (opens new window), a browser internal storage which is capable to store key-value pairs of string values. However the amount of data is limited to about 5-10MB (opens new window) in many browsers which is often too little to store all grids in AsTeRICS Grid.
  3. IndexedDB (opens new window), a browser internal storage for big amounts of structured data

AsTeRICS Grid uses LocalStorage for temporary data which should not synchronized with the cloud (e.g. hashed encryption password) and IndexedDB for the majority of configuration like e.g. data of stored grids. For online users all data that is stored in IndexedDB is synchronized with the cloud, for offline users (see Users) data is only stored locally on the device.

For accessing IndexedDB in a more comfortable manner and making synchronization with a remote CouchDB (opens new window) possible, the Javascript library PouchDB (opens new window) is used.

# Data storage abstraction layers

All Javascript modules regarding storage of data in AsTeRICS Grid can be found in the folder src/js/service/data (opens new window). Figure 1 shows the relevant files and their relationships:

Concept of data storage layers

Figure 1: Concept of data storage layers

These are the responsibilities and functions of the different modules:

  1. dataService.js: provides access to data objects on an application based abstraction level; implements methods that make it possible to get and save the objects described in chapter Data model, e.g. GridData objects.
  2. databaseService.js: implements CRUD (opens new window) methods for generic data model objects, methods for switching databases (users) and initializes databases with default data. This layer also does encryption and decryption of data using filterService.js. This level of abstraction is still quite independent of PouchDB (opens new window) or IndexedDB (opens new window) and theoretically could also use a different backend.
  3. pouchDbService.js: performs queries and actions on a PouchDB instance actually accessing the IndexedDB or CouchDB databases. It also caches queries for performance optimizations and broadcasts events if updates from the remote database are recognized.
  4. pouchDBAdapter.js: manages which actual database should be used, a local PouchDB instance or a remote CouchDB. Also sets up synchronization between both.
  5. undoService.js: is used by grid.js in order to pass updates on grids to dataService.js while keeping track of changes and providing an undo and redo functionality
  6. filterService.js: provides methods in order to convert objects used in the application to (encrypted) objects that should be saved to database and vice versa. If an object with an outdated data model version passes these methods, additional filter functions are inserted in order to upgrade the data model.
  7. encryptionService.js: provides methods for AES encryption and decryption and hasing of data objects and strings using the sjcl (opens new window) library
  8. localStorageService.js: accesses the browser internal LocalStorage (opens new window) for saving data like the last active user or hashed user passwords - in general data which should not be synchronized to cloud or aren’t suitable for a user’s database.

# One database per user

AsTeRICS Grid implements the idea of one database per user (opens new window). For each user a new database is created. Offline users have an own IndexedDB database within the browser and online users have both a local IndexedDB database and an online CouchDB database which are kept in sync by replication features (opens new window) of the PouchDB library. Advantages of this approach are:

  • access rights are easy to manage, each user can access their own database and nothing else
  • it’s easy to delete all data of a user, just delete their database
  • replication and synchronizing is simple, the whole user database can be replicated on the local device

# Authentication layer

The one-database-per-user approach needs a layer for managing user accounts and the corresponding CouchDB user databases (only for online users). For AsTeRICS Grid the framework superlogin (opens new window) is used. It provides an API in order to register and login users and creates the corresponding CouchDB databases in the background. However the original superlogin project (opens new window) seems to be no longer maintained and therefore AsTeRICS Grid uses a more up-to-date fork (opens new window) of it.

The file superlogin/start.js (opens new window) starts superlogin and contains it’s configuration (= server side). The file loginService.js (opens new window) is the client-side counterpart which uses the library superlogin-client (opens new window) in order to connect to and make use of the superlogin server.

In order to start superlogin a CouchDB instance must be running on the same machine and the correct CouchDB parameters have to be inserted in start.js:43 (opens new window). Then use npm run start-superlogin-dev or npm run start-superlogin-prod in order to start superlogin in development or production mode.

# Inspecting locally saved data

For examining the locally saved data of AsTeRICS Grid do the following (e.g. in Firefox or Chrome browser):

  1. open developer tools (Ctrl + Shift + I or right click -> Inspect)
  2. go to Application (Chrome) or Web-Storage (Firefox)
  3. open LocalStorage or IndexedDB to inspect the data saved in these storages

← Previous Chapter Next Chapter →

Back to Overview

← 07 I18n →