Application Architecture

This document breaks down WiMetrix’s application architecture. We will describe the pieces that make up our architecture. We will also detail how these pieces fit in together to create make sure our web applications are fast, secure, and reliable.

The Pieces

• ApiSix: Api gateway, Ingress controller
• KeyCloak: Authentication and Authorization provider
• REST Services: Microservice based REST API network
• Frontend Applications: React based Single Page Applications (SPA)
• Mobile Apps: React native based mobile and tablet applications
• Desktop Applications: Electron based desktop applications written with web technologies

Architecture Diagram

ApiSix

Apache ApiSix is being used as an API Gateway.

ApiSix provides rich traffic management features like:

• Load Balancing
• Rate Limiting
• Authentication
• Authorization
• Logging
• Dynamic Upstream
• Circuit Breaking
• Observability
• Canary Release

ApiSix offers these features by integrating with popular 3rd party libraries. This modular design enables us to integrate existing tools and workflows into ApiSix with ease.

ApiSix can be configured through a REST API, and it also comes with a web-based dashboard.

Where ApiSix fits in

• Enables a singular pathway for cross application communication (frontend-to-backend, backend-to-backend)
• Exposes our backend services and frontend applications over the network securely
• Secure backend service calls through an Auth Provider (See: KeyCloak)
• Enable zero config load balancing
• Provide simple setups for logging, monitoring, and observability

Setting up a route with ApiSix

ApiSix routes can be created a few different methods:

• With the ApiSix Admin UI
• Through hitting the REST Api PUT /apisix/admin/routes
• Passing the route yaml config as part of the the docker compose file

ApiSix routes have a lot of builtin plugins and config available to them.

Read the official docs for a full breakdown

Example Route config using a few plugins and setting up a single upstream:

uri: /${{PREFIX}}/*
name: ${{NAME}}
plugins:
  authz-keycloak:
    _meta:
      disable: false
      filter:
        - - request_uri
          - "!"
          - ~~
          - ^/[^/]*/ping$
    client_id: apisix
    client_secret: ${{CLIENT_SECRET}}
    discovery: http://${{KEYCLOAK_IP:PORT}}/realms/wimetrix/.well-known/uma2-configuration
    lazy_load_paths: true
  cors:
    _meta:
      disable: false
    allow_credential: false
    allow_headers: "*"
    allow_methods: "*"
    allow_origins: "*"
    expose_headers: "*"
    max_age: 5
  proxy-rewrite:
    regex_uri:
      - ^/[^/]*/(.*)$
      - /$1
  serverless-pre-function:
    functions:
      - |-
        return function()
          local core = require "apisix.core";
          local token = "token";
          core.request.set_header("x-apisix-token", token);
        end
    phase: before_proxy
  traffic-split:
    rules:
      - match:
          - vars:
              - - http_apisix-label
                - "=="
                - ${{LABEL}}
        weighted_upstreams:
          - upstream:
              nodes:
                ${{UPSTREAM_IP}}:${{UPSTREAM_PORT}}: 1
              type: roundrobin
            weight: 1
upstream:
  nodes:
    - host: ${{UPSTREAM_IP}}
      port: ${{UPSTREAM_PORT}}
      weight: 1
  type: roundrobin
status: 1

KeyCloak

KeyCloak is an Identity and Access Management solution. It is based on open standards like OpenID Connect, UMA, OAuth 2.0, and SAML.

KeyCloak provides:

• Single-Sign On
• User federation
• Strong authentication
• User management
• Fine-grained authorization
• Integrations with 3rd party auth/identity providers
• Social Login

KeyCloak can be managed through an Admin UI, as well as through a REST API.

Where KeyCloak fits in

KeyCloak handles every aspect of user authentication and authorization in our architecture. All applications authenticate directly or indirectly through KeyCloak.

Auth Flow

Frontend applications authenticate directly with KeyCloak through Openid-compliant REST API interfaces.

• Users login with a familiar username/password method from the login page
• Upon successful authentication, The application receives an Access Token, a Refresh Token, and User Info for the authenticated user
• Access Token has a short default expiry time of 1 minute
• Refresh Token has an idle expiry time 30 minutes
• Refresh Token has a maximum expiry time of 5 Hours
• When Access Token expires, the application uses the Refresh Token to obtain a new one
• Refresh Token expiry reset with every refresh, allowing the user to stay logged-in when they are actively working on the frontend
• The user is logged out after Refresh Token expires (which happens after user is inactive for the duration of the token’s expiry period)
• Active user sessions are visible and can be revoked from the KeyCloak admin dashboard at any time
• Access Token is forwarded in the Authorization header as a bearer token with every call to the backend services
• The API Gateway redirects the request to KeyCloak to be authorized
• The request is forwarded only if the token is valid and the user is authorized to access the route
• ApiSix which sits between the user/client and the services and redirects to KeyCloak for auth
• Backend services are only accessible through ApiSix

User Access Management With KeyCloak

KeyCloak is an un-opinionated and general purpose tool and allows many different general purpose entities. These entities can be mixed and matched to fit specific organizational structures.

See: Access Control

Initial Setup For KeyCloak

Read here for a detailed breakdown of KeyCloak

Starting Up KeyCloak

• Deploy the KeyCloak container
• Access the admin UI (the default port is 8080)
• Create a realm called wimetrix. We will use this realm for our auth
• Setup Users, Groups, and Realm Roles. (See: User Access Management)

Make First Name Required For Users

• Go to Realm Settings > User profile > firstName
• Set Required for to Both users and admins

Allow Updating Username

• Go to Realm Settings > Login
• Go to User info settings section
• Set Edit username to true

Configure Tokens

• Go to Realm Settings > Token > Access Tokens
• Set Access Token Lifespan to 1 Minute
• Go to Realm Settings > Token > Refresh Tokens
• Set Revoke Refresh Token to false
• Go to Realm settings > Sessions > SSO Session Settings
• Set SSO Session Idle to 1 Hours
• Set SSO Session Max to 30 Days

Adding access attribute to User Profile

• Go to Realm Settings > User Profile
• Click Create attribute button
• Enter access for Attribute [Name]
• Enter Access for Display Name
• Go down to the Validations section and click + Add validator
• Select length as the Validator Type
• Set Min to 0
• Set Max to 10000
• Click Save to add the validator
• Click on Create
• The attribute should show in User Profile list

Configuring Client Scopes

Add a mapper to include access in user info and tokens

1. Go to Client scopes
2. Click on the Profile scope
3. Go to Mappers tab
4. Click on Add mapper button and select by configuration. A modal should show
5. Select User Attribute from the list in the modal and click Add. A form should open to add mapper
6. Enter access for Name
7. Select access from the User Attribute dropdown
8. Enter access for Token Claim Name
9. Set Claim JSON Type to JSON
10. Set Add to access token to true
11. Set Add to Userinfo to true
12. Click Save
13. The new mapper should show in the Mappers tab

Add a mapper to include groups in user info and tokens

1. Go to Client scopes
2. Click on the Create client scope button
3. Enter groups as the name
4. Set Type to Default
5. Set Display on consent screen to false
6. Create the mapper and select it
7. Go to Mappers tab
8. Click on Add mapper button and select by configuration. A modal should show
9. Select Group Membership from the list in the modal and click Add. A form should open to add mapper
10. Enter groups for Name
11. Enter groups for Token Claim Name
12. Set Full group path to false
13. Set Add to ID token to false
14. Set Add to access token to true
15. Set Add to lightweight access token to false
16. Set Add to Userinfo to true
17. Set Add to token introspection to true
18. Click Save
19. The new mapper should show in the Mappers tab

Add a mapper to include user_id in user info and tokens

1. Go to Client scopes
2. Click on the Create client scope button
3. Enter user_id as the name
4. Set Type to Default
5. Set Display on consent screen to false
6. Create the mapper and select it
7. Go to Mappers tab
8. Click on Add mapper button and select by configuration. A modal should show
9. Select User Property from the list in the modal and click Add. A form should open to add mapper
10. Enter user_id for Name
11. Enter id for Property
12. Enter user_id for Token Claim Name
13. Set Claim JSON Type to String
14. Set Add to ID token to false
15. Set Add to access token to true
16. Set Add to lightweight access token to false
17. Set Add to Userinfo to true
18. Set Add to token introspection to true
19. Click Save
20. The new mapper should show in the Mappers tab

Add realm roles to user info and tokens, and client roles to tokens

1. Go to the Client Scope called roles
2. Open the realm roles mapper
3. Set Add to access token to true
4. Set Add to userinfo to true
5. Open the client roles mapper
6. Set Add to access token to true
7. Set Add to token introspection to true

Create A client for the frontend applications

1. Set Client ID and Name to frontend
2. Make sure Client Authentication and Authorization Stay off
3. Keep Standard flow and Direct access grants checked in Authentication flow
4. Set Valid redirect URIs to your frontend url (Set /* in dev)
5. Set Web origins to your frontend url (Set * in dev)
6. In Client scopes, add user_id and groups with assigned type Default

Or alternatively, you can import the frontend.json file to create the client.

Create A client for mobile apps

1. Set Client ID and Name to app
2. Make sure Client Authentication and Authorization Stay off
3. Keep Standard flow and Direct access grants checked in Authentication flow
4. Set Valid redirect URIs to your frontend url (Set /* in dev)
5. Set Web origins to your frontend url (Set * in dev)
6. In Client scopes, add user_id and groups with assigned type Default
7. In Advanced tab, go to Advanced Settings and set Access Token Lifespan to 1 Minute, set Client Session Idle to 365 Day, and set Client Session Max to 365 Days.

Or alternatively, you can import the app.json

Create A client for ApiSix

1. Set Client ID and Name to apisix
2. Set Client Authentication and Authorization to true
3. Uncheck everything in Authentication flow
4. In Client scopes, add user_id and groups with assigned type Default
5. Go to the Authorization tab
6. In the settings tab:
   1. Policy Enforcement should be Enforcing
   2. Decision Strategy should be Unanimous
   3. Remote Resource Management should be off
7. In the Resources tab
   1. Delete Default Resource
   2. Running pnpm gen:keycloak will add the required resources.
8. In the Policies tab
   1. Running pnpm gen:keycloak will add the required policies.
9. In Permissions tab
   1. Running pnpm gen:keycloak will add the required permissions.

Disable default authentication actions

1. Go to Authentication > Required actions and uncheck all required actions

This Postman collection can be used to test the auth flow (Make sure to update the Variables like url and client_secret for the collection)

REST Services

The Backend applications are divided into many independent REST microservices. Each microservice has a narrow scope and can be independently deployed.

The backend services use the following tech stack:

• JavaScript: Our backend programming language of choice
• TypeScript: Type-safe JavaScript
• Node.js: JavaScript Runtime for the backend services
• Fastify: Node.js framework for the backend services

Note

The REST services are accessible only through ApiSix. Here’s how that is enforced:

1. A random token is generated and exposed as an environment variable on each server. The variable is available to both ApiSix and the backend services
2. ApiSix adds the x-apisix-token header to each request. The value of the header is set to the environment variable
3. The backend service checks the value of the header and only approves the request carrying valid token

Frontend Applications

Frontend Web applications are written in react as Single Page Applications. They access the server/db through the backend services. The frontend is decoupled from the server side logic and can be deployed independently of the backend services.

The frontend applications use the following tech stack:

• JavaScript: The programming language of the web
• TypeScript: Type-safe JavaScript
• React: Frontend library for JavaScript
• Tanstack Router: Routing framework
• Material UI: Component Library and Design System
• Apache ECharts: Chart and Visualization components
• Material React Table: Data Grid and table component and hook
• Vite: Dev Server and Bundler

Mobile Applications

Mobile and Tablet applications are developed in React Native. This enables easy cross-platform availability and code reuse with the web frontend. It also allows us to streamline our tech stack and unify on a JavaScript/TypeScript Stack. This also enables us to integrate the mobile applications tightly into the monorepo architecture.

The apps use the following tech stack:

• JavaScript: The programming language of the web
• TypeScript: Type-safe JavaScript
• React: Frontend library for JavaScript
• React Native: Native components and JS-Native bridge to translate javascript into native code
• Expo: React Native meta-framework providing builtin workflows and libraries
• React Native Paper: Component Library and Design System

Desktop Applications

We aim to avoid desktop applications, preferring instead to go with web applications. For use-cases where a desktop applications is required, we develop them with Electron.js. This enables us to use web technologies to build the application, which keeps our tech stack unified.

The desktop applications use the following tech stack:

• Uses the frontend tech stack. (See: Frontend Applications)
• Electron: Cross-platform desktop apps with web technologies
• Electron Forge: Tool to scaffold, build, and publish electron applications