Skip to content
WiMetrix Docs

Cloud Infrastructure

This document provides a comprehensive overview of the architecture, design, and implementation of our cloud infrastructure. It is intended for use by technical stakeholders and system administrators.

Cloud Workflow

Our infrastructure is provisioned and managed using Terraform, an Infrastructure as Code (IaC) tool. Terraform enables declarative configuration using the HashiCorp Configuration Language (HCL). All infrastructure configurations are maintained in a single GitHub repository: terraform-infra-config.

Terraform Configuration

Section titled “Terraform Configuration”

Terraform Repository

The repository includes configurations for the following services:

  • Kubernetes (EKS)
  • Amazon ECR
  • Load Balancer
  • EC2
  • RDS
  • VPC
  • EKS Node Group
  • S3 Gateway
  • ACM (SSL Certificate Management)
  • Internet Gateway
  • NAT Gateway
  • ECR and Docker Registry Endpoints

We follow a modular structure instead of using wrapper modules. This modular approach ensures clarity, maintainability, and reusability by separating each service into its own folder. Environment-specific variables are centrally managed in environments/prod/terraform.tfvars.

Terraform tfvars

Learn more about the modular approach here.

Refer to the repository’s README file for usage instructions:

Terraform README

VPC (Virtual Private Cloud)

Section titled “VPC (Virtual Private Cloud)”

The VPC forms the foundation of the cloud infrastructure. It creates an isolated network within the AWS region, enabling internal resource communication while controlling external access.

Key components:

  • One Public Subnet
  • One Private Subnet per Availability Zone
  • One Database Subnet per Availability Zone
  • NAT Gateway for Private Subnet internet access
  • Internet Gateway for Public Subnet
  • Elastic IP associated with the NAT Gateway
  • Gateway Endpoint for S3 access via ECR
  • API Endpoints for ECR and Docker Registry

Configuration: modules/VPC/main.tf

EKS (Elastic Kubernetes Service)

Section titled “EKS (Elastic Kubernetes Service)”

Amazon EKS is used to manage Kubernetes clusters without the overhead of managing the control plane.

EKS configuration includes:

  • IAM Roles for Cluster and Node Group
  • Self-managed Node Groups
  • Cluster Initialization

Configuration: modules/EKS

ECR (Elastic Container Registry)

Section titled “ECR (Elastic Container Registry)”

ECR is used to host container images securely within AWS. It allows EKS to pull images efficiently over the AWS internal network.

Included configurations:

  • Registry Creation
  • Image Lifecycle Policies
  • VPC Endpoints for optimized access

Configuration: modules/EKS/ecr.tf

RDS (Relational Database Service)

Section titled “RDS (Relational Database Service)”

We initially considered using RDS but opted for a custom SQL Server deployment on EC2 due to the following limitations with RDS:

  • Restricted database-level access
  • No support for replication with on-premise SQL Servers
  • Limited administrative control

Alternative: We provisioned an EC2 instance and manually installed SQL Server.

Pros:

  • Full administrative access
  • Simplified on-premise replication
  • Flexibility in configuration

Cons:

  • Responsibility for backup and maintenance
  • No AWS support for patching or recovery
  • Increased operational overhead

Deprecated RDS Configuration: modules/RDS.OLD Current Configuration: modules/RDS

EC2 (Elastic Compute Cloud)

Section titled “EC2 (Elastic Compute Cloud)”

We use EC2 to host a publicly accessible instance that serves as a VPN gateway. It enables secure access to private subnets and database instances.

Capabilities:

  • Connects to Private and Database Subnets
  • Access to EKS cluster
  • Acts as a VPN server for remote connections

Configuration: modules/EC2/main.tf

Network Load Balancer

Section titled “Network Load Balancer”

We utilize a Network Load Balancer to expose internal Kubernetes services securely via HTTPS. nginx_ingress acts as a reverse proxy.

Exposed endpoints include:

Configuration: nginx.tf > nginx.yaml > Ingress Rules

ACM (AWS Certificate Manager)

Section titled “ACM (AWS Certificate Manager)”

ACM is used to issue and manage SSL/TLS certificates. Our domain is managed by Cloudflare, and ACM is authenticated using a Cloudflare API token to issue wildcard certificates.

Nginx Ingress automatically attaches ACM certificates to all hosted portals.

Configuration: modules/ACM

Kubernetes Workloads

Section titled “Kubernetes Workloads”

We manually deploy core services on Kubernetes:

  • Argo CD for GitOps-based CI/CD
  • APISIX as an API Gateway
  • Keycloak for Identity and Access Management
  • Custom Ingress Rules using nginx_ingress

Argo CD

Section titled “Argo CD”

Argo CD is used to manage Kubernetes deployments using GitOps. It watches for updates in the container registry and deploys updated pods automatically.

Argo CD Architecture

Deployment guide: Code Fresh Guide

APISIX

Section titled “APISIX”

APISIX is used as the primary API Gateway. We use custom manifests located at:

More details: Documentation

Keycloak

Section titled “Keycloak”

Keycloak is used for authentication and authorization using modern identity standards (OpenID Connect, OAuth2, SAML, UMA).

Configuration: Keycloak Manifests Documentation: Keycloak

NGINX Ingress Rules

Section titled “NGINX Ingress Rules”

Ingress rules define how traffic is routed to Kubernetes services via the Load Balancer.

  • Each exposed application has its own YAML file.
  • New services should follow the same structure.

Rules: Ingress YAML Files

Ingress Rules

The diagram below illustrates how NGINX Ingress works in the Kubernetes environment:

Kubernetes Ingress NGINX