Architecture Overview

In this production setup, we move away from the development server (manage.py runserver) to a robust architecture designed for high availability.

• Django (OpenIMIS): The application code.

• Gunicorn: A production-grade WSGI HTTP server that runs the Python code. We run multiple "workers" (instances) on different ports to handle concurrent traffic.

• Supervisor: A process control system. It monitors Gunicorn. If Gunicorn crashes, Supervisor restarts it immediately. It also provides a Web Dashboard for admins.

• Client Layer: A web server (like Nginx) acts as a reverse proxy, distributing incoming traffic to the Gunicorn instances.

• Process Management Layer (Supervisor): The heart of the setup. It starts, monitors, and manages the Gunicorn processes. If a Gunicorn instance crashes (as shown in red), Supervisor detects it and automatically restarts it (shown in green). It also provides a Web GUI for monitoring.

• Application Layer (Gunicorn & Django): Multiple independent Gunicorn instances run the OpenIMIS Django application on different ports, allowing for parallel processing of requests.

Phase 1: System Preparation

1. Install Dependencies

Update the system and install the required tools.

sudo apt update
sudo apt install -y python3-venv python3-pip supervisor

2. Prepare Python Environment

We use a Virtual Environment (venv) to isolate OpenIMIS dependencies from the system Python.

Bash

# Change to the new base directory
cd /var/www/openimis

# Create virtual environment named 'venv'
python3 -m venv venv

# Activate the environment
source venv/bin/activate

# Install dependencies
pip install --upgrade pip
pip install -r requirements.txt

Phase 2: Database Configuration

OpenIMIS requires specific ODBC configurations to talk to SQL Server, especially regarding encryption and SSL certificates.

1. Create Environment File

nano /var/www/openimis/openimis-be_py/openIMIS/.env

2. Configure Variables

⚠️ Important: The TrustServerCertificate option is critical if your SQL Server uses a self-signed certificate.

DB_HOST=
DB_PORT=
DB_OPTIONS={"driver": "ODBC Driver 18 for SQL Server", "encrypt": "no", "TrustServerCertificate": "yes", "unicode_results": true}

3. Verify Connection

Before setting up Gunicorn, ensure Django can actually connect.

python manage.py migrate
python manage.py runserver 0.0.0.0:8000

If the server starts without errors, press Ctrl+C to stop it and proceed.

Phase 3: Supervisor Configuration

Supervisor will manage Gunicorn processes.

1. Enable Supervisor Service

sudo systemctl enable supervisor
sudo systemctl start supervisor

2. Enable Supervisor Web GUI

This allows non-technical admins to see if the server is running and restart it via a browser.

Edit the main config:

sudo nano /etc/supervisor/supervisord.conf

Add block to the end of the file:

[inet_http_server]
port = 0.0.0.0:{PORT}
username = admin
password = admin123

Restart Supervisor to load changes:

sudo service supervisor restart

Action: Open your browser and go to http://YOUR_SERVER_IP:{PORT}. You should see the Supervisor Dashboard.

Application Configuration (Manual Method)

Note: For automated management, skip to Phase 5. This section explains how the configuration works under the hood.

Create a configuration file to tell Supervisor how to run OpenIMIS.

sudo nano /etc/supervisor/conf.d/openimis.conf

Configuration Block Example

Here is how we define 4 separate instances listening on ports 8001-8004, using the new directory structure.

[program:openimis_8001]
# Updated paths
command=/var/www/openimis/venv/bin/gunicorn openIMIS.wsgi:application --bind 0.0.0.0:8001 --workers 6 --timeout 120
directory=/var/www/openimis/openimis-be_py/openIMIS
user=fhir
autostart=true
autorestart=true
stdout_logfile=/var/log/openimis/openimis_8001.out.log
stderr_logfile=/var/log/openimis/openimis_8001.err.log

# Repeat this block for openimis_8002, openimis_8003, etc., changing the port number.

Apply Changes

Whenever you change a .conf file, run:

sudo supervisorctl reread
sudo supervisorctl update
sudo supervisorctl start all

Phase 5: Automated Instance Management (Scripts)

To make scaling easier, we use custom scripts to add or remove server instances without manually editing config files.

Directory Structure

Ensure your scripts are located here:

/var/www/openimis/
├── add_openimis_instance.sh
└── remove_openimis_instance.sh

Adding a New Instance

This script generates the Supervisor config and starts the process.

Script: add_openimis_instance.sh

Usage:

# Syntax: ./add_openimis_instance.sh <port> [workers]

# Example: Add a server on port 8005
./add_openimis_instance.sh 8005

# Example: Add a server on port 8006 with 5 workers
./add_openimis_instance.sh 8006 5

Removing an Instance

This script stops the process and deletes the configuration.

Script: remove_openimis_instance.sh

Usage:

# Syntax: ./remove_openimis_instance.sh <port>

# Example: Remove the server running on port 8005
./remove_openimis_instance.sh 8005

Phase 6: Maintenance & Logs

How to View Logs

If the application errors out (Internal Server Error), check the Gunicorn error logs.

# View live logs for instance 8001
sudo tail -f /var/log/openimis/openimis_8001.err.log

How to Update OpenIMIS Code

When you have new code changes (git pull), you must restart the processes for changes to take effect.

1. Pull Code, into respective project:

   
   cd /var/www/openimis/openimis-be_py/openIMIS
   git pull
   

2. Restart Instances:

   You can restart specific instances or all of them.

   # Restart specific instances
   sudo supervisorctl restart openimis_8001 openimis_8002
   
   # OR Restart everything
   sudo supervisorctl restart all
   

Phase 7: NGINX Load Balancer Setup

Now that multiple Gunicorn instances are running via Supervisor (on ports 8001–8008), we need a single entry point to distribute traffic efficiently across these instances. We will configure NGINX to act as a Load Balancer and Reverse Proxy.

1. Strategy & Configuration Explanation

In this setup, NGINX listens on port 8000 and acts as the "Traffic Cop." It accepts incoming requests and forwards them to one of the backend workers defined in the upstream block.

• Load Balancing Algorithm (Round Robin): By default (since ip_hash is commented out, it is needed for persistent client to host connection), NGINX uses Round Robin. This means request #1 goes to port 8001, request #2 goes to 8002, and so on. This ensures even distribution of load across all CPU cores.

• Upstream Block: Defines the pool of available backend servers (our Gunicorn instances).

• Proxy Pass: The directive proxy_pass http://modular_backend; tells NGINX to hand off the actual processing to the defined upstream group.

• Header Forwarding: We explicitly forward headers like Host, Upgrade, and Connection. This is crucial for modern applications to ensure the backend knows the original request details and can handle protocol upgrades (though standard Django is WSGI, these headers ensure compatibility if future async features are added).

2. Create Configuration File

Create a new server block file for the backend application.

sudo nano /etc/nginx/sites-available/modular_backend

3. Add Configuration

Paste the following configuration. This sets up the upstream group and defines a custom log format to track which specific port handles each request.

# Define a custom log format to include upstream details
# This helps debug exactly which Gunicorn instance (e.g., 8001 vs 8005) handled a specific request
log_format log 'request [$request_method]:$server_port $request_uri status:$status upstream:$upstream_addr date:$time_local';

upstream modular_backend {
    # ip_hash; # Uncomment this line if you need "Sticky Sessions" (User A always goes to Port 8001)
    
    # List all Node backend ports managed by Supervisor
    server 127.0.0.1:8001;
    server 127.0.0.1:8002;
    server 127.0.0.1:8003;
    server 127.0.0.1:8004;
    server 127.0.0.1:8005;
    server 127.0.0.1:8006;
    server 127.0.0.1:8007;
    server 127.0.0.1:8008;
}

server {
    listen 8000;
    server_name _;

    # Access and error logs for this site
    # Note: Ensure the directory /var/modular_backend_log/ exists
    access_log /var/modular_backend_log/modular_backend_access.log log;
    error_log /var/modular_backend_log/modular_backend_error.log warn;

    location / {
        proxy_pass http://modular_backend;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_cache_bypass $http_upgrade;
    }
}

4. Create Log Directory (Critical Validation Step)

Your configuration uses a custom log path /var/modular_backend_log/. NGINX will fail to start if this directory does not exist or if permissions are wrong.

# Create the directory
sudo mkdir -p /var/modular_backend_log

# Assign ownership to the www-data user (which NGINX runs as)
sudo chown -R www-data:www-data /var/modular_backend_log

5. Enable the Configuration

Link the file from sites-available to sites-enabled to make it active.

sudo ln -s /etc/nginx/sites-available/modular_backend /etc/nginx/sites-enabled/

6. Verify and Reload

Test the configuration for syntax errors and reload the NGINX service.

# Check for syntax errors
sudo nginx -t

# If successful (displays "syntax is ok"), reload NGINX
sudo systemctl reload nginx

7. Validation & Monitoring

To confirm the load balancer is working, you can tail the access logs while making requests to the server. You should see the upstream IP change (e.g., :8001, :8002) as you refresh the page.

Command to watch traffic distribution:

sudo tail -f /var/modular_backend_log/modular_backend_access.log

Expected Log Output Example:

request [GET]:8000 /api/login status:200 upstream:127.0.0.1:8001 ... request [GET]:8000 /api/login status:200 upstream:127.0.0.1:8002 ...