MO2.2 Install openIMIS without using Docker

1 General comments about the installation procedure
- 1.1 TO DO
- 1.2 Database
2 1 - Minimal Linux setup
3 2 - Fetching and installing openIMIS apps
4 3 - Install the mobile REST API
- 4.1 3.1 - Setting up the REST API
- 4.2 3.2 - Making sure the REST API is working

General comments about the installation procedure

We have seen in the previous section that Docker packaging is managing all components of the modular openIMIS. If you wish to install the modular openIMIS without Docker, you will have to install separately the backend, the frontend and the gateway (and other optional modules).

Installing openIMIS directly is a tradeoff: it is more complex to get running, has more components to update and monitor but it is also much more flexible for low-bandwidth environments where downloading large docker images is an issue.

TO DO

This guide is still missing:

configuration of the images folder

Database

This guide does not cover the database setup. Both PostgreSQL and Microsoft SQL Server are supported here, unless the REST API is needed.

1 - Minimal Linux setup

This takes Ubuntu as example but can easily be adapted to most other distributions.

mkdir oi
cd oi
sudo apt install openssh-server curl
sudo apt install git python3-venv python3-wheel libpq-dev python3-dev gcc g++ make

If you are going to use the Microsoft database, install the corresponding driver:

Install the Microsoft ODBC driver for SQL Server (Linux) - ODBC Driver for SQL Server

Here is the version for Ubuntu until 2021:

if ! [[ "16.04 18.04 20.04 21.04 21.10" == *"$(lsb_release -rs)"* ]];
then
    echo "Ubuntu $(lsb_release -rs) is not currently supported.";
    exit;
fi

sudo su
curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add -

curl https://packages.microsoft.com/config/ubuntu/$(lsb_release -rs)/prod.list > /etc/apt/sources.list.d/mssql-release.list

exit
sudo apt-get update
sudo ACCEPT_EULA=Y apt-get install -y msodbcsql17
# optional: for bcp and sqlcmd
sudo ACCEPT_EULA=Y apt-get install -y mssql-tools
echo 'export PATH="$PATH:/opt/mssql-tools/bin"' >> ~/.bashrc
source ~/.bashrc
# optional: for unixODBC development headers
sudo apt-get install -y unixodbc-dev

Now, let’s create a Python virtual environment. This is not mandatory but strongly advised to avoid interactions between Python applications:

cd ~/oi  # if you are not in this folder anymore
python3 -mvenv venv
source venv/bin/activate

2 - Fetching and installing openIMIS apps

We will fetch the main application components, backend and frontend:

git clone https://github.com/openimis/openimis-be_py.git
git clone https://github.com/openimis/openimis-fe_js.git

2.1 - Configure the backend

cd openimis-be_py
# if you are not going to work from the main branch (latest release): git checkout develop
# You might need here to adapt the requirements.txt file. If pandas fails to install below, change to pandas==1.4.2

2.1.1 - Backend requirements installation

Adapt openimis.json to suit your needs of openIMIS modules to deploy.

pip install -r requirements.txt
python modules-requirements.py > modules-requirements.txt
pip install -r modules-requirements.txt

Now, you will need to create an .env file with the main parameters of the backend (to check meaning of env variables - please check this page MO2.2.3 Environment variables - openimis-be_py):

DB_HOST=mssql-host-server
DB_PORT=1433
DB_NAME=database-name
DB_USER=database-user
DB_PASSWORD=database-password
DB_ENGINE=mssql

INSUREE_NUMBER_LENGTH=10
INSUREE_NUMBER_MODULE_ROOT=7
SITE_ROOT=api
MASTER_PASS=')(#$1HsD'

# Database PSQL
PSQL_DB_USER=IMISuser
PSQL_DB_PASSWORD=<DB-PASSWORD>
PSQL_DB_PORT=5432
PSQL_DB_ENGINE=django.db.backends.postgresql
PSQL_DB_HOST=127.0.0.1
PSQL_DB_NAME=test_imis
# Database MSSQL
MSSQL_DB_PORT=1433
MSSQL_DB_ENGINE=mssql
MSSQL_DB_USER=SA
MSSQL_DB_PASSWORD=<DB PASSWORD>
MSSQL_DB_NAME=test_imis
MSSQL_DB_HOST=127.0.0.1

DB_NAME=test_imis
DB_TEST_NAME=test_imis

# Site root that will prefix all exposed endpoints. It's required when working with openIMIS frontend
SITE_ROOT=api
# Should the debug be on (i.e. debug information will be displayed)
MODE=DEV
# this will also show the DB request in the console
DJANGO_DB_LOG_HANDLER=console
# Photo path root used in insuree module. Only used if InsureeConfig value not specified. Comment out for default.
#PHOTO_ROOT_PATH=<photo path>
# Should the database be migrated before start (entrypoint.sh - docker setup). Will be migrated anyway if $SITE_ROOT=api. Comment out for False
DJANGO_MIGRATE=True

PROJECT_NAME=dev
# set up you main domain
#DOMAIN=dev-openimis.org
HTTP_PORT=80
HTTPS_PORT=443
DB_DEFAULT=postgresql
# Comment  if you don't want to initialize with the demo dataset
DEMO_DATASET=true

DB_BRANCH=develop
GW_BRANCH=develop
BE_BRANCH=develop
FE_BRANCH=develop

# Lockout mechanism
# Allowed login failures before lockout
LOGIN_LOCKOUT_FAILURE_LIMIT=5
# Lockout duration in minutes
LOGIN_LOCKOUT_COOLOFF_TIME=5 

PASSWORD_MIN_LENGTH=8
# Minimum number of uppercase letters
PASSWORD_UPPERCASE=1
# Minimum number of lowercase letters
PASSWORD_LOWERCASE=1 
# Minimum number of digits
PASSWORD_DIGITS=1
# Minimum number of symbols
PASSWORD_SYMBOLS=1 
# Maximum number of spaces allowed
PASSWORD_SPACES=1 

# Define the trusted origins for CSRF protection, separated by commas
CSRF_TRUSTED_ORIGINS=http://localhost:3000,http://localhost:8000 

# Rate limiting settings
# The cache alias to use for rate limiting
RATELIMIT_CACHE=default  
# Key to identify the client; 'ip' means it will use the client's IP address
RATELIMIT_KEY=ip  
# Rate limit (150 requests per minute)
RATELIMIT_RATE=150/m  
# HTTP methods to rate limit; 'ALL' means all methods
RATELIMIT_METHOD=ALL  
# Group name for the rate limit
RATELIMIT_GROUP=graphql 
# Whether to skip rate limiting
RATELIMIT_SKIP_TIMEOUT=False

OPENSEARCH_ADMIN=<OPENSEARCH-ADMIN-NAME>
OPENSEARCH_PASSWORD=<OPENSEARCH-ADMIN-PASS>

Now, we just need to generate the static files:

cd openIMIS
./manage.py collectstatic

Then, try to run the backend server:

cd ~/oi/openimis-be_py/openIMIS  # If not already there from last step

# if migrations are not applied - run them
./manage.py migrate

# run the server
./manage.py runserver

If the server takes a really long time to start, the database connection parameters are probably wrong and it is waiting for a database timeout to print an error.

2.1.2 - Configure the automated restart

For systemd, create a file /lib/systemd/system/openimis.service:

[Unit]
Description=openIMIS backend
After=network-online.target

[Service]
WorkingDirectory=/home/openimis/oi/openimis-be_py/openIMIS/
ExecStart=/home/openimis/oi/venv/bin/python manage.py runserver
Restart=always
RestartSec=15s
KillMode=process
TimeoutSec=infinity
User=openimis
Group=openimis

[Install]
WantedBy=multi-user.target

Be careful to adapt the 4 occurences of the “openimis” if your username is different:

two /home/openimis
User/Group

Start the server manually:

sudo systemctl start openimis.service

To make it start automatically at boot:

sudo systemctl enable openimis.service

If the service fails to start, check the logs with:

sudo journalctl -u openimis.service

2.2 - Configure the frontend

2.2.1 - node & yarn installation

The first step is to install node.js. openIMIS versions 1.2 to 1.5 are using node 16. The node installation instructions are available here, but a generic solution like nvm is also a good option.

For Ubuntu with stock Node:

curl -fsSL https://deb.nodesource.com/setup_16.x | sudo -E bash -
sudo apt-get install -y nodejs

Then, install yarn:

curl -sL https://dl.yarnpkg.com/debian/pubkey.gpg | gpg --dearmor | sudo tee /usr/share/keyrings/yarnkey.gpg >/dev/null
echo "deb [signed-by=/usr/share/keyrings/yarnkey.gpg] https://dl.yarnpkg.com/debian stable main" | sudo tee /etc/apt/sources.list.d/yarn.list
sudo apt-get update && sudo apt-get install yarn

2.2.2 - Frontend requirements installation

Go the frontend folder:

cd ~/oi/openimis-fe_js

You can now edit the openimis.json for the frontend to select the components to install.

When done, load the configuration:

yarn load-config
yarn install
yarn build
yarn global add serve

If building yarn fails with an out of memory error, use export GENERATE_SOURCEMAP=false. You can also add this export to your .bashrc as it is available only for your current session.

At this stage, you should be able to run the app in dev mode with yarn start.

This is not recommended for production !!

So instead, let’s build a static bundle and deploy it in nginx’s path:

yarn build  # If not already done above
sudo apt install nginx  # We'll set it up below, so if you didn't install it already...
sudo mkdir /var/www/html/front/
sudo chown $USER /var/www/html/front/
cp -r build/* /var/www/html/front/

2.3 - Configure nginx (gateway/reverse proxy)

The docker-compose version uses an openresty image that only relays, here we will be deploying a regular nginx pointing to the static site.

Nginx will also be handling the TLS (HTTPS) with Letsencrypt.

sudo apt install nginx
sudo rm /etc/nginx/sites-enabled/default  # remove the default site (it’s just a link, don’t worry)
sudo vi /etc/nginx/sites-available/openimis.conf
sudo ln -s /etc/nginx/sites-available/openimis.conf /etc/nginx/sites-enabled/openimis.conf

Here is a sample configuration, to be tuned to your setup. The SSL parts are commented out, they will be updated by Letsencrypt’s certbot:

upstream docker-backend {
        server localhost:8000;
}
upstream docker-frontend {
        server localhost:3000;
}
upstream restapi {
        server localhost:8080;
}
server {
        server_name example.openimis.org; 
        # Don't forget to edit the URL
#        return 301 https://$host$request_uri;
#}
#server {
#
#        listen       443 ssl;
#        server_name example.openimis.org;
#
##        ssl_certificate /etc/ssl/certs/example.openimis.org.crt;
##        ssl_certificate_key /etc/ssl/private/example.openimis.org.key;

        client_max_body_size 100M;

        location /.well-known {
                root /var/www/html;
        }

        location /LegacyHome {
                return 204;
        }

        location /keepLegacyAlive {
                return 204;
        }

        location / {
                return 301 /front/;
        }
        
        location /home {
                return 301 /front/;
        }

        location /Home.aspx {
                return 301 /front/;
        }

        location ~/front/(.*) {
                root /var/www/html;
                try_files $uri $uri/ /front/index.html;
                #error_page 404 $scheme://$host/front/;
        }

        location /iapi/ {
                proxy_pass http://docker-backend/api/;
                proxy_set_header   Host $host;
                proxy_set_header   X-Real-IP $remote_addr;
                proxy_set_header   X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header   X-Forwarded-Host $server_name;
        }

        location /api/ {
                proxy_pass http://docker-backend/api/;
                proxy_set_header   Host $host;
                proxy_set_header   X-Real-IP $remote_addr;
                proxy_set_header   X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header   X-Forwarded-Host $server_name;
        }

        location /rest/ {
                proxy_pass http://restapi/;
                proxy_set_header   Host $host;
                proxy_set_header   X-Real-IP $remote_addr;
                proxy_set_header   X-Forwarded-For $proxy_add_x_forwarded_for;
		proxy_set_header   X-Forwarded-Proto https;
                proxy_set_header   X-Forwarded-Host $server_name;
        }
}

client_max_body_size was added in March 2023 to allow uploads of large pictures

Now, restart nginx:

sudo systemctl restart nginx.service

Finally, enable TLS/SSL with Letsencrypt:

sudo apt-get install python3-certbot-nginx
sudo certbot --nginx -d example.openimis.org # don't forget to edit the URL

You will then need to choose whether or not to redirect HTTP traffic to HTTPS:

Please choose whether or not to redirect HTTP traffic to HTTPS, removing HTTP access.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1: No redirect - Make no further changes to the webserver configuration.
2: Redirect - Make all requests redirect to secure HTTPS access. Choose this for
new sites, or if you're confident your site works on HTTPS. You can undo this
change by editing your web server's configuration.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Select the appropriate number [1-2] then [enter] (press 'c' to cancel):

Redirecting HTTP to HTTPS is recommended.

2.4 - Configure nginx (frontend/opensearch proxy)

Overview

In recent versions of the openIMIS Docker distribution, Nginx is now integrated as part of the frontend container. It acts as a reverse proxy that routes traffic between the frontend, backend (API), and optional services such as OpenSearch Dashboards.

New Nginx Configuration Location

Instead of managing Nginx in a separate container, configuration is now maintained within the openimis-fe_js repository:

📂 conf/locations

This folder contains reusable location blocks for routing specific paths (like /api, /graphql, /opensearch) to internal services.

Serving Additional Services (e.g. OpenSearch)

To support OpenSearch Dashboards under the same domain, you need to:

Mount the updated Nginx config into the frontend container (enabled in docker-compose.yml):
volumes: - ./conf/nginx:/conf/
Ensure the correct location block is included, such as opensearch.conf under conf/locations.
Set OPENSEARCH_PROXY_ROOT="opensearch" in the frontend's environment.

This allows you to access OpenSearch Dashboards at:

https://your-domain/opensearch

3 - Install the mobile REST API

At the moment, the REST API is the last part of the openIMIS system that was not adapted to Postgres. It can be installed on Linux or other systems via Docker. This guide will show how to set it up as a stand-alone docker.

The REST API does NOT work with Postgres. You must use a Microsoft database for the REST API.

3.1 - Setting up the REST API

First, go to oi and clone the REST API repository:

cd  /oi
git clone https://github.com/openimis/rest_api_c-sharp
cd rest_api_c-sharp

At this stage, you should be on the main branch.

Currently (May 4 2023), there are some issues that must be fixed on the main branch, so you should use the fix/docker_compose branch until it has been merged into main.

In order to to that, you can enter git switch fix/docker_compose.

Before starting the REST API, you will need to edit the docker-compose and Dockerfile files with the database information.

Here is a sample docker-compose file. Replace the 5 database values in the environment section:

version: '2.4'
services:
  restapi:
    build: 
      context: ./
      args:
        BUILD-FLAVOUR: ${BUILD-FLAVOUR:-Release}
    environment:
      - DB_HOST=Server
      - DB_NAME=IMIS
      - DB_USER=IMISuser
      - DB_PASSWORD=IMISuser@1234
      - DB_PORT=1433
    ports:
      - 8080:80
    volumes:
      - ./OpenImis.RestApi/config:/app/config
      - ./OpenImis.RestApi/logs:/app/log
      - ./OpenImis.RestApi/Certificates:/app/wwwRoot/Certificates
      - ./photos:/app/photos
      - ./Escape:/app/Escape
      - ./FromPhone:/app/FromPhone
    restart: always

Here is a sample Dockerfile file. Replace the 5 ENV database values:

FROM mcr.microsoft.com/dotnet/core/sdk:2.1 AS build-env
WORKDIR /app

COPY /OpenImis.RestApi/*.csproj ./
RUN dotnet restore
ARG BUILD-FLAVOUR=Release
COPY . ./
RUN dotnet publish OpenImis.RestApi/OpenImis.RestApi.csproj -c $BUILD-FLAVOUR -o out

FROM mcr.microsoft.com/dotnet/core/aspnet:2.1
WORKDIR /app


ENV DB_HOST=Server
ENV DB_NAME=IMIS
ENV DB_USER=IMISuser
ENV DB_PASSWORD=IMISuser@1234
ENV DB_PORT=1433

# copy appsettings templates
COPY ./OpenImis.RestApi/config/appsettings.Production.json.dist /app/tpl/
COPY ./OpenImis.RestApi/config/appsettings.json /app/config/
COPY ./scripts/entrypoint.sh /app/
RUN chmod a+x /app/entrypoint.sh
COPY --from=build-env /app/OpenImis.RestApi/out .
RUN echo 'deb http://archive.debian.org/debian/ stretch main' > /etc/apt/sources.list \
    && echo 'deb http://archive.debian.org/debian-security/ stretch/updates main' >> /etc/apt/sources.list \
    && apt-get -o Acquire::Check-Valid-Until=false update \
    && apt-get install gettext -y \
    && rm -rf /var/lib/apt/lists/*

ENTRYPOINT /app/entrypoint.sh

Then, you can start the Docker service:

docker-compose up -d

3.2 - Making sure the REST API is working

Once the container has finished building and has started, you can check that the REST API is properly working:

First, you can type docker ps. In the STATUS column, you should see something similar to Up xx minutes.
Then you can access the REST API and see if you see any data fetched from the database, for instance by accessing the following page: <your-openIMIS-URL>/rest/api/claim/Controls

Previous step: MO2.1 Install openIMIS using Docker

Table of content

Next step: