Installation guide

This installation guide will first guide you through the installation of NDR Core. If your data and source imagery is stored in different places, you’re all set. If you want to store your data and source imagery in the same place, you need to install a data service and a IIIF server. This guide will also guide you through the installation of MongoDB and Cantaloupe IIIF Image Server. The latter is meant for ongoing projects. You should at least be able to get your source imagery from a long term repository like InvenioRDM.

Install NDR Core for production

This guide will help you install NDR Core on a Ubuntu server. The setup is as follows:

  • Ubuntu 22.04 LTS server

  • Nginx webserver

  • Gunicorn

  • Django and NDR Core application

Optional:

  • MongoDB

  • Cantaloupe IIIF Image Server

This guide is tested for a Ubuntu 22.04 LTS server installation. It should work for other versions of Ubuntu as well but the mongodb installation may differ.

Most of the steps in this guide are not specific to NDR Core. This guide provides a full setup for a django application, serving it with gunicorn and nginx.

Prerequisites

This guide assumes you have a working Ubuntu 22.04 LTS server installation. If you don’t have one you can follow this guide to install one: https://www.digitalocean.com/community/tutorials/initial-server-setup-with-ubuntu-22-04

Basic Setup

First you need to install the basic packages. You need to install Python, pip, virtualenv, git, nginx and postgresql.

Django is a Python web framework. You need to install Python and pip. Django and NDR Core are installed within a virtual python environment. Nginx is a webserver that will serve our application.

Django can run with different database backends (This is not your data, but your website configuration). We will use SQLite for this tutorial. You can use any other database backend you like. For production use you can use PostgreSQL or MySQL. The django-database stores the page contents, your search and api configuration, user messages and configuration values, so it is not particularly large or heavily used and SQLite is fine most of the time. For more information on other databases see https://docs.djangoproject.com/en/5.0/ref/databases/. The following command also installs PostgreSQL if you want to use it.

sudo apt update
sudo apt install python3-venv python3-dev libpq-dev postgresql postgresql-contrib nginx

Create Your Project

Create a directory for your project, change its ownership and change into it.

sudo mkdir /var/www/<project_root>
sudo chown <username> /var/www/<project_root>
cd /var/www/<project_root>

Now create a virtual environment and activate it.

python3 -m venv venv
source venv/bin/activate

In your virtual environment install django, gunicorn and ndr_core. The psycopg2-binary package is not needed if you don’t use PostgreSQL.

pip install gunicorn psycopg2-binary django-ndr-core

Create a Django Project

Now you can create a django project. Replace <projectname> with the name of your project. This can be the same name as the project_root with your virtual environment, but it hasn’t to be. Choose a short name without spaces or special characters.

django-admin startproject <projectname>

This will create a directory with the name of your project. Change into it. You’ll find another directory with the same name inside. Change into that directory.

cd <projectname>/<projectname>

There you’ll find a file called settings.py. Open it with your favorite editor and make the following changes:

Add the following lines to the top of the file with the other imports:

Add the following line int the INSTALLED_APPS list:

Save the file and exit your editor.

Now change back to your <projectname> directory (cd ..) and initialize your NDR Core installation.

python manage.py init_ndr_core

This will initialize your NDR_CORE system. You will be asked to enter some values, but don’t worry, you can change them later. The script does the following:

  • Initializes the database. Creates the tables and adds some initial values.

  • Creates an app called “ndr”. This is your website.

  • Creates an admin user. You can use this user to log in to the admin interface.

The default database backend is SQLite. If you want to use PostgreSQL or MySQL you’ll have to change the DATABASES setting in the settings.py file. See https://docs.djangoproject.com/en/5.0/ref/databases/ for more information.

Now we need to collect all the static files for our project. This will create a directory called static in your project directory.

python manage.py collectstatic

Your django installation is now ready to run and all necessary settings have been made. For production use, you’ll have to change more settings: Set the ALLOWED_HOSTS setting to include the host name of your server and set the DEBUG flag to False. Also, you might want to configure your captcha api key or other settings. See the django documentation for more information: https://docs.djangoproject.com/en/5.0/howto/deployment/checklist/

If you are doing a local installation, run the following command and then visit http://localhost:8000 in your browser. This most likely won’t work if you are running your server in a virtual machine.

python manage.py runserver

On a virtual machine, you can try to open port 8000 and then visit http://<your-server-ip>:8000

sudo ufw allow 8000
python manage.py runserver 0.0.0.0:8000

You should assign ownership of the project directory to the user that will run the django project. This is ideally www-data or something similar.

sudo chown -R www-data /var/www/<project_root>

To now run your django project with gunicorn, follow the next steps.

Configure Nginx and Gunicorn

First we test, if we can serve the page with gunicorn. Run the following command (Replace <projectname> with the name of your project):

gunicorn --bind 0.0.0.0:8000 <projectname>.wsgi

If gunicorn starts without errors, visit http://<your-server-ip>:8000 to check if your page is served. It is normal that stylesheets and images are missing. We’ll fix that later.

Stop gunicorn with Ctrl-C. Exit your virtual environment and create a systemd socket and service file for gunicorn.

deactivate
sudo nano /etc/systemd/system/gunicorn.socket

Paste the following lines into the file:

[Unit]
Description=gunicorn socket

[Socket]
ListenStream=/run/gunicorn.sock

[Install]
WantedBy=sockets.target

Now create a systemd service file for gunicorn:

sudo nano /etc/systemd/system/gunicorn.service

Paste the following lines into the file:

[Unit]
Description=gunicorn daemon
Requires=gunicorn.socket
After=network.target

[Service]
User=www-data
Group=www-data
WorkingDirectory=/var/www/<project_root>/<projectname>
ExecStart=/var/www/<project_root>/venv/bin/gunicorn \
          --access-logfile - \
          --workers 3 \
          --bind unix:/run/gunicorn.sock \
          <projectname>.wsgi:application

[Install]
WantedBy=multi-user.target

Replace <project_root> with the name of the directory where your project is located. Replace <projectname> with the name of your project. Replace www-data with the user and group that should run the gunicorn process. Usually this is www-data, but it might be different on your system.

Now start and enable the gunicorn socket:

sudo systemctl start gunicorn.socket
sudo systemctl enable gunicorn.socket

You can check if the socket is running with the following command. It should show the status of the socket and the file that it is listening on. If the socket is not running, check the systemd logs for errors. You can also check if the file exists. If it doesn’t, there is probably an error in your gunicorn.service file.

sudo systemctl status gunicorn.socket
file /run/gunicorn.sock

With the following command, you can access the gunicorn logs:

sudo journalctl -u gunicorn.socket

Until now, we have only started the gunicorn socket. The gunicorn service is not running yet because it is only started when a connection is made to the socket. Let’s proceed to configure Nginx to Proxy Pass to the gunicorn socket.

sudo nano /etc/nginx/sites-available/<projectname>

Paste the following lines into the file:

server {
    listen 80;
    server_name your-server.org;

    location = /favicon.ico { access_log off; log_not_found off; }
    location /static/ {
        root /var/www/<project_root>/<projectname>;
    }
    location /media/ {
        root /var/www/<project_root>/<projectname>;
    }

    location / {
        include proxy_params;
        proxy_pass http://unix:/run/gunicorn.sock;
    }
}

Now enable the site, remove the default setting and test the configuration:

sudo ln -s /etc/nginx/sites-available/<projectname> /etc/nginx/sites-enabled
sudo rm /etc/nginx/sites-enabled/default
sudo nginx -t

You should see the following output:

nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

Now restart nginx:

sudo systemctl restart nginx

Now we need to configure the firewall to allow connections to port 80. We can delete the configuration for port 8000, because we won’t need it anymore.

sudo ufw delete allow 8000
sudo ufw allow 'Nginx Full'

Now you should be able to visit your page in your browser but it is served with http. To enable https, we need to install certbot.

Install certbot

Certbot is provided by the certbot snap package.

sudo snap install core; sudo snap refresh core

If you’re working on a server that previously had an older version of certbot installed, you should remove it before going any further.

sudo apt-get remove certbot

Now install certbot:

sudo snap install --classic certbot

Finally, link the certbot command to certbot-auto:

sudo ln -s /snap/bin/certbot /usr/bin/certbot

Now we can request a certificate from Let’s Encrypt. Replace <your-domain> with your domain name. If you have multiple domains, you can add them with the -d option. Certbot will ask you to enter your email address and to agree to the terms of service. Certbot will also ask you if you want to redirect all http traffic to https. If you want to do that, choose option 2.

sudo certbot --nginx -d <your-domain>

You should activate the certificate renewal service. Currently it is not active. Check the status of the timer:

sudo systemctl status snap.certbot.renew.service

To test the renewal process, you can run the following command:

sudo certbot renew --dry-run

If you see no errors, the renewal process is working fine. When necessary, Certbot will renew your certificates and reload Nginx to pick up the changes. If the automated renewal process ever fails, Let’s Encrypt will send a message to the email you specified, warning you when your certificate is about to expire.

Note

Your NDR Core installation is now complete. If your data and source imagery is stored in different places, you’re all set.

If you want to store your data and source imagery in the same place, you need to install a data service and a IIIF server. See the next sections for instructions.

This is only recommended for ongoing projects. Finished projects should store their data and source imagery in a long term repository.

Install MongoDB

To install MongoDB Community Edition, you can follow the instructions on the MongoDB website or follow the instructions below.

sudo apt-get install gnupg

curl -fsSL https://pgp.mongodb.com/server-6.0.asc | \
   sudo gpg -o /usr/share/keyrings/mongodb-server-6.0.gpg \
   --dearmor

echo "deb [ arch=amd64,arm64 signed-by=/usr/share/keyrings/mongodb-server-6.0.gpg ] https://repo.mongodb.org/apt/ubuntu jammy/mongodb-org/6.0 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-6.0.list

sudo apt-get update

sudo apt-get install -y mongodb-org

You now have installed gnupg, added the MongoDB GPG key to your system, created a list file for MongoDB, updated the local package list and installed the MongoDB packages.

Reload systemd and start MongoDB:

sudo systemctl daemon-reload
sudo systemctl start mongod
sudo systemctl status mongod

You can stop it with the following command:

sudo systemctl start mongod

If you want it to run as a service, you can enable it with the following command:

sudo systemctl enable mongod

Your MongoDB installation is now complete.

Cantaloupe IIIF Server

Cantaloupe is an open-source IIIF image server. It is written in Java and uses the Java Advanced Imaging (JAI) library. It is fast, scalable, and easy to deploy.

First, we need to install Java or check if it is installed. We will work with OpenJDK 11.

Check if Java is installed: .. code-block:: bash

java -version

If it is not installed, install it with the following command: .. code-block:: bash

sudo apt install default-jre

Change into the /usr/local/ directory and download the latest version of Cantaloupe:

cd /usr/local
sudo mkdir cantaloupe
cd cantaloupe
sudo wget https://github.com/cantaloupe-project/cantaloupe/releases/download/v5.0.5/cantaloupe-5.0.5.zip

Unzip the file, cd into the directory and copy the cantaloupe.properties.sample file:

unzip cantaloupe-5.0.5.zip
cd cantaloupe-5.0.5
cp cantaloupe.properties.sample cantaloupe.properties

Create a directory to store the images:

sudo mkdir /var/www/<project_root>>/images

Open the cantaloupe.properties file and change at least the following settings:

FilesystemSource.BasicLookupStrategy.path_prefix = /var/www/<project_root>/images/

If you want you can activate the admin interface:

# Enables the Control Panel, at /admin.
endpoint.admin.enabled = true
endpoint.admin.username = admin
endpoint.admin.secret = s3cr3t

Now you can test if cantaloupe is working:

java -Dcantaloupe.config=cantaloupe.properties -Xmx2g -jar cantaloupe-5.0.5.jar

If it works, we can create a service file for Cantaloupe:

sudo nano /etc/systemd/system/cantaloupe.service

Add the following content to the file:

[Unit]
Description=Cantaloupe IIIF Service

[Service]
User=www-data
WorkingDirectory=/usr/local/cantaloupe/cantaloupe-5.0.5
ExecStart=/usr/local/cantaloupe/cantaloupe-5.0.5/start-cantaloupe
SuccessExitStatus=143
TimeoutStopSec=10
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

Now we need to create the start script:

sudo nano /usr/local/cantaloupe/cantaloupe-5.0.5/start-cantaloupe

Add the following content to the file:

#!/bin/bash
/usr/bin/java -Dcantaloupe.config=cantaloupe.properties -Xmx2g -jar cantaloupe-5.0.5.jar

Make the script executable:

sudo chmod u+x /usr/local/cantaloupe/cantaloupe-5.0.5/start-cantaloupe

Now change ownership of the cantaloupe directory:

sudo chown -R www-data:www-data /usr/local/cantaloupe

Reload systemd and start Cantaloupe:

sudo systemctl daemon-reload
sudo systemctl start cantaloupe
sudo systemctl status cantaloupe

Enable it as a service:

sudo systemctl enable cantaloupe

Allow the cantaloupe port in the firewall:

sudo ufw allow 8182/tcp

Add images to your image directory and test if Cantaloupe is working. Say you have an image called test.jpg in your image directory. You can now access it with the following URL: http://<your_domain>:8182/iiif/3/test.jpg/full/full/0/default.jpg

You can also access the admin interface with the following URL: http://<your_domain>:8182/admin

Next Steps

You now have installed NDR Core and if needed MongoDB and a IIIF image server. The next steps are to populate the database with data and the images folder with images.

See Example Database Setup for an example on how to populate the database.

Sources

This guide heavily relies on the following sources: