Docker, Podman & Kubernetes

Docker

For convenience, a Docker container can be used to run Python scripts that make use of OSKAR without needing to compile or install any code. The container is on Docker Hub and can be accessed under the tag docker.io/fdulwich/oskar-python3:latest (or a named version).

The container should be deployed by attaching the directory containing the input data files, settings and/or scripts as a bind mount to a directory inside the container, for example to /data. (Output data files will also be written here.) Using docker from the command line, the bind mount can be set using the -v option, and the work directory inside the container specified using the -w option.

To make sure the output files are created with the correct ownership (since Docker runs everything as the root user by default), a user is created on-the-fly by reading the value of the environment variable LOCAL_USER_ID. This should be set to the numeric ID of the user running the script, and can be obtained by running id -u $USER in the shell. Run docker with the -e flag to pass the environment variable into the container.

Combining all these to run a Python script called hello-world.py (from the “hello world” example) in the container, use:

docker run -v $PWD:/data -w /data -e LOCAL_USER_ID=`id -u $USER` \
    docker.io/fdulwich/oskar-python3 python3 hello-world.py

Podman

If using podman instead of docker to run the container, note that filesystem permissions on bind mounts are handled differently between the two systems. With docker, files written as root inside the container will appear as if they had been written by root on the host (not a good idea!), whereas files written as root inside a container using podman will appear on the host as if they had been written by the user that started the container.

To work around this issue when using docker, an entrypoint script is used to create a user on-the-fly with a matching numerical UID whenever the container is started, and then executes code as that user. Although podman handles the mapping between users differently so this is no longer required, unfortunately the same container started in the same way using podman will break, as the newly-created user will not have permission to write to the specified location on the host. (Inside the container, this location will instead appear to be owned by the root user.) Therefore if using podman, the entrypoint script must be bypassed by specifying an empty entrypoint when running the container, as follows:

podman run -v $PWD:/data -w /data --entrypoint= \
    docker.io/fdulwich/oskar-python3 python3 hello-world.py

Kubernetes

To run a Python script using Kubernetes (for example on P3-ALaSKA), the shell script below can be used to run the container as a Kubernetes Job by setting the required parameters automatically. The helper script takes care of mounting the current directory into the container, setting the work directory and passing in the LOCAL_USER_ID environment variable. Use the command line arguments to the script to specify what should be run inside the container.

For example, if the helper script is saved as oskar_run_k8s, the example Python script hello-world.py could be run using:

oskar_run_k8s python3 hello-world.py

The helper script is included below (oskar_run_k8s):

#!/bin/bash
#
# Runs an OSKAR application or Python script in a container using Kubernetes.
#
# Usage:
#
#     oskar_run_k8s <application> <args> ...
#
# This script runs a Kubernetes Job using "kubectl create".
# The application inside the container is run as the current user,
# and the current directory is mounted into it so that data files
# can be easily read and written by the application.
#
# All command line arguments to the script will be passed to the application.
#
# Examples:
#
#     To run the 'oskar_sim_interferometer' application
#     with a parameter file 'sim.ini' and data files in the
#     current directory, use:
#
#         oskar_run_k8s oskar_sim_interferometer sim.ini
#
#     To run an OSKAR Python script in the current directory
#     called 'hello-world.py', use:
#
#         oskar_run_k8s python3 hello-world.py
#
#     (For convenience the container includes the astropy, matplotlib,
#     numpy and oskar Python modules. To use other ones, you can build a
#     new container based on this one.)
#
cat <<END | kubectl create -f -
apiVersion: batch/v1
kind: Job
metadata:
  generateName: oskar-
spec:
  backoffLimit: 0
  ttlSecondsAfterFinished: 60
  template:
    spec:
      containers:
      - name: oskar-container
        image: docker.io/fdulwich/oskar-python3:latest
        imagePullPolicy: IfNotPresent
        workingDir: /data
        volumeMounts:
        - mountPath: /data
          name: host-storage
        args:
`for var in "$@"; do echo "        - \"$var\""; done`
        env:
        - name: LOCAL_USER_ID
          value: "`id -u $USER`"
        resources:
          limits:
            nvidia.com/gpu: 4 # requesting 4 GPUs
      restartPolicy: Never
      volumes:
      - name: host-storage
        hostPath:
          path: $PWD
          type: Directory
END