Migration

Foreword

If you need help, please create an issue on GitHub.

To Docker Compose

Situation

You are using the old virtool.tar.gz installation (not Docker) and want to migrate to Docker Compose.

Overview

The Virtool data directory and MongoDB database need to be made available to the Docker container.

After the migration, the built-in software update system will no longer work.

MongoDB

First, we will deploy MongoDB as a Docker container and copy your existing database to the container.

  1. Start with a Docker Compose configuration for MongoDB.

    docker-compose.yml

    version: "3.1"
    services:
      mongo:
        image: mongo:4.0
        volumes:
          - mongo:/data/db
    volumes:
      # MongoDB data is stored in a volume.
      mongo: null
  2. Start MongoDB with Docker Compose

    docker compose -p virtool up -d
  3. Ensure MongoDB is configured and running in Docker.

    You can double-check MongoDB is running by running docker ps. You should see something similar to:

     CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                      NAMES
     1a2b3c4d5e6f        mongo:4.0           "docker-entrypoint.s…"   2 hours ago         Up 2 hours
  4. Stop your existing Virtool server. This will prevent any changes being made to your existing MongoDB database while we migrate.

  5. Dump your existing MongoDB database into a file called virtool.dump.

    This will not affect the existing database!

    mongodump --db virtool --archive=virtool.dump
  6. Restore the MongoDB database into the new MongoDB Docker container.

    docker compose exec -T mongo sh -c 'mongorestore --archive' < virtool.dump

Virtool

Next, we will deploy Virtool as a Docker container and connect it to your existing data directory.

The data directory is the directory configured in Virtool with the --data-path option.

The simplest way to make the data directory available to the Virtool Docker container is to use a bind mount. This will map a location on your host machine to a location in the Docker container.

  1. Update the docker-compose.yml to include the Virtool service with a bind mount.

    docker-compose.yml

    version: "3.1"
    services:
    mongo:
       image: mongo:4.0
       volumes:
          - mongo:/data/db
    virtool:
       image: ghcr.io/virtool/virtool:4.4.0
       command: ["python", "run.py", "--no-setup"]
       environment:
          VT_DB_CONNECTION_STRING: "mongodb://mongo:27017"
          VT_DB_NAME: "virtool"
          VT_DATA_PATH: "/data"
          VT_HOST: "0.0.0.0"
       ports:
          - "9950:9950"
       volumes:
          # Bind mount the data directory from the host machine to the container. It will
          # be located at /data in the container.
          - type: bind
          source: /mnt/virtool_data_on_host
          target: /data
    volumes:
    # MongoDB data is stored in a volume.
    mongo: null

    Things to note:

    • We use the --no-setup option to skip the initial Virtool setup.
    • We use the VT_DATA_PATH environment variable to configure the data directory.
    • We use the VT_DB_CONNECTION_STRING and VT_DB_NAME environment variables to configure the MongoDB connection.
    • We define a bind mount to our existing data directory on the host machine. In this example it is located at /mnt/virtool_data_on_host.
    • The location of the data directory in the container is /data. This points to the bind mount we defined above.
  2. Start Virtool with Docker Compose.

    docker compose -p virtool up -d

    Using docker compose up will bring up any new service defined in docker-compose.yml and will not affect existing services.

  3. You should now be able to access Virtool at localhost:9950 and login with your existing credentials.

To a Newer MongoDB Version

MongoDB 3.6.0 or newer is required in Virtool 4.0.0

  • Virtool will not start if it detects that the configured database does not meet this requirement
  • Virtool versions prior to 4.0.0 are not compatible with MongoDB 4.0.0

In Docker Compose

Follow these instructions if you have MongoDB running in Docker Compose.

  1. Backup the Database

First, backup your database using the following command (assuming your database is called virtool). This will dump the entire database to a folder that can later be restored if necessary.

docker compose exec -T mongo sh -c 'mongodump -d virtool --archive' > virtool.dump
  1. Check Compatibility Version using mongo.

    Open a Mongo shell:

    mongo

    In the shell, check the current featureCompatibilityVersion value:

    db.adminCommand({ getParameter: 1, featureCompatibilityVersion: 1 });

    The response should contain the current version compatibility information:

    // Nothing needs to be done in this case.
    { "featureCompatibilityVersion" : { "version" : "3.4" }, "ok" : 1 }
    
    // Feature compatibility must be set to 3.4 in this case.
    { "featureCompatibilityVersion" : { "version" : "3.2" }, "ok" : 1 }
  2. Update Compatibility Version

    If the version value is not 3.4, the compatibility version need to be updated:

    db.adminCommand({ setFeatureCompatibilityVersion: "3.4" });

    The featureCompatibility value should now be set to 3.4. Check using:

    db.adminCommand({ getParameter: 1, featureCompatibilityVersion: 1 });

    The response should have the value set to 3.4.

    { "featureCompatibilityVersion" : { "version" : "3.4" }, "ok" : 1 }
  3. Update Database Software

    Install MongoDB 3.6 according to the instructions from MongoDB:

  4. Update Compatibility Version

    The featureCompatibilityVersion now needs to be set to 3.6. This process is the same as in step 2 except the value is being set to 3. 6 instead of 3.4.

    Open the Mongo shell and set the compatibility version:

    db.adminCommand({ setFeatureCompatibilityVersion: "3.6" });

    The featureCompatibility value should now be set to 3.6. Check using:

    db.adminCommand({ getParameter: 1, featureCompatibilityVersion: 1 });

    The response should show the version value set to 3.6.

    { "featureCompatibilityVersion" : { "version" : "3.6" }, "ok" : 1 }

On Host

Follow these instructions if you have MongoDB running directly on the your machine. Not in a Docker container.

  1. Backup the Database

    First, backup your database using the following command (assuming your database is called virtool). This will dump the entire database to a folder that can later be restored if necessary.

    mongodump -d virtool
  2. Check Compatibility Version

    Open a Mongo shell:

    mongo

    In the shell, check the current featureCompatibilityVersion value:

    db.adminCommand({ getParameter: 1, featureCompatibilityVersion: 1 });

    The response should contain the current version compatibility information:

    // Nothing needs to be done in this case.
    { "featureCompatibilityVersion" : { "version" : "3.4" }, "ok" : 1 }
    
    // Feature compatibility must be set to 3.4 in this case.
    { "featureCompatibilityVersion" : { "version" : "3.2" }, "ok" : 1 }
  3. Update Compatibility Version

    If the version value is not 3.4, the compatibility version need to be updated:

    db.adminCommand({ setFeatureCompatibilityVersion: "3.4" });

    The featureCompatibility value should now be set to 3.4. Check using:

    db.adminCommand({ getParameter: 1, featureCompatibilityVersion: 1 });

    The response should have the value set to 3.4.

    { "featureCompatibilityVersion" : { "version" : "3.4" }, "ok" : 1 }
  4. Update Database Software

    Install MongoDB 3.6 according to the instructions from MongoDB:

  5. Update Compatibility Version

    The featureCompatibilityVersion now needs to be set to 3.6. This process is the same as in step 2 except the value is being set to 3. 6 instead of 3.4.

    Open the Mongo shell and set the compatibility version:

    db.adminCommand({ setFeatureCompatibilityVersion: "3.6" });

    The featureCompatibility value should now be set to 3.6. Check using:

    db.adminCommand({ getParameter: 1, featureCompatibilityVersion: 1 });

    The response should show the version value set to 3.6.

    { "featureCompatibilityVersion" : { "version" : "3.6" }, "ok" : 1 }

To Kubernetes

We haven’t figured this out yet. For now, stick with Virtool 4.