Documentation

asyncio

The asyncio library supports asynchronous programming in Python.

Important Sections:

• Event Loops
  • get_event_loop()
  • loop.run_until_complete()
  • loop.run_in_executor()
  • loop.set_default_executor()
• Coroutines and Tasks
  • Coroutines
  • Awaitables
  • Creating Tasks
  • Sleeping
  • Running Tasks Concurrently
  • Shielding From Cancellation
  • Task Object
• Exceptions

aiohttp

An HTTP server and client written for asyncio.

Virtool contributors should understand all parts of the aiohttp server components. Most features of the server are used in Virtool.

The aiohttp client has minimal use in Virtool. It is used for making simple requests to GitHub and NCBI.

MongoDB

Virtool currently uses MongoDB >= 4.4.

The official MongoDB documentation includes detailed information about the following:

• CRUD Operations
• Aggregation
• Indexes
• Security

motor

Virtool uses the motor asynchronous MongoDB driver to connect to the database and perform operations. We suggest starting by following the tutorial here.

You have to refer to the asyncio-focussed API documentation for motor. Sections related to Virtool are:

• AsyncIOMotorClient
• AsyncIOMotorDatabase
• AsyncIOMotorCollection
• AsyncIOMotorCursor

PostgreSQL

We use SQLAlchemy as a Postgres database and object-relation mapping (ORM) library.

Always use Postgres asyncio support when executing queries.

Redis

We use aioredis==1.3.1 for connecting to Redis with asyncio support.

We use a limited set of Redis features:

• PubSub for telling jobs to cancel themselves. Job IDs published to channel:cancel should be cancelled.
• Redis lists with names like jobs_pathoscope and jobs_create_sample to queue up job IDs that should be taken up by job runners.