Specify 7 Test Panel

A dashboard for deploying Specify 7 instances for QA purposes

Test Panel is a Dashboard for configuring a cluster of docker containers of Specify 7, with an automatic deployment feature.

The Test Panel is used to easily test different versions of the software and to speed up the QA process for bug fixes by automatically deploying bug fixes that are ready to be tested.

Notable features:

These features are described in more detail below:

Automatic deployments

A GitHub Webhook has been configured for the Specify 7 repository which pings the test panel to check if some bug is ready to be tested.

A ready-to-be-tested bug is defined as a branch in the Specify 7 repository, for which all automated tests have passed, and which has an associated pull request that has been assigned for review to the QA team (or a member of the team), and has not yet been reviewed. If a pull request has been assigned for review both to a member of the development team and a member of the QA team, the test panel deploys the branch only after the developer has approved the pull request, so as not to waste the QA team's time testing code that may be rejected.

If a maximum number of deployments has already been reached (defined in the config file), the test panel tries to destroy old deployments that haven't been accessed recently.

Automatic deployments

Custom deployments

Besides the automated deployments, there is often a need to test a specific branch (e.i. production) in a specific database to replicate a bug or get everything ready for a release.

For these purposes, any deployment can have its configuration changed. Each deployment has an associated DockerHub tag (created from a HEAD of a GitHub branche), a database, and a data model version

Custom deployments (with dark mode)

Database Management

Besides all deployments running in Docker, the test panel itself is Dockerized. Docker composition comes with a MariaDB server to provide databases for deployments.

The dashboard provides a list of databases, a list of users in each database (needed for authentication into a Specify 7 instance), and an ability to upload a new database, download an existing one, or drop it.

Database management (with dark mode)

Online demo

For security purposes, the test panel is protected behind a GitHub OAuth authentication, which only permits signing it with accounts that are members of the "specify" GitHub organization. Thus, even though a live version is available at test.specifysystems.org, the dashboard itself is inaccessible. If you want to try out the test panel, I encourage you to deploy it on your machine.

Exhaustive deployment instructions are documented in the README.md

It should be possible to reconfigure the dashboard to serve deployments of software other than Specify 7.

screen (with dark mode)

Technologies used

Things learned

While this project was nice in terms of usability and features (especially after a few incremental updates), it did fall short in one important aspect.

A primary goal of the test panel is to emulate a production environment so that testers can catch bugs before they get discovered by users.

However, as we eventually discovered, the test panel environment differed from production in several important ways.

As it often happens, once the test panel came into heavy usage, the issues mentioned above were discovered. Fixing them is an ongoing incremental process, but it showed me the importance of the tool being really good at the core thing it is indented to do.