Skip to content

AWS Apppack setup

We currently have pairs of Apppack environments with the following URLs:

The 'main' and 'api' ones serve our Django and FastAPI servers respectively. The only difference between them is the Procfile at the top of the project, which tells the buildpack which is the entry point web process. This works because a script bin/post_compile swaps in either a Procfile.main or a Procfile.api into the Procfile - after build and before deployment.

Each environment is configured by some environment variables as follows:

$ apppack -a ot-main-prod config list
=== ot-main-prod Config Vars
DATABASE_URL:       mysql://ot-main-prod:<insert-password>@apppack-database-opentrack-cluster-1uy49ker94x74.cluster-c4z06eo8x7sa.eu-west-2.rds.amazonaws.com:3306/ot-main-prod
MONGODB_URL:        mongodb://administrator:<insert-password>@opentrack.cluster-c4z06eo8x7sa.eu-west-2.docdb.amazonaws.com:27017/ot-main-prod
NODE_ENV:       dev                                                             
OT_AWS_ENVIRONMENT: ot-main-prod 
UWSGI_PROCESSES:    4

The key one is OT_AWS_ENVIRONMENT, which passes into Django which environment it is. Thus the live server sees ot-main-prod. The settings file looks at this, and realises it's a live server not a test server, which affects things like whether you see tracebacks, and whether it can send emails.

Subdmain names and urlconfs

Route 53 (the DNS manager) can point public domains like malta.opentrack.run or data.opentrack.run at an Amazon stack, or possibly at Cloudfront in front of one. The Django code looks at the HTTP_HOST. So, if called as https://malta.opentrack.run/, the code splits off "malta", realises it's a federation host, selects project.federation.urls as the URLConf for the rest of the request. Then, there's a quick query to attach the Malta object to the request, which tells the templates what styles to load.

Database and DocumentDB setup

We have Aurora (MySQL-compatible) and DocumentDB (Mongo-compatible) clusters. Each has multiple database instances within it, which are named to match: ot-main-prod, ot-main-test etc.

Here's some notes on loading up the Mongo data in the transition period.

Lots of help from Peter Baumgartner.

The objective at this stage is to load up Amazon DocumentDB, which turned out not to be 100% Mongodb compatible. Since apppack doesn't know about MongoDB, Peter had set up a DocumentDB cluster for the app to connect to (via the settings file) and we need to get data in repeatedly

  1. Logged into ec2 instance with Session Manager (web shell) - cd to the /home/ssm-user folder

  2. Created a .ssh directory and manually pasted the otr-download key between terminals.

  3. Grabbed the database and untarred with: ssh -i ~/.ssh/otr-download rptlab@app1.reportlab.com dump-mongo > otr-mongo.tar.gz gunzip -df otr-mongo.tar.gz tar -xvf otr-mongo.tar

  4. installed wget with sudo yum install wget

  5. Downloaded an Amazon certificate chain bundle with wget https://s3.amazonaws.com/rds-downloads/rds-combined-ca-bundle.pem

  6. Start a docker shell with docker run --rm -it -v /home/ssm-user:/home/ssm-user mongo bash. The ec2 instance apparently exists just to start Docker instances and there's one called mongo which is allowed to connect to the DocumentDB virtual private network. We mount our directory when connecting, so the docker shell can see the database dump we downloaded.

  7. See what 'secrets' apppack holds - the mongo password it was first created with - which turned out to be adminpassword:

    MacBook-Pro:raceresults andy$ apppack -a dev config list
    === dev Config Vars
    DATABASE_URL:       mysql://dev:WB6hWll6qeGnX9m5jZ0mwd6GV3Md79@apppack-database-opentrack1-cluster-1corq7slxtbbw.cluster-c4z06eo8x7sa.eu-west-2.rds.amazonaws.com:3306/dev
    MONGODB_URL:        mongodb://administrator:adminpassword@opentrack.cluster-c4z06eo8x7sa.eu-west-2.docdb.amazonaws.com:27017/otr?tls=true&tlsAllowInvalidCertificates=true  
    NODE_ENV:       dev                                                             
    PUBLIC_S3_BUCKET_NAME:  apppack-app-dev-publics3bucket-1x359x0ywjdw8
  1. Run a mongo command to connect and query interactively: 
    mongo -u administrator --password adminpassword --host opentrack.c4z06eo8x7sa.eu-west-2.docdb.amazonaws.com:27017 --ssl --sslAllowInvalidCertificates otr --eval "printjson(db.dropDatabase())"
  1. Now, after hand-editing some indexes in the ec2 shell with vi, start restoring:

mongorestore -u administrator --password adminpassword --host opentrack.c4z06eo8x7sa.eu-west-2.docdb.amazonaws.com:27017 --ssl --sslAllowInvalidCertificates --db otr /home/ssm-user/tmp/dump/otr_0