Amazon Web Services

History pre-Amazon

Things started out on ReportLab's dedicated servers, app01.reportlab.com and app04.reportlab.com. We moved to AWS during the pandemic with help from Pete Baumgarter of Lincoln Loop LLC.

One or two things still depend on ReportLab:

UKA proxy server: For security they require the IP address we call from, and AWS didn't make static addresses easy. So requests to them go through a thin https://ukaproxy.reportlab.com/ service, which is just a block in the Nginx config file.

• Code repos for docs and staff docs *

Apppack

(Apppack.io)[https://apppack.io] is our CI/CD framework, which can be learned in a few hours, and saves us from needing months of learning AWS. It is designed to deploy Django sites with optimum security, performance and ease.

We pay $25 per month for each apppack user, who log in with Google accounts.

In general AWS replaces programming with configuration, but some things we set up ourselves. So it's really important that we keep track of any custom settings. Keep these notes up to date!

AWS logins and security

We have a master AWS account, and within it each employee has their own account using IAM (Amazon's security module). Nobody should ever log in with the master account, nor is it needed.

Two factor authentication is essential to log in.

Billing

We pay up to $1500 per month. There is a billing console with a detaild breakdown. We should analyse it regularly and see what can be pruned, and alos try to "turn down"

Registry

This is where we pay for the name (e.g.) "opentrack.run", and a couple of others. When the .run domains went on sale, I managed to back a couple for the future.

Route 53

This is where we define all the names and where they point e.g. https://malta.opentrack.run`

Simple Storage Service (S3)

This is cheap, almost infinite file storage which is the backbone of AWS We have a number of buckets

files.opentrack.run: public stuff, like images, which can be accessed on a link. This has live/ and test/ subdirectories, and a copy job between them, since sometimes static assets go with the code.

privatebucket.opentrack.run: used for things uploaded

Appack also has a bunch of buckets we don't touch with things that help build the servers. AWS deployment consists of creating a "machine image", storing it in S3, then installing it as "the disk" of multiple running servers.

backups

Nightly, we dump out the MySQL and MOngo databases to zipped files. There is a folder for the "latest" which our developers can pull down without impacting the live server

We also have dated folder e.g 2025/03/28/<backup-file.zip> so in theory we could look back at what happened on a particular day.

todo: we ought to prune these as we don't need EVERY day going back years; could maybe just store one per month if more than a year old.

EC2 (Elastic Container Service) and scaling

This is where our servers are run. All managed by apppack. We tend to run 2 web and 2 worker processes when thing are quiet

The recommended way to scale things is

apppack -a ot-main-prod scale web 6 apppack -a ot-main-prod scale worker 4

If you scale from the web interface, be awae that any code deployment will cause it to drop back, as Amazon stores a "plan" of what the resources should be, and apppack adjusts this, and any deployment will see what's different from the plan.

DocumentDB

This is Amazon's Mongo clone. On a busy weekend, when DocDb gets busy, we add read replicas. Just select the cluster and click to add an instance, and it comes on-stream within a few minutes. These days we rarely need more than one read replica on the weekend.

DIRECTORY

• Key account details
• IAM user accounts
• Caching solution
  • Cloudfront settings
  • Django middleware and settings required
• AWS CLI configuration
• DNS migration

Key account details

http://console.aws.amazon.com/ or many other URLs

• Login email: admin@opentrack.run (forwards to Andy and Gus currently).

• Password: in Andy’s green book, Gik***21 (random 5 syllables plus yy).

• Option: Developer support, $29 per month, 12 hour response times.

IAM user accounts

We create IAM users as we go with just enough privilege to do each job.

Account Number	601589172507
User account	Username: Credentials will be provided upon creation Password: Credentials will be provided upon creation
Access billing info, analytics, usage (not yet set up)	ot-billing / 4******i

Caching Solution

We have about fifteen subdomains,

• HOSTS_SPECIAL_SITE = ['bmaf', 'euroath', 'data', 'results']

• HOSTS_FEDERATION_SITES = ['bih', 'bmafs', 'estonia', 'belarus', 'malta', 'cyprus', 'belgium', 'serbia', 'norway', 'wales']

Each has 4 possible server environments it can point to:

• HOSTS_PREFIXES = [‘live-’, 'test-', 'staging-', dev-']

Each of these has a DNS entry to map it to somewhere. Henceforth the names without a prefix can be set to point to the ‘live-’ site directly; or to a web cache which in turn explicitly forwards to live-.opentrack.run.

Formally the ‘live-’ DNS addresses are not needed by Cloudfront, but they let us debug things, and allow a workaround if a request exceeds the 60 second response limit.

Full domain name (browse to it!)	Where it points
	SWITCHABLE: either to live-data.opentrack.run (directly) OR, to cloudfront (dos3b38mjx5bt.cloudfront.net) which then points to live-data.opentrack.run
live-data.opentrack.run	App1 (82.103.138.164) live environment
test-data.opentrack.run	App1 (82.103.138.164) test environment
dev-data.opentrack.run	App1 (82.103.138.164) dev environment
data.opentrack.run	Either: App1 directly (no cache) OR, Cloudfront, cache in front of live-data.opentrack.run

We can switch back and forth in 5 minutes or so, with zero downtime. And then we can tune the cache rules with point and click - what content they strip or forward, how old files can be and so on.

General Cloudfront settings

Main tab

• Domain name: dos3b38mjx5bt.cloudfront.net
• Alternate CNames: data.opentrack.run
• SSL Certificate: our standard

Origin Tab

• Origin: app1.reportlab.com
• Origin Protocol Policy: Match Viewer
• Origin Response Timeout: 60 seconds (Amazon’s max allowable)

Behaviour Tab

• Viewer Protocol Policy: Redirect HTTP to HTTPS
• Allowed HTTP methods: GET, HEAD, OPTIONS, PUT, POST, PATCH, DELETE
• Cache ...settings: Use legacy cache settings
• Cache based on headers: Whitelist: Accept, Authorization, CloudFront-Forwarded-Proto, Host, Origin, Referer, X-Logged-In (a custom one of ours)
• Object Caching: Use Origin Cache Headers
• Forward Cookies: Whitelist: sessionid, csrftoken

Specific Cloudfront Distributions

Currently two ‘distributions’, settings likely no or blank if left out.

files.opentrack.run

Distribution ID: ARN: Delivery Method: Cookie Logging: Distribution Status: Price Class: State: Alternate Domain Names (CNAMEs): SSL Certificate: Domain Name: Custom SSL Client Support: Security Policy: Supported HTTP Versions: IPv6:	E3FMFLSJFDCD4U arn:aws:cloudfront::601589172507:distribution/E3FMFLSJFDCD4U Web Off Deployed Use All Edge Locations (Best Performance) Enabled files.opentrack.run, file.opentrack.run opentrack.run (3eddb1cf-fb21-477f-a387-77060a8f8265) d39z31or3pgb07.cloudfront.net Clients that Support Server Name Indication (SNI) - (Recommended) TLSv1.2_2019 HTTP/2, HTTP/1.1, HTTP/1.0 Enabled
Origin Domain Name: Enable Origin Shield: Origin ID: Restrict Bucket Access: Origin Connection Attempts: Origin Connection Timeout:	files.opentrack.run.s3.amazonaws.com No S3-files.opentrack.run No 3 10
Origin or Origin Group: Viewer Protocol Policy: Allowed HTTP Methods: Cache and origin request settings: Cache Based Selected Request Headers: Object Caching: Maximum TTL: Default TTL: Forward Cookies: Query String Forwarding and Caching:	S3-files.opentrack.run HTTP and HTTPS GET, HEAD Use legacy cache settings None (Improves Caching) Use Origin Cache Headers 31536000 86400 None (Improves Caching) None (Improves Caching)

main site

Distribution ID: ARN: Delivery Method: Cookie Logging: Distribution Status: Price Class: State: AlternateDomainNames(CNAME): . . . . SSL Certificate: Domain Name: Custom SSL Client Support: Security Policy: Supported HTTP Versions: IPv6:	EIQFKHLN9CGQS arn:aws:cloudfront::601589172507:distribution/EIQFKHLN9CGQS Web Off Deployed Use All Edge Locations (Best Performance) Enabled data.opentrack.run, estonia.opentrack.run, belarus.opentrack.run, norway.opentrack.run, belgium.opentrack.run, malta.opentrack.run, cyprus.opentrack.run, bmafs.opentrack.run, serbia.opentrack.run, bmaf.opentrack.run, euroath.opentrack.run, wales.opentrack.run results.opentrack.run, bih.opentrack.run opentrack.run (3eddb1cf-fb21-477f-a387-77060a8f8265) dos3b38mjx5bt.cloudfront.net Clients that Support Server Name Indication (SNI) - (Recommended) TLSv1.2_2019 HTTP/2, HTTP/1.1, HTTP/1.0 Enabled
Origin Domain Name: Enable Origin Shield: Origin ID: Minimum Origin SSL Protocol: Origin Protocol Policy: Origin Connection Attempts: Origin Connection Timeout: Origin Response Timeout: Origin Keep-alive Timeout: HTTP Port: HTTPS Port:	app1.opentrack.run No proxy-live-data.otr TLSv1 Match Viewer 3 10 60 5 80 443
Origin or Origin Group: Viewer Protocol Policy: Allowed HTTP Methods: Cache and origin request settings: CacheBasedSelectedRequestHeaders: Whitelist headers: . Object Caching: Maximum TTL: Default TTL: Forward Cookies: Whitelist Cookies: Query String Forwarding and Caching:	proxy-live-data.otr Redirect HTTP to HTTPS GET, HEAD, Options, Put, Post, Patch, Delete Use legacy cache settings Whitelist Accept, Authorization, CloudFront-Forwarded-Proto, Host, Origin, Referer, X-Logged-In Use Origin Cache Headers 31536000 86400 Whitelist sessionid, csrftoken Forward all, cache based on all

Django middleware and settings required

CSRF_TRUSTED_ORIGINS = [  # otherwise no forms or logins
    'd3lqiw5qm4073k.cloudfront.net', #AWS OpenTrack account
    '.opentrack.run'  # for when we switch and the raw one has another name 
    ]

Federation middleware detects the host, and tells Django which host/country-site to serve. This now handles live-, test-, dev-, stating- and prefix-less domains.

project.comp.middleware.AuthenticatedNoCacheMiddleware adds cache control headers to say “Cache anon stuff for up to 5 minutes, never cache logged in responses”.

AWS CLI configuration

Everyone will need the secret key on Andy’s home laptop to configure the aws cli We can then save and version-control all our settings

DNS migration

Two steps:

1. Move DNS Management: Move zone file from wherever it is now (dnsmadeeasy etc) to AWS Route 53, and change the name server
2. Move the registration of the domain name

Prototyped with “steady.run” which belongs to me and is unused on GoDaddy. I used a CNAME to point this at the echo service, so you can see stuff here:

• http://echo.steady.run/echo