Restarting the server

Normally, when we deploy code, Amazon creates some new servers, then 'swaps out' and recycles the old ones.

If you have an AWS console login, you can trigger a build from the AWS web site from this page. Click "Start Build" and it should happen in 10 minutes or so.

Developers can also do it with this command from their own machines:

`apppack -a ot-main-prod build start`

The only reason we might do this is the dreaded "Heisenbug"

Heisenbug history

For years we have been plagued by something we call the "Heisenbug". The GridFields get into a state where they seem to be receiving a list-of-dictionary structure rather than a list-of-lists, or vice versa. On saving, the user gets a pink Illegal numeric pattern error and nothing is saved.

In 2021/2022 we have mostly seen this with the Relay form.

It seems some ReportLab staff wrote some code to work around and accept either data representation, after being unable to find a rational cause. This was applied to most results forms but not the relays. Andy aplied it to the relay form on 21 Jan 2022, so we will see if/when it recurs.

Heisenbug approach

Weeks have been invested by ReportLab in trying to solve the above bug, including writing tracing code and running it under heavy loads to trap data when the problem happens (which is rare).

Long term, we should ideally write new grids directly in Handsontable, and write Python code to handle them, which have far less "degrees of freedom" and options than docengine's Gridfield widget.