Unfortunately tonight there were a few more unexpected Application errors and delayed or longer running jobs between 1:25am–4:55am CEST (4:25pm–7:55pm) in the US region.
We have experimented with different storage drives (swapping from SSD to throughput optimized HDD) which lead to initial issues with building Custom Science apps. Attempts to provision further resources lead to too many running jobs at once (you could see "SQLSTATE[HY000] [1040] Too many connections" in the failed app events) and removing some of the additional resources could have yielded some other Application errors too.
Currently we're running SSD drives again with enough resources to process all workloads. Please restart your failed jobs.
We hope we'll be able to stabilize this whole unfortunate situation as soon as possible and we're very sorry for inconvenience.