since 3:50am CET we are experiencing server queues overload. We are still investigating the issue and will inform about the progress.
UPDATE 5:45am CET- We found the possible root of the cause, Snowflake queries are being unusually queued, furthermore we are unable to raise power or number of cluster of our Snowflake warehouses and we waiting for Snowflake Support findings.
UPDATE 6:10am CET from Snowflake support: Engineering has started mitigation steps to address this issue, we will provide another update shortly.
UPDATE 7:00am CET: Snowflake queries queues has got empty and everything looks to be back to normal.
UPDATE RESOLVED 9:00am CET: We confirm the issue has been resolved, the cause was the unexpected queueing of Snowflake queries. Snowflake confirmed and rolled back their latest release. However, as a consequence our waiting jobs reached backoff threshold and timeout, which resulted in orchestration failures. We recommend to review your project orchestrations jobs and take action if needed.