Post-mortem: Failing transformation in EU region

This is a post-mortem of the Failing transformation in EU region incident. 

We received a root-cause analysis from Snowflake and learned that the problem was caused by a DNS resolution issue. The nodes could not reach the NTP service, causing some clock skew and failed jobs for several Keboola customers. Snowflake replaced the nodes that were affected and resolved the issue. 

We learned that even the jobs that did not fail might have produced incorrect results as a result of a few seconds of time skew. Server-time-related functions (current_timestamp(), current_time() and current_date()) returned values that were skewed by a few seconds from the actual time. The data would only be affected if the query modifying it used the above mentioned functions.

We're working on a report of affected queries internally. You can contact us via standard support channels if you think your queries are using server time in a way in which a few seconds of skew would affect the results. We'll work with you on checking the queries as soon as we have the report ready.