Keboola Status

Updated Components

• MS SQL Server extractor has had a driver update.  Advanced queries now have slightly different behaviour so it is recommended to switch to simple table and column selected configurations.  Where that is not possible, please see the documentation for further help.
• Snowflake writer - minor bugfixes and enhancements
• primary key is added when creating a table
• nonexistent schema/warehouse validation
• warehouse detected correctly for users with different login name and user name
• Database extractors - column datatypes are now shown in UI

Minor Improvements

• You can now change the sharing type of a shared bucket in the storage console.

Around 1:30am CEST one of the job worker instances stopped processing assigned jobs. This could have lead to jobs being stuck in the processing state for a long time without any activity.

At 11:15am CEST the worker instance was terminated and all unfinished jobs started processing on other instances.

We're sorry for this inconvenience.

Database storing locks was restarted at 03:49 UTC which caused the job failures. Also some of the jobs were queued after this failure.

The backlog of all jobs was cleared at 06:15 UTC. The system is fully operational now. We're working on infrastructure changes which should prevent similar issues.

The maintenance period in the US region has just finished. All systems are back online and working properly.

Thank you for your patience.

The maintenance period in the EU region has just finished. All systems are back online and working properly.

Thank you for your patience. 

The pre-announced maintenance period for the US and EU regions has just started. The estimated downtime is 5 hours. 

There will be a maintenance period on Saturday, October 6th, 2018 from 8:00am CEST and should take less than 5 hours.

We will be upgrading component job indices and metadata databases.

All projects in connection.keboola.com and connection.eu-central-1.keboola.com will be inaccessible during the maintenance.

Since September 25 we're experiencing degraded Snowflake performance affecting all Snowflake operations. 

We're sorry for this inconvenience, we're working with Snowflake to fix this issue.

Update, October 8

Snowflake Engineering team has discovered and fixed the issue (waiting for an official statement from Snowflake). We're seeing operation times going back to normal.

Storage jobs processing was delayed between 04:30 - 06:04 UTC. Worker nodes didn't started as expected to process the workload. 

We are investigating the issue.

We are experiencing minor job queue issues in US region which may result in delaying orchestrations. We are investigating this issue at the moment. We are sorry for inconvenience.

UPDATE, Sep 28, 06:30 UTC - We have confirmed from Snowflake support that they are tracking the issue in the increase in the compilation time of the queries and their engineering is actively investigating the issue.

UPDATE, Sep 28, 10:45 UTC - Snowflake support reported that they are experiencing a degradation of service with elevated query failures in our US region. https://support.snowflake.net/s/feed/0D50Z00008QI7AqSAL

UPDATE, Sep 28, 15:00 UTC - Snowflake support reported that they have resolved the issue and the US west region is again operating at 100%