There was a short Snowflake outage between 10:30 and 10:35 CEST (09:30am and 09:35am UTC) in US region.

• Sandboxes might have lost their data and worksheets
• Transformation jobs might have finished with an error
• Async data loads and exports were unaffected
  

We're investigating the impact and root cause and will update this post as soon as we know more. Snowflake is now back fully operational.

UPDATE Jan 30 2018: Snowflake released their RCA.

You haven't heard from us for a while. We're sorry. Here's what's new.

New Components

• Mailchimp Writer by Robin Nemeth

Updated Components

• SSH Proxy in Generic Extractor
• AdWords Extractor is available in the EU region
• Tree Depth Application was refactored and is significantly faster

Minor Improvements

• SSL configuration of all database extractors is on the same page with credentials and the SSH tunnel configuration
• Database extractors give a warning if your connection is invalid
• Apify Extractor accepts a list of urls from a table in Storage
• New part about ad-hoc data exploration in Jupyter was added to the tutorial on Ad-hoc Data Analysis

Fixes

• Database extractors automatically changes table names to lowercase
• Fixed a bug affecting non-incremental import of sliced tables in the BigQuery and Snowflake extractors 

Blog

Our developers have published 2 blog posts

• Application logging in AWS Lambda with Serverless plugin and Papertrail
• Xdebug for a CLI App in Docker (and PHPStorm)

Community News

• Docker powered GoodData CLI for less used GoodData API features (dataset synchronization and SLI Hashes optimization)

We're experiencing slower Docker components jobs processing, many jobs stalled in waiting state. Finding the root cause, hopefully we'll be back online soon.

UPDATE 9:40 AM CET: All operations are back to normal, the stalled jobs were caused by a misbehaving Redshift cluster. We're going to investigate the root cause. 

We're very sorry for this inconvenience.

We have been experiencing temporary technical difficulties today around 9:15 AM CET and 9:20 AM CET.

Some component jobs may have failed as a result. We're investigating the issue and post an update when the root cause is found.

UPDATE 11:35 AM CET Jobs storage was temporarily unavailable for about two minutes. Jobs scheduling wasn't affected and running jobs were waiting until storage came up again so these jobs weren't affected. Unfortunately few orchestrations have failed, we'll do step to prevent this in the future.

Support for MySQL in Keboola Connection is coming to its end. Here's what will happen.

Effective immediately

• New projects and projects without existing MySQL transformations will not be able to create new MySQL transformations.

MySQL Storage Backend (supported until January 2018)

• The default storage backend for all projects is immediately switched from MySQL to Snowflake.
  
• All MySQL buckets in all projects will be migrated to Snowflake in January 2018. This will not affect any operations, only a short maintenance on the project will be required.
• No changes in the project are required.
• You can apply for a sooner migration at support@keboola.com.

MySQL Transformations (supported until April 2018)

• Your existing MySQL transformations will need to be migrated to Snowflake by the end of April 2018.
  
• If you need any help migrating your MySQL transformations, contact support@keboola.com.

These steps will allow us to deprecate a piece of the legacy infrastructure and focus on the state of the art technologies. The Snowflake storage backend and transformations have significant performance and scaling benefits, so your projects will run faster than on MySQL without any extra charge.

New Features

• Finer granularity in all input mappings. The Days parameter was renamed to Changed in last and you can set the interval of changed data to as short as 10 minutes. Please note, that combining a legacy Days value with a new Changed in last is not allowed (eg. in a transformation chain, where the same table gets imported multiple times, all input mappings have to use the same filter type).

Minor Improvements

• All new transformations come with a predefined boilerplate that will help you at the start. It is especially useful for Python and R transformations.
• Sliced tables in Storage console (eg. exports from a Snowflake backend) are now merged when downloaded.
• Database extractors display the name of the query that failed in their error messages.

Updated Components

• Oracle DB Extractor and Writer are now available in the EU region.

Fixes

• Input Mapping modal has autofocus in the Source field when opened.

Today, October 4, 2017, we experienced several jobs failures between 12:07--14:40 PM CEST. One of our worker servers became unresponsive which led to the jobs failures. We have shut down the instance and launched a new one. Everything is working properly now and we will further investigate the root cause of the issue. 

UPDATE 09:00 PM CEST 
We're still encountering rare job failures. We're monitoring the issue closely and trying to find the root cause.

We are sorry for any inconvenience.

On Tuesday September 19th around 7am CEST a new version of Restbox was deployed. This release contained a bug affecting all wildcard downloads from S3. The older version was restored at 5:20pm CEST.

We're sorry for this inconvenience.

All Snowflake transformations now have SQLdep enabled. 

SQLdep allows you to visualize the data flow in your SQL. You can now easily see the impact of your changes or track the lineage of each column. 

SQLdep Snowflake support is still in beta, so do not hesitate to contact support@keboola.com with any questions or bugs.

New Features

• Generic Extractor introduces some backward incompatible changes, but your current running configurations will remain unaffected. Learn more about Generic Extractor compatibility levels.
• Generic Extractor supports stopping pagination using an attribute in the response, see more on the Limit Stop feature.

Minor Improvements

• Processors configuration is now editable where eligible.
• Transformation scrips and queries editors link to the documentation. 

Fixes

• Transformations UI refreshes properly on transformation or bucket delete or revert/rollback.
• Transformation copy or restore does not delete transformation name and description.