Week in Review -- September 19, 2018

Updated Components

Minor Improvements

  • Link to a configuration from the Job Detail page has been improved for the Transformation component and also for some components which support configuration using Configuration Rows
  • From now you can switch to another Keboola Connection region directly from the Login Page

  • We unified search inputs across application, so searching will work in more unified way for you

Client libraries

SQL Server Extractor Driver Update

The Microsoft SQL Server Extractor will be updated to use a new driver on October 1, 2018.

This update may effect the format of datetime columns if you are using the advanced query option in your configuration.
Standard configurations (not advanced queries) will not have any change to their output.

The old driver exports datetime columns in the format `yyyy-mm-dd hh:mm:ss` however the new driver includes milliseconds when outputting datetime values `yyyy-mm-dd hh:mm:ss.ms`. 

There are 2 options to mitigate this change:

1) Recommended: Switch the configuration to select the tables and columns that you need and not use an advanced query.

2) Convert the datetime output column in the SQL query, ex: CONVERT(DATETIME2(0), my_column)

If you have any concerns or questions regarding this update please contact us using the support button in your Keboola Connection project.

Thank you

Increased Error Rate in MySQL Extractor

We have encountered an increased error rate in MySQL Extractor after a new version release on Friday, Sept 14, 16:00 UTC.

These errors happened only when the job had a connection issue and had to reconnect to the MySQL server.

The previous version is now rolled back and we're working on a fix.

We're sorry for this inconvenience.

Update Sunday, Sept 16, 2018, 08:15 UTC 

The MySQL Extractor has been updated to resolve this issue. 

Snowflake Outage (US Region)

We have encountered an increased number of Snowflake connection errors between 04:11 UTC and 04:21 UTC. This may have caused failed storage and component jobs.

Furthermore Snowflake announced possible SQL query failures between 02:24 UTC and 03:30 UTC. 

We're sorry for this inconvenience, all systems are operational now. 

Snowflake connection issues (US region)

We're experiencing random connection errors to Snowflake started on Sep 12, 22:08 UTC.

These errors caused a few transformations failures. We're investigating the issue and we'll update this post with our findings.

Update, Sep 14, 07:15 UTC

These rare errors are still occuring.

Update, Sep 14, 13:07 UTC

Snowflake confirms the issue is on their side and are investigating it.

Investigating incident (US region)

We're currently investigating an incident in the US region that may have caused component job and orchestration failures today, Tuesday, Sep 11, between 02:30 UTC and 06:30 UTC.

Update 08:30 UTC

The DB storing job locks (preventing jobs from running multiple times on multiple workers) was restarted at 02:33 UTC. All connections were terminated and all component jobs and transformations running at that time were disconnected. This could lead to any or both of the following situations:

  • Failure at any time later during the job execution
  • A new parallel job execution

Any jobs started after the DB restart were not affected by this issue.

We apologize for this inconvenience. We're planning an infrastructure change to prevent such huge impact during similar situations. 

If you have any further questions, please use the support button in your project.

Week in Review -- August 22, 2018

Updated Components

Storage

  • Added Configuration Row detail API resource

Slack Extractor

  • Fixed pagination. The extractor was only downloading 1000 records from a channel history.
    Existing configuration of the extractor needs to be re-saved to apply this fix.


New Features

  • When creating new API tokens, you can now choose Orchestrator from the Custom Access components dropdown

Minor Improvements

  • Added Incremental label to Output Mapping
  • Storage API and Manage API tokens are now generated from alphanumeric characters instead of hexadecimal
  • Added Documentation badge to component detail. Additional external documentation is available for component marked with this badge. 
  • Filter expression operators were renamed to be more intuitive

    EQ (equals) -> IN
    NE (not equal) -> NOT IN
  • It's no longer necessary to click the <-Back button when adding tasks to Orchestration. The New Task modal will reset its state and after reopening it shows the list of components. Previously, after reopening, it showed a list of configurations of the previously selected component

September, October 2018 Maintenance Schedule

US Region

Sat, September 22, 2018

A maintenance will start at 6:00 am UTC. We'll be migrating metadata storage of Storage Files during the maintenance.
The maintenance won't be system wide. It will be performed for each project in connection.keboola.com separately and it should take few minutes for each project depending on the number of project's files. It should not affect project's running jobs, these will be queued during the short project maintenance. Only projects created before August 16, 2018 will be affected by this migration.

Sat, October 6, 2018

A maintenance will start at 6:00 am UTC and should take less than two hours. We'll be migrating metadata of Jobs during the maintenance. All jobs will be stopped during the maintenance and should be restarted after the maintenance. Orchestrations scheduled during the maintenance period won't be started.

All projects in connection.keboola.com will be inaccessible during the maintenance.

EU Region

Sat, October 6, 2018

A maintenance will start at 6:00 am UTC and should take less than two hours. We'll be migrating metadata of Jobs during the maintenance. All jobs will be stopped during the maintenance and should be restarted after the maintenance. Orchestrations scheduled during the maintenance period won't be started.

All projects in connection.eu-central-1.keboola.com will be inaccessible during the maintenance.

Week in Review -- August 2, 2018

Improvements

  • Jobs and Storage Files search syntax hints
  • Configurations quick search on component detail page

Updated components

Developers

Storage Form Uploads Deprecation

We are deprecating rarely used Storage HTTP Form Upload. This type of upload is supported only in US KBC region and is using legacy AWS Signature Version 2. The upload can be replaced by direct S3 upload using AWS SDK or by Importer Service for POCs or smaller files. We’ll notify projects using this type of upload to help them migrate.