Week in Review -- October 24, 2018

Complexity Badges

There are components in KBC which you can setup in just a few clicks with only basic knowledge. But there are also components which require you to have deep knowledge about how it works. For this reason we have introduced Complexity Badges to be able distinguish between these types of components even before you start setting them up.

Not all components have assigned Complexity Badges yet, but we are continuously working on updating the components catalog.

Minor Improvements

  • When you are retrying failed orchestrations it will now continue from where it failed by default.
  • Transformation backends have new labels with icons.

Components

Restbox shutdown

We would like to remind you that the Restbox component was deprecated in July 2018 and will be completely shut down on November 1, 2018.

If you still have some Restbox configurations in your projects, please migrate them to one of following replacements:


Table primary key consistency incident

We have identified a bug in the primary key implementation in Storage which could lead to improper data deduplication. Only a very limited number of tables is affected by this bug – 7 tables in all KBC projects in all stacks. We'll be contacting owners of the affected projects soon to help fixing the affected tables.

Root Cause

The deduplication stopped working when a column used in a compound primary key was deleted.

During this operation, the information about the whole primary key was unknowingly dropped in the Snowflake backend and this was not propagated correctly to our Storage metadata that still contained the primary key (minus the deleted column). In Snowflake, commands such as ALTER TABLE ... DROP COLUMN ... immediately drop the whole primary key if it’s dropping a column of a compound primary key. The deduplication process retrieves primary key information from the DESCRIBE TABLE ... Snowflake command which shows no primary key in the affected tables, but our metadata still incorrectly shows that a primary key is set.

We are implementing a fix that will store and retrieve primary keys from a single source.

Operating with Primary Key Columns

  • Deleting columns which are part of the table primary key is no longer supported in Storage.
  • To delete a primary key column please drop the primary key first.
  • To change primary key of a table you will need to first remove the primary key and then set it again.

Week in Review -- October 08, 2018

Updated Components

  • MS SQL Server extractor has had a driver update.  Advanced queries now have slightly different behaviour so it is recommended to switch to simple table and column selected configurations.  Where that is not possible, please see the documentation for further help.
  • Snowflake writer - minor bugfixes and enhancements
    • primary key is added when creating a table
    • nonexistent schema/warehouse validation
    • warehouse detected correctly for users with different login name and user name
  • Database extractors - column datatypes are now shown in UI

Minor Improvements

    • You can now change the sharing type of a shared bucket in the storage console.

    Failed and delayed jobs in EU region

    Database storing locks was restarted at 03:49 UTC which caused the job failures. Also some of the jobs were queued after this failure.

    The backlog of all jobs was cleared at 06:15 UTC. The system is fully operational now. We're working on infrastructure changes which should prevent similar issues.

    Jobs delay issues in US region

    We are experiencing minor job queue issues in US region which may result in delaying orchestrations. We are investigating this issue at the moment. We are sorry for inconvenience.

    UPDATE, Sep 28, 06:30 UTC - We have confirmed from Snowflake support that they are tracking the issue in the increase in the compilation time of the queries and their engineering is actively investigating the issue.

    UPDATE, Sep 28, 10:45 UTC - Snowflake support reported that they are experiencing a degradation of service with elevated query failures in our US region. https://support.snowflake.net/s/feed/0D50Z00008QI7AqSAL

    UPDATE, Sep 28, 15:00 UTC - Snowflake support reported that they have resolved the issue and the US west region is again operating at 100%

    Week in Review -- September 25, 2018

    Keboola Connection in AUS region

    We have launched Keboola Connection in AUS region. You can now create your projects in the AUS region. Please contact info@keboola.com if you are interested.


    Feature Wishlist

    According to our open-source philosophy, we always want to hear new ideas from you. To make communication easier, we have added a Feature Wishlist link to Keboola Connection sidebar which leads to productboard site where you can request new features.



    Google Sign In issues

    Some users are not able authenticate to KBC using Google Sign In. We are investigating the issue.

    In the meantime, please sign in using the standard email and password method.

    UPDATE, Sep 22, 07:47 UTC - We have identified the issue. Google users with some special characters in name were affected. We have made changes in interaction with Google. It should resolve this issue. Fix should be deployed in two hours.

    UPDATE, Sep 22, 09:55 UTC - Fix was deployed. You should be able to authenticate to KBC using Google Sign In. 


    Scheduled Maintenance (US Region)

    Scheduled maintenance will start at 06:00 am UTC. 

    The maintenance won't be system wide. It will be performed for each project in connection.keboola.com separately and it should take few minutes for each project depending on the number of project's files. It should not affect project's running jobs, these will be queued during the short project maintenance. Only projects created before August 16, 2018 will be affected by this migration.

    We'll update this status about the progress.

    Update, Sep 22, 06:00 UTC - Migrations have started.

    Update, Sep 22, 10:11 UTC - All projects have been migrated.


    Failing Oracle Extractor

    Friday, September 26, 2018 04:30 UTC

    Due to a configuration error the Oracle extractor was failing to process jobs from 22:00 UTC Thursday, September 20 to around 04:20 UTC on Friday, September 21, 2018.

    The error has now been corrected and all new jobs will process correctly.

    Sincere apologies for the error and inconvenience.