Week in review -- March 27, 2019

New Features

  • Added fulltext search in API tokens.
  • Added fulltext search and sorting by name in Orchestrations.

Minor Updates

  • We have discontinued Productboard and all your new ideas come directly into our inbox. Please feel free to share with us anything on your mind.
  • Python and R transformations show the link to the transformation configuration.
  • After successfully validating a SQL transformation you can immediately run it. 


Power BI Desktop not showing Snowflake tables

We're investigating issues with Power BI not showing tables in your Snowflake workspace.

We're sorry for this inconvenience, we'll be updating this post with our findings.

UPDATE 17:26 CET

Snowflake confirmed they can reproduce the issue and are investigating further. 

The issue affects transient tables (CREATE TABLE statement), but does not affect views. As a temporary workaround we suggest to log in directly into the Snowflake workspace and create views from all tables in the schema. These views should be immediately visible in Power BI.


UPDATE 11:20 CET

Snowflake are preparing a new version of the ODBC Driver which allows Power BI to see the tables. Use the support button in your project to contact us to obtain this driver.


Speeding up transformation outputs in your projects

We're working hard every day to make Keboola Connection faster and minimize the time you spend waiting for your results. This effort includes a wide variety of components of the platform architecture. Some of the changes are straightforward and transparent to the end users, but others are unexpectedly complicated. 

This is the case of transformations. We rolled out an update earlier this year and were forced to immediately rollback to the previous version as it broke the data flow in a few projects. This time, we're more prepared. We have identified the source of the incompatibility, and we'll be rolling out the update silently, only for those projects that will not be adversely affected. The projects that would be affected will not be updated. Instead, they will be notified to take steps to fix the incompatibilities. Then they'll become eligible for the update as well. 

Parallel output processing

In the original system, when all transformations in a phase are executed, the output processing starts. It takes all the transformations sequentially and processes the outputs one by one. The order of the executions is not defined, but it is predictable and, most importantly, it doesn't change between runs. Some projects rely on a specific order of output processing to achieve certain goals.

To speed up the output, we have decided to queue all outputs at once and let Storage handle all jobs as fast as possible. But, as you already may have noticed, this mean the transformations can run in any order or even in parallel. This may affect the result of the output if you relied on a particular order.

Updating the project

The project overview and all affected transformations will notify you about multiple outputs being written to the same table in Storage. You will be easily navigated to the places that need to be fixed. Please contact our support if you need any help doing that. 

Once you have fixed all instances where multiple outputs are written to the same table in your project, you can immediately contact us using the support button. We will turn on the update in your project. Or, wait until we update your project automatically (we're watching all projects regularly).



Failing Jobs

Since 2018-12-19, 19:13:00 UTC we're experiencing a higher rate of application errors in all regions due to an outage in AWS ECR. 

We're investigating the issue and update this post once there.

We apologize for the inconvenience.

Update 19:30 UTC

This outage also affects the Developer Portal.

Update 20:05 UTC

As of 19:51 UTC the issue is resolved by Amazon

Stalled jobs in EU region

Around 1:30am CEST one of the job worker instances stopped processing assigned jobs. This could have lead to jobs being stuck in the processing state for a long time without any activity.

At 11:15am CEST the worker instance was terminated and all unfinished jobs started processing on other instances.

We're sorry for this inconvenience.

Scheduled US and EU Maintenance

There will be a maintenance period on Saturday, October 6th, 2018 from 8:00am CEST and should take less than 5 hours.

We will be upgrading component job indices and metadata databases.

All projects in connection.keboola.com and connection.eu-central-1.keboola.com will be inaccessible during the maintenance.

Degraded Snowflake Performance (US region)

Since September 25 we're experiencing degraded Snowflake performance affecting all Snowflake operations. 

We're sorry for this inconvenience, we're working with Snowflake to fix this issue.

Update, October 8

Snowflake Engineering team has discovered and fixed the issue (waiting for an official statement from Snowflake). We're seeing operation times going back to normal.