UI Improvements

• Action buttons are now directly accessible when hovering over list items in transformations and components which use generic input or output mappings.
  

• We added a new modal to improve the orchestration set-up experience.  You can now more easily schedule orchestrations on an hourly, daily or weekly basis. There's still an option to set up a custom schedule.

• When you want to edit tables or edit credentials in your database writers, you no longer have to click on the “Edit” button, you can directly edit the values and push “Save“ button.
  

  

• We added a new modal for database writers that support provisioned credentials(Redshift, Snowflake). You can now directly create provisioned credentials.

  

Minor Improvements

• Julia transformation and sandbox have been updated to julia1.4

We’re currently experiencing a transformation failures, we are investigating the problem. Next update in one hour.

Since March 26, 4:00 PM UTC we are experiencing failures for starting transformations in US and EU regions with error Storage API bucket 'configuration_id' with configuration not found.

Error was caused by incorrect configuration.

We're investigating the issue and will update this post with our findings.

We apologize for the inconvenience.

UPDATE Mar 24, 10:10 UTC  - Performance should be back to normal, we're closely monitoring the situation.

Announcements

New Features

• Keboola Connection in-app news — we now display important news inside the application. We aim to replace the current status with the in-application news:

  
• Search in transformations now searches the individual transformations again:

  
• Orchestrations can now be copied (from the configuration versions page); the new orchestrations are created as disabled so that they don't run unexpectedly:

  
• An entire bucket can be added in input mapping in transformations: 

  
• Orchestrator now supports setting the timezone for the orchestration schedule:

  
• We're working on making the UI less cluttered; therefore, we're hiding numerous action buttons into an action menu. In the following releases, some of the actions that are now in the action menu will be directly accessible when hovering over the item in the list:

  

Community News

For Czech speaking folks - we're participating in Covid-19 CZ activity.

Minor Improvements

• DWHM Manager now has the option to reset users' passwords; the password link will be returned in job events. Keep in mind that the link can only be clicked once: 

  

Fixes

• Microsoft SQL Server writer now correctly creates the primary key when creating a table.

• Thoughtspot writer now creates the target database & schema if it does not exist.

Developers

• Developer portal now shows the link to your development project: 

  

If you are uploading a file to Storage (manually or automatically), there's an option to upload it with the Public flag. The file can then be accessed publicly outside of Keboola Connection.

Only a minority of Keboola Connection users take advantage of this feature, and they do so in a very non-standard way (e.g., for HTML files). That's why we decided to deprecate it. Also, the new File Storage we have prepared (Azure Blob Storage) doesn't support public File Uploads, and we want to make this behavior consistent across all supported File Storage Backends.

The option to create Public Files from the UI has been removed (effective with the publication of this post).

The option to create Public Files via an API will be removed in about three months, by the end of June, 2020.

An alternative solution could be the AWS S3 Writer component, but we don't recommend relying on Public Files at all. Not even outside of Keboola Connection.

We are investigating decreased performance of Snowflake in EU region which unfortunately reoccured after previous resolution. We are in touch in with Snowflake support. Next update in three hours.

UPDATE Mar 16, 11:58 CET  - Snowflake confirmed the issue and will fix it by end of business day today. Meanwhile backlog of jobs cleared but we still see performance degradation. Next update in 6 hours or when the issue will be resolved by Snowflake.

UPDATE Mar 16, 18:21 CET  - Snowflake is performing update which should resolve the issue. We already see performance improvements and there are no jobs in backlog. Next update tomorrow morning.

UPDATE Mar 17, 08:08 CET  - All operations are back to normal all jobs were processed during the night without any delays. We are working together with Snowflake to avoid these issues happening again.

We're experiencing degraded Snowflake performance affecting all operations in the EU region.

This problem can affect our users in the following ways:

• Job execution time may be longer than usual.
• Jobs are randomly failing with the error message "Result not found".

This is similar to the issue we observed last week, which was caused by the Snowflake Cloud Service Layer in EU region being overloaded.

We have reported it to Snowflake support and will keep you posted as soon as we have an update.

UPDATE Mar 3, 10:43 CET - The performance of Snowflake operations has currently improved. We're still working with Snowflake support on mitigating this issue permanently.

UPDATE Mar 4, 19:50 CET - We were hit again today with degraded performance of Snowflake's Cloud Service Layer. Between 3:30 PM - 5 PM CET we observed a large increase in compilation time of Snowflake queries. After 5 PM it returned back to normal. We updated Snowflake with our latest observations and are hopeful for a quick resolution. We will update here with any more information within the next 12 hours.

UPDATE Mar 5, 06:35 CET  - We hit again Snowflake performance degradation in last six hours. Snowflake is working on issue mitigation with highest severity. We will provide an update in three hours.

UPDATE Mar 5, 09:49 CET  - Performance degradation peaks still occurs. We are in touch with Snowflake on resolution. We will provide an update in three hours.

UPDATE Mar 5, 13:43 CET  - There has been no change of performance for the past few hours and its degradation still occurs. We will provide an update in three hours.

UPDATE Mar 5, 16:18 CET - We've been working with Snowflake support on mitigating the performance degradation. From our monitoring it seems that Keboola platform in EU region is now stabilised (since around 14:00). We will provide next update in three hours. 

UPDATE Mar 5, 16:43 CET - The issue is a highest priority case with Snowflake support. We're closely collaborating on finding a permanent solution to the performance degradation. Status update from Snowflake: 

Snowflake acknowledges the performance degradation issue reported by Keboola Czech SRO since February 24, 2020 and tracks it under the case #00097987 as a critical incident.

Snowflake Support together with Snowflake Engineering are continuously collaborating and conducting an in-depth investigation to identify the root cause and provide a viable solution to you on priority.

At this time, additional resources have been allocated in the Snowflake Services Layer which should show improved performance values.

We will provide next update in three hours. 

UPDATE Mar 5, 20:55 CET - The performance is now stabilised and jobs are running as expected. We are monitoring the situation and We will provide next update in 12 hours.

UPDATE Mar 6, 09:23 CET - The performance has worsened slightly during the night job peak. We are monitoring the situation and  will provide next update in 3 hours.

UPDATE Mar 6, 12:55 CET - The Snowflake Engineering team made some optimizations of the Cloud Service Layer for our account.  Since 7:40 AM CET there have been no "Result not found" errors and job performance should return to normal state.

We will continue monitoring the situating and will provide another update in 6 hours.

UPDATE Mar 6, 21:45 CET - All operations are back to normal. Thanks for your patience and understanding.

We're experiencing degraded Snowflake performance affecting all operations in the EU region.

We are investigating this issue and will keep you posted as soon as we have an update.

UPDATE Feb 24, 17:07 CET - After more detailed investigation there is a rapid increase of SQL compilation time of queries on snowflake side. It was reported to Snowflake Support and we are waiting for response from their side.

UPDATE Feb 25, 9:00 AM CET - We have confirmation from Snowflake support about high workload in their Cloud Service Layer on yesterday. Their engineering team taken steps mitigate the issue.

UPDATE Feb 25, 2:05 PM CET - After a detailed investigation we found that job performance was impacted from between 8 AM and 6 PM CET yesterday. We are still monitoring the situation. Please contact our support if your jobs are still having any problems with performance.