UI Improvements

• Action buttons are now directly accessible when hovering over list items in transformations and components which use generic input or output mappings.
  

• We added a new modal to improve the orchestration set-up experience.  You can now more easily schedule orchestrations on an hourly, daily or weekly basis. There's still an option to set up a custom schedule.

• When you want to edit tables or edit credentials in your database writers, you no longer have to click on the “Edit” button, you can directly edit the values and push “Save“ button.
  

  

• We added a new modal for database writers that support provisioned credentials(Redshift, Snowflake). You can now directly create provisioned credentials.

  

Minor Improvements

• Julia transformation and sandbox have been updated to julia1.4

We’re currently experiencing a transformation failures, we are investigating the problem. Next update in one hour.

Since March 26, 4:00 PM UTC we are experiencing failures for starting transformations in US and EU regions with error Storage API bucket 'configuration_id' with configuration not found.

Error was caused by incorrect configuration.

We're investigating the issue and will update this post with our findings.

We apologize for the inconvenience.

UPDATE Mar 24, 10:10 UTC  - Performance should be back to normal, we're closely monitoring the situation.

Announcements

New Features

• Keboola Connection in-app news — we now display important news inside the application. We aim to replace the current status with the in-application news:

  
• Search in transformations now searches the individual transformations again:

  
• Orchestrations can now be copied (from the configuration versions page); the new orchestrations are created as disabled so that they don't run unexpectedly:

  
• An entire bucket can be added in input mapping in transformations: 

  
• Orchestrator now supports setting the timezone for the orchestration schedule:

  
• We're working on making the UI less cluttered; therefore, we're hiding numerous action buttons into an action menu. In the following releases, some of the actions that are now in the action menu will be directly accessible when hovering over the item in the list:

  

Community News

For Czech speaking folks - we're participating in Covid-19 CZ activity.

Minor Improvements

• DWHM Manager now has the option to reset users' passwords; the password link will be returned in job events. Keep in mind that the link can only be clicked once: 

  

Fixes

• Microsoft SQL Server writer now correctly creates the primary key when creating a table.

• Thoughtspot writer now creates the target database & schema if it does not exist.

Developers

• Developer portal now shows the link to your development project: 

  

We are investigating decreased performance of Snowflake in EU region which unfortunately reoccured after previous resolution. We are in touch in with Snowflake support. Next update in three hours.

UPDATE Mar 16, 11:58 CET  - Snowflake confirmed the issue and will fix it by end of business day today. Meanwhile backlog of jobs cleared but we still see performance degradation. Next update in 6 hours or when the issue will be resolved by Snowflake.

UPDATE Mar 16, 18:21 CET  - Snowflake is performing update which should resolve the issue. We already see performance improvements and there are no jobs in backlog. Next update tomorrow morning.

UPDATE Mar 17, 08:08 CET  - All operations are back to normal all jobs were processed during the night without any delays. We are working together with Snowflake to avoid these issues happening again.

We're experiencing degraded Snowflake performance affecting all operations in the EU region.

We are investigating this issue and will keep you posted as soon as we have an update.

UPDATE Feb 24, 17:07 CET - After more detailed investigation there is a rapid increase of SQL compilation time of queries on snowflake side. It was reported to Snowflake Support and we are waiting for response from their side.

UPDATE Feb 25, 9:00 AM CET - We have confirmation from Snowflake support about high workload in their Cloud Service Layer on yesterday. Their engineering team taken steps mitigate the issue.

UPDATE Feb 25, 2:05 PM CET - After a detailed investigation we found that job performance was impacted from between 8 AM and 6 PM CET yesterday. We are still monitoring the situation. Please contact our support if your jobs are still having any problems with performance.

New Components

• Mailgun v2 — Mailgun is an email automation service. It offers a complete cloud-based email service for sending, receiving and tracking email sent through your websites and applications.
• Drive CX — Downloads Drive CX data about locations, purchase details, employees, and surveys.
  
• AzureML Model Deployment — Allows you to deploy your trained model to Azure Container Instances and query it via an API.
• MS SharePoint Lists writer — SharePoint empowers teamwork with dynamic and productive team sites. A list in SharePoint is a collection of data that gives you and your co-workers a flexible way to organize information.

New Features

• Orchestrator supports setting the timezone over the API (support in Keboola Connection's UI will follow soon).
• MongoDB extractor supports the SRV protocol for specifying connection, the so called seed list.
• Snowflake Data Warehouse Manager now creates schemas with "MANAGED ACCESS" and grants "ON FUTURE" to all objects of the schema. This means that recreating tables (drop and load) doesn't require re-granting for all roles/users previously granted.
• ThoughtSpot writer now checks if a database and a schema exist.

UI Improvements

• When creating new Input mapping in a Snowflake transformation, the load type Clone is used if the source table is cloneable and bigger than 100 megabytes.
• JSON editor now shows all fields, not only the required ones.
  
• Transformation detail page warns about unsaved queries/scripts if you attempt to run a transformation or leave/close a page.

• Orchestration detail page supports the load of older orchestration jobs.
  

• In Transformation, you can see the basic Input Mapping setting without the opening detailed modal.
  

• Create buttons (table, column) in the Storage explorer were moved to the top right corner.

• Transformation Output Mapping modal is now more compact.
  

• Organization selector shows the Maintainer name ("KBC Internal" in the example below).

  

Minor Improvements

• Python environment is 3.8.1 and includes Java and H2O.
  
• Jupyter notebooks now save notebook files to Keboola Storage — both on a manual and auto save.
  
• Input mappings that are using CSV files(e.g. python/r transformation components) now have a global limit of 100 GB per table.
  
• GoodData extractor uses more verbose logging.
• Most emails from Keboola Connection have a new design.

Summary

On 2020-01-28 09:00 UTC, we deployed a version of Keboola Connection containing a bug. It resulted in loads from transformations to storage were missing our internal _timestamp value. This issue was hard to detect and persisted till 2020-01-29 08:00 UTC. Backfill was applied and all missing _timestamp fields were set to value 2020-01-29 00:00:00 UTC at 2020-01-31 16:30 UTC.  The effect of the tables not having the _timestamp set was that jobs which used this table for incremental loading had no reference for the newest data.

What Happened?

There was an error in our upgrade of the library responsible for loads. An incorrect parameter set resulted in timestamps not being set during load. Such a scenario was not covered by our tests, and this situation was not caught during our peer review process. We immediately deployed the previous functioning version of Keboola Connection as soon as the problem was identified. That itself took about 15 minutes. This was an issue that affected some customers' data so backfill was carefully discussed and tested. Unfortunately, we were also impacted with an issue in the 3rd party build system we use which prevented us from performing the backfill of the missing timestamps on 30th January. Finally, between 2020-01-31 09:30 UTC and 2020-01-31 16:30 UTC all impacted project were back filled.

Timetable

•  2020-01-28 09:00 Version containing a bug deployed
•  2020-01-29 08:00 Rollback
•  2020-01-29 Investigation of issue, impact assessment
  
•  2020-01-30 Testing of backfill
  
•  2020-01-31 09:30 Start data backfill
•  2020-01-31 16:30 Data backfill done

What Are We Doing About It?

We're extending the software tests to include more scenarios including test of _timestamp presence on all types of load. We're also working on improving our public incident response to post more frequent updates. 

Original status of issue: https://status.keboola.com/investigating-problems-with-incremental-lods

If your data was affected with this issue, our backfill is not enough for your specific case and you are not in contact with our support yet, feel free to get in touch. Our professional services team will provide all necessary help.