Since 2021-04-21 10:00 UTC we're seeing increased error rate in Snowflake connections. Users may experience failed Snowflake transformation jobs. We're investigating the root cause. 

UPDATE 18:00 UTC: The last error occurred on 2021-04-22 11:24 UTC and we haven't seen any further failures since then. All operations back to normal. We're in touch with Snowflake support to find the root cause and prevent this from happening in the future. 

We're sorry for this inconvenience and thanks for your understanding.

March 15, 2021, 20:16 UTC We're seeing an authentication errors of multiple Microsoft or Azure related services. This affects all stacks and all regions. In AWS regions only some component jobs are affected (mainly PowerBI writer, OneDrive Excel Sheets writer, OneDrive Excel Sheets extractor). in Azure regions more services may be affected.

You may see application errors or increased job running times.

See https://status.azure.com/en-us/status for more details.

Next update in 60 minutes.

UPDATE March 15, 2021, 21:44 UTC - Engineers at Azure are currently rolling out mitigation worldwide. Full mitigation expected within 60 minutes.

UPDATE March 16, 2021, 0:16 UTC - Engineers at Azure have rolled out a fix to all affected regions for Azure AD. Internal telemetry and customer reports suggests that the error rate for dependent services is rapidly decreasing. Microsoft services are in the process of recovery. Some services may have varying times of recovery following the underlying fix. The next update will be provided at 8:00 UTC.

UPDATE March 16, 2021, 7:52 UTC - The issue related to Azure Active Directory has been resolved. Azure reports problems with authentication to the Storage Service now, but we don't see any impact on Keboola Connection. The next update will be provided at 12:00 UTC.

UPDATE March 16, 2021, 11:23 UTC - Issues on the Azure services are solved. The incident is now resolved.

We have identified a system wide failure of the deprecated RTB House extractor by Revolt BI (revolt-bi.ex-rtbhouse). The API version used in this extractor was just deprecated on the service side.

Please upgrade your configurations to the new RTB House extractor - https://components.keboola.com/components/revolt-bi.ex-rtb-house-v5. 

A scaling script running at 12:00 AM CEST failed to scale up the Snowflake warehouse in EU region. All storage and transformation jobs in the EU were affected by this issue and were significantly slower than usual. 

To help process the queued load we have scaled up the warehouse at 9:45 AM CEST and will keep it running until all load is processed.

We're sorry for this inconvenience and we'll be implementing safeguards to prevent this from happening again. 

We are investigating decreased performance of Snowflake in EU region which unfortunately reoccured after previous resolutions. We are in touch in with Snowflake support. Job performance and sandbox loading times may be affected. Next update at 12:30pm UTC.

Update 12:30 UTC: We are handling the performance issue with Snowflake support, we've offset the slowdown by scaling up the cluster. We'll have more information in about an hour. We caught the issue early on, so we hope it will have minimal impact on jobs, apart from a small slowdown. So far we've seen 3 job failures because of this across the whole EU region. We'll post another update at 14:30 UTC or sooner if there are any new information or situation changes. 

Update 14:30 UTC: We are still working with Snowflake on resolving the issue. The situation is currently stable and we did not see any jobs failing since the last update. Our main goal is currently to mitigate the issue before the midnight job surge. Next update in 18:30 UTC or sooner if there are any new information or situation changes. 

Update 18:15 UTC: We are still working on mitigating the slowdown. We've seen only 3 related job failures since last update, so we still consider the situation stable. We think that the issue will be resolved in the following hour.

Update 19:30 UTC: We're monitoring the situation and the performance is improved and close to previous values. We should have fresh aggregated monitoring data an aprroximately 15 minutes and we expect them to show complete recovery to the standard performance. 

Update 20:01 UTC: The issue has been resolved. 

On Dec 1st, 2019, Snowflake will turn on strict string-to-date conversion for all Keboola Connection projects.

By introducing this change, Snowflake wants to address issues such as invalid date strings rolling over. For example, 07-32 (July 32nd) has been interpreted as August 1st. Rollover will no longer be supported for dates, times, or timestamps.

A full description of affected cases and tips on what to do are provided in the Snowflake 3.45 Release Notes.

We have a list of affected queries in the Keboola Connection projects ready and will be contacting all users that may be affected by this change. You'll have plenty of time to check everything and fix any potential issue. 

Thanks a lot for your cooperation. 

Since October 11, 2:00 PM UTC we are experiencing failures on the Transformation Overview page and on the Storage Table Graph page in all regions with the following error message:

Cannot load GoodData Writer: Client error response [status code] 410 [reason phrase] Gone

We're investigating the issue and will update this post with our findings.

We apologize for the inconvenience.

UPDATE October 15, 5:15 PM UTC: a new version was released and the bug was fixed.

New Components

• Smartsupp extractor by Revolt BI downloads data from the live chat platform.

Updates

• Input mapping filter can be set on an ignored column.
• Sandboxes with duration longer than 5 days display the expiration date correctly.

Updated Components

• Oracle and PostgreSQL extractors use optimized tables and columns loading.
• R transformation and sandbox updated to version 3.6.1.
R transformation supports R Markdown.
• GoodData Writer fixes the link to the GoodData project in the EU region.

Deprecations

• GoodData Writer V2, old OAuth Broker and the original AdForm extractor were successfully turned off.

Since October 10, 3:00 am UTC we are experiencing higher rate of "Obsolete output mapping" user errors in transformation jobs. This issue is caused by a recent update of the transformation component. We'll be rolling back the release as soon as possible and keep you updated on further events.

UPDATE 8:45 am UTC: We have successfully deployed a previous version and all affected configurations should be working now. 

UPDATE October 13, 6:11 PM UTC: A new version with a fix has been released and deployed.

We apologize for the inconvenience.

New Features

• Added fulltext search in API tokens.
• Added fulltext search and sorting by name in Orchestrations.

Minor Updates

• We have discontinued Productboard and all your new ideas come directly into our inbox. Please feel free to share with us anything on your mind.
• Python and R transformations show the link to the transformation configuration.
• After successfully validating a SQL transformation you can immediately run it.