Stuck Jobs in US Stack

2022-06-23 22:30 UTC: We're investigating cases of jobs not processing in the US stack (https://connection.keboola.com/). Next update in 30 minutes.

2022-06-23 07 23:00 UTC: The situation is now under control and our services are running normally. Processing jobs will start to gradually speed up.

We are sorry for the inconvenience.

Job Processing Slowing down

Since approximately 13:30 we're seeing some jobs processing longer then usual in the the us-east and eu-central stack (connection.keboola.com and connection.eu-central-1.keboola.com). We're investigating the issue. Next update in 30 minutes.

Update 14:30 UTC: We have identified the root cause of the issue. We're actively working on fixing the issue. Next update in 30 minutes.

Update 15:20 UTC: We are still working on the fix. Next update in 30 minutes.

Update 15:55 UTC: We have deployed the fix to both affected stacks and the issues should be resolved now. We will continue monitoring the situation.

Weeks in review -- May 20, 2019

Improvements

  • Transformation buckets can now run in parallel. That means when you put multiple transformation buckets in one orchestration phase, they will execute in parallel - the same way extractors or writers work. Previously the transformation jobs would be queued to execute sequentially.
  • The editor of transformation queries now uses the official Snowflake syntax highlighter, so the syntax highlighting should now be consistent with the Snowflake Sandbox.

  • The organization settings now offer a new feature to require MFA (Multi-Factor Authentication) for all organization members.

  • A new GoodData User Management Application was released. It replaces part of the functionality of the old GoodData writer and allows to manage users in the GoodData project, invite new users to the project, change roles and assign data permissions for each user.

Fixes

  • A more readable Roboto font is now used in Keboola Connection and Developer Portal UI.
  • Fixed error when adding metadata to columns with special characters in name.
  • Column metadata are now displayed even when they cannot be edited (due to insufficient permissions, table being an alias or table being hosted on Redshift).

Developers

  • Removed support for deprecated private Quay.io and private Docker Hub repositories (no active components were using it).


Week in Review -- October 9th, 2017

The longstanding issue with slow SHOW DATABASES query executions on Snowflake has finally been resolved. We will now gradually enforce snowflake query execution limits where applicable by contract. Before turning the limits back on, we will check the query history and notify you if you may be affected.

(Query times in milliseconds)

New Features

Minor Improvements

We continue with cleaning up the KBC UI. So far, we have fixed many little annoying issues:

  • List of configurations now shows when the configuration was created (with exact date on hover) 

  • Cleaned up Job list (removed unused stuff).
  • Backend in Transformations is now preselected to default project backend.
  • CSV importer now supports .TSV and .GZ files too.
  • "Test Credentials" button and result polished.
  • Removed 'sticky' buttons when editing configurations.

Fixes

  • Fixed files input mapping not showing in certain configurations.
  • Fixed a couple of errors with the Save button when editing transformation queries.
  • Fixed bug in Redshift writer with mixed case table names.

Developers

  • The encryption API is simplified to a single endpoint. If you manually encrypt values for KBC, please follow the docs. We will be contacting you if you are using the deprecated API calls.


New Generic Extractor Documentation

Long neglected, long awaited, it's here!

We are done with a major overhaul of our Generic Extractor documentation. It is now completely moved to developers.keboola.com. The new documentation contains over 100 runnable examples. We have also created a tutorial which guides you through how to configure the Generic Extractor for a new API.

We have also a configuration map with links to documentation of all parts of the configuration.

In case you haven't heard of the Generic Extractor, it is our universal REST API client. It reads data (in JSON format) from an API according to how you configure it, then converts the results to CSV files and imports them into KBC Storage. This way you can extract data from new APIs in tens of minutes. You can use the Generic Extractor to create a new extractor or perform ad-hoc extractions from an API. 

If you are interested, give a try to the Tutorial.

Week in Review -- March 6, 2017

Minor Improvements

  • We have upgraded the Google Sign-in to the latest most secure version.

Fixes

  • Dropping a empty primary key in Storage no longer causes an error.

  • Creating a Storage table with a missing S3 file now returns a proper error message.

  • Handle correctly a situation when a primary key is supplied as an array to a Storage table.


AWS S3 Outage Resolved

Amazon services are almost restored. Our infrastructure recovered too, everything should be running now. Though the waiting times are expected to be longer. 

We're sorry for this inconvenience and thanks a lot for your understanding and patience.

Call for testers: RStudio and Jupyter Sandboxes

We have been working on RStudio and Jupyter Notebook sandboxes (these are the missing sandboxes for R and Python transformations) for some time and we now feel that it is time to move to the next phase. 

We can now enable these on a per-user basis for early adopters. If you are interested in playing with these Sandboxes please contact us at support@keboola.com (don't forget to tell us your KBC email).

Limitations:

  • Sandboxes are available only as plain sandboxes (not in a transformation bucket yet)
  • HTTPS is not yet supported.
  • Sandbox disk space is limited to 10GB.
  • Memory is limited to 8GB.
  • The UI allows loading only tables to the Sandbox. Loading input files and transformation script is supported only by the API.
  • Sandboxes will be deleted after 5 days.
  • Adding data to existing sandboxes is not supported.

Otherwise, it's like RStudio connected to your KBC project:

The CSV file locations are the same as in transformations (/data/in/tables/). In the meantime, we will work on:

  • HTTPS Support.
  • Creating better UI and integration with transformations.
  • Allowing tables and files to be added to a running sandbox.


Week in review -- December 27, 2016

Hello everyone,

Last week we have updated the R environment for both custom science and transformations to version 3.3.2.

Apart from that not much has happened last week because our office has been seized:


If everything goes as planned, this should be our last post of the year. So we wish you Happy New Year!

Job failures

Job Failures

There were jobs failures between 4 AM - 9 AM UTC caused by one of our servers dying in a horrible slow and painful death.

We're sorry for this inconvenience, we have restarted the affected orchestrations.