In the following weeks, starting tomorrow 2.3.2022, we will be continuously migrating projects in our North Europe stack to the new version of the Job Queue.

At first glance, these are mostly cosmetic changes, but under the hood we have totally reworked how jobs are processed in order to make them more reliable, faster, and makes it easier for us to bring more features to you:

• Dynamic backends
• Parallel execution of configuration rows
• Long running jobs
• Versioning and trash for Orchestrator configurations
• More concise API of the individual services

Please read more about new Job Queue and the migration here.

We will start the migration with our internal projects and in the following weeks we will migrate projects which don't have any incompatible components. (The list of incompatible components can also be found here). 
If you are using one of the components from the list in you project and want your project to be migrated to the new Job Queue, please contact our support.

The migration of all projects should be completed within a month or two.

Please contact us on support@keboola.com if you have any questions about the upcoming migration or the new Job Queue in general.

We will keep you posted about the progress of the migration.

2022-01-19 07:45 UTC We're investigating issue in Legacy Sandboxes, internal error is being thrown when user tries to create sandbox from transformation

2022-01-19 10:57 UTC We've identified the issue and prepared a fix. We will let you know as soon as the fix is deployed.

2022-01-19 15:52 UTC The fix has been deployed and Sandboxes creation works as expected. We are sorry for any inconvenience.

2022-01-18 18:15 UTC We're investigating issue, where rows of configuration, which was copied from another configuration cannot be deleted nor updated. 

2022-01-18 18:55 UTC We've identified the issue is related to internal database migration. Only configurations of components with rows, such as DB writers and extractors are affected.

2022-01-18 19:36 UTC We know what causes the problem and we are working on a fix. We will let you know as soon as the fix is deployed.

2022-01-19 9:30 UTC The fix has been deployed. We are sorry for any inconvenience.

2022-01-19 11:32 UTC We're preparing a report with the list of affected configurations and next steps. At the moment, we advise users to delete affected (copied) configurations and create new copy.

2022-01-21 10:32 UTC We have fixed all affected configurations with 'configuration cannot be deleted nor updated' problem.

Unfortunately, we also found a problem with mismatch updated date time between configuration and rows versions. We are working on solution. We are sorry for any inconvenience.

2022-01-24 15:05 UTC All project on connection.eu-central-1.keboola.com have now fixed author and created time of configuration rows versions. 

Rest of platform will be fixed 2022-01-25.

2022-01-25 08:00 UTC connection.keboola.com and connection.north-europe.azure.keboola.com have now fixed author and created time of configuration rows versions. 

2022-01-18 15:04 UTC We have identified a bug in latest release of Keboola Connection. Because of it buckets cannot be created from the UI. They can still be created via API. We are reverting to previous version. We will inform you once the previous working version is deployed.

UPDATE 2022-01-18 15:53 UTC The issue has been resolved, everything is working as expected.

2022-01-18 10:30 UTC We are investigating an issue when a created workspace is inaccessible from UI and the request for workspace detail ends with error: 

New Components

AppsFlyer Extractor

• We’re happy to welcome the AppsFlyer Extractor to the family!

• AppsFlyer is the market leader in mobile advertising attribution and analytics, helping marketers to pinpoint their targeting, optimize their ad spend, and boost ROI.

• This component was developed by KDS Team.

DynamoDB Writer

• Another great addition to our component collection is DynamoDB Writer!

• This component lets you write data from our Storage into AWS' DynamoDB, which is a key-value, document-based database. It's fully managed, multiregion, multimaster, durable with built-in security, backup and restore, and in-memory caching for internet-scale applications.
  

  

• This component was developed by KDS Team.
  

CSV to XLSX Processor

• Highly demanded utility to convert data in CSV format to XLSX is here!

• This component was also developed by KDS Team.
  

Updated Components

AWS S3 Extractor

• Uses parallel download and it's more than 20 times faster!

Snowflake Extractor

• Supports incremental load.
  

• Supports config rows.
  

Oracle Extractor

• Supports incremental load.
  

• Supports configuration rows.

• Implemented retry mechanism for all types of queries

Redshift Extractor

• Supports incremental load.

MySQL Extractor

• Implemented retry mechanism for all types of queries

PostgreSQL Extractor

• Added support for PostgreSQL Server v12
  

• Transformations are collapsed by default.
  

  

Data Catalog

• Improved "empty" page
  
  
• Type of sharing is now shown and will be improved further. 
  
  
• Improved “initial" modal
  
  
• Bucket description is shown where applicable. 
  

  

Storage

• When creating an alias table, you can now filter data by the EMPTY_STRING or SPACE characters.
  

  

Fixes

• Fixed a bug in the Darksky extractor date conversion. The bug caused dates in the date column for places in UTC+ timezones to be one day less than the actual forecast date.
  

Developers

• php-csv-options - A library for setting CSV options
  

• php-db-import-export - A library for loading/unloading data between Snowflake and Azure Blob Storage

Since July 26 2019 there was an issue with authorisation of our Google related components using OAuth in the EU region.

We needed to undergo a verification process of our apps by Google. This process has been recently made more strict and we weren't able to finish it before the deadline.

We are very sorry about this, we should have paid more attention to this process and the consequences related to not complying with it.

We have now successfully verified most of our Google components. We are still waiting for the verification of GMail Extractor and GMail Attachments Extractor.

New Features

• Oracle Writer - supports setting custom schema in credentials configuration
• GitHub Extractor - adds organization to downloaded commits and issues
• Storage API - supports multiple where filters and order by statement inside datapreview and asynchronous table export. You can quickly search for your data in datapreview with multiple conditions. We are going to add this feature to UI soon.
• BigQuery Extractor - supports extracting data from the EU region

UI

• full page table preview for Storage tables

Bug fixes

• OAuth Broker API - fixed missing component credentials
• Oracle Extractor - fixed manifest for exported tables if there were more than one table with the same name in different schemas
• Transformations 
• unpaired closing comment tag ( */ ) in SQL query is now properly identified as a user error
  
• added additional retries when creating a workspace

UPDATE 2019-02-25 9:00 AM UTC
A fix has beed deployed and the problem now longer occurs. We suppose, that around 50 jobs suffered from this issue.
We do sincerely apologize for the trouble this may have caused to you. Don't hesitate to contact our support for help.

2019-02-24 10:00 PM UTC
We're experiencing issues with new OAuth Broker API. 

In some cases it might not return the authorised credentials for a component's job. Re-running the job might be successful.

The fix will be deployed very soon.

If you haven't migrated to the new version, please wait until the fix is deployed.

We're terribly sorry for any inconvenience.

We have just released a new version of our OAuth Broker API.

OAuth Broker is a KBC service, which handles the authorisation flow for all KBC components (extractors, writers, ...) using OAuth authorisation and also stores the credentials (tokens) for them.

The new version was needed to simplify integration with KBC and allows us to implement new features into this API more easily.
The features we are preparing are for example automatic refreshing of OAuth tokens if needed, using more than one OAuth client id for better quota limits handling and so on.

The old OAuth Broker is now deprecated, and we ask you to migrate affected configurations credentials before May 1, 2019. We can't migrate these credentials automatically because we cannot modify configurations in your project without your consent.

In the project Overview, you can see whether your project contains any configurations needing migration: