Deprecating Redshift Bucket Credentials

In November 2014, we were really happy to announce a new cool feature -- Read-only bucket credentials to your Redshift backend. Time has passed and we have decided to deprecate this feature.

Though accessing data in a bucket is very useful, with the current scenario it can lead to an assumption how data is stored and how it'll be stored in future. The latter is much worse, because any change in our backend can lead to breakage of the system, which was programmed to access the data in a bucket.

Thus, we recommend you to access your data in a more robust way:

  1. Create a new Redshift Writer
  2. Choose a database provided by Keboola or your own Redshift database
  3. Configure which tables you need to synchronize
  4. Run the newly configured writer
  5. Copy the credentials created in step 2 to the software which needs to access your data

The existing Bucket credentials continue to work until April 18, 2018. However, we strongly recommend to migrate to the Redshift Writer as soon as possible.

The creation of new credentials and credentials listing will be disabled in the Storage section with the publication of this post.

Also, the Bucket credentials have been marked as deprecated in our Storage API.

Week in Review -- March 27, 2017

Trash for configurations

One of the most wanted features in Keboola Connection was the option to restore deleted configuration.

We are happy to announce this feature is now avalable for all users.

This feature allows you to:

1. Check who and when deleted a configuration

2. Restore configuration

Configuration Restore is supported by all components, except Orchestrator, GoodData Writer and some old components which have configuration in a bucket created in `sys` stage.

3. Delete configurations permanently

If you use "Empty Trash" button, all configurations will be deleted permanently. There is no other way to restore them!

As always, you can use mentioned functionality by calling our Storage API: http://docs.keboola.apiary.io/#reference/component-configurations/configuration-restore/restore-deleted-configuration


Fixes and improvements

  • added optional parameter to specify warehouse in Snowflake Database Writer
  • fixed credentials editing in DB writers

Week in Review -- March 20, 2017

New Components

  • Facebook Extractors have been rewritten. For more information see this post.

New Features

  • We added Table Data Filter to latest versions of our database writers. So it's possible to filter data by "days" and "conditions"

  • New option of bucket sharing has been added. Now you can share buckets to your whole organization (by setting new sharing option) and anyone in your organization be able to link bucket to his project.

Minor Improvements & Fixes

  • From now, attempt to set nonexistent column as Primary Key (in Storage API) will cause an user error

Week In Review -- January 16, 2017

Our last week was filled with some bug fixes, performance improvements and old infrastructure deprecation.

Shared Buckets

Shared bucket loading was tweaked to load data directly on Snowflake backend. (available to everyone)

Loading to Sanboxes

Loading data to sandboxes (mainly used by Transformations and Sandboxes themselves) has been improved, which means multiple tables can be loaded at once.

You should notice speed improvement when using more tables in input mapping and loading larger amount of data. (testing phase, available on request)

Also we would like to mention:


Detail of a job with enabled "Multi Load"

Project Power Reset

Since the Project Power feature has been developed/improved in a continuous manner, and the last thing we want to do is provide partial (or worse, inaccurate) results, we have decided to begin measuring credit consumption from the beginning of this year, 2017.

So all Project Power statistics from 2016 will no longer be shown, and the Project Power era officially begins for everybody from Jan 5, 2017 (as in image bellow).

Thanks for your understanding.

Error on job detail page

Due to recent update in one of our components few of you may noticed "Internal Error" message while waiting for job to finish its processing.

Issue affected users browsing job detail page between 17:20 and 17:40 CEST (Jan 5, 2017).

We are very sorry for inconvenience.

Weeks in Review -- September 12, 2016

Google Drive Extractor

Google Drive Extractor has been revamped. Not much has changed, this was standard update due to our container based infrastructure. Just one thing to mention -- its UI is much faster. Read more about New incarnation of Google Drive Extractor.

Migration tool for Google Analytics Extractor

Another important thing is Migration Tool for new Google Analytics Extractor.

Transformation/Sandboxes Provisioning

Also, we have changed the way how Redshift transformations and sandboxes are provisioned. Migration of existing projects is prepared to week from September 26th to September 30th.

Custom SQL Aliases

We decided to drop support for custom SQL aliases (existing aliases will continue to work), due to theirs rare usage. You can still use Simple Aliases, which are now supported for Redshift backend too.


As always, we did many small improvements:

  • Search in components while creating new orchestration (UI)

  • AdWords Extractor update to new AdWords API v201607 (Extractor Backend)
  • Automated Configuration Adjustment (UI)
  • Show Multi-factor Authentication (MFA) status in users list -- not enabled MFA produces warning (UI)

Extractor Failures

We have experienced some extractor failures between 2016-09-09 21:00 CEST and 2016-09-10 12:00 CEST.

Affected extractors were Gmail, Twitter, MongoDB and LinkedIn.

Due to these errors, some of the runs exited with failure or wrote to a wrong bucket stage.

All affected orchestrations have been checked and started again.

We are sorry for any inconvenience.

If you have any concerns about this, please contact us at support@keboola.com.