Week In Review -- January 16, 2017

Our last week was filled with some bug fixes, performance improvements and old infrastructure deprecation.

Shared Buckets

Shared bucket loading was tweaked to load data directly on Snowflake backend. (available to everyone)

Loading to Sanboxes

Loading data to sandboxes (mainly used by Transformations and Sandboxes themselves) has been improved, which means multiple tables can be loaded at once.

You should notice speed improvement when using more tables in input mapping and loading larger amount of data. (testing phase, available on request)

Also we would like to mention:

Detail of a job with enabled "Multi Load"

Project Power Reset

Since the Project Power feature has been developed/improved in a continuous manner, and the last thing we want to do is provide partial (or worse, inaccurate) results, we have decided to begin measuring credit consumption from the beginning of this year, 2017.

So all Project Power statistics from 2016 will no longer be shown, and the Project Power era officially begins for everybody from Jan 5, 2017 (as in image bellow).

Thanks for your understanding.

Error on job detail page

Due to recent update in one of our components few of you may noticed "Internal Error" message while waiting for job to finish its processing.

Issue affected users browsing job detail page between 17:20 and 17:40 CEST (Jan 5, 2017).

We are very sorry for inconvenience.

Weeks in Review -- September 12, 2016

Google Drive Extractor

Google Drive Extractor has been revamped. Not much has changed, this was standard update due to our container based infrastructure. Just one thing to mention -- its UI is much faster. Read more about New incarnation of Google Drive Extractor.

Migration tool for Google Analytics Extractor

Another important thing is Migration Tool for new Google Analytics Extractor.

Transformation/Sandboxes Provisioning

Also, we have changed the way how Redshift transformations and sandboxes are provisioned. Migration of existing projects is prepared to week from September 26th to September 30th.

Custom SQL Aliases

We decided to drop support for custom SQL aliases (existing aliases will continue to work), due to theirs rare usage. You can still use Simple Aliases, which are now supported for Redshift backend too.

As always, we did many small improvements:

  • Search in components while creating new orchestration (UI)

  • AdWords Extractor update to new AdWords API v201607 (Extractor Backend)
  • Automated Configuration Adjustment (UI)
  • Show Multi-factor Authentication (MFA) status in users list -- not enabled MFA produces warning (UI)

Extractor Failures

We have experienced some extractor failures between 2016-09-09 21:00 CEST and 2016-09-10 12:00 CEST.

Affected extractors were Gmail, Twitter, MongoDB and LinkedIn.

Due to these errors, some of the runs exited with failure or wrote to a wrong bucket stage.

All affected orchestrations have been checked and started again.

We are sorry for any inconvenience.

If you have any concerns about this, please contact us at support@keboola.com.

MongoDB Extractor

There's a new extractor available in our group of Docker extractors - MongoDB Extractor.

This extractor allows you fetch data from your MongoDB databases. By specifying collection, query, sort, limit and mapping you're able to extract exactly which parts of your data you want.

On the other hand, MongoDB extractor is very similar to our new set of standard database extractors, so there's a chance you are already familiar with some parts of its UI.

Main features of the MongoDB extractor:

  • you can specify query, sort and limit to filter your data (as in mongoexport commnad)
  • each export has to be named, which helps you identify your exports better
  • there's a mapping section (actually most important) through which you can specify how your data will be processed, how they'll be split to multiple tables and which columns will be exported (thus you can join them with ease)
  • as always, there's an option to have multiple exports in one configuration

Here's a sneak peak of sample configuration:

For more information about its configuration follow the guide at our help site.

Week in Review -- April 4, 2016

Last week was shorter for us due to Easter, but we managed to do more things in a shorter time.

Versions Management

We introduced simple management for transformation buckets. You are now able to rollback to the previous version of your transformation. Or simply copy one version to another.

Read more about Versions Management in previous post.


We're still working on our documentation to provide you with best support. Whether for easier components configurations or diving more into Keboola Connection in general.

Check our post about Documentation Changes.


Thanks to you, our users, we can detect anomalies (or bugs) in components faster. Thus after applying patches, components will be better for all, not only for issue reporters.

  • Small fixes have been applied to Keboola Connection UI
  • Bings Ads extractor, provided by David Ešner, has been updated

Gmail Extractor

We are proud to announce a new member of our group of Docker extractors - Gmail Extractor.

This extractor helps you fetch messages from your Inbox by specifying simple queries. These queries are identical to those you're using in Gmail's web interface. If you're more into advanced search, check this Gmail's Help site.

Main features of the Gmail Extractor:

  • you can specify multiple queries
  • output is divided to multiple tables, which gives you more freedom when selecting/joining data
  • multiple runs of the same query won't cause downloading the same data
  • you can specify the headers you want to download

The process for adding the Gmail Extractor is very similar to the other extractors (e.g. Adwords Extractor v2)

1. Find Gmail Extractor in the list of extractors

2. Create a new configuration for this extractor

3. Authorize the extractor to access your inbox

4. Configure the extractor and specify the queries you want to run

For more detailed information about the extractor configuration options please see the documentation.