Keboola Status

Trash for configurations

One of the most wanted features in Keboola Connection was the option to restore deleted configuration.

We are happy to announce this feature is now avalable for all users.

This feature allows you to:

If you use "Empty Trash" button, all configurations will be deleted permanently. There is no other way to restore them!

As always, you can use mentioned functionality by calling our Storage API: http://docs.keboola.apiary.io/#reference/component-configurations/configuration-restore/restore-deleted-configuration

Fixes and improvements

• added optional parameter to specify warehouse in Snowflake Database Writer
• fixed credentials editing in DB writers

New Components

• Facebook Extractors have been rewritten. For more information see this post.

New Features

• We added Table Data Filter to latest versions of our database writers. So it's possible to filter data by "days" and "conditions"

• New option of bucket sharing has been added. Now you can share buckets to your whole organization (by setting new sharing option) and anyone in your organization be able to link bucket to his project.

Minor Improvements & Fixes

• From now, attempt to set nonexistent column as Primary Key (in Storage API) will cause an user error

There were few job failures between 20:55 and 21:15 CEST, due to problems on one of our workers.

We're sorry for this inconvenience, we have restarted the affected orchestrations.

Our last week was filled with some bug fixes, performance improvements and old infrastructure deprecation.

Shared Buckets

Shared bucket loading was tweaked to load data directly on Snowflake backend. (available to everyone)

Loading to Sanboxes

Loading data to sandboxes (mainly used by Transformations and Sandboxes themselves) has been improved, which means multiple tables can be loaded at once.

You should notice speed improvement when using more tables in input mapping and loading larger amount of data. (testing phase, available on request)

Also we would like to mention:

• Old server providing MySQL sandboxes has been replaced due to failures.
  
• Snowflake Database Writer has been published
  
• and Migration Tool for Google Drive Extractor arrived too
  

Detail of a job with enabled "Multi Load"

There were few problems with MySQL sandboxes between 12:40 and 13:20 CEST.

Affected server has been deactivated and all operations should run normally right now.

We're sorry for inconvenience.

Since the Project Power feature has been developed/improved in a continuous manner, and the last thing we want to do is provide partial (or worse, inaccurate) results, we have decided to begin measuring credit consumption from the beginning of this year, 2017.

So all Project Power statistics from 2016 will no longer be shown, and the Project Power era officially begins for everybody from Jan 5, 2017 (as in image bellow).

Thanks for your understanding.

Due to recent update in one of our components few of you may noticed "Internal Error" message while waiting for job to finish its processing.

Issue affected users browsing job detail page between 17:20 and 17:40 CEST (Jan 5, 2017).

We are very sorry for inconvenience.

Google Drive Extractor

Google Drive Extractor has been revamped. Not much has changed, this was standard update due to our container based infrastructure. Just one thing to mention -- its UI is much faster. Read more about New incarnation of Google Drive Extractor.

Migration tool for Google Analytics Extractor

Another important thing is Migration Tool for new Google Analytics Extractor.

Transformation/Sandboxes Provisioning

Also, we have changed the way how Redshift transformations and sandboxes are provisioned. Migration of existing projects is prepared to week from September 26th to September 30th.

Custom SQL Aliases

We decided to drop support for custom SQL aliases (existing aliases will continue to work), due to theirs rare usage. You can still use Simple Aliases, which are now supported for Redshift backend too.

As always, we did many small improvements:

• Search in components while creating new orchestration (UI)

• AdWords Extractor update to new AdWords API v201607 (Extractor Backend)
• Automated Configuration Adjustment (UI)
• Show Multi-factor Authentication (MFA) status in users list -- not enabled MFA produces warning (UI)

We have experienced some extractor failures between 2016-09-09 21:00 CEST and 2016-09-10 12:00 CEST.

Affected extractors were Gmail, Twitter, MongoDB and LinkedIn.

Due to these errors, some of the runs exited with failure or wrote to a wrong bucket stage.

All affected orchestrations have been checked and started again.

We are sorry for any inconvenience.

If you have any concerns about this, please contact us at support@keboola.com.

There's a new extractor available in our group of Docker extractors - MongoDB Extractor.

This extractor allows you fetch data from your MongoDB databases. By specifying collection, query, sort, limit and mapping you're able to extract exactly which parts of your data you want.

On the other hand, MongoDB extractor is very similar to our new set of standard database extractors, so there's a chance you are already familiar with some parts of its UI.

Main features of the MongoDB extractor:

• you can specify query, sort and limit to filter your data (as in mongoexport commnad)
• each export has to be named, which helps you identify your exports better
• there's a mapping section (actually most important) through which you can specify how your data will be processed, how they'll be split to multiple tables and which columns will be exported (thus you can join them with ease)
• as always, there's an option to have multiple exports in one configuration

Here's a sneak peak of sample configuration:

For more information about its configuration follow the guide at our help site.