Week in Review -- March 27, 2017

Trash for configurations

One of the most wanted features in Keboola Connection was the option to restore deleted configuration.

We are happy to announce this feature is now avalable for all users.

This feature allows you to:

1. Check who and when deleted a configuration

2. Restore configuration

Configuration Restore is supported by all components, except Orchestrator, GoodData Writer and some old components which have configuration in a bucket created in `sys` stage.

3. Delete configurations permanently

If you use "Empty Trash" button, all configurations will be deleted permanently. There is no other way to restore them!

As always, you can use mentioned functionality by calling our Storage API: http://docs.keboola.apiary.io/#reference/component-configurations/configuration-restore/restore-deleted-configuration

Fixes and improvements

  • added optional parameter to specify warehouse in Snowflake Database Writer
  • fixed credentials editing in DB writers

Week in Review -- March 20, 2017

New Components

  • Facebook Extractors have been rewritten. For more information see this post.

New Features

  • We added Table Data Filter to latest versions of our database writers. So it's possible to filter data by "days" and "conditions"

  • New option of bucket sharing has been added. Now you can share buckets to your whole organization (by setting new sharing option) and anyone in your organization be able to link bucket to his project.

Minor Improvements & Fixes

  • From now, attempt to set nonexistent column as Primary Key (in Storage API) will cause an user error

Week In Review -- January 16, 2017

Our last week was filled with some bug fixes, performance improvements and old infrastructure deprecation.

Shared Buckets

Shared bucket loading was tweaked to load data directly on Snowflake backend. (available to everyone)

Loading to Sanboxes

Loading data to sandboxes (mainly used by Transformations and Sandboxes themselves) has been improved, which means multiple tables can be loaded at once.

You should notice speed improvement when using more tables in input mapping and loading larger amount of data. (testing phase, available on request)

Also we would like to mention:

Detail of a job with enabled "Multi Load"

Project Power Reset

Since the Project Power feature has been developed/improved in a continuous manner, and the last thing we want to do is provide partial (or worse, inaccurate) results, we have decided to begin measuring credit consumption from the beginning of this year, 2017.

So all Project Power statistics from 2016 will no longer be shown, and the Project Power era officially begins for everybody from Jan 5, 2017 (as in image bellow).

Thanks for your understanding.

Error on job detail page

Due to recent update in one of our components few of you may noticed "Internal Error" message while waiting for job to finish its processing.

Issue affected users browsing job detail page between 17:20 and 17:40 CEST (Jan 5, 2017).

We are very sorry for inconvenience.

Weeks in Review -- September 12, 2016

Google Drive Extractor

Google Drive Extractor has been revamped. Not much has changed, this was standard update due to our container based infrastructure. Just one thing to mention -- its UI is much faster. Read more about New incarnation of Google Drive Extractor.

Migration tool for Google Analytics Extractor

Another important thing is Migration Tool for new Google Analytics Extractor.

Transformation/Sandboxes Provisioning

Also, we have changed the way how Redshift transformations and sandboxes are provisioned. Migration of existing projects is prepared to week from September 26th to September 30th.

Custom SQL Aliases

We decided to drop support for custom SQL aliases (existing aliases will continue to work), due to theirs rare usage. You can still use Simple Aliases, which are now supported for Redshift backend too.

As always, we did many small improvements:

  • Search in components while creating new orchestration (UI)

  • AdWords Extractor update to new AdWords API v201607 (Extractor Backend)
  • Automated Configuration Adjustment (UI)
  • Show Multi-factor Authentication (MFA) status in users list -- not enabled MFA produces warning (UI)

Extractor Failures

We have experienced some extractor failures between 2016-09-09 21:00 CEST and 2016-09-10 12:00 CEST.

Affected extractors were Gmail, Twitter, MongoDB and LinkedIn.

Due to these errors, some of the runs exited with failure or wrote to a wrong bucket stage.

All affected orchestrations have been checked and started again.

We are sorry for any inconvenience.

If you have any concerns about this, please contact us at support@keboola.com.

MongoDB Extractor

There's a new extractor available in our group of Docker extractors - MongoDB Extractor.

This extractor allows you fetch data from your MongoDB databases. By specifying collection, query, sort, limit and mapping you're able to extract exactly which parts of your data you want.

On the other hand, MongoDB extractor is very similar to our new set of standard database extractors, so there's a chance you are already familiar with some parts of its UI.

Main features of the MongoDB extractor:

  • you can specify query, sort and limit to filter your data (as in mongoexport commnad)
  • each export has to be named, which helps you identify your exports better
  • there's a mapping section (actually most important) through which you can specify how your data will be processed, how they'll be split to multiple tables and which columns will be exported (thus you can join them with ease)
  • as always, there's an option to have multiple exports in one configuration

Here's a sneak peak of sample configuration:

For more information about its configuration follow the guide at our help site.