There's a new extractor available in our group of Docker extractors - MongoDB Extractor.

This extractor allows you fetch data from your MongoDB databases. By specifying collection, query, sort, limit and mapping you're able to extract exactly which parts of your data you want.

On the other hand, MongoDB extractor is very similar to our new set of standard database extractors, so there's a chance you are already familiar with some parts of its UI.

Main features of the MongoDB extractor:

• you can specify query, sort and limit to filter your data (as in mongoexport commnad)
• each export has to be named, which helps you identify your exports better
• there's a mapping section (actually most important) through which you can specify how your data will be processed, how they'll be split to multiple tables and which columns will be exported (thus you can join them with ease)
• as always, there's an option to have multiple exports in one configuration

Here's a sneak peak of sample configuration:

For more information about its configuration follow the guide at our help site.

We have a few user-friendliness announcements for transformations.  First, transformations will automatically apply a convenient description to the versions it stores:

Further, all new transformation buckets will allow editing names!

In other news, the data-takout function will now export all versions of all component configurations.

There's also been a little re-organisation of the storage API documentation for tables https://docs.keboola.apiary.io:

Hope you're enjoying some fine spring weather.

We have released bunch of new database extractors supporting wide range of vendors:

• MySQL
• Microsoft SQL Server
• PostgreSQL
• Oracle
• Cloudera Impala
• Firebird
• IBM DB2
• MongoDB (coming very soon)

All these new Extractors are running on container based architecture and supports SSH tunneling, so there's no need for tedious VPN setups.

The old Database Extractor is now deprecated, but we will continue to  support if for 3 more months.  After this period, we will migrate any remaining "old" configurations to the new extractors.

We are now preparing a migration tool to help with this task, and when it is ready we will share with you so you can easily migrate your existing configurations to the new extractors.

If you'd like to migrate your configurations immediately, it will have to be done manually, so please let us know at support@keboola.com and we can try to convince you to wait for the tool :)

We've encountered an issue running newly created transformations. Please stay with us, the fix currently being tested and will be deployed within the next hour. 

The symptom is a failing job with error Transformation 'x' not found. Only newly created transformations are affected by this bug, all old transformations should work.

We're sorry for this inconvenience!

EDIT 4.00 pm CEST: The bugfix has been deployed to production. 

New features

- If you have a limited project, we will now send you an email notification a week before the project expires.

- We have updated R in Transformations and Custom Science to R version 3.2.5 (April 2016).

Bug fixes

- Snapshots of Redshift tables with non-lowercase primary keys are now working correctly.

- Project Backup and Takeout now exports configuration rows (e.g. transformation queries) and works for large projects too.

Other posts this week

- Redshift Transformation Input Mapping Update

- Orchestrator table deletion announcement

As of now we're removing type input mapping attribute in Redshift transformations. 

All tables which were set to be created using CREATE VIEW from another schema will now be automatically created using CREATE TABLE and the data will be copied.

There are many reasons behind this update, the most important being a deadlock when deleting tables in Storage that had a view created in a transformation or in an orphaned sandbox.

You may experience slightly slower transformation times and heavier disk usage caused by physically copying data.

Should you experience any unexpected behaviour or a major slowdown, please let us know.

PS: No action required, we're glad you made it here safely.

All Orchestration configurations have been migrated to dedicated configuration storage and old configuration tables in sys.c-orchestrator bucket are no longer used.

Therefore we will automatically delete the configuration bucket sys.c-orchestrator (including its contents) on next Wednesday, April 18. After deletion, you will be informed by KBC notification.

To access orchestration configurations programatically, you must use the Orchestration API.

Docker app configuration with JSON Schema

This feature is mainly useful for 3rd party developers. 

It is now possible to define configuration UI with JSON Schema.

Read more about this feature in documentation.

Slack Extractor

Another addition to our fast growing family of extractors! 

Slack is a great real time messaging app, read more about it.

Use this extractor and other KBC tools to analyse your conversations, see which users / clients are most active in what channels and when. You can do sentiment analyses of messages and much more.

GitHub Extractor

And another new extractor. This time for most popular Git repository hosting service.

The GitHub API provides access to the many public activities that happen on GitHub.com. 

Read more about GitHub Data. Now you can use these in Keboola Connection environment.

S3 Writer fixes

• default path 'kbc_upload' is now used when no remote path is specified or it's just '/'. 
• region 'us-east-1' is now set as default region
• updated documentation

Tableau Writer improvements

You can now use our standard data filter in table export configuration

Geneea NLP Analysis improvements

As in previous case, input date filtering is available

Redshift Transformations

The option for storing persistent tables was removed, because nobody is using it anymore

GoodData Writer improvements

Tables from "in" data stage are newly allowed in configuration

Project settings page redesign

People from Keboola Support are identified by small badge before their icon in users list

Developers

Actions were added to our Docker Runner. It is way to run quick synchronous tasks in your KBC extensions

We have encountered problem with SSO links generated by our GoodData Writer.

UPDATE: This issue has been solved. All SSO links should be valid now.

List of products, which are hit by this problem:

- On demand SSO access to GoodData in Keboola Connection 

- Keboola Academy

- Keboola Sardine

We are working on the fix of the problem. We are sorry for any inconvenience.