Keboola Status

New Features and Updates

Incremental Fetching

The following extractors now support Incremental Fetching:

• Snowflake Extractor
• Oracle Extractor
• Redshift Extractor
  
• Firebird Extractor
  
• Cloudera Impala Extractor 

Job Performance

When a job is finished, we show you how long it took. If a job takes 10% longer (or less) than the average, we'll show you the difference.

Transformation Buckets

The list of Transformation Buckets is no longer collapsible and looks like other lists of configurations in an application.

Bucket details in Data Catalog

In a source project (a project which shares a bucket and "owns" data), you are able to see the bucket detail.

• You can see how the bucket is shared (to organization members, projects members, selected projects, etc.).
• You can edit the description of the bucket — it will be available for those users who want to link the bucket to their project.
• You can see a list of tables and bucket properties (size, row count, etc.).
• Also, if someone uses the bucket, you can see which project it is (Used by section).

In a destination project (where you can see and use a bucket):

• You can see the description of the bucket and decide if you want to use it.
• You can see a list of tables and bucket properties (size, row count, etc.).
• If you decide to use the bucket, you will also see the link to Storage.

Show tables searched by metadata in Writers

In case you use our new feature Scaffolds, related writers (e.g., the Snowflake database writer) will show you all input tables that are searched by metadata.

Environment updates

• R updated to 3.6.2
• Python updated to 3.7.5

New Components

• Deal Prediction application by Bootiq predicts deal outcomes based on metrics from CRM software of your choice.
• Google My Business extractor gets reviews and media related information from your Google My Business account.
  

We started redesigning Keboola Connection in June, 2019. The first pages we changed were the pages related to authentication, followed by moving and redesigning the navigation. Today, we're happy to announce that the 3rd part of Keboola Connection redesign is ready and deployed. The content itself hasn't changed that much, but we invested a lot of time and effort into making all parts look similar to bring you better user experience.

Let's summarize the biggest changes:

• New project dashboard - shows what's happening in the project.
• Scaffolds - predefined blocks of configurations for specific use cases
• Data Catalog - a new way of sharing data in the organization
• Option to share buckets for a specific project in the organization
• Option to share individual tables from different buckets (buckets are created automatically)

Brand New Dashboard

The new dashboard is the entry point to your project and you should find the most important information on the project there. You can learn what users are active in your project, who created a configuration recently, which jobs are currently running, etc.

Extractors, Writers and Applications are Components

We grouped all components together in the main navigation.

Each section has two parts:

• Directory - list of all components available for the selected type (extractor, writer, and application)
• My components - list of all components you have configured (also for the selected type)

Scaffolds (New in Keboola Connection)

Scaffolds allow you to use a group of prepared configurations for a specific use case by filling in only a few details (e.g., credentials). No additional configuration is required.

Data Catalog (New in Keboola Connection)

This is a new way you can now see the data shared from/to your project:

• Data which you can use in your project (the section Shared with you)
• Data you shared from your project (the section Shared from this project)

Bucket Sharing

You can share a bucket directly from the Data Catalog section:

Simple sharing - when you already have a prepared bucket you want to share.

Advanced sharing - when you want to create a new bucket with specific tables.

Also, a new type of sharing has been added. From now on, you can share a bucket to a specific project in your organization.

Storage, Transformations, Orchestrations

These sections have been slightly redesigned, but you should be able to find everything in the same place.

There are a few things we want to mention:

• In the orchestration detail, the configured token is more visible.
• Jobs in the orchestration detail can be filtered by their status (successful vs. failed).
• In the transformation detail, you can find the information about the backend you used and the phase on the right side.

Jobs

The Job detail page has also been redesigned.

Again, as with Transformations or Orchestrations, the content stays the same. We only reorganized it a bit. The event detail is shown in modal. Job parameters and Mapping (information about input/output tables) are shown in modal as well. Just click "Show all" in the particular section.

User and Project menus

These menus haven't changed much either. They were moved to the same line (as was Trash).

Anyway, we're going to continue to improve Keboola Connection for you, there are definitely more things to come. We hope you like our new UI and would love to hear your feedback!

New Features, Improvements and Minor Fixes

• When checking events in the job detail, the Load More button tries to load up to 1,000 events at once.
• You can search in component configurations when adding a new task to an orchestration phase.

• MongoDB extractor supports an optional Authentication Database parameter.
  
  

• Refreshing your token no longer breaks access to your project.
• Remove Empty Files and Folders processor has a new option available to remove files with whitespace characters only: remove_files_with_whitespace
  

New Components

• Bitbucket extractor for getting data about commits to your team's repositories
• PowerBI writer
• Sendinblue application for relationship marketing 
• SurveyMonkey extractor
  
• MapboxUtils application by CleverAnalytics for manipulating vector tilesets on Mapbox

Some of our users experienced errors in Keboola Connection (in both regions) yesterday when saving configurations of several components.

An error was displayed to users who tried to save their configuration between Jul 9, 2019, 12:35 CET and Jul 10, 2019, 14:18 CET.

Mainly users who have an older version of Chrome Browser or any version of the Firefox browser were affected.

The issue was caused by wrong response headers (unsupported by the mentioned browsers) being sent to a preflight request on one of our API services.

We sincerely apologize for the inconvenience caused.

Three weeks ago we announced the start of Keboola Connection redesign.

We're glad we can share another important update with you.

We moved the navigation from the left side to the top

This allows you to more quickly navigate within a project and should improve user experience. All sections remain the same and the content hasn't changed.

• Project Settings can be found on the right side (by clicking on the "cogs")
  

• The Project Selector is almost the same as before

• The User Settings menu has a new home on the top right side

• Also, we added a shortcut to the Trash, so you no longer need to navigate there through User & Settings

We hope you'll like this update as much as we do.

New Features and Improvements

Require MFA for organization

There's a new option in organization-level settings which allows you to require Multi-Factor Authentication for each member of your organization and project.

Users not having the Multi-Factor Authentication enabled will be prompted to enable it.

Set same data type to all columns at once

When setting data types in transformation's Input Mapping you can set same data type to all columns at once.

GoodData Writer data types

When configuring tables in GoodData writer, we try to look for metadata for columns and set types for you automatically.

New Components

Google Translate

Translates text into any supported language.

https://components.keboola.com/components/kds-team.ex-google-translation

Google NLP

New component which uses Google Cloud Natural Language API to extract text features using pre-trained machine learning models.

https://components.keboola.com/components/kds-team.ex-google-nlp

Many of you have noticed that our website was redesigned a few months ago. We have been continuing to prepare and apply a new design to all remaining parts of our system:

Four months ago, we introduced the Refined Storage Console, even though the main point of the changes was to remove old code. Four weeks ago, we started using the Roboto font in Keboola Connection and Developer Portal.

We're now happy to announce that the first part of the redesign has been applied to all pages you see during the authentication process.

The authentication process itself wasn't affected by the changes and you should be able to go through it with ease.

There are many more things to come from us. We're already working on the second phase of the redesign. Stay tuned.

We're happy to announce a small technology update of our Storage Console. Several months in the making, this will allow us to bring new features in the near future.

Even though the primary purpose of this update is to bring the code up to date and align the design with the rest of the UI, we have already made some small improvements:

• You can no longer see an additional loading page when navigating to Storage from other pages.
• Search in buckets (or tables) highlights the matched parts of your search query in yellow.

• An active bucket is highlighted on the left side when its detail or a detail of its table is active.

• Files and Jobs sections are automatically reloaded every 20 seconds.
• Event sections have predefined searches, so you can filter events faster.

• Buttons Create Bucket, Link Bucket and Reload are now bigger.

• There's an option to create Table Alias directly from a table detail (in Actions).

• Other minor cosmetic things like navigation, buttons, etc.

Components

Google Drive Writer (wr-google-drive)

• We announced the new version of Google Drive Writer on June 28th, 2017.
• This is the last call, the old component will be shut down by the end of this month - January 31st, 2019.

Lucky Guess (rt-lucky-guess)

• Enhanced Analysis for Redshift backend is no longer available.
• This component will be shut down on January 31st, 2019.

SalesForce Extractor (ex-salesforce)

• This component is already deprecated.
• This component will be shutdown on January 31st, 2019.

YouTube Extractor (ex-youtube)

• This component is already deprecated and no jobs were run in last months.
• This component will be shut down immediately.

Zendesk Extractor (ex-zendesk)

• A new version of the Zendesk extractor was announced on July 27, 2016.
• Like the YouTube Extractor, no jobs were run in last months and we will shut it down immediately.

Restbox (restbox)

• This component was deprecated on July 16, 2018 with multiple replacements.
• Same here, no jobs were run in last months and it will be shut down immediately.

MySQL Transformations

MySQL Transformations were deprecated on November 13, 2017. As we promised, MySQL Transformations were supported in 2018 and will be finally shut down on January 31st, 2019.

Deprecated Bucket and Table attributes

We are deprecating Bucket and Table attributes. These attributes were used as configuration storage for legacy components. If you need to store additional information with buckets or tables, please use the Metadata API.

There was an outage of the Keboola Connection platform in the US region from Dec 13, 2018, 23:19 CET to Dec 14, 2018, 02:28 CET. It was caused by an update of the Elasticsearch Service.

It was a self-service update of the managed Amazon Elasticsearch Service, and we decided to do it because updates were tested and all other clusters we're using were updated successfully - without any issues.

Updates like this usually go smoothly, but not this time. Unfortunately, the cluster froze and refused to accept any requests.

After detecting this, we decided to start a new service and restore from backup.

All services were fully restored to their normal state by Dec 14, 2018, at 02:28 CET.

We have already taken action to prevent this kind of failure from happening again, and we'll be testing all future updates on testing clusters (using snapshots from production clusters).

Any orchestrations in the US region that were scheduled to start during the outage were not started. Therefore you will need to run them manually or wait until their next scheduled run.

We want to sincerely apologize for the inconvenience caused by this outage.