Keboola Status

Today between 12:13 and 13:43 we had an error in one of our compontents, which caused some jobs to fail with message: 

Running container exceeded the timeout of 3600 seconds.

even if the job did not in fact exceed a timeout. Failed orchestrations have been restared. We are really sorry about this.

I am very proud to announce that there is a new addition to the family of Keboola Extractors. This time it is Google Calendar Extractor.

It can download all your calendars available in your account with events and details. It is also usable for downloading Google's calendars with all national holidays as well. 

This extractor is a result of cooperation between Prague and Singapore Keboola. In case of questions or need to clarify something, please, contact vokurka@keboola.com.

I am proud to announce the launch of a new Keboola Connection component - the Mailkit extractor.

With this component you can easily download all valuable insights about your e-mail campaigns provided by this powerful e-mail marketing tool.

Simple configuration supporting the new generic UI based on the configuration schema makes it easy and comfortable to set up. You can simply choose datasets you are interested in or let the component download all of them. The time range of retrieved data can be easily set by specifying either relative period or you can set the period based on the last run of a configuration, which is especially useful in orchestration scenarios. 

The component is developed as 3^rd party extension. For more information on configuration, please refer to the documentation. In case of any issues or questions, please contact me at esnerda@gmail.com.

P.S. The Mailkit Writer component to accompany this one is coming soon, so stay tuned.

As we have announced earlier, we have launched a bunch of new DB extractors, supporting various vendors. The "old" Database Extractor became deprecated and as we promised, we've prepared a tool for you, to help you migrate your configurations to the new extractors.

To initiate migration, click "Proceed to Migration" button in any configuration of the "old" Database Extractor.

A dialog window will open, where you will see what configuration will be migrated, status of the migration and also affected orchestrations.

If you need any assistance, contact support@keboola.com.

On Jun 13, 2016 from 5:50am till 6:20am UTC+2 we encountered issues with DNS that may have led to failure of some running jobs at the time. We are sorry for any inconvenience.

We've launched a new extractor for Pingdom.

This extractor will fetch data collected by the Pingdom service about your web application's uptime and performance metrics. This allows you to directly examine your application's performance and what effects that may have on your campaigns, sales activities, or other business metrics.

Detailed descriptions of the data structure and a guide to help get you started are available in the KBC User Documentation.

Let's start the week with a resume of features/improvements introduced in the last week.

New/Updated Components

• MongoDB extractor
• Prague's stand up comedian and node.js developer Radek Tomasek created FTP/FTPS Writer and added visual configuration for his SFTP/WebDAV writer

UI Improvements

• Primary key in input mappings is automatically populated when the destination table already exists
• Performance improvements in generic extractor (and in extractors based on generic extractor, eg. all templated extractors) - parsing large JSON responses is significantly faster
  

Geocoding Augmentation Extractor has been rewritten as Docker component and the old version will be switched off by the end of June. Till then its users are kindly requested to migrate their configuration to the new version. Here is how to do it:

1.  Choose Geocoding v2 from list of extractors
   

   
2. Create new configuration and give it some name as usual
   

   
3. Add extraction configuration
   
1. Add one or more input tables to input mapping. Please note that the table must have exactly one column with locations (or two columns with latitudes and longitudes in case of reverse geocoding) or you have to map the one (or two) columns in the input mapping.
   
2. Add exactly one table to output mapping which will be filled with result of geocoding.
   
3. Fill parameters configuration. It is in json format and must contain method of geocoding, data provider and other optional parameters like api key or locale. See https://github.com/keboola/geocoding-augmentation#configuration for more details
   

It's my pleasure to announce another addition to Keboola Connection - a FTP/FTPS Writer. 

I was asked to extend the SFTP/WebDAV Writer and add a support for FTP/FTPS protocols. However, after some discussion with Keboola developers, a decision of making a separated connector was made.

The use-case is simple, you can upload your data from Keboola Storage to either FTP or FTPS location. The great thing from my perspective is that this writer supports the new configuration schema that helps to pass the input configuration in a very convenient way.

This writer is developed independently by Blue Sky Media. For more information on how to use the writer, please refer to the documentation. In case you run into some issues or you have more questions, please contact me directly (radek@bluesky.pro). 

On Jun 2, 2016 3:40pm UTC+2 a new version of Storage API was released containing the following bug.

Incremental loads into Redshift tables with primary keys do not correctly deduplicate data - rows with duplicate primary key may exist in the table.

We'll be deploying the original version shortly and then we'll dedup all affected tables.

We're sorry for this inconvenience, we'll keep you updated in this post. 

UPDATE Jun 3, 2016, 2:30pm UTC+2

Original version deployed. We're starting recovery process, no service outage will be required.

UPDATE Jun 4, 2016, 9:15am UTC+2

Recovery process for all affected tables is finished, all duplicate records should be mitigated. We're now investigating the root cause of the issue to prevent similar incidents in the future.