We have released new versions of these database writers:

• MySQL
• Microsoft SQL Server
• Redshift
• Oracle

Also, we are introducing Cloudera Impala database writer.

All these writers are running on container based architecture with SSH tunnel support.

Here's what last week was about:

• Fixed bug in Redshift backend: Exporting a table with column of maximum allowed length (64KB) no longer causes an error.
• MariaDB, which is used as MySQL backend in transformations, was updated to version 5.5.53. This update enhances the handling of 4-byte UTF-8 characters. In previous version 5.5.44 a string was cut after the first utf8mb4 character. Now, all these characters will be converted to '?'. Full support of 4-byte UTF-8 characters is unfortunately unavailable in MySQL.
• Fixed bug in MySQL extractor: When query returned an empty result, extractor ended up with 'Orphaned manifest' error

Some jobs have failed between 2016-09-26 17:30 CEST and 2016-09-26 17:40 CEST due to problem with one of our DB servers.

All affected orchestrations have been checked and restarted.

We are very sorry for any inconvenience.

If you have any concerns about this, please contact us at support@keboola.com.

As some of you might know, Google Analytics API doesn't always return precise data. Under certain circumstances, the data returned are sampled. Read more about sampling here.

Anti-Sampling

You can choose from one of two anti-sampling algorithms - DailyWalk or Adaptive. Both are based on the same principle, to divide the wanted date range into smaller chunks.

DailyWalk as the name suggest divides the date range by days. So the extractor needs to make as many request as there are days in the date range.

Adaptive algorithm is using more sophisticated approach, and divides the date range into few smaller date ranges. Read in-depth explanation of the algorithm here, if you are interested. 

DailyWalk algorithm might be more precise in some cases, but usually you will get the same results faster with the Adaptive algorithm.

Experiment with them and use what suits you best.

We have encountered a bug in the Google Analytics Extractor. It was downloading data only for one profile, even if the query was set to "All profiles".

This bug was introduced on 16th of September and it is now fixed.

We are very sorry for any inconvenience. Please adjust your date ranges and download the missing data if needed. 

As you may have noticed, we have launched a new version of Google Analytics Extractor.

To help you migrate your existing configurations from the old extractor, we prepared a Migration Tool.

The migration process will migrate all your existing configurations, which are properly authorised and also replace the old extractor in all orchestration tasks in the project.

How to

1. Click "Proceed to Migration" button in any configuration of the old Google Analytics Extractor
2. A dialog window will open with a list of all configurations in the project
3. Click "Migrate" button and a migration job will be created
4. You can watch progress of the migration job either in the migration dialog window or in the Jobs list
5. After job is finished, you can see the result of the migration for every configuration and also affected orchestrations on the "Affected Orchestrations" tab

Backward compatibility

The only thing that is not migrated automatically are the Date Range settings.

In the old extractor, the date ranges were specified in parameters of a orchestration task by "since" and "until" attributes.

In the new extractor, the date ranges are specified in the extractors configuration and you can have multiple date ranges per query.

After migration, the orchestration task parameters will remain as they were. If present, the orchestration task with new extractor will fail. Simply move your date range settings from the orchestration task parameters to the extractors query configuration.

Orchestration error:

Move "since" and "until" parameters from orchestration task to query configuration: 

For more information about new Google Analytics Extractor see documentation.

If you have any troubles or questions, please contact support@keboola.com.

New version of Google Drive Extractor, running on container based architecture, is now available. The old one has been marked deprecated.

Features

Not much has changed in terms of functionality, but the configuration UI is now much faster :) Also the extractor now uses the newest version of the Google Drive and Google Sheets API.

Migration

To help you migrate your configuration to the new version of this extractor, we have prepared a migration tool, which will be available very soon.

New version of Google Analytics Extractor is now available. The old one has been marked deprecated.

Features

Google Analytics Extractor now works with the newest Google Analytics API V4, providing these key features:

• Metrics expressions

  The API allows you to request not only built-in metrics but also combination of metrics expressed in mathematical operations. For example, you can use the expression ga:goal1completions/ga:sessions to request the goal completions per number of sessions.

• Multiple date ranges

  The API allows you to get data from 2 date ranges with a single request.

• Multiple segments

  The API enables you to get data from multiple segments with a single request.

Migration

SalesForce Writer

Heureka.cz and Zbozi.cz Extractors

Get marketing data from two of the most popular price comparison and recommendation shopping engines in Czech Republic. These extractors was developed by Sun Marketing, s.r.o.

Storage API - Load Data Into Workspace

This new feature is only available for Snowflake backend.

Use this API call to load data from tables in Storage API into a Snowflake workspace.

For more details check the documentation.

Transformations - direct data load in Snowflake input mapping

Input mapping from a Snowflake storage table into a Snowflake transformation utilise the Load Data Into Workspace function mentioned above. 

Input mapping now performs faster. 

In some rare cases, we have encountered problems with incremental loads into larger tables (>5 GB), in Storage API buckets with Redshift backend. 

It seems that the update of Redshift to version 1.0.1070 is the culprit here.

We have contacted Amazon AWS support regarding this issue.