Google Analytics Extractor Bug

We have encountered a bug in the Google Analytics Extractor. It was downloading data only for one profile, even if the query was set to "All profiles".

This bug was introduced on 16th of September and it is now fixed.

We are very sorry for any inconvenience. Please adjust your date ranges and download the missing data if needed. 

Week in Review -- September 19, 2016

Call for testers: OpenRefine Transformations BETA

Our new  OpenRefine transformations need some testers. Do you want early access and want to play with OpenRefine in Keboola Connection? Please contact us at support@keboola.com.

Google Analytics Extractor

The Google Analytics Extractor can now parse urls for queries.  So you can create your query with the very convenient  Google Analytics Query Explorer and simply copy and paste the url.

Versions management

Simplified access to latest version diff.

Announced Redshift maintenance canceled

We are cancelling previously announced maintenance of Redshift projects 

Other improvements and bugfixes

  • New Hubspot extractor published
  • Adwords extractor - more verbose logging
  • PostgreSQL extractor - fixed retry mechanism which previously caused invalid CSVs with duplicated header
  • Storage - fixed table size and rows count for Snowflake backed projects
  • Storage configurations - numeric ids are generated, fixed bug in duplicate ids creation
  • GoodData Writer - fixed grain settings for tables with custom identifiers 
  • Elasticsearch Writer - SSH tunnel support added


New HubSpot Extractor

We've created new HubSpot extractor which enables you to extract basic data from your CRM.

It is based on our Generic Extractor with predefined templates to helps download your data easily. That also brings an option to adjust the configuration to your needs by switching to JSON editor.

If you want to learn more about the configuration of the extractor, how to get your HubSpot API Token and details of the extractor output tables, look into our new documentation.

Feel free to try the new extractor and in if you have any questions or something is missing, please contact us at support@keboola.com.

Weeks in Review -- September 12, 2016

Google Drive Extractor

Google Drive Extractor has been revamped. Not much has changed, this was standard update due to our container based infrastructure. Just one thing to mention -- its UI is much faster. Read more about New incarnation of Google Drive Extractor.

Migration tool for Google Analytics Extractor

Another important thing is Migration Tool for new Google Analytics Extractor.

Transformation/Sandboxes Provisioning

Also, we have changed the way how Redshift transformations and sandboxes are provisioned. Migration of existing projects is prepared to week from September 26th to September 30th.

Custom SQL Aliases

We decided to drop support for custom SQL aliases (existing aliases will continue to work), due to theirs rare usage. You can still use Simple Aliases, which are now supported for Redshift backend too.


As always, we did many small improvements:

  • Search in components while creating new orchestration (UI)

  • AdWords Extractor update to new AdWords API v201607 (Extractor Backend)
  • Automated Configuration Adjustment (UI)
  • Show Multi-factor Authentication (MFA) status in users list -- not enabled MFA produces warning (UI)

Extractor Failures

We have experienced some extractor failures between 2016-09-09 21:00 CEST and 2016-09-10 12:00 CEST.

Affected extractors were Gmail, Twitter, MongoDB and LinkedIn.

Due to these errors, some of the runs exited with failure or wrote to a wrong bucket stage.

All affected orchestrations have been checked and started again.

We are sorry for any inconvenience.

If you have any concerns about this, please contact us at support@keboola.com.

Redshift Transformation/Sandbox Provisioning

We're changing the way Redshift transformations and sandboxes are provisioned.

During the week from September 26th to September 30th all projects will be migrated. What does that mean for you?

Faster with less errors

In certain situations (eg. table locks) creating of a sandbox would take a long time. After the migration provisioning will no longer depend on any locks in the whole Redshift cluster.

Provisioning and data loading

We're offloading the input mapping work to Storage API. Storage API is now in charge of creating the sandbox or transformation workspace for you as well as loading the data into it. Storage API will decide the fastest way to load required data into your workspace.

Credentials change

Username, password and schema for your Redshift sandbox will change and new sandboxes will be created. Your current sandboxes will be deleted 7 days after the migration. UI will no longer serve credentials for your current sandbox, only for the newly generated one. You will not be able to load data into your current sandbox.

No direct access to bucket data

If your transformation uses data directly from a Redshift schema, this won't be supported. Queries like

SELECT * FROM "in.c-mybucket"."table" 

will no longer have access to in.c-mybucket Redshift schema. If your transformation contains such queries, please adjust the query so that it uses a table specified in the input mapping.

Transparent and hassle free migration

There will be no service interruption and no expected delay in your transformations. Unless you're using direct access to bucket data there is no action required. In case of any problems the whole migration is reversible. 

If you are concerned about your operations please get in touch with us at support@keboola.com. We can try out the migration or change the date of migration. 

Redshift Transformation Failures

One of yesterday's updates has introduced a bug in transformation engine. Output mappings with boolean columns are failing with the following error:

Only a few projects are affected by this bug and we're working on a fix that will be ready within next couple hours. We will update this post when the fix is deployed. 

We're sorry for this inconvenience.

UPDATE 1:36 CEST: The fix was deployed into production. Everything should work as expected.

Google Analytics Extractor Migration

As you may have noticed, we have launched a new version of Google Analytics Extractor.

To help you migrate your existing configurations from the old extractor, we prepared a Migration Tool.

The migration process will migrate all your existing configurations, which are properly authorised and also replace the old extractor in all orchestration tasks in the project.

How to

  1. Click "Proceed to Migration" button in any configuration of the old Google Analytics Extractor
  2. A dialog window will open with a list of all configurations in the project
  3. Click "Migrate" button and a migration job will be created
  4. You can watch progress of the migration job either in the migration dialog window or in the Jobs list
  5. After job is finished, you can see the result of the migration for every configuration and also affected orchestrations on the "Affected Orchestrations" tab

Backward compatibility

The only thing that is not migrated automatically are the Date Range settings.

In the old extractor, the date ranges were specified in parameters of a orchestration task by "since" and "until" attributes.

In the new extractor, the date ranges are specified in the extractors configuration and you can have multiple date ranges per query.

After migration, the orchestration task parameters will remain as they were. If present, the orchestration task with new extractor will fail. Simply move your date range settings from the orchestration task parameters to the extractors query configuration.


Orchestration error:


Move "since" and "until" parameters from orchestration task to query configuration: 


For more information about new Google Analytics Extractor see documentation.

If you have any troubles or questions, please contact support@keboola.com.

New incarnation of Google Drive Extractor

New version of Google Drive Extractor, running on container based architecture, is now available. The old one has been marked deprecated.

Features

Not much has changed in terms of functionality, but the configuration UI is now much faster :) Also the extractor now uses the newest version of the Google Drive and Google Sheets API.

Migration

To help you migrate your configuration to the new version of this extractor, we have prepared a migration tool, which will be available very soon.

The old extractor is now deprecated, but it will remain functional until January 2017.

We are always glad to receive feedback from our users, so if you have any questions or ideas how to improve this component, don't hesitate to contact us.