Between Oct 07 02:00 (GMT+2) and Oct 07 02:24 (GMT+2) some transformation jobs were failing. We have identified the root cause of this issue and fixed it. All affected orchestrations will be restarted shortly.

We're sorry for this inconvenience.

Snowflake backend is down since 11:02am GMT+2. We're trying to fix it, post mortem will follow once we recover platform.

UPDATE:

Dead DWH was switched, everything is running on new backend. All running jobs, employing Snowflake backend, crashed. You have to run it again - everything will be OK for now. In case you're suffering under some other issues, contact us at support@keboola.com, please. 

We'll also publish official post-mortem as soon as Snowflake guys resolve our Issue. Stay tuned and thanks for patience!

UPDATE II:

If you're experiencing errors in your personal sandboxes, reset your credentials under the "Sandbox" link on the right-top of Transformations (https://connection.keboola.com/admin/projects/$pid$/transformations/sandbox). 

Some jobs have failed between 2016-09-26 17:30 CEST and 2016-09-26 17:40 CEST due to problem with one of our DB servers.

All affected orchestrations have been checked and restarted.

We are very sorry for any inconvenience.

If you have any concerns about this, please contact us at support@keboola.com.

As some of you might know, Google Analytics API doesn't always return precise data. Under certain circumstances, the data returned are sampled. Read more about sampling here.

Anti-Sampling

You can choose from one of two anti-sampling algorithms - DailyWalk or Adaptive. Both are based on the same principle, to divide the wanted date range into smaller chunks.

DailyWalk as the name suggest divides the date range by days. So the extractor needs to make as many request as there are days in the date range.

Adaptive algorithm is using more sophisticated approach, and divides the date range into few smaller date ranges. Read in-depth explanation of the algorithm here, if you are interested. 

DailyWalk algorithm might be more precise in some cases, but usually you will get the same results faster with the Adaptive algorithm.

Experiment with them and use what suits you best.

We have encountered a bug in the Google Analytics Extractor. It was downloading data only for one profile, even if the query was set to "All profiles".

This bug was introduced on 16th of September and it is now fixed.

We are very sorry for any inconvenience. Please adjust your date ranges and download the missing data if needed. 

Call for testers: OpenRefine Transformations BETA

Our new  OpenRefine transformations need some testers. Do you want early access and want to play with OpenRefine in Keboola Connection? Please contact us at support@keboola.com.

Google Analytics Extractor

The Google Analytics Extractor can now parse urls for queries.  So you can create your query with the very convenient  Google Analytics Query Explorer and simply copy and paste the url.

Versions management

Simplified access to latest version diff.

Announced Redshift maintenance canceled

We are cancelling previously announced maintenance of Redshift projects 

Other improvements and bugfixes

• New Hubspot extractor published
  
• Adwords extractor - more verbose logging
• PostgreSQL extractor - fixed retry mechanism which previously caused invalid CSVs with duplicated header
• Storage - fixed table size and rows count for Snowflake backed projects
• Storage configurations - numeric ids are generated, fixed bug in duplicate ids creation
• GoodData Writer - fixed grain settings for tables with custom identifiers 
• Elasticsearch Writer - SSH tunnel support added

We've created new HubSpot extractor which enables you to extract basic data from your CRM.

It is based on our Generic Extractor with predefined templates to helps download your data easily. That also brings an option to adjust the configuration to your needs by switching to JSON editor.

If you want to learn more about the configuration of the extractor, how to get your HubSpot API Token and details of the extractor output tables, look into our new documentation.

Feel free to try the new extractor and in if you have any questions or something is missing, please contact us at support@keboola.com.

Google Drive Extractor

Google Drive Extractor has been revamped. Not much has changed, this was standard update due to our container based infrastructure. Just one thing to mention -- its UI is much faster. Read more about New incarnation of Google Drive Extractor.

Migration tool for Google Analytics Extractor

Another important thing is Migration Tool for new Google Analytics Extractor.

Transformation/Sandboxes Provisioning

Also, we have changed the way how Redshift transformations and sandboxes are provisioned. Migration of existing projects is prepared to week from September 26th to September 30th.

Custom SQL Aliases

We decided to drop support for custom SQL aliases (existing aliases will continue to work), due to theirs rare usage. You can still use Simple Aliases, which are now supported for Redshift backend too.

As always, we did many small improvements:

• Search in components while creating new orchestration (UI)

• AdWords Extractor update to new AdWords API v201607 (Extractor Backend)
• Automated Configuration Adjustment (UI)
• Show Multi-factor Authentication (MFA) status in users list -- not enabled MFA produces warning (UI)

We have experienced some extractor failures between 2016-09-09 21:00 CEST and 2016-09-10 12:00 CEST.

Affected extractors were Gmail, Twitter, MongoDB and LinkedIn.

Due to these errors, some of the runs exited with failure or wrote to a wrong bucket stage.

All affected orchestrations have been checked and started again.

We are sorry for any inconvenience.

If you have any concerns about this, please contact us at support@keboola.com.

We're changing the way Redshift transformations and sandboxes are provisioned.

During the week from September 26th to September 30th all projects will be migrated. What does that mean for you?

Faster with less errors

In certain situations (eg. table locks) creating of a sandbox would take a long time. After the migration provisioning will no longer depend on any locks in the whole Redshift cluster.

Provisioning and data loading

We're offloading the input mapping work to Storage API. Storage API is now in charge of creating the sandbox or transformation workspace for you as well as loading the data into it. Storage API will decide the fastest way to load required data into your workspace.

Credentials change

Username, password and schema for your Redshift sandbox will change and new sandboxes will be created. Your current sandboxes will be deleted 7 days after the migration. UI will no longer serve credentials for your current sandbox, only for the newly generated one. You will not be able to load data into your current sandbox.

No direct access to bucket data

If your transformation uses data directly from a Redshift schema, this won't be supported. Queries like

SELECT * FROM "in.c-mybucket"."table" 

will no longer have access to in.c-mybucket Redshift schema. If your transformation contains such queries, please adjust the query so that it uses a table specified in the input mapping.

Transparent and hassle free migration

There will be no service interruption and no expected delay in your transformations. Unless you're using direct access to bucket data there is no action required. In case of any problems the whole migration is reversible. 

If you are concerned about your operations please get in touch with us at support@keboola.com. We can try out the migration or change the date of migration.