AWS Connectivity Issues

We're experiencing connectivity issues from AWS to some parts of the outer world (aka the Internet). As our transformation sandbox server (ovh-tapi.keboola.com) is not in AWS you may experience failures when creating sandboxes and credentials. If this is not resolved within AWS shortly, we'll be switching the sandbox server into AWS network. 

We will keep this post updated with the current status.

UPDATE (2:00pm PST): Connectivity seems to work fine now.

Facebook Extractor: invalid account or token

We're changing the way Facebook Extractor reacts to invalid accounts and tokens. Previously all invalid accounts/tokens were automatically disabled (with an error event in Storage) and the extractor continued with extraction. 

As this event was easy to miss and there was no other notification about an invalid token/account, we switched to a more strict behavior. Any invalid account/token will stop the execution of the whole job/orchestration with an explaining message:

You need then to change the token or disable the account manually. 

For any questions and comments do not hesitate to contact us at support@keboola.com.

Provisioning improvements: MySQL DB names and logging

We changed the naming conventions for MySQL provisioning. Instead of tapi_3800_sand and tapi_3800_tran, where 3800 is my token ID, the new database names are sand_232_3800 and tran_3800, where 232 is project ID for easier distinction between projects in your sandbox userspace. Existing credentials keep their database names.

We also added a little more information to events:

MailGun.com issues

Mailgun.com, backend API powering our Pigeon Extractor (email data import), has an incidents with their infrastructure (caused by Rackspace maintenance). Their status page is here, twitter account here. When they stabilise their platform, everything should be OK. Please be patient.

Table Aliases with your custom SQL

A few weeks ago, we silently launched ability to create Storage API Aliases by using your own SQL code. These Alias Tables with custom SQL can be created with Redshift backend only.

Create New Alias Table:

Define your own SQL code:

Why?

Alias Tables can help you structure your data. Imagine it as a "Transform on Demand" - everything is happening on-the-fly (aka real-time). Say we have business transactions in table "data". This is an example how to define "derived" table with weekly sum of all transactions, that can't be joined with our Customer (alarm, wrong data!! :-)

Raw Result of this simple alias table:

"year","week","total"
"2014","1","1314788.27"
"2014","2","3719694.16"
"2014","3","3907852.92"
"2014","4","4013945.26"
"2014","5","3884234.84"

Thanks to the almost unlimited power of the Redshift cluster, you can also create much more complex examples. For instance, this one creates a denormalised table of transactions that occur during the night at weekends, in EUR, outside of Czech Republic and not having one specific product code:

AdWords and Sklik extractors issues

There has been a bug in both AdWords and Sklik extractors which caused undelivery of data to input tables since Feb 17 13:00 UTC. Actually the data had been saved to wrong bucket: in.c-ex-adwords instead of in.c-ex-adwords-[config] (similarly for Sklik). You can get the data there if you have only one configuration for the extractor. Otherwise data for more configurations will be mixed in this one bucket and so it will be safer to run the extractions again. Now the problem is fixed and should not occur again. We are sorry for any inconvenience.