We are experiencing mysql transformations processes to be running much slower than usually. We are working on fixing this issue. However, this does not affect the redshift based transformations which should be running normally.

Sorry for any incovience. 

Update (11:30am PST): Issue has been resolved, everything is running fine now.

Replace attributes

All attributes of table or bucket can be set by one API call 

File Storage RunId

RunId identifiing running job is associated to all Storage API Files generated by this job. At the moment files can be filtered by RunId http://docs.keboola.apiary.io/#get-%2Fv2%2Fstorage%2Ffiles

This is a building block for upcoming aggregation API which will be able to provide aggregated informations about jobs such as data size transfered in and out, imported or created tables etc. 

We're experiencing connectivity issues from AWS to some parts of the outer world (aka the Internet). As our transformation sandbox server (ovh-tapi.keboola.com) is not in AWS you may experience failures when creating sandboxes and credentials. If this is not resolved within AWS shortly, we'll be switching the sandbox server into AWS network. 

We will keep this post updated with the current status.

UPDATE (2:00pm PST): Connectivity seems to work fine now.

We're changing the way Facebook Extractor reacts to invalid accounts and tokens. Previously all invalid accounts/tokens were automatically disabled (with an error event in Storage) and the extractor continued with extraction. 

As this event was easy to miss and there was no other notification about an invalid token/account, we switched to a more strict behavior. Any invalid account/token will stop the execution of the whole job/orchestration with an explaining message:

You need then to change the token or disable the account manually. 

For any questions and comments do not hesitate to contact us at support@keboola.com.

Some orchestrations have failed around 09:46 AM CET, it was caused by SSL issues in Storage API.

Problem is fixed now, failed orchestrations are restarted. Sorry for inconvenience.

We changed the naming conventions for MySQL provisioning. Instead of tapi_3800_sand and tapi_3800_tran, where 3800 is my token ID, the new database names are sand_232_3800 and tran_3800, where 232 is project ID for easier distinction between projects in your sandbox userspace. Existing credentials keep their database names.

We also added a little more information to events:

Mailgun.com, backend API powering our Pigeon Extractor (email data import), has an incidents with their infrastructure (caused by Rackspace maintenance). Their status page is here, twitter account here. When they stabilise their platform, everything should be OK. Please be patient.

For all columns without a defined datatype in a Redshift transformation we added a default LZO compression. This will lead to a marginally slower transformation times but significantly less memory and disk space usage. 

You can override this behavior by defining a datatype without a compression.

A few weeks ago, we silently launched ability to create Storage API Aliases by using your own SQL code. These Alias Tables with custom SQL can be created with Redshift backend only.

Create New Alias Table:

Define your own SQL code:

Why?

Alias Tables can help you structure your data. Imagine it as a "Transform on Demand" - everything is happening on-the-fly (aka real-time). Say we have business transactions in table "data". This is an example how to define "derived" table with weekly sum of all transactions, that can't be joined with our Customer (alarm, wrong data!! :-)

Raw Result of this simple alias table:

Tables can be  aliased between out and in stage in both ways from now. It is no longer limited to only in -> out direction.