Inaccessible Storage API files

Some files were not accessible between 7 PM - 10 PM PST November 4. It caused failures of loads to storage API tables and thus also orchestration failures.

Example of failed orchestration:


It was caused by failed Elasticsearch cluster node. We are still investigating the cause of this issue. However, our whole infrastructure works smoothly at this time. To finish your tasks, just re-run your orchestrators. We're sorry for any inconvenience!

Direct (r/o) Access to any Redshift Bucket

Today we're announcing new Storage API feature: Bucket Credentials (api here). 

If you're using Keboola Connection w/ Redshift backend, you can have read-only credentials (direct sql access) to any Redshift bucket. 

In Storage API Console, go to Bucket Detail > Credentials and press "Create new credentials" button:

Describe new credentials (you can have multiple credentials assigned to each bucket!):

When you create credentials, carefully copy&paste credentials to you SQL client or preferred remote service (jackdb.comchartio.com, etc.). After closing displayed credentials, you can't display it's settings:

In case you need to re-use already created credentials, you have to delete it and create new combination of username and password. All existing credentials are listed under it's bucket:


Limits:
  1. credentials can be used for accessing just one bucket
  2. write access isn't supported 

WARNING: Always employ SSL when accessing your data. Generated credentials are opening your dedicated AWS Redshift Cluster. Please read "Configure Security Options for Connections". Redshift Cluster's CA certificate can be downloaded here.

Orchestration UI Update

We released a new version of Orchestration API and UI. The UI is completely rebuildm is lightning fast and everything can be now configured within a single UI. 

Here's a short list of the most exciting features:

  • orchestrations dashboard with the most important details about every orchestration
  • new orchestrations get a token and configuration table automatically (no more Storage Console)
  • orchestration tasks have a configuration UI, components are identified by names and icons instead of URLs

Happy orchestrating!

UPDATE: A short outage occurred while migrating Orchestrator configurations causing some of the orchestrations to fail to start on schedule. We restarted all failed orchestrations manually. We're sorry for this inconvenience.

End of Life Announcement for old Twitter Extractor

Today, 30 October 2014, Keboola is announcing an End-of-Life date of January 31, 2015 for all instances of the Old Twitter Extractor.

Customers that are currently using this version should evaluate our shiny, very powerful and fully configurable Twitter Extractor as a newer alternative. Information on the ex-twitter can be found in our documentation.

If you need any support, please contact us.

Tableau Writer

For those who love Tableau, we're officially launching "Tableau Writer". At this time, it's able to provision MySQL database or push data to your own MySQL server.

Our Tableau Writer allows you to rename tables, it's columns and define data types for better handling on Tableau side (which configuration is pretty straight forward too - just let Tableau connect to MySQL server with using proper credentials). 

If you're "Tableau Desktop Personal Edition" user, you can enjoy TDE files very soon! (delivered to your Google Drive account automatically). 

SSL issues

We're experiencing some issues in our internal infrastructure. 

You can see following errors on affected orchestrations:

  • Error response from component
  • [curl] (#35)

We're working on a fix and restarting all failed orchestrations. We're sorry for this inconvenience.

Google Analytics Extractor: Metrics and dimensions will be renamed

A few months ago Google Analytics renames several metrics and dimensions (Release 2014-04-16)

https://developers.google.com/analytics/devguides/reporting/core/v3/changelog

Our GA extractor was still using the old names, but according to Googles Data Deprecation Policy, the old metric and dimension names stops working on November 1st.

The Extractor will be using the new names, hence the old ones still remains in the data input buckets. So in the input table will be two columns - old one with old name and old data and new one.

We will be helpful with the transition, but it is on KBC users to solve this by them selfs. For example by merging the old column with the new one in transformation, or move data from the old column to the new one and make respectful changes in transformations working with that input bucket.

Don't hesitate to contact us, if you need any assistance.


Transformation MySQL server down

Our MySQL transformation server TAPI-A is down. We're offloading all work to the backup server, rerunning all failed orchestrations and restarting the TAPI-A server. 

We're sorry for this inconvenience, all operations should be back to normal soon and your transformations will switch to the restarted server automatically.

Provisioning API Update

We'll be updating Provisioning API today at 1:15 pm PST. This shouldn't take longer than 15 minutes. Transformations will be resumed, your credentials will still be valid, only creating of new sandboxes in the Transformation UI will be disabled during this period. 

1:30 pm PST: Migration completed. Thanks for your patience.