Direct (r/o) Access to any Redshift Bucket

Today we're announcing new Storage API feature: Bucket Credentials (api here). 

If you're using Keboola Connection w/ Redshift backend, you can have read-only credentials (direct sql access) to any Redshift bucket. 

In Storage API Console, go to Bucket Detail > Credentials and press "Create new credentials" button:

Describe new credentials (you can have multiple credentials assigned to each bucket!):

When you create credentials, carefully copy&paste credentials to you SQL client or preferred remote service (jackdb.comchartio.com, etc.). After closing displayed credentials, you can't display it's settings:

In case you need to re-use already created credentials, you have to delete it and create new combination of username and password. All existing credentials are listed under it's bucket:


Limits:
  1. credentials can be used for accessing just one bucket
  2. write access isn't supported 

WARNING: Always employ SSL when accessing your data. Generated credentials are opening your dedicated AWS Redshift Cluster. Please read "Configure Security Options for Connections". Redshift Cluster's CA certificate can be downloaded here.

Orchestration UI Update

We released a new version of Orchestration API and UI. The UI is completely rebuildm is lightning fast and everything can be now configured within a single UI. 

Here's a short list of the most exciting features:

  • orchestrations dashboard with the most important details about every orchestration
  • new orchestrations get a token and configuration table automatically (no more Storage Console)
  • orchestration tasks have a configuration UI, components are identified by names and icons instead of URLs

Happy orchestrating!

UPDATE: A short outage occurred while migrating Orchestrator configurations causing some of the orchestrations to fail to start on schedule. We restarted all failed orchestrations manually. We're sorry for this inconvenience.

End of Life Announcement for old Twitter Extractor

Today, 30 October 2014, Keboola is announcing an End-of-Life date of January 31, 2015 for all instances of the Old Twitter Extractor.

Customers that are currently using this version should evaluate our shiny, very powerful and fully configurable Twitter Extractor as a newer alternative. Information on the ex-twitter can be found in our documentation.

If you need any support, please contact us.

Tableau Writer

For those who love Tableau, we're officially launching "Tableau Writer". At this time, it's able to provision MySQL database or push data to your own MySQL server.

Our Tableau Writer allows you to rename tables, it's columns and define data types for better handling on Tableau side (which configuration is pretty straight forward too - just let Tableau connect to MySQL server with using proper credentials). 

If you're "Tableau Desktop Personal Edition" user, you can enjoy TDE files very soon! (delivered to your Google Drive account automatically). 

SSL issues

We're experiencing some issues in our internal infrastructure. 

You can see following errors on affected orchestrations:

  • Error response from component
  • [curl] (#35)

We're working on a fix and restarting all failed orchestrations. We're sorry for this inconvenience.

Google Analytics Extractor: Metrics and dimensions will be renamed

A few months ago Google Analytics renames several metrics and dimensions (Release 2014-04-16)

https://developers.google.com/analytics/devguides/reporting/core/v3/changelog

Our GA extractor was still using the old names, but according to Googles Data Deprecation Policy, the old metric and dimension names stops working on November 1st.

The Extractor will be using the new names, hence the old ones still remains in the data input buckets. So in the input table will be two columns - old one with old name and old data and new one.

We will be helpful with the transition, but it is on KBC users to solve this by them selfs. For example by merging the old column with the new one in transformation, or move data from the old column to the new one and make respectful changes in transformations working with that input bucket.

Don't hesitate to contact us, if you need any assistance.


Transformation MySQL server down

Our MySQL transformation server TAPI-A is down. We're offloading all work to the backup server, rerunning all failed orchestrations and restarting the TAPI-A server. 

We're sorry for this inconvenience, all operations should be back to normal soon and your transformations will switch to the restarted server automatically.

Provisioning API Update

We'll be updating Provisioning API today at 1:15 pm PST. This shouldn't take longer than 15 minutes. Transformations will be resumed, your credentials will still be valid, only creating of new sandboxes in the Transformation UI will be disabled during this period. 

1:30 pm PST: Migration completed. Thanks for your patience.

Randomly Failing Orchestrations

We're experiencing an AWS-wide DNS problem. Every 01 minute of each hour the DNS does not respond, which can cause failing attempts to connect between components in Keboola Connection. This will cause the running orchestration to fail.

It is reported all across AWS:

https://forums.aws.amazon.com/thread.jspa?threadID=161837&tstart=0

If your orchestration fails with an application error around 01 minute of an hour (it can propagate to the job a bit later, eg. at XX:02), you're most likely affected by this issue. Just re-running the orchestration will solve the problem. 

Please bear with us while we're trying to solve this issue with AWS Support. Here are some examples of failed orchestrations that might help you identify this issue.