Week in Review -- May 23rd, 2016


We have a few user-friendliness announcements for transformations.  First, transformations will automatically apply a convenient description to the versions it stores:

Further, all new transformation buckets will allow editing names!

In other news, the data-takout function will now export all versions of all component configurations.

There's also been a little re-organisation of the storage API documentation for tables https://docs.keboola.apiary.io:

Hope you're enjoying some fine spring weather.

Week(s) in Review -- March 21, 2016

Writer News:

The GoodData Writer has successfully migrated all of its configurations from sys buckets to the components API.  The buckets no longer in use will be removed this week.

Also, we're thrilled to now officially support writing to Looker and Qlik.

Extractor News:

Get your hardhats on!  Our DB2 and Firebird extractors now support SSH tunnelling.

In Other News:

Events and Notifications:
  • Newly invited users will not see old project notifications when they join, they will now only see notifications from the time of their joining the project
  • Checking the mark all as read ckeckmark will now mark all notifications as read, not just those loaded to the display
  • Newly joining users will now see all project events, whereas before they'd only see the last 200 events prior to joining the project.
Transformations and Applications:
  • Inputs in the standard interface for input and output mappings now support [space] character 
  • We've noticed that SQL comments larger than 8kb will fail without exception id, so please take note if your SQL is very comment heavy.  
  • We also now support docker containers from private repositories on quay.io

Storage API Client For R

Want to play with your KBC data in your local R environment?

Install the keboola-sapi-r-client and you can.  
(The package is on GitHub so it is installed via the devtools package) 

install.packages("devtools")
library(devtools)

We need to install a github dependency
for aws request signature generation

devtools::install_github("cloudyr/aws.signature")

Now we can install the Storage api client and load it into our R session

devtools::install_github("keboola/sapi-r-client")
library(keboola.sapi.r.client)

Just like any other R package, once installed, it can be invoked in any future session with the library() command.

To instantiate the client just give it a KBC token.
We'll use the token for the currency exchange rates for demonstration purposes.

client <- SapiClient$new('452-33945-de5bb7fecb818901f0834b2431564003296a4b05')

Now we can import data to our R session

currencyData <- client$importTable('in.c-ex-currency.rates')

Just for fun, let's make a simple plot of EUR vs USD using the ggplot2 library
if not installed on your R use install.packages("ggplot2")

# prepare our data
eurVsUsd <- currencyData[which(currencyData$toCurrency == "USD"),]
eurVsUsd$date <- as.Date(eurVsUsd$date)

# load the libraries needed to make our plot
library(ggplot2)
library(scales) # for prettier x-axis labeling

p <- ggplot(eurVsUsd, aes_string(x="date", y="rate")) + geom_point()
# add x-axis scaling and title
p <- p + scale_x_date(breaks="1 year", labels=date_format("%Y"))
p <- p + ggtitle("EUR vs USD")
print(p)

The code for this sample is here in this gist

The Storage API client gives full read and write access to your KBC project within the comforting power of your local R environment.

Imagine the possibilities!

* small print *  This is a development tool in Beta, use at your own risk!