Failing data loads due to a bug in GoodData WebDav

Some data loads are failing due to a bug in WebDav storage on GoodData's site introduced in their Saturday release. They are working on fix, be patient please. Jobs affected by this problem fail with message "Csv file has not been uploaded to 'https://secure-di.gooddata.com/uploads/....csv'". We apologize for all the inconveniences.

Update 16:15 UTC: According to GoodData support this problem unfortunately won't be fixed earlier than on November 26 9:00 UTC. See their maintenance announcement

On-demand SSO access to GoodData projects from KBC

Access to GoodData projects from their writers in Keboola Connection has been changed. Once you enter a project in KBC you are not automatically added to any GoodData projects any more. Instead there is new button "Enable Access to Project" in each writer which needs to be explicitly confirmed first.


After that you can directly access the project using sso link (no need to login first any more) or you can Disable access to the project.


Also when you leave a KBC project, you are removed from all of it's GoodData projects automatically. 

GoodData Writer automatically creates one GoodData account for each KBC user. Unfortunately because of security reasons we cannot connect this feature to your existing GoodData accounts but once you are logged in GoodData using a sso link you can switch between other projects you joined earlier so it should not be a big obstacle.

Access to projects is provisioned with editor role by default but you can ask support to allow you access with admin roles if you are sure you need it.

aLook Analytics App

Getting actionable insights from data is usually more discussed than actually implemented. In order to succeed in this task, you need to combine three completely different worlds. Data, analytics and business. Practically, it means creating a team that consists of someone who knows the data, someone with the relevant analytical skill-set (call it data science, machine learning or predictive modelling) and someone with the domain knowledge. These people often speak different languages and use different tools.

Thanks to Keboola and the new aLook Analytics app, the main integration and collaboration issues are gone. The beauty of the solution at hand lies in a simple integration of custom analytics into the standard data processes. All the issues around data processing, data transfer and model deployment have been already sorted out, and therefore the primary focus really is on solving the business problem. No middle-man needed. Your data guy will deal with data processing, our data scientist will train the best possible model in R and finally your business person will make sure that you will be able to make more money using this new information.

Predictive modelling embedded in data processes

The job of aLook Analytics is to provide the client with tailor-made predictive models developed in R that directly support their KPIs. Some specific examples of these models are:

  • Propensity models – predicting, which customers are likely to buy a product, start using a product, decrease their activity or even churn completely
  • Recommendation engines – providing a personalized product offer for each customer
  • Direct campaign optimization – models indicating who to target, when, with which offer and through which channel

We have expertise in building such models in various industries – retail banking and e-commerce, but also behavioral talent analytics and sport.

The setting step for the end-user of aLook Analytics app in Keboola Connection is very simple. It includes just mapping of input and output tables. Any other extra configurations need to be added only in case when multiple complex models are used in parallel.


All models are kept in our private repository on Bitbucket and their management is very easy. We can develop a new version of a model and switch it to production in no time.

If you want to give it a try, start by contacting us at adam@alookanalytics.com or kbcapp@alookanalytics.com. The process is fairly straightforward. After initial check-in of what the business problem is, we put together a solution plan, exchange a data sample to develop an initial model and after a few very quick iteration loops [days] we can deploy the model into your standard data processes to start acting upon it. It’s easy and fun. You will see :).

Docker updates

Quay Integration

(Read on if you build components for KBC, feel free to go on with your life if you don't):

Our technically brave customers and partners might have noticed an outage in DockerHub automated build system that affected our docker-based components on the 16th and 17th of November. We were not happy with Docker's (lack of) communication regarding this outage. To prevent from similar occurrences in the future (and to provide alternative) we have implemented integration with Quay as well. Quay provides the same functionality as DockerHub, offering nicer UI on top of it. In case a future DockerHub outage affects you in any way, you can switch your images to Quay and we can hot-swap component configuration.

Support for image tags

On November 9th DockerHub changed default tag of the latest automated image build from latest to master (branch name). When adding a Docker image as a component in Keboola Connection you can now choose which tag will be pulled from the repository.

Curious about Docker integration in Keboola Connection? You can build your own (data) apps and run them on our infrastructure, read more here.


Deleting Projects

To simplify cleanup when a KBC project no longer serves its purpose, we added simple method for deleting it. You'll find that in the "Users & Settings" tab.

We maintain a recoverable backup copy of each project for 60 days following the deletion. After that period, it's gone for good.


YouTube Analytics and Reporting API - Extractor

Google released recently a new Bulk API in order to retrieve viewing statistics, popularity metrics, and more for YouTube videos and channels more conveniently.

The use-case of this API is very simple. It is possible to list jobs and schedule processes that generate new data on a daily basis. As we work with Youtube in our company a lot, we wanted to adapt this new API as quickly as possible, despite its significant limitation (e.g. it is not possible to download historic data before the reports started to be generated by this Bulk API).

We implemented and integrated the first iteration of the YouTube Analytics and Reporting API Extractor for Keboola Connection and started using heavily. 

In a nutshell, the extractor expects that the jobs have already been scheduled and data has been generated (the configuration of the jobs must be done outside the Keboola extractor). You can specify particular report types as well as content owner id and timeframes.  

Configuration is done by Keboola generic GUI that expect a valid JSON object (we plan to build a custom GUI for the configuration part for the further releases as well as make the whole user experience better). The params related to credentials utilise the new encryption feature and are stored safely on Keboola Connection Backend. You can check the documentation to learn more about the configuration part. Documentation contains also important note about the current limitation of this extractor. 

In case of any question/issue, feel free to contact me at any time (radek@bluesky.pro). I am more than happy to help you with the configuration/fixing issues if any. Thank you and enjoy!


KBC as a Data Science Brain Interface

The Keboola Data App Store has a fresh new addition. That brings us to total of 16 currently available apps, three of which provided by development partners.

This new one is called “aLook Analytics”, and technically it is a clone of our development project, a “Custom Science” app (not available yet, but soon!). It facilitates connection to a GitHub/Bitbucket repository of a specific data science shop, which you can “hire” via the app and enable them to safely work on your project.

This first instance is connected to Adam Votava’s company aLook Analytics (check them out at http://www.alookanalytics.com/).

How does it work?

Let’s imagine you want to build something data-science-complex in your project. You get in touch with aLook and agree on what it is you want them to do for you. You exchange some data, the boys there will do some testing on their side, set up the environment and once they’re done, they’ll give you a short configuration script that you will enter into their app in KBC. Any business agreement regarding their work is to be made directly between you and aLook, Keboola stays on the sidelines for this one.


When you run the app, your data gets served to aLook’s prepared model and scripts, saved in aLooks repository get executed on Keboola servers. All the complex stuff happens and the resulting data gets returned into your project. The app can be (like any other) included in your Orchestrations, which means it can run automatically as a part of your regular workflow.

The user of KBC does not have direct access to the script, protecting aLook’s IP (of course, if you agree with them otherwise, we do not put up any barriers).

Very soon we will enable the generic “Custom Science” app mentioned above. That means that any data science pro can connect their GitHub/Bitbucket themselves - that gives you, our user, the freedom to find the best brain in the world for your job.

Why people and not just machines?

No “Machine Learning Drag&Drop” app provides the same quality as a bit of thought by a seasoned data scientist. We’re talking business analytics here! People can put things in context and be creative, while all machines can do is to adjust (sometimes thousands of) parameters and tests the results against a training set. That may be awesome for facial recognition or self-driving car AI, but in any specific business application, a trained brain will beat the machine. Often you don’t even have enough of a test sample so a bit of abstract thinking is critical and irreplaceable.

Configuration encryption

To address security of passwords and other components that require stronger protection, KBC now allows to encrypt certain values in stored configurations. All attributes prefixed with a hashmark sign (#) are automatically encrypted during save. The key is derived from the used component and project and there are no means in any UI or API to decrypt the value. The original value is available only internally and only to the app during its runtime.

What does that mean? When you save your password as an encrypted attribute, even you cannot decrypt it. It becomes available only in the application and in the project it was encrypted and the values cannot be transferred to any other apps or projects. Your passwords are safe and cannot be retrieved even by user with admin rights to your KBC project.

We hope this makes you feel safer! :-)

Note to developers and tech partners: The encryption is completely transparent. You only need 2 simple things: 

  1. tell us that your component uses encryption
  2. prefix all encrypted attributes with # (eg. password => #password)

The infrastructure takes care of the rest. Your application will "see" the decrypted value.

Stopped Docker jobs

Due to a spike in AWS SPOT instances price our Docker workers were shut down around 12am UTC. This affects all jobs that are running on Docker components. We're working on fixing this issue and hope to resume all operations shortly. Thanks for your patience.

Update 04:30am UTC: All operations back to normal, all jobs should have resumed their execution. There was a minor failure with a Docker image for Generic Extractor, some of its jobs have failed with this error

User error: Container 'keboola/docker-generic-extractor:latest' failed: no such file or directory Error response from daemon: Cannot start container 08763383d5370bcdd6e1479da00ae369fe5d845c33485df5337239cc7bdd9c90: [8] System error: no such file or directory

This issue is now fixed and if you have encountered this error, please restart the job. 

Thanks for bearing with us and we're sorry for the inconvenience.