Facebook Insights Extractor Bug in pivoted configurations

There was a bug affecting insightsPages_pivoted and insightsPosts_pivoted configurations for pages or posts with no values in some metrics. Under certain situations the zero value was replaced with the value of 1.

To fix any affected data you can run the extractor with parameters to backfill a given period of time, eg.

{
  "since":"2012-01-01",
  "until":"2015-12-10"
}

Put these parameters in your orchestrator configuration or use a separate API call to create a job. 

We're sorry for this bug and for any trouble caused, read more about Facebook Insights Extractor for additional info.



New version of Sklik Extractor

Sklik Extractor has been rewritten as Docker component and the old version will be switched off by the end of December. Till then its users are kindly requested to migrate their configuration to the new version. Here is how to do it:

  1. Choose Sklik v2 from list of extractors
  2. Give it some name as usual
  3. Add extraction configuration to the text area. It is in json format and must contain Sklik account's username and password and bucket where you want the data to be saved. Optionally it may contain parameters since and until to specify dates interval for stats (it is "-1 day" i.e. "yesterday" by default)

Please note that the new Extractor saves data incrementally which means that you have to add primary keys if you want to fill data to existing tables. You can find the required format of data tables in documentation.


iTunes Connect Extractor - update

A few months ago I made my first contribution to Keboola Ecosystem and created a component to Keboola Connection that downloaded data from iTunes Connect. It was working well, but despite my limited knowledge of how to really work with Keboola Connection (and my limited knowledge of practical programming as well), the functionality of the component was quite limited and was possible to download just one type of report. Two days ago I decided to improve this extractor significantly and I can proudly share the results of this update.

From the business perspective the major update is related to possibility of downloading more types of reports. Last version allowed you to download only Sales data. Now is possible to extract Earnings data as well. There is also an automatic fiscal calendar generator (5-4-4) that make sure the download of the earnings data will be handled in the correct order. It's been generated dynamically and results of the fiscal calendar generator were validated against Apple calendar and data is set properly. Another improvement was about adding more options for downloading of Sales data. More types of grain is supported in the new version.

From the technical perspective, there is an avalanche of various improvements. In a nutshell, scripts were simplified by removing unnecessary logic (as my understanding of Keboola is better) & rewrote the handling of asynchronous flow completely. Files are also uploaded dynamically and for that reason a need for specifying static Table Output Mapping is removed completely. That helped to simplify the input parameters. Credentials are now encrypted.   

Check the documentation & source code for more information. And in case of any question/issue, don't hesitate to contact me at my email (radek@bluesky.pro). I am happy to provide more details if you are interested in. 

Transformation Configuration (Public Beta)

If you have long list of transformations or they're just quite complex, you may find yourself somewhat annoyed with the responsiveness of the UI. If that is the case, you will be delighted to know that we have solution ready to go that significantly improves the user experience there, maintaining the same functionality.

Before we roll out this feature to all projects we'd like to invite you to beta-test this feature. If you're crunching through a vast list of transformations, you can benefit from this today. The migration process is easy, let us know at support@keboola.com. We'll migrate your configuration for you. There are no changes in the UI or anywhere else, nothing will be stopped or lost. Numerous backups are made along the way and rollback is easy.  

Thanks for participating!

Failing data loads due to a bug in GoodData WebDav

Some data loads are failing due to a bug in WebDav storage on GoodData's site introduced in their Saturday release. They are working on fix, be patient please. Jobs affected by this problem fail with message "Csv file has not been uploaded to 'https://secure-di.gooddata.com/uploads/....csv'". We apologize for all the inconveniences.

Update 16:15 UTC: According to GoodData support this problem unfortunately won't be fixed earlier than on November 26 9:00 UTC. See their maintenance announcement

On-demand SSO access to GoodData projects from KBC

Access to GoodData projects from their writers in Keboola Connection has been changed. Once you enter a project in KBC you are not automatically added to any GoodData projects any more. Instead there is new button "Enable Access to Project" in each writer which needs to be explicitly confirmed first.


After that you can directly access the project using sso link (no need to login first any more) or you can Disable access to the project.


Also when you leave a KBC project, you are removed from all of it's GoodData projects automatically. 

GoodData Writer automatically creates one GoodData account for each KBC user. Unfortunately because of security reasons we cannot connect this feature to your existing GoodData accounts but once you are logged in GoodData using a sso link you can switch between other projects you joined earlier so it should not be a big obstacle.

Access to projects is provisioned with editor role by default but you can ask support to allow you access with admin roles if you are sure you need it.

aLook Analytics App

Getting actionable insights from data is usually more discussed than actually implemented. In order to succeed in this task, you need to combine three completely different worlds. Data, analytics and business. Practically, it means creating a team that consists of someone who knows the data, someone with the relevant analytical skill-set (call it data science, machine learning or predictive modelling) and someone with the domain knowledge. These people often speak different languages and use different tools.

Thanks to Keboola and the new aLook Analytics app, the main integration and collaboration issues are gone. The beauty of the solution at hand lies in a simple integration of custom analytics into the standard data processes. All the issues around data processing, data transfer and model deployment have been already sorted out, and therefore the primary focus really is on solving the business problem. No middle-man needed. Your data guy will deal with data processing, our data scientist will train the best possible model in R and finally your business person will make sure that you will be able to make more money using this new information.

Predictive modelling embedded in data processes

The job of aLook Analytics is to provide the client with tailor-made predictive models developed in R that directly support their KPIs. Some specific examples of these models are:

  • Propensity models – predicting, which customers are likely to buy a product, start using a product, decrease their activity or even churn completely
  • Recommendation engines – providing a personalized product offer for each customer
  • Direct campaign optimization – models indicating who to target, when, with which offer and through which channel

We have expertise in building such models in various industries – retail banking and e-commerce, but also behavioral talent analytics and sport.

The setting step for the end-user of aLook Analytics app in Keboola Connection is very simple. It includes just mapping of input and output tables. Any other extra configurations need to be added only in case when multiple complex models are used in parallel.


All models are kept in our private repository on Bitbucket and their management is very easy. We can develop a new version of a model and switch it to production in no time.

If you want to give it a try, start by contacting us at adam@alookanalytics.com or kbcapp@alookanalytics.com. The process is fairly straightforward. After initial check-in of what the business problem is, we put together a solution plan, exchange a data sample to develop an initial model and after a few very quick iteration loops [days] we can deploy the model into your standard data processes to start acting upon it. It’s easy and fun. You will see :).

Docker updates

Quay Integration

(Read on if you build components for KBC, feel free to go on with your life if you don't):

Our technically brave customers and partners might have noticed an outage in DockerHub automated build system that affected our docker-based components on the 16th and 17th of November. We were not happy with Docker's (lack of) communication regarding this outage. To prevent from similar occurrences in the future (and to provide alternative) we have implemented integration with Quay as well. Quay provides the same functionality as DockerHub, offering nicer UI on top of it. In case a future DockerHub outage affects you in any way, you can switch your images to Quay and we can hot-swap component configuration.

Support for image tags

On November 9th DockerHub changed default tag of the latest automated image build from latest to master (branch name). When adding a Docker image as a component in Keboola Connection you can now choose which tag will be pulled from the repository.

Curious about Docker integration in Keboola Connection? You can build your own (data) apps and run them on our infrastructure, read more here.


Deleting Projects

To simplify cleanup when a KBC project no longer serves its purpose, we added simple method for deleting it. You'll find that in the "Users & Settings" tab.

We maintain a recoverable backup copy of each project for 60 days following the deletion. After that period, it's gone for good.