Keboola Status

A few months ago I made my first contribution to Keboola Ecosystem and created a component to Keboola Connection that downloaded data from iTunes Connect. It was working well, but despite my limited knowledge of how to really work with Keboola Connection (and my limited knowledge of practical programming as well), the functionality of the component was quite limited and was possible to download just one type of report. Two days ago I decided to improve this extractor significantly and I can proudly share the results of this update.

From the business perspective the major update is related to possibility of downloading more types of reports. Last version allowed you to download only Sales data. Now is possible to extract Earnings data as well. There is also an automatic fiscal calendar generator (5-4-4) that make sure the download of the earnings data will be handled in the correct order. It's been generated dynamically and results of the fiscal calendar generator were validated against Apple calendar and data is set properly. Another improvement was about adding more options for downloading of Sales data. More types of grain is supported in the new version.

From the technical perspective, there is an avalanche of various improvements. In a nutshell, scripts were simplified by removing unnecessary logic (as my understanding of Keboola is better) & rewrote the handling of asynchronous flow completely. Files are also uploaded dynamically and for that reason a need for specifying static Table Output Mapping is removed completely. That helped to simplify the input parameters. Credentials are now encrypted.   

Check the documentation & source code for more information. And in case of any question/issue, don't hesitate to contact me at my email (radek@bluesky.pro). I am happy to provide more details if you are interested in. 

Google released recently a new Bulk API in order to retrieve viewing statistics, popularity metrics, and more for YouTube videos and channels more conveniently.

The use-case of this API is very simple. It is possible to list jobs and schedule processes that generate new data on a daily basis. As we work with Youtube in our company a lot, we wanted to adapt this new API as quickly as possible, despite its significant limitation (e.g. it is not possible to download historic data before the reports started to be generated by this Bulk API).

We implemented and integrated the first iteration of the YouTube Analytics and Reporting API Extractor for Keboola Connection and started using heavily. 

In a nutshell, the extractor expects that the jobs have already been scheduled and data has been generated (the configuration of the jobs must be done outside the Keboola extractor). You can specify particular report types as well as content owner id and timeframes.  

Configuration is done by Keboola generic GUI that expect a valid JSON object (we plan to build a custom GUI for the configuration part for the further releases as well as make the whole user experience better). The params related to credentials utilise the new encryption feature and are stored safely on Keboola Connection Backend. You can check the documentation to learn more about the configuration part. Documentation contains also important note about the current limitation of this extractor. 

In case of any question/issue, feel free to contact me at any time (radek@bluesky.pro). I am more than happy to help you with the configuration/fixing issues if any. Thank you and enjoy!

iTunes Connect helps to manage the content sold on the Apple iTunes, iBooks Store and App Store. If you are working with the content for Apple devices, you have the basic analytics available in iTunes Connection Web Application where you can track the standard information. However, if there is a need for more detailed information and usage in deeper context (e.g. to make a mashup with other data sources), it may be handy to use Keboola Connection (as we needed in DigitalAirways.tv) and that was the main reason for writing of this extension.

iTunes Connect Extractor is based on Apple Autoingestion Tool and written in Node.js and deployed with Docker.

For successful login you need to pass the iTunes Connect username, password and vendor id assigned to your iTunes account. How to pass these credentials and other params is written in Github repository. Apple's official guide is also handy for deeper understanding of all parameters.

The limitation of current version is that there is only possible to extract the Sales data only (with all available fields, described in documentation in Github repository as well) and the configuration have to be passed via JSON. You can download data within specified date period or use the daily increments.The next major version will have functionality for downloading Earnings data and have a proper user interface.

Feel free to use the iTunes Extractor and you find any issue or have any question or suggestion, don't hesitate to contact me at radek@digitalairways.tv.