Between 5:29pm CET and 5:46pm CET there was an outage in one of the Transformations components. All transformations starting, ending or going between phases during that period were terminated with an application error. All affected transformations were resumed and all other transformations remain unaffected.
We're sincerely sorry for this inconvenience!
It's difficult for me to write a quick summary about Dropbox Extractor - project that I had been developing and tuning for a while. But I'll try anyway and I am sure the wait for the result was worth it!
First of all, thank you for your patience! :-) The main reason for such a long development time was that I wanted to reach the best user experience I could. Of course, the first idea was to use a default GUI for Docker infrastructure. But I guess this approach would be enough only for specific connectors like iTunes Connect or Youtube Reporting Api, not for Dropbox, where a bigger adoption is expected (if the UX is good enough).
For that reason I was sure I wanted to write my own GUI that makes the work with the Dropbox very straightforward. The idea sounded great, but my problem was I had little (almost none, in other words) knowledge of frontend development and zero knowledge of React/Flux. However, my eagerness to reach a certain level of JavaScript full-stack development competence helped me to start with actual development and learn everything on the fly. And I never regret that decision, despite the fact, it took me a little bit more time to really understand what is going on, how to write a React/Flux code and how components in Keboola Connector are composed.
A month ago a production version was completed, but I still wanted to add some more features and improve the overall user-experience. Now I can proudly share the result of that effort. In a nutshell, you can simply select files from your Dropbox and upload them to your Keboola storage, but after a deeper look you will find features like:
- Selecting CSV files and assign them to various KBC buckets (one file can be assigned to multiple buckets).
- In case of selecting two files with the same name, they will either keep the actual name (if two different buckets had been selected) or distinguish between each other by adding a simple hash.
- Bulk or single file upload.
- Output table details in configuration list.
Well, I would love to say thanks to Keboola developers, to Martin Halamicek in particular. His hints and debugging skills helped to define what the Dropbox Extractor is now today and I just hope you enjoy it in the same way as I do! And of course, any feedback is welcome! Enjoy!
Keboola Connection projects now need to be specifically enabled to create "production" (permanent) GoodData projects. The UI will prompt you to contact support in those cases.
On Dec 16 between 5pm and 9pm CEST some MySQL and Redshift queries containing comments were malformed in our parser.
This bug has been resolved and deployed, all affected orchestrations were resumed. We're sorry for this inconvenience!
Tables are not automatically added to GoodData Writer UI any more and you have to add them there explicitly. Naturally you can also delete no longer needed tables. This improvement should benefit clarity and unites behaviour across other writers. Non-configured tables (i.e. those with all columns ignored) will be removed from your existing configurations soon by automated script.
There was a bug affecting insightsPages_pivoted and insightsPosts_pivoted configurations for pages or posts with no values in some metrics. Under certain situations the zero value was replaced with the value of 1.
To fix any affected data you can run the extractor with parameters to backfill a given period of time, eg.
{
"since":"2012-01-01",
"until":"2015-12-10"
}
Put these parameters in your orchestrator configuration or use a separate API call to create a job.
We're sorry for this bug and for any trouble caused, read more about Facebook Insights Extractor for additional info.
Sklik Extractor has been rewritten as Docker component and the old version will be switched off by the end of December. Till then its users are kindly requested to migrate their configuration to the new version. Here is how to do it:
Please note that the new Extractor saves data incrementally which means that you have to add primary keys if you want to fill data to existing tables. You can find the required format of data tables in documentation.
A few months ago I made my first contribution to Keboola Ecosystem and created a component to Keboola Connection that downloaded data from iTunes Connect. It was working well, but despite my limited knowledge of how to really work with Keboola Connection (and my limited knowledge of practical programming as well), the functionality of the component was quite limited and was possible to download just one type of report. Two days ago I decided to improve this extractor significantly and I can proudly share the results of this update.
From the business perspective the major update is related to possibility of downloading more types of reports. Last version allowed you to download only Sales data. Now is possible to extract Earnings data as well. There is also an automatic fiscal calendar generator (5-4-4) that make sure the download of the earnings data will be handled in the correct order. It's been generated dynamically and results of the fiscal calendar generator were validated against Apple calendar and data is set properly. Another improvement was about adding more options for downloading of Sales data. More types of grain is supported in the new version.
From the technical perspective, there is an avalanche of various improvements. In a nutshell, scripts were simplified by removing unnecessary logic (as my understanding of Keboola is better) & rewrote the handling of asynchronous flow completely. Files are also uploaded dynamically and for that reason a need for specifying static Table Output Mapping is removed completely. That helped to simplify the input parameters. Credentials are now encrypted.
Check the documentation & source code for more information. And in case of any question/issue, don't hesitate to contact me at my email (radek@bluesky.pro). I am happy to provide more details if you are interested in.
If you have long list of transformations or they're just quite complex, you may find yourself somewhat annoyed with the responsiveness of the UI. If that is the case, you will be delighted to know that we have solution ready to go that significantly improves the user experience there, maintaining the same functionality.
Before we roll out this feature to all projects we'd like to invite you to beta-test this feature. If you're crunching through a vast list of transformations, you can benefit from this today. The migration process is easy, let us know at support@keboola.com. We'll migrate your configuration for you. There are no changes in the UI or anywhere else, nothing will be stopped or lost. Numerous backups are made along the way and rollback is easy.
Thanks for participating!
We have finally updated the MySQL transformation server to MariaDB 5.5.44, the same version as the sandbox server.
Everything should be running as expected, no migration is required, but if you encounter any issues or incorrect behaviour, let us know.