Keboola Status

In our effort to bring you all updates we're now starting a regular weekly post that will include a list of all changes, that didn't make it to a separate post.

Input mapping UI bugfix

Creating a duplicate destination in input mapping is now displayed as an error.

Input mapping UI improvement

When selecting a table, the destination is automatically populated with table name (previously with full table id).

Storage: Redshift primary keys bugfix

Setting a primary key on a Redshift table failed when there was another table in the project with the same name and different columns.

Storage: MySQL long primary key bugfix

Setting a compound primary key with more than 4 columns failed with an internal error. This was fixed and it shows a warning that MySQL primary key is limited to 4 columns.

Other posts this week

• End of Life Announcement for Telfa Extractor
  
• End of Life Announcement for Elasticsearch Writer/Extractor
  
• End of Life Announcement: Remote Transformations (eg. Text Splitter)

As of today, January 22th 2016 we won't be making any further improvements or updates on Remote Transformations (including the executor interface in Transformations and all remote transformations, eg. Text Splitter)

As of February 22th 2016 Remote Transformations functionality will be completely removed from Transformations.

Text Splitter functionality can be replaced by an R transformation, see the demo code.

Contact our support, if you need any assistance with migration to R transformation.

The MySQL sandbox server (sh-tapi.keboola.com) is currently not available, we're investigating the cause.

Update Jan 20 5:35pm CET

The backup MySQL sandbox server tapi-sandbox-fallback-3.keboola.com is up and serving all sandbox requests. All previous sandboxes were deleted and all credentials cleared. We'll be transitioning back to the original server sh-tapi.keboola.com tomorrow. Please check your sandbox credentials to be sure you're logging on the right server.

We're sorry for this inconvenience and thanks for your understanding

Update Jan 21 5.33pm CET

The original MySQL sandbox server (sh-tapi.keboola.com) is back up. All new sandboxes will be created on this server, all currently existing sandboxes will remain on tapi-sandbox-fallback-3.keboola.com and will be purged on Sunday night.

Between 5:29pm CET and 5:46pm CET there was an outage in one of the Transformations components. All transformations starting, ending or going between phases during that period were terminated with an application error. All affected transformations were resumed and all other transformations remain unaffected. 

We're sincerely sorry for this inconvenience!

On Dec 16 between 5pm and 9pm CEST some MySQL and Redshift queries containing comments were malformed in our parser. 

This bug has been resolved and deployed, all affected orchestrations were resumed. We're sorry for this inconvenience!

There was a bug affecting insightsPages_pivoted and insightsPosts_pivoted configurations for pages or posts with no values in some metrics. Under certain situations the zero value was replaced with the value of 1.

To fix any affected data you can run the extractor with parameters to backfill a given period of time, eg.

{
  "since":"2012-01-01",
  "until":"2015-12-10"
}

Put these parameters in your orchestrator configuration or use a separate API call to create a job. 

We're sorry for this bug and for any trouble caused, read more about Facebook Insights Extractor for additional info.

If you have long list of transformations or they're just quite complex, you may find yourself somewhat annoyed with the responsiveness of the UI. If that is the case, you will be delighted to know that we have solution ready to go that significantly improves the user experience there, maintaining the same functionality.

Before we roll out this feature to all projects we'd like to invite you to beta-test this feature. If you're crunching through a vast list of transformations, you can benefit from this today. The migration process is easy, let us know at support@keboola.com. We'll migrate your configuration for you. There are no changes in the UI or anywhere else, nothing will be stopped or lost. Numerous backups are made along the way and rollback is easy.  

Thanks for participating!

We have finally updated the MySQL transformation server to MariaDB 5.5.44, the same version as the sandbox server. 

Everything should be running as expected, no migration is required, but if you encounter any issues or incorrect behaviour, let us know.

Quay Integration

(Read on if you build components for KBC, feel free to go on with your life if you don't):

Our technically brave customers and partners might have noticed an outage in DockerHub automated build system that affected our docker-based components on the 16th and 17th of November. We were not happy with Docker's (lack of) communication regarding this outage. To prevent from similar occurrences in the future (and to provide alternative) we have implemented integration with Quay as well. Quay provides the same functionality as DockerHub, offering nicer UI on top of it. In case a future DockerHub outage affects you in any way, you can switch your images to Quay and we can hot-swap component configuration.

Support for image tags

On November 9th DockerHub changed default tag of the latest automated image build from latest to master (branch name). When adding a Docker image as a component in Keboola Connection you can now choose which tag will be pulled from the repository.

Curious about Docker integration in Keboola Connection? You can build your own (data) apps and run them on our infrastructure, read more here.

To address security of passwords and other components that require stronger protection, KBC now allows to encrypt certain values in stored configurations. All attributes prefixed with a hashmark sign (#) are automatically encrypted during save. The key is derived from the used component and project and there are no means in any UI or API to decrypt the value. The original value is available only internally and only to the app during its runtime.

What does that mean? When you save your password as an encrypted attribute, even you cannot decrypt it. It becomes available only in the application and in the project it was encrypted and the values cannot be transferred to any other apps or projects. Your passwords are safe and cannot be retrieved even by user with admin rights to your KBC project.

We hope this makes you feel safer! :-)

Note to developers and tech partners: The encryption is completely transparent. You only need 2 simple things: 

1. tell us that your component uses encryption
2. prefix all encrypted attributes with # (eg. password => #password)

The infrastructure takes care of the rest. Your application will "see" the decrypted value.