Transformation configuration migration

Following an extensive testing with volunteer users (thank you all, you know who you are!), we will migrate transformation configuration storage for all projects. The migration will happen in several batches processed during a 5 days period next week. We'll be monitoring the process but we don't expect any failures, outages or other bugs.

Expected period of migration is Monday Feb 15th to Friday Feb 19th. Once your project is migrated, you'll get a notification (for each project in which you are an admin). 

During the migration process all your transformations will be serialized and uploaded to your project's File uploads for safekeeping.

Why?

The current method of storing configuration in sys.tr-* buckets has become obsolete and practically prohibits new features. Storing transformation configuration natively in a dedicated configuration storage (part of Storage API) opens, besides having some immediate benefits, the door for new exciting features. 

Benefits

  • faster saving and loading of transformations (faster & smoother UI)
  • elimination of all sys.c-tr-* buckets (cleaner Storage console)
  • plenty of upcoming cool features (versioning, anyone?)

Should you encounter any issues during the migration or in case you have any concerns/questions regarding your project, don't hesitate to contact us at support@keboola.com.


Introducing notifications

We're adding a new channel to inform you about important events in Keboola Connection or any of your projects. Notifications show up as a bell icon next to your name or the Keboola logo. 

When there's a new notification waiting for you the bell has a red badge. Clicking on the icon you'll go to the Notifications page, where you'll see all notifications. Unread notifications are highlighted and you can mark each notification as read or mark all notifications as read in a single click. 

Notifications will inform you

  • about new significant features
  • about important events regarding your project (outages, migrations)
  • when your project reaches contract limits

It's not a Facebook feed, we promise we won't bother you too often.

Keboola Connection Week in Review: January 23rd

In our effort to bring you all updates we're now starting a regular weekly post that will include a list of all changes, that didn't make it to a separate post.

Input mapping UI bugfix

Creating a duplicate destination in input mapping is now displayed as an error.

Input mapping UI improvement

When selecting a table, the destination is automatically populated with table name (previously with full table id).

Storage: Redshift primary keys bugfix

Setting a primary key on a Redshift table failed when there was another table in the project with the same name and different columns.

Storage: MySQL long primary key bugfix

Setting a compound primary key with more than 4 columns failed with an internal error. This was fixed and it shows a warning that MySQL primary key is limited to 4 columns.

Other posts this week

End of Life Announcement: Remote Transformations (eg. Text Splitter)

As of today, January 22th 2016 we won't be making any further improvements or updates on Remote Transformations (including the executor interface in Transformations and all remote transformations, eg. Text Splitter)

As of February 22th 2016 Remote Transformations functionality will be completely removed from Transformations.

Text Splitter functionality can be replaced by an R transformation, see the demo code.

Contact our support, if you need any assistance with migration to R transformation.

MySQL sandbox server outage

The MySQL sandbox server (sh-tapi.keboola.com) is currently not available, we're investigating the cause.

Update Jan 20 5:35pm CET

The backup MySQL sandbox server tapi-sandbox-fallback-3.keboola.com is up and serving all sandbox requests. All previous sandboxes were deleted and all credentials cleared. We'll be transitioning back to the original server sh-tapi.keboola.com tomorrow. Please check your sandbox credentials to be sure you're logging on the right server.

We're sorry for this inconvenience and thanks for your understanding

Update Jan 21 5.33pm CET

The original MySQL sandbox server (sh-tapi.keboola.com) is back up. All new sandboxes will be created on this server, all currently existing sandboxes will remain on tapi-sandbox-fallback-3.keboola.com and will be purged on Sunday night.

Brief Transformation Outage

Between 5:29pm CET and 5:46pm CET there was an outage in one of the Transformations components. All transformations starting, ending or going between phases during that period were terminated with an application error. All affected transformations were resumed and all other transformations remain unaffected. 

We're sincerely sorry for this inconvenience!

Transformation failures

On Dec 16 between 5pm and 9pm CEST some MySQL and Redshift queries containing comments were malformed in our parser. 

This bug has been resolved and deployed, all affected orchestrations were resumed. We're sorry for this inconvenience!



Facebook Insights Extractor Bug in pivoted configurations

There was a bug affecting insightsPages_pivoted and insightsPosts_pivoted configurations for pages or posts with no values in some metrics. Under certain situations the zero value was replaced with the value of 1.

To fix any affected data you can run the extractor with parameters to backfill a given period of time, eg.

{
  "since":"2012-01-01",
  "until":"2015-12-10"
}

Put these parameters in your orchestrator configuration or use a separate API call to create a job. 

We're sorry for this bug and for any trouble caused, read more about Facebook Insights Extractor for additional info.



Transformation Configuration (Public Beta)

If you have long list of transformations or they're just quite complex, you may find yourself somewhat annoyed with the responsiveness of the UI. If that is the case, you will be delighted to know that we have solution ready to go that significantly improves the user experience there, maintaining the same functionality.

Before we roll out this feature to all projects we'd like to invite you to beta-test this feature. If you're crunching through a vast list of transformations, you can benefit from this today. The migration process is easy, let us know at support@keboola.com. We'll migrate your configuration for you. There are no changes in the UI or anywhere else, nothing will be stopped or lost. Numerous backups are made along the way and rollback is easy.  

Thanks for participating!