Transformation jobs forking

Between 11:08am CEST and 2:03pm CEST under certain circumstances transformation jobs were forking from existing processing transformation jobs. We have found the root cause, fixed it, terminated all forks and restarted all failed orchestrations. If you're still seeing this issue, please let us know.

UPDATE: This issue occurred again between 4:49pm CEST and 8:05pm CEST.

We're sincerely sorry for this inconvenience.

Important Facebook Ads extractor update

On Friday April 8th we'll be switching our Facebook Ads extractor to Facebook Marketing API from v2.4 to v2.5. If your configuration is using a fixed API version (api_version bucket attribute), please update it manually.

Please see the Graph API changelog to verify your configuration won't break, here's a shortlist:

Marketing API Changes from v2.4 to v2.5

  • Consistent naming for three-level campaign structure between API and UI. This changes naming at endpoint, params, fields, and enum level. For the full list of changes, refer here.
    • Change /adcampaign_groups to /campaigns
    • Change /adcampaigns to /adsets
    • Change /adgroups to /ads
    • In the write path, change campaign_group_status, campaign_status, adgroup_status to status
  • Campaign
    • Change naming of objectives
    • WEBSITE_CLICKS to LINK_CLICKS
    • WEBSITE_CONVERSIONS to CONVERSIONS
  • Targeting
    • Response for call to /search?type=adgeolocation will contain only the 'City' instead of 'City, State'
    • Change {cpc, cpm, cpa}_{min, max, median}, such as cpc_min fields in /reachestimate to bid_amount_min, bid_amount_median, bid_amount_max. Change bid_for to optimize_for

Should you encounter any issues after the update, please let us know at support@keboola.com

In case you need a bit more time to migrate your configuration, use bucket attribute api_version and set it to v2.4. This will give you time until Facebook shuts down this version completely (Monday April 11th).

Transformation Versions Management

We're introducing a simple versions management for transformation buckets. You can quickly access some functions via a dropdown on transformation bucket page.

This dropdown shows only last 5 versions and allows to quickly rollback any of the previous versions or to copy the current version to a new bucket. All versions are available on a separate page via Show all versions menu option.

There you can see all versions, do a fulltext search within their descriptions, dates or authors, rollback to any version in history or copy any version to a new bucket.

Versions always work with the whole bucket, you cannot rollback or copy a single transformation. 

Please consider this as a first version - we're too excited to present you what we have so far we didn't want to wait to have everything ready. There are plenty of enhancements to come:

  • better change descriptions - we'll be fixing the messages to be more accurate and descriptive
  • versions for all components and their configurations
  • access to versions from transformations and not only buckets
  • diffing
  • dev workspace and committing changes
  • ability to set your own change description

Please let us know your thoughts, happy versioning! And of course, if you experience any unexpected behavior or find a bug, reach out to us at support@keboola.com.

Week in Review -- March 29, 2016

Another week has gone by at lightning speed. Here's what's new in Keboola Connection.

Documentation

We started overhauling our docs. About time. Read more...

Temporary access to projects

Our support staff (that's us, developers, btw) will stop littering your projects with unused accounts. We're now only allowed to enter your project for a limited period and we must provide a reason. Everything is saved, so you can see what's happening. Thoughts on security, control and privacy? Talk to us!

Bugfixes
  • Truncating Redshift tables now correctly updates bucket stats
New components or component changes
  • Apache Impala extractor
  • Bing Ads extractor, a 3rd party extractor provided by David Ešner
  • S3 writer allows to specify the format of timestamp suffix and optionally compresses the files

Transformation UI bug

A recent minor update in transformation UI introduced a bug, where certain input / output mappings didn't open. The bug has been fixed and deployed. If you're still experiencing this issue please refresh your browser.

We're sorry for this inconvenience.

Transformation failures

Feb 25th 10:00 GMT+0100

We're investigating issues when working with transformation configurations (creating, saving, deleting). We're sorry for this inconvenience, it will be fixed shortly.

UPDATE Feb 25th 10:10 GMT+0100

The issue has been resolved. Thank you for your patience.

Transformation configuration migration

Following an extensive testing with volunteer users (thank you all, you know who you are!), we will migrate transformation configuration storage for all projects. The migration will happen in several batches processed during a 5 days period next week. We'll be monitoring the process but we don't expect any failures, outages or other bugs.

Expected period of migration is Monday Feb 15th to Friday Feb 19th. Once your project is migrated, you'll get a notification (for each project in which you are an admin). 

During the migration process all your transformations will be serialized and uploaded to your project's File uploads for safekeeping.

Why?

The current method of storing configuration in sys.tr-* buckets has become obsolete and practically prohibits new features. Storing transformation configuration natively in a dedicated configuration storage (part of Storage API) opens, besides having some immediate benefits, the door for new exciting features. 

Benefits

  • faster saving and loading of transformations (faster & smoother UI)
  • elimination of all sys.c-tr-* buckets (cleaner Storage console)
  • plenty of upcoming cool features (versioning, anyone?)

Should you encounter any issues during the migration or in case you have any concerns/questions regarding your project, don't hesitate to contact us at support@keboola.com.


Introducing notifications

We're adding a new channel to inform you about important events in Keboola Connection or any of your projects. Notifications show up as a bell icon next to your name or the Keboola logo. 

When there's a new notification waiting for you the bell has a red badge. Clicking on the icon you'll go to the Notifications page, where you'll see all notifications. Unread notifications are highlighted and you can mark each notification as read or mark all notifications as read in a single click. 

Notifications will inform you

  • about new significant features
  • about important events regarding your project (outages, migrations)
  • when your project reaches contract limits

It's not a Facebook feed, we promise we won't bother you too often.

Keboola Connection Week in Review: January 23rd

In our effort to bring you all updates we're now starting a regular weekly post that will include a list of all changes, that didn't make it to a separate post.

Input mapping UI bugfix

Creating a duplicate destination in input mapping is now displayed as an error.

Input mapping UI improvement

When selecting a table, the destination is automatically populated with table name (previously with full table id).

Storage: Redshift primary keys bugfix

Setting a primary key on a Redshift table failed when there was another table in the project with the same name and different columns.

Storage: MySQL long primary key bugfix

Setting a compound primary key with more than 4 columns failed with an internal error. This was fixed and it shows a warning that MySQL primary key is limited to 4 columns.

Other posts this week