Components errors

We observe components failures hosted on quay.io from 08:25  AM CET. Python Transformations, Facebook Extractor, Google Sheets Writer are between affected components.

Quay.io outage is not reported yet. To avoid this issue in future we are moving all components to Keboola provided AWS repository.

We will inform about the progress.

9:09 AM CET RESOLVED Last error was seen at 08:53 AM CET. Quay.io has confirmed and fixed the issue.

New Intercom Extractor

Due to a change in the Intecom API authorization, the current extractor stopped working. It took us a while to go through the Intercom OAuth registration, but the result is here - the new Intercom Extractor. The current version is immediately deprecated, please configure and authorize the new extractor. You can start downloading your Intercom data in seconds.

The full documentation is available here.

End of life announcement for synchronous table exports

The deprecated table synchronous exports will be shut down on 21st Aug 2017. 

This API call was replaced by Asynchronous table export for full table unloads and Data Preview for small samples of table data.

All Keboola Connection components use the new methods. As there are only a few projects currently using the deprecated method directly, these projects will be notified individually. For all other projects there is no action required.

Week in Review - July, 31, 2017

New Features

  • RStudio and Jupyter Sandboxes now support (and enforce) HTTPS connections.

Updated Components

Qlik and Looker writers were updated:

  • Improved and unified UI

  • Optional provisioned Redshift database

Fixes

  • Fixed freezing of columns in Google Sheets Writer


Job failures

There were jobs failures between 4:30 AM - 5:30 AM CET. Failures were caused by low disk space of one of worker servers.

We are sorry for this inconvenience and we're taking steps to mitigate this problem in the future.

Deprecated Facebook Ads API v2.8 and below

Facebook is deprecating all Marketing API versions prior to v2.9 on Wednesday, July 26, 2017 and recommends upgrading to version v2.9 immediately. Facebook Ads Extractor uses v2.8 by default for all newly created configurations but after July 26 all newly created configurations will have default api-version set to v2.9.

The migration should only require changing the api-version parameter in all existing configurations of our Facebook Ads Extractor. However, since there may be some breaking changes, we strongly recommend to change the api-version in your configurations manually, review any possible changes and take the appropriate actions. For more details on the new version, please read Marketing API Changes in v2.9 in the Facebook API changlelog. Of those changes we'd like to highlight the deprecation of date_preset values which are replaced with new ones, e.g. last_3_days, replaced by last_3d .

In other Facebook news, they have announced version v2.10 of Facebook API and Facebook Marketing API.

Week in review -- July 19, 2017

GoodData Writer

  • Upload Project job isn't executing reports anymore. This caused troubles on Vertica backend. You can add report execution API call into orchestrator if needed

Python sandboxes

  • Updated to Jupyter 5.0.0 
  • Current directory is now */data/*

    Transformations (SQLdep)

    • Fixed issue with lineage on aliased cloumns

    Generic Extractor

    • Delimiter character can now be configured in Data Field, see documentation
    • Functions can now be used in Login Authentication in API Request, see documentation

    Storage API CLI

    • Added create-bucket command
    • It is now possible to run this tool as Docker container
    • Latest version is available on GitHub

    Storage API Client for Python

    • Alpha version of the client is available on GitHub.  
    • Call for contributions: any pythonistas out there want to contribute, please do so:) 

    Storage Jobs Stuck in Processing State

    We have noticed an increased number of Storage jobs stuck in the processing state, in rare cases causing a complete halt and queuing of all new Storage jobs in a project.

    We have identified the root cause. Due to the Snowflake incident last night it seems some of the database transactions were still open and they blocked queries from consecutive jobs. We have terminated all orphaned transactions and jobs started processing and restarting. 

    Please note, that queued jobs are processed in random order. 

    We are sorry for this inconvenience.

    Sudden Jobs Failures on July 6th, 2017

    On July 6th, 2017, between 9:16-9:18am CET one of our internal databases was forced to update and restart by AWS. Result of this action was Application error failure of all running components jobs. The error may show up significantly later than the restart.  We recommend to review your orchestrations jobs and take action if needed. We are sorry for this inconvenience and we're taking steps to mitigate this problem in the future.