Week in Review -- August 28, 2017

New Features

  • List of deprecated components in the Project Overview:

    In the Project Overview you may now see a warning that you are using deprecated components in your project. 

    To prevent any inconveniences, please keep your components up-to-date. 

    Clicking on a component from the list will take you to a component configuration, from where you can start a migration process.

    Fixes

    • Fixed issue in Orchestrator, when you changed notification settings during orchestrators run, the settings will return to previous values after orchestrator has finished. (github issue in Czech language)

    Week in review -- July 19, 2017

    GoodData Writer

    • Upload Project job isn't executing reports anymore. This caused troubles on Vertica backend. You can add report execution API call into orchestrator if needed

    Python sandboxes

    • Updated to Jupyter 5.0.0 
    • Current directory is now */data/*

      Transformations (SQLdep)

      • Fixed issue with lineage on aliased cloumns

      Generic Extractor

      • Delimiter character can now be configured in Data Field, see documentation
      • Functions can now be used in Login Authentication in API Request, see documentation

      Storage API CLI

      • Added create-bucket command
      • It is now possible to run this tool as Docker container
      • Latest version is available on GitHub

      Storage API Client for Python

      • Alpha version of the client is available on GitHub.  
      • Call for contributions: any pythonistas out there want to contribute, please do so:) 

      New Google Drive Writer

      As we announced earlier, we have decided to split the current Google Drive Writer into two separate components to simplify their usage:

      • Google Sheets Writer - as its name implies, is designed to upload tabular data from Storage to Google Sheets.
      • (New) Google Drive Writer - will handle uploading general files to Google Drive.

      Google Sheets Writer has already been available for some time and now we are proud to release the new version of Google Drive Writer.

      The Google Drive Writer is best for uploading general files into Drive. It allows you to create a new file on each run and therefore is ideal for backup purposes.

      Features

      • Advanced Input Mapping
        • choose which columns will be uploaded
        • filter data by date or by column values

      • Update existing files or create new files


      Migration

      The existing Google Drive Writer will be deprecated soon and a migration tool will be made available to help you transfer your existing configurations to new Google Drive or Google Sheets Writer respectively.

      Don't hesitate to give us feedback or ask a question, write to support@keboola.com.

      New Google Sheets Writer

      Google Sheets Writer is now available. It is meant to replace a portion of the current Google Drive Writer.

      We have decided to split the current Google Drive Writer into two separate components to simplify their usage:

      • Google Sheets Writer - as its name implies, is designed to upload tabular data from Storage to Google Sheets.
      • (New) Google Drive Writer - will handle uploading general files to Google Drive. It will be released very soon.



      Features

      • Advanced Input Mapping
        • choose which columns will be uploaded
        • filter data by date or by column values

      • Upload into existing or new Spreadsheet

      • Update or append data in a Sheet
        • you can write into existing Sheet within a Spreadsheet or create a new one


      Migration

      The existing Google Drive Writer will be deprecated soon and a migration tool will help you transfer your existing configurations to new Google Drive or Google Sheets Writer respectively.

      Don't hesitate to give us feedback or ask a question, write to support@keboola.com.

      Revision of Database Writers, new Impala Writer

      We have released new versions of these database writers:

      • MySQL
      • Microsoft SQL Server
      • Redshift
      • Oracle

      Also, we are introducing Cloudera Impala database writer.

      All these writers are running on container based architecture with SSH tunnel support.

      The Database writer and the old version of MSSQL writer are now marked as deprecated. We will continue to  support them for at least 3 months from now. After this period, we will migrate any remaining old configurations to the new versions.

      We are now preparing a migration tool to help you migrate your existing configurations to the new versions.

      If you have any questions or need help, please contact us at support@keboola.com.

      Week In Review -- November 14, 2016

      Here's what last week was about:

      • Fixed bug in Redshift backend: Exporting a table with column of maximum allowed length (64KB) no longer causes an error.
      • MariaDB, which is used as MySQL backend in transformations, was updated to version 5.5.53. This update enhances the handling of 4-byte UTF-8 characters. In previous version 5.5.44 a string was cut after the first utf8mb4 character. Now, all these characters will be converted to '?'. Full support of 4-byte UTF-8 characters is unfortunately unavailable in MySQL.
      • Fixed bug in MySQL extractor: When query returned an empty result, extractor ended up with 'Orphaned manifest' error

      Failed Jobs

      Some jobs have failed between 2016-09-26 17:30 CEST and 2016-09-26 17:40 CEST due to problem with one of our DB servers.

      All affected orchestrations have been checked and restarted.

      We are very sorry for any inconvenience.

      If you have any concerns about this, please contact us at support@keboola.com.

      Anti-Sampling for Google Analytics Extractor

      As some of you might know, Google Analytics API doesn't always return precise data. Under certain circumstances, the data returned are sampled. Read more about sampling here.

      To work around this problem and get more precise results, we are introducing a much anticipated feature into Google Analytics Extractor:

      Anti-Sampling

      You can choose from one of two anti-sampling algorithms - DailyWalk or Adaptive. Both are based on the same principle, to divide the wanted date range into smaller chunks.

      DailyWalk as the name suggest divides the date range by days. So the extractor needs to make as many request as there are days in the date range.

      Adaptive algorithm is using more sophisticated approach, and divides the date range into few smaller date ranges. Read in-depth explanation of the algorithm here, if you are interested. 

      DailyWalk algorithm might be more precise in some cases, but usually you will get the same results faster with the Adaptive algorithm.

      Experiment with them and use what suits you best.

      Google Analytics Extractor Bug

      We have encountered a bug in the Google Analytics Extractor. It was downloading data only for one profile, even if the query was set to "All profiles".

      This bug was introduced on 16th of September and it is now fixed.

      We are very sorry for any inconvenience. Please adjust your date ranges and download the missing data if needed.