Week in Review -- May 28, 2018

New Components

OneDrive Extractor (Beta)
  • You can now download your documents from OneDrive. The component was developed by Jakub Bartel.

Google Big Query Writer (Beta)

  • Write data from KBC to Google Big Query with our new writer. The component was developed by us, Keboola s.r.o. 


Updated Components

Currency Extractor

  • Now offers exchange rates of GBP.

Oracle Extractor

  • Now supports exporting columns of the LOB datatype family.

Gmail Attachments Extractor

  • Added support for processors, see example.

Create Manifest Processor 

  • Will test, if all the slices of a CSV file have the same header columns.

Flatten Folders Processor 

  • Throws a user exception when the flattened filename is too long (longer than 255 characters).
  • Has now configurable flatten strategies. Added `hash-sha256` strategy, which solves an issue with 255 characters filename limit in default `concat` strategy.


New Features

  • Component badges.
  • Validate your SQL in transformations with new Validate feature, provided by SQLDep.

  • MySQL extractor now supports incremental fetching. You can extract just the most recent records from a database table and write them incrementally into Storage.


    Minor Improvements

    • Show more / show less button added to the list of inputs and outputs of a job. It now shows all the tables and the view is more compact.

    Week in Review -- March 5, 2018

    Improvements

    • Schema in Snowflake Extractor is no longer a required connection parameter. If not set, the table selector allows you select tables from the whole database.
    • Snowflake extractor now supports importing these semi-structured data types: `VARIANT`, `OBJECT`, and `ARRAY` 

    • Updated two factor authentication in Keboola Developer Portal. The SMS authentication is now deprecated. All new users will have to use either Google Authenticator or Duo Mobile app.
    • MySQL extractor now has an option to enable compression of data sent over network
    • Enhanced Output Mapping selector
    • R in Sandbox and Transformations is updated to 3.4.3, also the Tidyverse package is now installed by default.

    Bug fixes

    • Data Takeout was randomly failing on backing up your data to S3.
    • Task editor in Orchestrator produced errors when orchestration had configured dozens of tasks.
    • In Twitter extractor template, if a user made a mention of your account, the details of that user account weren't downloaded. Edit and save existing configuration to remedy this issue.

    Google Analytics Extractor - Custom Authorization

    Recently we have been hitting quota limits of the Google Analytics API with our extractor. As these limits are shared between all users of the extractor, we decided to add a new authorization option - Custom Authorization.

    With Custom Authorization, you can provide your own API keys for the Google Analytics extractor, so you will have your own quota limits. This way you can monitor these limits as you need, and other users won't be affected.

    Please check out our step-by-step guide on how to set up Custom Authorization in the Google Analytics Extactor.

    In case of any problems or questions, please contact us on support@keboola.com.

    End of life of Keboola Academy

    All things have to come to an end and Keboola Academy’s GoodData training is no exception. But don’t mourn, we are working hard on a new education program. Its main focus will be Keboola Connection.

    Keboola Academy courses will be available for purchase until December 31, 2017. After that, you won’t be able to purchase new courses, but you’ll still be able to access your already purchased courses for another two months. Courses will be shut down completely on February 28, 2018.

    If you have any questions, please contact us via support@keboola.com.

    Week in Review -- August 28, 2017

    New Features

    • List of deprecated components in the Project Overview:

      In the Project Overview you may now see a warning that you are using deprecated components in your project. 

      To prevent any inconveniences, please keep your components up-to-date. 

      Clicking on a component from the list will take you to a component configuration, from where you can start a migration process.

      Fixes

      • Fixed issue in Orchestrator, when you changed notification settings during orchestrators run, the settings will return to previous values after orchestrator has finished. (github issue in Czech language)

      Week in review -- July 19, 2017

      GoodData Writer

      • Upload Project job isn't executing reports anymore. This caused troubles on Vertica backend. You can add report execution API call into orchestrator if needed

      Python sandboxes

      • Updated to Jupyter 5.0.0 
      • Current directory is now */data/*

        Transformations (SQLdep)

        • Fixed issue with lineage on aliased cloumns

        Generic Extractor

        • Delimiter character can now be configured in Data Field, see documentation
        • Functions can now be used in Login Authentication in API Request, see documentation

        Storage API CLI

        • Added create-bucket command
        • It is now possible to run this tool as Docker container
        • Latest version is available on GitHub

        Storage API Client for Python

        • Alpha version of the client is available on GitHub.  
        • Call for contributions: any pythonistas out there want to contribute, please do so:) 

        New Google Drive Writer

        As we announced earlier, we have decided to split the current Google Drive Writer into two separate components to simplify their usage:

        • Google Sheets Writer - as its name implies, is designed to upload tabular data from Storage to Google Sheets.
        • (New) Google Drive Writer - will handle uploading general files to Google Drive.

        Google Sheets Writer has already been available for some time and now we are proud to release the new version of Google Drive Writer.

        The Google Drive Writer is best for uploading general files into Drive. It allows you to create a new file on each run and therefore is ideal for backup purposes.

        Features

        • Advanced Input Mapping
          • choose which columns will be uploaded
          • filter data by date or by column values

        • Update existing files or create new files


        Migration

        The existing Google Drive Writer will be deprecated soon and a migration tool will be made available to help you transfer your existing configurations to new Google Drive or Google Sheets Writer respectively.

        Don't hesitate to give us feedback or ask a question, write to support@keboola.com.

        New Google Sheets Writer

        Google Sheets Writer is now available. It is meant to replace a portion of the current Google Drive Writer.

        We have decided to split the current Google Drive Writer into two separate components to simplify their usage:

        • Google Sheets Writer - as its name implies, is designed to upload tabular data from Storage to Google Sheets.
        • (New) Google Drive Writer - will handle uploading general files to Google Drive. It will be released very soon.



        Features

        • Advanced Input Mapping
          • choose which columns will be uploaded
          • filter data by date or by column values

        • Upload into existing or new Spreadsheet

        • Update or append data in a Sheet
          • you can write into existing Sheet within a Spreadsheet or create a new one


        Migration

        The existing Google Drive Writer will be deprecated soon and a migration tool will help you transfer your existing configurations to new Google Drive or Google Sheets Writer respectively.

        Don't hesitate to give us feedback or ask a question, write to support@keboola.com.

        Revision of Database Writers, new Impala Writer

        We have released new versions of these database writers:

        • MySQL
        • Microsoft SQL Server
        • Redshift
        • Oracle

        Also, we are introducing Cloudera Impala database writer.

        All these writers are running on container based architecture with SSH tunnel support.

        The Database writer and the old version of MSSQL writer are now marked as deprecated. We will continue to  support them for at least 3 months from now. After this period, we will migrate any remaining old configurations to the new versions.

        We are now preparing a migration tool to help you migrate your existing configurations to the new versions.

        If you have any questions or need help, please contact us at support@keboola.com.