New Google Sheets Writer

Google Sheets Writer is now available. It is meant to replace a portion of the current Google Drive Writer.

We have decided to split the current Google Drive Writer into two separate components to simplify their usage:

  • Google Sheets Writer - as its name implies, is designed to upload tabular data from Storage to Google Sheets.
  • (New) Google Drive Writer - will handle uploading general files to Google Drive. It will be released very soon.



Features

  • Advanced Input Mapping
    • choose which columns will be uploaded
    • filter data by date or by column values

  • Upload into existing or new Spreadsheet

  • Update or append data in a Sheet
    • you can write into existing Sheet within a Spreadsheet or create a new one


Migration

The existing Google Drive Writer will be deprecated soon and a migration tool will help you transfer your existing configurations to new Google Drive or Google Sheets Writer respectively.

Don't hesitate to give us feedback or ask a question, write to support@keboola.com.

Week in Review -- April 4, 2017

New Component - Papertrail Extractor

We’re happy to welcome the Papertrail Extractor to the family.  Papertrail manages billions of log messages for operations-savvy companies. It has been our log management system of choice for years. If you log to Papertrail and realise that log messages contain important information, feel free to incorporate this unstructured data into your data strategy. By using our Papertrail Extractor, you can download all records matching your search query within the retention period. The extractor can also incrementally add new records each run.

Discrete sessions from e-commerce, low-level transactions, developer stack traces or operational data - everything can fit in your Keboola project! 

We will cover this topic in an upcoming blog post next week. If you're interested in how we perform complex analytic deep-dive into our logs, follow our Medium account at https://500.keboola.com/

Minor Improvements

  • S3 extractor now displays how many files were downloaded in the specific job; it is very handy especially in case of wildcard rules

  • MSSQL Writer now supports the BCP method - you can activate it in the table settings - it can write your data to desired MSSQL DB at supersonic speed, but take note that it doesn't handle weird UTF8 characters properly

  • Transformations with Snowflake backend now support FLOAT data type in input mapping -> no hacking with NUMBER data type anymore



Updated UI for S3 Extractor and CSV Import

We have updated the UI for the S3 Extractor and the CSV Import tool. We're keen on trying new things to improve your user experience with Keboola Connection and hope that these improvements offer a simpler and easier approach to editing your component configurations.

There is no difference between editing and non-editing state, everything is directly editable and the state of the Save button indicates if the configuration has been changed or not. 

If this approach proves useful and stands the test of time, we'll carry it on to other components as well.

Let us know what you think!

Strict time-out in Snowflake Writer

Between Mar 29 10:03 UTC and Mar 29 13:10 UTC all Snowflake Writer jobs had accidentally 15 seconds limit and potentially ended-up by error message:

Statement reached its statement or warehouse timeout of 15 second(s) and was canceled.

All Snowflake writers should be fully operational now. Only four jobs were affected by this bug - Please accept our sincere apologies for this inconvenience!

Week in Review -- March 27, 2017

Trash for configurations

One of the most wanted features in Keboola Connection was the option to restore deleted configuration.

We are happy to announce this feature is now avalable for all users.

This feature allows you to:

1. Check who and when deleted a configuration

2. Restore configuration

Configuration Restore is supported by all components, except Orchestrator, GoodData Writer and some old components which have configuration in a bucket created in `sys` stage.

3. Delete configurations permanently

If you use "Empty Trash" button, all configurations will be deleted permanently. There is no other way to restore them!

As always, you can use mentioned functionality by calling our Storage API: http://docs.keboola.apiary.io/#reference/component-configurations/configuration-restore/restore-deleted-configuration


Fixes and improvements

  • added optional parameter to specify warehouse in Snowflake Database Writer
  • fixed credentials editing in DB writers

Facebook Ads API v2.7 deprecation from April 25, 2017

tldr; If you're using the old Facebook Ads (Deprecated) extractor then migrate to new extractor or set the `api_version` attribute to v2.8 in corresponding configuration sys table of the old extractor before April 25 2017.

Marketing API(aka Facebook Ads API) version v2.7 has been marked as deprecated and will stop working from April 25, 2017. The latest version is v2.8, see Marketing API Changes in v2.8.

The New Facebook Ads extractor uses version v2.8 by default so no actions are required.

The Old Facebook Ads extractor uses version v2.7 by default. The best thing to do would be to migrate to the new extractor. Another possibility is to override the api version setting: Go to the configuration sys table and update the attribute api_version with value v2.8. There should be no incompatibilities, but we offer no guarantees.




Week in Review -- March 20, 2017

New Components

  • Facebook Extractors have been rewritten. For more information see this post.

New Features

  • We added Table Data Filter to latest versions of our database writers. So it's possible to filter data by "days" and "conditions"

  • New option of bucket sharing has been added. Now you can share buckets to your whole organization (by setting new sharing option) and anyone in your organization be able to link bucket to his project.

Minor Improvements & Fixes

  • From now, attempt to set nonexistent column as Primary Key (in Storage API) will cause an user error

New Facebook Extractors

As you might have already noticed, new versions of Facebook Extractor and Facebook Ads Extractor, that runs on container based architecture,  are available. The old ones have been marked deprecated.

Improvements compared to the old extractors

  • UI templates allow you to easily configure the extraction. 
  • More customisation possible (more endpoints, different api versions)
  • The output tables have more intuitive structures.  Unfortunately this means that direct data transfer from the old extractor is not possible
  • Tables are always incrementally imported with auto-detected primary keys

Migration

Due to various differences in output tables structure and configuration between new and old Facebook extractors we don't provide automatic migration tool, however there is basic tutorial on how to create configuration with same authorized account. See here for Facebook and here for Facebook Ads extractor.

We are always glad to receive feedback, so if you have any questions or ideas how to improve this component, please don't hesitate to contact us. Go ahead and try it ;)

Deprecation of MySQL database provisioning for database writers

We no longer provide Keboola MySQL database provisioning for database writers. Existing Keboola provisioned MySQL databases will be kept intact till the end of Jun 2017. Existing MySQL users will have to move to their own MySQL instances. Here are a few suggestions of vendors which provide MySQL provisioning:

Amazon AWS

- DigitalOcean

- Microsoft Azure

Google Cloud

- Rackspace

You can keep your data inside KBC and connect to them directly. We believe that our Snowflake Writer is a great replacement for your existing MySQL Writer, so please check it out!

Do not hesitate to contact if you have any questions!