We have been working on RStudio and Jupyter Notebook sandboxes (these are the missing sandboxes for R and Python transformations) for some time and we now feel that it is time to move to the next phase. 

We can now enable these on a per-user basis for early adopters. If you are interested in playing with these Sandboxes please contact us at support@keboola.com (don't forget to tell us your KBC email).

Limitations:

• Sandboxes are available only as plain sandboxes (not in a transformation bucket yet)
• HTTPS is not yet supported.
• Sandbox disk space is limited to 10GB.
• Memory is limited to 8GB.
• The UI allows loading only tables to the Sandbox. Loading input files and transformation script is supported only by the API.
• Sandboxes will be deleted after 5 days.
• Adding data to existing sandboxes is not supported.

Otherwise, it's like RStudio connected to your KBC project:

The CSV file locations are the same as in transformations (/data/in/tables/). In the meantime, we will work on:

• HTTPS Support.
• Creating better UI and integration with transformations.
• Allowing tables and files to be added to a running sandbox.

Sandboxes

We have introduced garbage collection for MySQL sandboxes. All sandboxes will be automatically discarded after 14 days. Redshift and Snowflake sandboxes will follow soon.

BigQuery Extractor

The Extractor allows you run query with BigQuery legacy SQL or standard SQL dialect.

Snowflake Writer

Empty values are now converted to NULL when the column is set to nullable.

GoodData Extractor

Old version of GoodData extractor has been deprecated in December and will be switched off in upcoming days. Please migrate to the new version if you haven't done it yet. More information here.

Project Access Approval

Now you can require that visiting support administrators will have their access request approved by an existing project member in order to temporarily join your project. See this article for more information.

In an effort to increase account sovereignty we've introduced a new feature that will give you more control over who can access your projects and for what purposes.   It is now possible to require that visiting support administrators have their access request approved by an existing project member in order to temporarily join a project.

The setting can be found in your Account settings at the organization level.  This setting will apply to all projects within the organization.

When the support user completes the request form, an email describing the access request is sent to each project member.  

The requested access will be listed in the project settings Users section and can be approved or rejected by any active project member.

There were few job failures between 20:55 and 21:15 CEST, due to problems on one of our workers.

We're sorry for this inconvenience, we have restarted the affected orchestrations.

Snowflake backend is experiencing service stability issues which may manifest itself as failed queries or service unavailability. Snowflake engineers are working on fix.

Read more on Snowflake status page.

Update (11:34 PM PST) - Snowflake is still in a recovery process and don't have a defined ETA. 

Update (12:38 AM PST) - Service is back online

A few months ago we released a new generation of database writers. 

If you are still using the old, deprecated, database writers in your KBC projects, it's time to migrate!

To help with this, we have just released a migration tool for old MySQL, Oracle and Redshift database writers.

If you have any questions or need assistance please contact support@keboola.com.

Our last week was filled with some bug fixes, performance improvements and old infrastructure deprecation.

Shared Buckets

Shared bucket loading was tweaked to load data directly on Snowflake backend. (available to everyone)

Loading to Sanboxes

Loading data to sandboxes (mainly used by Transformations and Sandboxes themselves) has been improved, which means multiple tables can be loaded at once.

You should notice speed improvement when using more tables in input mapping and loading larger amount of data. (testing phase, available on request)

Also we would like to mention:

• Old server providing MySQL sandboxes has been replaced due to failures.
  
• Snowflake Database Writer has been published
  
• and Migration Tool for Google Drive Extractor arrived too
  

Detail of a job with enabled "Multi Load"

There were few problems with MySQL sandboxes between 12:40 and 13:20 CEST.

Affected server has been deactivated and all operations should run normally right now.

We're sorry for inconvenience.

A couple of months ago we launched a new version of Google Drive Extractor and promised to release a migration tool.

Well it took a bit more time then we expected, but as promised, here it is. This tool will help you migrate your configurations to the new version of the extractor.

How to

1. Click "Proceed to Migration" button in any configuration of the old Google Drive Extractor
2. A dialog window will open with a list of all configurations in the project
3. Click the "Migrate" button and a migration job will be created
4. You can watch the progress of the migration job either in the migration dialog window or in the Jobs list
5. When the job has finished, you can see the result of the migration for every configuration and also any affected orchestrations will be listed in the "Affected Orchestrations" tab

Backward compatibility

The much anticipated Snowflake Database Writer is now available!

Key features:

• container based architecture
• upload data to your own Snowflake database or use a Keboola provisioned database. 
  
• SSH tunnel
  
• incremental load
  

Provisioned database limits:

You can use a Snowflake database provisioned by us, and this database is included in the price of your Keboola Connection project.  Please note, however, that there is a 15 second timeout per query, and an additional fee will be required in order to increase this timeout.

If you have any questions or troubles setting up this writer, please contact our support at support@keboola.com.