Jupyter and RStudio Sandboxes are not starting

3:45pm CEST: We're investigating the issue.

3:55pm CEST: Not only starting, existing sandboxes do not seem to respond.

4:10pm CEST: We're shutting down existing sandbox instances and spinning up new ones. This will take a couple of minutes before the sandboxes will be available again. All existing sandboxes will be unfortunately deleted.

4:20pm CEST: Sandboxes are starting again. All previous sandboxes are deleted. We're sorry for this inconvenience.

Week in Review -- March 29, 2018

New Components

New Features

  • We have released the Guide Mode, an interactive tutorial for Keboola Connection
  • "Sudo" mode - important changes are protected by requiring password

Updated Components

  • Google Drive Extractor, Google Drive Writer and Google Sheets Writer all support Team Drives
  • Google Sheets Writer preserves formatting when writing into an existing sheet
  • Generic Extractor supports arrays as properties in child jobs.

Minor Improvements

  • Encrypted values are now filtered from component events. This prevents accidental leak of credentials from a component, e.g. when it crashes as prints its stack trace or other internal logs to events
  • keboola.processor-orthogonal is now available to fix malformed CSVs. Handy if you encounter Load error: Line 1 - Extra column(s) found errors

New HTTP(S) Extractor

Another one joins the band.

In our effort to replace Restbox with modern components, the next logical step was the HTTP(s) extractor. It allows you to download a single CSV file or a compressed, publicly available file and import them into a single table in Storage. In case you have more public files to download from a single domain, the UI allows you to reuse the same base URL for more files.

The UI of the new extractor supports many features out of the box, but the extractor is not limited by its UI: it's another component that supports processors. So your CSV file can be invalid, in a weird charset, pivoted or mutilated in some other way, and there's tooling to get that fixed.

The list of available processors will be kept and updated in the Developer Portal list of components. A full description of the extractor is available in our documentation.

Introducing Guide Mode

We are happy to announce the immediate availability of Guide Mode. In Guide mode, the Keboola Connection user interface will switch to an interactive tutorial which will guide you through the basics of using Keboola Connection. 

Guide mode is designed for new users and works best on empty projects. Therefore, when you invite a new person to Keboola Connection, they will receive a special link in their invitation email:

The link leads to the try.keboola.com page. By following the link, they will receive a 15day demo project with the Guide mode activated. 

The Guide Mode is the very first step in creating a replacement the old Academy. We are gradually going to fill it with more advanced content, but in the mean time try it out and let us know what you think.


New version of AdWords Extractor

We have just released a new version of AdWords Extractor. It works with AdWords API v201802 (see the Release notes).

The previous version of the extractor is deprecated and you can use our migration tool which will migrate your AWQL queries. However, you have to reauthorize the extractor and give it access to your AdWords data again. The previous version uses AdWords API v201710 which will be switched off on 11 July 2018.

Week in Review -- March 19, 2018

New Components

Asana Extractor

We’re happy to welcome the Asana Extractor to our family. It can extract your projects and tasks from the Asana application which is designed to help teams track their work. This component was developed by Leo Chan.

Thoughtspot Writer

We're likewise delighted to announce a new writer to Thoughtspot that is now available for public use. 
Thoughtspot is a "search and AI-driven analytics platform".

DynamoDB Extractor

We also released a beta version of the DynamoDB extractor. It does not have any UI yet, and has to be configured via JSON. If you are feeling adventurous, please give it a try and let us know how it goes.

Marketing Miner Extractor

Lastly, but in no way least, we have a new extractor for Marketing Miner that allows you to fetch your project rank tracking data from Marketing Miner. 

New Features

  • The project API Tokens section now shows when a token was refreshed: 

Minor Improvements

  • We've modified the storage job polling to reduce component job run times.  The greatest speedups will be observable in small to medium sized data loads.  
  • Artificial limits were removed from CSV file import. Previously the upload had to go through in 10 minutes. Now it's left to the decission of your web browser. Please note that it still holds that large files should be uploaded through the API.

  • Further improvements to Output mapping. The destination bucket is now prefilled from the transformation name.

Fixes

  • The MSSQL extractor was updated to correctly handle databases with case-sensitive collations.

  • The Email Attachments extractor now supports incremental and addresses in angle brackets, ex: `Joe <email@example.com>`

  • Developer portal vendors can now approve requests to join via the request email.



New S3 Extractor

This one took us a while, but we believe it's worth it. We carefully gathered feedback and made the most commonly used features accessible through a new streamlined UI. And there's even more under the hood.

The original AWS S3 extractor was renamed to Simple AWS S3. It stays fully supported and is not being deprecated. There's no need to migrate your configurations.

There are several major differences between the original and the new extractor. The new AWS S3 extractor

  • can download multiple files/tables using a single set of credentials.
  • fully supports incremental loads.
  • is more flexible.

The UI of the new extractor supports many features, but the extractor is not limited by its UI: it is the first component that openly supports processors. Opening the JSON editor (aka Power User Mode) opens up the configuration to endless possibilities. The extractor itself does only a simple job – downloads a set of files from S3. All other jobs (decompression, CSV fixing, setting the manifest file, etc.) are delegated to processors. You can order and configure the processors so that they handle the files as required. You can even develop your own processor in case you're missing something. We're fully aware that this is not an easy concept to grasp, but it's intended for advanced users. Not advanced? Use the UI.

The list of available processors will be kept and updated in the Developer Portal list of components. A full description of the extractor is available in our documentation.

One step closer to replacing legacy Restbox. The HTTP extractor will follow shortly. 

Week in Review -- March 5, 2018

Improvements

  • Schema in Snowflake Extractor is no longer a required connection parameter. If not set, the table selector allows you select tables from the whole database.
  • Snowflake extractor now supports importing these semi-structured data types: `VARIANT`, `OBJECT`, and `ARRAY` 

  • Updated two factor authentication in Keboola Developer Portal. The SMS authentication is now deprecated. All new users will have to use either Google Authenticator or Duo Mobile app.
  • MySQL extractor now has an option to enable compression of data sent over network
  • Enhanced Output Mapping selector
  • R in Sandbox and Transformations is updated to 3.4.3, also the Tidyverse package is now installed by default.

Bug fixes

  • Data Takeout was randomly failing on backing up your data to S3.
  • Task editor in Orchestrator produced errors when orchestration had configured dozens of tasks.
  • In Twitter extractor template, if a user made a mention of your account, the details of that user account weren't downloaded. Edit and save existing configuration to remedy this issue.

New Email Attachments Extractor

There’s a new version of Email attachments extractor (previously known as Pigeon extractor) you can use from the Keboola Connection’s Extractors tab. It serves for importing csv files to the Storage by sending them as attachments to a generated email address.

Email address for sending csv attachments is generated automatically and the new extractor has a fresh and simpler UI.

The old version is deprecated and will be discontinued on April 6. Please migrate to the new version in upcoming weeks. There is no automatic migration script because you need to generate new email addresses but the switch should be very easy.