We have been experiencing temporary technical difficulties today since this morning (~ 3:00 AM CET)
Some jobs may have failed as a result.
We're working on resolving the problem and will update here with further information when we have it.
Thank you for your understanding.
Update 14:00 CET, October 22, 2017: All systems have returned to normal operations and all jobs are now being processed normally.
In November 2014, we were really happy to announce a new cool feature -- Read-only bucket credentials to your Redshift backend. Time has passed and we have decided to deprecate this feature.
Though accessing data in a bucket is very useful, with the current scenario it can lead to an assumption how data is stored and how it'll be stored in future. The latter is much worse, because any change in our backend can lead to breakage of the system, which was programmed to access the data in a bucket.
Thus, we recommend you to access your data in a more robust way:
- Create a new Redshift Writer
- Choose a database provided by Keboola or your own Redshift database
- Configure which tables you need to synchronize
- Run the newly configured writer
- Copy the credentials created in step 2 to the software which needs to access your data
The existing Bucket credentials continue to work until April 18, 2018. However, we strongly recommend to migrate to the Redshift Writer as soon as possible.
The creation of new credentials and credentials listing will be disabled in the Storage section with the publication of this post.
Also, the Bucket credentials have been marked as deprecated in our Storage API.
The longstanding issue with slow SHOW DATABASES query executions on Snowflake has finally been resolved. We will now gradually enforce snowflake query execution limits where applicable by contract. Before turning the limits back on, we will check the query history and notify you if you may be affected.
(Query times in milliseconds)
New Features
R in Transformations and Custom Science is updated to 3.4.2. Also we internally switched to rocker distribution so R guts are more standard.
CSP (Content Security Policy) is now used in entire Keboola Connection.
Minor Improvements
We continue with cleaning up the KBC UI. So far, we have fixed many little annoying issues:
- List of configurations now shows when the configuration was created (with exact date on hover)
- Cleaned up Job list (removed unused stuff).
- Backend in Transformations is now preselected to default project backend.
- CSV importer now supports .TSV and .GZ files too.
- "Test Credentials" button and result polished.
- Removed 'sticky' buttons when editing configurations.
Fixes
- Fixed files input mapping not showing in certain configurations.
- Fixed a couple of errors with the Save button when editing transformation queries.
- Fixed bug in Redshift writer with mixed case table names.
Developers
- The encryption API is simplified to a single endpoint. If you manually encrypt values for KBC, please follow the docs. We will be contacting you if you are using the deprecated API calls.
Today, October 4, 2017, we experienced several jobs failures between 12:07--14:40 PM CEST. One of our worker servers became unresponsive which led to the jobs failures. We have shut down the instance and launched a new one. Everything is working properly now and we will further investigate the root cause of the issue.
UPDATE 09:00 PM CEST
We're still encountering rare job failures. We're monitoring the issue closely and trying to find the root cause.
We are sorry for any inconvenience.
New Features and Improvements
- Geneea NLP app now supports relations - e.g. My pizza was completely burned. > burn(OBJECT:pizza)
- Redshift transformations input mapping now supports ZSTD compression.
- Improved handling of events search errors
New Applications
GeoIP
GeoIP is latest transformation component power by MaxMind. It helps you to obtain location data from your IP address inputs.
Please check out the BitBucket Documentation to get you started.
Data Health Application
Data Health Application is an app designed to aid users to produce a clean data file. For more details, please check out Keboola Blog.
Today, September 25, 2017, we experienced jobs failures between 15:11 - 16:15 PM CET. One of our worker servers went down and while launching a new one automatic job restart mechanism didn't seem to work properly and this led to the jobs failures. Everything is working properly now and we will further investigate the cause of the issue. However we recommend to review your orchestrations and take appropriate action if needed.
We are sorry for any inconvenience.
On Tuesday September 19th around 7am CEST a new version of Restbox was deployed. This release contained a bug affecting all wildcard downloads from S3. The older version was restored at 5:20pm CEST.
We're sorry for this inconvenience.
A few Orchestrations which were scheduled to run at exactly midnight UTC may not have started due to an issue with the Orchestration Scheduler. The issue arose on the 15th of September and has now been resolved.
If you have orchestrations which are scheduled to run at 0:00 UTC, please check that these orchestrations have run correctly.
Sincere apologies for the inconvenience.
All Snowflake transformations now have SQLdep enabled.
SQLdep allows you to visualize the data flow in your SQL. You can now easily see the impact of your changes or track the lineage of each column.
SQLdep Snowflake support is still in beta, so do not hesitate to contact support@keboola.com with any questions or bugs.
We are investigating Snowflake partial outage. Queries are now queued.
We will inform about the progress.
UPDATE 11:21 AM PDT Snowflake warehouses are up and running. Queued queries are being processed. We will update the post when we'll have more information about incident cause from Snowflake engineering.