14:00 UTC: All Azure stacks are currently experiencing issues starting Python Workspaces, Data Apps and Jobs. The root cause appears to be a failure to pull the required Docker images.
On startup you may see a long start which ends with an internal error. We apologize for the situation.
UPDATE 14:52 UTC: We’ve applied a fix and all systems are now operating normally. We apologize for the disruption.
We would like to inform you that our Buffer API is being deprecated as part of our commitment to providing you with the best tools and services. Effective 1.3.2025, the Buffer API will no longer be available.
To ensure a seamless transition, we are introducing Data Streams, full replacement and powerful and improved service designed to enhance your data integration experience. You can learn more about Data Streams and its benefits HERE.
We’ve created a migration tool to help you move current Buffer API endpoints to Data Streams quickly and easily. Our support team is also here to provide guidance and support throughout the process.
You can read more about Data Streams and how to set it up HERE.
Our support team is here to help you every step of the way. If you have any questions or concerns about the migration, feel free to reach out to us.
Thank you for your understanding and for being a valued part of the Keboola community. We’re excited for you to experience the benefits of Data Streams!
We noticed that from about 12:30 CET they stopped working Data Apps on GCP stacks:
We apologize for the inconvenience; we are currently investigating the issue and will provide an update in 30 minutes.
UPDATE 2024-12-11 13:01 UTC: We have identified the root cause, which only affected configurations with authorization enabled or a secret value set. This caused failures during the decryption process. We have fixed the permissions required for decryption in the runtime job, and all systems are now operating normally. We sincerely apologize for the inconvenience.
2024-10-14 16:30 UTC We are observing a small number of instances where errors occur during job creation, and you may encounter the error message: “Decryption failed: Deciphering failed.” As a result, orchestrations may become stuck in the terminate state. If you experience this issue, please contact our support team.
We are actively investigating the situation and will provide an update later this evening.
2024-10-14 21:40 UTC We have successfully identified the affected orchestrations and deployed a fix that automatically terminates them. We now consider this incident resolved. We sincerely apologize once again for the inconvenience caused.
We are experiencing problems on our AWS EU stack (https://connection.eu-central-1.keboola.com/). We are deeply sorry for the inconvenience this may cause. In the user interface, you can issue an error alert or task slowdown processing jobs. Next update 30 minut.
Sep 26 08:34 UTC: We identified and fixed an overload on one of our Kubernetes node. All systems are now running normally. We’ve implemented measures to prevent recurrence.
Thank you for your patience.
2024-08-06 We have experienced some jobs stuck to continue and failed between 02:22 - 04:45 UTC. The root cause was invalid job node, we had to shut it down to prevent further issues.
2024-04-25 02:23 UTC - We experienced another incident on connection.eu-central-1.keboola.com between 01:30 UTC and 02:30 UTC. Some jobs ended in error due to an underlying node failure. We're still investigating the root cause and taking measures to prevent future incidents.
We noticed a slowdown in our AWS EU stack’s job queue from 12:00 to 14:00 UTC, due to a temporary service performance issue. We sincerely apologize for any inconvenience this may have caused.
Our team has resolved the problem and we are taking steps to prevent future occurrences. Thank you for your understanding and patience.
7:20 UTC: We are investigating an error when installing a python package in the transformations.
You may see errors such as: Job "XXXXXXX" ended with a user error "Failed to install package: io".
Next update in 15 min.
UPDATE 7:38 UTC: We have rolled back a previous version and all operations are back normal. We're sorry for this inconvenience.
Today at 15:30 UTC we noticed a problem with the listing tables in Storage. Tables fail to display only to new users in the project. There can be related issues such as not being able to load data to workspace. All of which apply only to new users. We are seeing this problem across all stacks and regions.
We are working on a fix, next update in 30 min.
UPDATE 16:10 We have identified the root cause and are working on a fix. Next update in 2 hours.
UPDATE 19:00 The incident is now resolved, and tables are displayed correctly in storage for all users.
We apologize for the inconvenience.