Degraded Performance on GCP Stacks

We are currently experiencing degraded performance on our GCP stacks due to ongoing global GCP infrastructure issues

The following stacks are affected:

As a result, you may observe random job failures or delays. Jobs may fail with intermittent errors unrelated to the actual job configuration or data.

Our team is actively monitoring the situation and working to restore full functionality. We appreciate your patience and will provide updates as more information becomes available.

[Update – 2025-06-12 19:37 UTC] GCP is in the process of restoring its services. We are observing that our GCP stacks are beginning to operate as expected. API latency has returned to normal levels, and the job queue is starting to clear.

We will continue to monitor the situation closely.

[Update – 2025-06-12 20:25 UTC] The incident has been resolved. All GCP stacks are stable and the platform is fully operational. Thank you for your patience.

Python Workspace Outage on AWS EU

We are currently experiencing an outage of the Python Workspace on the stack https://connection.eu-central-1.keboola.com. Our team is actively working on resolving the issue. Please monitor this status page for further updates.

Update 2025-06-10 14:45 UTC: Python Workspaces are fully operational after a brief overload-related outage. Apologies for the inconvenience.

Update 2025-06-10 14:55 UTC: Due to the earlier Python Workspace outage, job processing was delayed by approximately 15 minutes. The issue has been stabilized.

Update 2025-06-10 14:59 UTC: The root cause of the incident was a failover on the MySQL RDS database. We apologize for the disruption and appreciate your patience.

Google Login to Keboola Not Working

We are currently experiencing an issue across all stacks where login to Keboola Connection is not possible. Attempting to access the application results in a 404 Not Found error.

Workaround: After login, if redirected to a 404 URL ending with `/admin`, manually remove the `/admin` part and reload the page.

Our engineering team is actively investigating the root cause. We will share an update as soon as we know more or within the next 30 minutes.

Update 2025-06-10 07:17 UTC: The issue affects only login via Google. Other login methods remain functional. We are continuing to investigate the root cause. Next update will follow within 30 minutes.

Update 2025-06-10 07:42 UTC: The issue has been resolved. Login via Google is now working as expected.

Thank you for your patience.

Azure Stacks: Workspace, Data Apps and Jobs Startup Failures

14:00 UTC: All Azure stacks are currently experiencing issues starting Python Workspaces, Data Apps and Jobs. The root cause appears to be a failure to pull the required Docker images.

On startup you may see a long start which ends with an internal error. We apologize for the situation.

UPDATE 14:52 UTCWe’ve applied a fix and all systems are now operating normally. We apologize for the disruption.

Buffer API Deprecation & Migration to Data Streams

We would like to inform you that our Buffer API is being deprecated as part of our commitment to providing you with the best tools and services. Effective 1.3.2025, the Buffer API will no longer be available.

To ensure a seamless transition, we are introducing Data Streams, full replacement and powerful and improved service designed to enhance your data integration experience. You can learn more about Data Streams and its benefits HERE.

Why the change?

  • Enhanced Performance: Faster and more reliable data streaming
  • Improved Functionality: Supports more use cases with advanced features.
  • Streamlined Experience: Fresh UI with more intuitive and flexible integration process.

How to migrate:

We’ve created a migration tool to help you move current Buffer API endpoints to Data Streams quickly and easily. Our support team is also here to provide guidance and support throughout the process.

What you need to do:

  1. Manually migrate existing Buffer API endpoint to Data Streams - Create new Data Streams, copy the stream URL and replace the current Buffer API endpoint within your integration with it
  2. Contact our support team - we will run a script to migrate old Buffer API endpoints to new Data Streams so you can just copy paste the Data Stream URL and replace the current Buffer API endpoint within your integration with it

You can read more about Data Streams and how to set it up HERE.

Need help?

Our support team is here to help you every step of the way. If you have any questions or concerns about the migration, feel free to reach out to us.

Thank you for your understanding and for being a valued part of the Keboola community. We’re excited for you to experience the benefits of Data Streams!

Keboola Team

Broken Data Apps on GCP stacks [EU, US]

We noticed that from about 12:30 CET they stopped working Data Apps on GCP stacks:

We apologize for the inconvenience; we are currently investigating the issue and will provide an update in 30 minutes.

UPDATE 2024-12-11 13:01 UTC: We have identified the root cause, which only affected configurations with authorization enabled or a secret value set. This caused failures during the decryption process. We have fixed the permissions required for decryption in the runtime job, and all systems are now operating normally. We sincerely apologize for the inconvenience.

Orchestrator stuck on application error during job creation on north-europe.azure.keboola.com

2024-10-14 16:30 UTC  We are observing a small number of instances where errors occur during job creation, and you may encounter the error message: “Decryption failed: Deciphering failed.” As a result, orchestrations may become stuck in the terminate state. If you experience this issue, please contact our support team.

We are actively investigating the situation and will provide an update later this evening.

2024-10-14 21:40 UTC We have successfully identified the affected orchestrations and deployed a fix that automatically terminates them. We now consider this incident resolved. We sincerely apologize once again for the inconvenience caused.

Errors in the AWS EU Stack

We are experiencing problems on our AWS EU stack (https://connection.eu-central-1.keboola.com/). We are deeply sorry for the inconvenience this may cause. In the user interface, you can issue an error alert or task slowdown processing jobs. Next update 30 minut.

Sep 26 08:34 UTC: We identified and fixed an overload on one of our Kubernetes node. All systems are now running normally. We’ve implemented measures to prevent recurrence.

Thank you for your patience.