Failed jobs

There were  jobs failures between 1:30 AM - 3:30 AM UTC caused by flooded disk on one of our servers. We are investigating the causes of the issue.

We're sorry for this inconvenience.

Job failures

There were  jobs failures between 1:30 AM - 6:15 AM UTC caused by flooded disk on one of our servers. We are making steps to avoid this issue happening again.

We're sorry for this inconvenience.

Job failures

There were jobs failures between 10:30 AM - 12:50 PM caused by low disk space on one of the jobs workers.

We're sorry for this inconvenience.

Snowflake backend project errors

We are investigating connection errors in projects with Snowflake backend. It is related to today's snowflake maintenance, we are in contact with their support.

We will update this post when we have more information. Sorry for any inconvenience.


UPDATE 09:30 PM PDT Storage is fixed and failed orchestrations were restarted. We are working on Snowflake transformations fix.

UPDATE 10:15 PM PDT Snowflake transformations are also working. Snowflake did the rollback of release which caused problems.

Waiting jobs

We are investigating a problem with waiting orchestrations jobs.

We will update this post when we have more information.


UPDATE 12:02 AM PDT We have found and fixed the issue, waiting orchestrations are now starting.  

UPDATE 12:26 AM PDT All waiting jobs were processed. Everything should now be working normally. If you encounter some problem, please let us know.

Sorry for any inconvenience.

Orchestration manual run issue

UPDATE: this issue has been solved and one shall not encounter the problem described below.

We have encountered problem when running orchestration manually from the orchestration detail page: 

error message detail: "Error. [Task 0] Job task is different from orchestration task"

We are working on the fix of the problem which should be released within a short while.

Temporal workaround: if you are experiencing such a problem, you can fix it by resaving the orchestration tasks(Edit Tasks->Save).

All other orchestration features are working normally. We are sorry for any inconvenience.