We changed the naming conventions for MySQL provisioning. Instead of tapi_3800_sand and tapi_3800_tran, where 3800 is my token ID, the new database names are sand_232_3800 and tran_3800, where 232 is project ID for easier distinction between projects in your sandbox userspace. Existing credentials keep their database names.
We also added a little more information to events:
For all columns without a defined datatype in a Redshift transformation we added a default LZO compression. This will lead to a marginally slower transformation times but significantly less memory and disk space usage.
You can override this behavior by defining a datatype without a compression.
Wait... what is R?
We've developed the backend and UI for transformations in R. All R transformations run in our public Docker image and have 1 GB memory allowance (will increase in near future).
There's a guide how to develop and test your R scripts locally and a list of best practices and limitations.
Start typing you R scripts now!
AWS recently issued some information about connectivity issue in US-EAST-1 Region, where majority of our infrastructure is located. This may result in 500, 503 and 504 application errors within the infrastructure (our components) or when reaching out to other APIs (extractors).
We're sorry for any inconvenience. we'll keep this post updated with current status. You can also check the current status at http://status.aws.amazon.com/, row Amazon Elastic Compute Cloud (N. Virginia).
---
9:23 AM PST We are investigating possible Internet connectivity issues in the US-EAST-1 Region.
10:09 AM PST We are continuing to investigate Internet connectivity issues in the US-EAST-1 Region.
From now on you can use all OUT tables in all transformations. Previous restrictions on using OUT tables in Redshift transformations were removed.
We had some issues in one of our components (job handling queue) between 10pm and 10:15pm PST on Sunday night (03:00–03:15 UTC Monday). That had resulted in some failed orchestrations (scheduled at that time or starting it's tasks at that time) . We're planning to upgrade the component in the coming weeks, but if the error occurs again, we'll upgrade it immediately. The upgrade will be accompanied with a short maintenance downtime.
We're sorry for any inconvenience.
After the maintenance window we've started experiencing some technical difficulties with our Elasticsearch cluster resulting in orchestration processing glitches. We're working on it and monitoring all processes closely. Please let us know at support@keboola.com if you encounter any unexpected behavior.
We have slightly changed the layout of the transformation detail (sections moved to tabs) and added an Overview graph.
The graph shows the data flow around the current transformation, including all required transformations, input/output tables and transformations from the same bucket, that work with any of the tables as the current transformation. This allows you to get a bird's eye view on what you're working with.
The difference between this and Visual SQL (or SQLdep) is that Visual SQL goes deeper in the transformation and analyses the queries, whereas Overview shows you the environment around the transformation.
There's a direct link to GoodData LDM Visualizer in the model page in GoodData Writer. So in a single click you can compare what is defined in Keboola Connection and what has been already uploaded to GoodData and how the model is interpreted.
We're now showing bucket and transformation descriptions in the UI. Currently there's no way to change the bucket description - it is defined setup when the bucket is created, but we're working on a way to make them editable. Transformation descriptions can be changed easily in the transformation detail
