News In Transformation UI

Here's a list of things that have changed in last couple weeks, in case you didn't notice in the UI directly.

No Custom Credentials

Support for custom transformation or sandbox credentials was dropped in the UI. Everything in the UI is provisioned. You can still use your own credentials using the API.

Input Mapping Within Redshift

When adding an input mapping in a Redshift transformation from a Redshift (Storage) backend, the engine chooses the fastest data transfer route. As the transformations and storage share the same cluster, the table never leaves the database and is transferred directly within the cluster. You have an additional option to set, if the input mapping is performed as a CREATE VIEW or CREATE TABLE from the original table. View is faster (there is no data transfer at all), but might cause memory issues, if you start piling up views on top of each other, table is slower (the data is duplicated), but may perform better on larger set of queries.

Also COPY Options pane disappeared from input mappings within Redshift, they didn't make any sense there.

Note: Be careful with mismatching datatypes.

Sandbox Credentials

Sandbox Credentials page now shows both MySQL and Redshift credentials with all their details. 

Running Processes

You can now terminate both MySQL and Redshift processes. 

Note: Terminating a database process is not immediate, it might take some time - there are some rollbacks on the backend.

Note: Running a Redshift transformation uses both MySQL and Redshift credentials. Terminating any of them will end the transformation.