We show a PARTIAL label for jobs which didn't run the whole configuration, but only part of it. Typically these are jobs when only one transformation is run from a bucket or when one file is exported using AWS S3 extractor, HTTP extractor, etc.
Transformation description, Last Runs, and Updates
- There's a new option to save transformation with description.
- The Last Runs section and Updates section have been added also to the Transformation Detail page and all components which support "single runs" (e.g. when HTTP extractor or AWS S3 extractor is extracting one file)
Input Mapping data types
In transformations with Snowflake backend, data types should be populated automatically for tables created with the components which set data types - MySQL extractor, Oracle extractor, MSSQL Server extractor, PostgreSQL extractor, DB2 extractor, and Snowflake extractor.
Input Mapping load type
For Snowflake backend, you can set Clone Table load type which leads to an ultra-fast load to a workspace. Most of your tables should be loaded under 10 seconds.
You can find more about this feature in our documentation.
- You can see a component type in Orchestration Tasks, thus having both MySQL writer and MySQL extractor in task won't be confusing anymore
- API Token can be Refreshed, Sent or Deleted directly from the token detail page
- We improved how data is loaded to Storage from our components - compression is used by default. Loads to Storage are about 40% faster for sliced files. So it impacts Credit Consumption too.
- MSSQL Server Extractor has new option WITH(NOLOCK) and supports incremental fetching with smalldatetime column
- Orchestration Notifications page has been improved, and notifications can be set easier
New components and component updates
- In RStudio/Jupyter sandboxes there's a hard memory limit set to 8GB. This limit is equal to memory limit in Transformations. With this change, you can be sure that what worked in Sandbox will work in Transformation too.
- Rename Files processor to rename files using regular expressions: https://components.keboola.com/components/jakub-bartel.processor-rename-files
- XLS processor to convert .xls(x) files to .csv files: https://components.keboola.com/components/jakub-bartel.processor-xls2csv
- Snowflake writer uses atomic swap of tables during full load (prevents empty table issue)