17th of May, 2022

Service

New Features

Reconciliation

You can now create Reconciliation rules in Loome Monitor. Reconciliation rules mean you can compare two datasets in one rule. You can then view these two datasets in the results table and see highlighted mismatches and orphan records from each dataset.

Results View

We have updated the results table in Loome Monitor so that it is more intuitive and retrieves records faster than before. It is now in a new grid, where slicers are now available beside each column heading, and includes conditional slicers, sorting, and paging. You can also hide columns, pin columns to the left or right of the table, and auto size the results table. You can load 20,000 records at a time, and can easily view a dataset of 100,000 records.

Metrics

Each results table in Loome Monitor will now display the metrics of records in that rule at the top of the page. It will show how many records are under each status. It will also be available as a table in your target connection.

Export to Excel

You can now export the results table of a rule to Excel in Loome Monitor. You can find an ‘Export to Excel’ button at the top right of the results page. The exported file will contain the records from the default statuses that appear on your results page in a similar format. Custom fields, that are editable, are in yellow and source data are read-only in grey.

String Custom Field

You can now add a String custom field to rules in Loome Monitor. Similar to a notes custom field, you can provide characters to this field in the results table but the string field is for shorter fields. A string custom field has a limit of 255 characters.

Dataverse Source Connector

We have added a Dataverse source connector as a preview feature in Loome Integrate, please let us know if you would like to try it out.

Task Level Overrides for Tenant Settings

You can now override any tenant level setting at the task level, such as bulk copy timeout etc, in Loome Integrate. Each setting also includes a description so you can easily understand what each setting will change. When creating or editing a task, rather than clicking ‘Submit’, you can click on ‘Next’ to find these individual task settings that will override the global settings set for the tenant.

Azure Function

You can now execute Azure functions from within Loome Integrate without having to write PowerShell. You can run an Azure function app by providing the URL of the function app into the Azure Function task type, as well as by choosing your authentication type.

Improvements

  • Monitor: You can now choose multiple statuses to be displayed by default in results view.
  • Monitor: Loome will display an error message if you have selected a key column that isn’t unique that will identify the values causing the issue.
  • Monitor: The ignored status will now load before the resolved status by default to avoid the incorrect application of statuses.
  • Monitor: You will now find that connections are shown in alphabetical order in the dropdown when creating a rule.
  • Monitor: We have added a complete list of countries to the Country field on the tenant registration page.
  • Monitor: We added a link to our documentation for more information about Regex in Data Quality rules.
  • Monitor: We have changed the agent bar and it is now a scrollable single line if you have multiple agents.
  • Monitor: Users in application roles will now be sorted alphabetically.
  • Monitor: You will now be provided the option to also remove the output tables and views when you delete a rule. When confirming that you would like to delete a rule, you can choose to either only delete the rule and keep the output tables and views, or you can delete both the rule and output tables.
  • Monitor: As slicers are now available beside each column heading, we have removed the slicer slide-out.
  • Monitor: We have added a notification to inform users that a manual field cannot be deleted if it is used as a key column.
  • Integrate: The tenant level settings page has been improved significantly to provide descriptions of what each setting does and what it applies to, and we have added data type specific controls rather than plain text boxes. These descriptions will explain what the setting will do to provide a more friendly user experience.
  • Integrate: We have added all countries to the Tenant registration ‘Country’ field. Previously it only listed AU and US as selectable countries.
  • Integrate: We have reworked the CRM connector to use OAuth instead of a username and password due to deprecation by Microsoft.
  • Integrate: Many agents in a tenant would previously cause the agent bar at the bottom to overlap content, so we have changed the agent bar to scroll across horizontally.
  • Integrate: We have added alphabetical sorting to dropdowns for;
    • File definitions
    • Cluster definitions
    • Tenant users
    • Agents
    • Source and target file definitions when creating a task
  • Integrate: We have added the schema switch preview feature into the tenant, agent, and task level settings. Previously, this would have to be configured via the agent config file on the server.
  • Integrate: We have moved the tenant global settings to its own settings tab under the heading ‘Tenant Settings’.
  • Integrate: We have also added a ‘Tenant Name’ tab where you can change the name of your tenant.
  • Assist: We have improved the PowerShell processor for solution collection action scripts in Assist. This will allow the install and import of modules in these scripts.
  • Assist: Modified Virtual Networks to be able to share address spaces across accounts. Assist will also calculate the next available subnet for a new project in an address space.
  • Assist: Added API endpoints to retrieve network information for workstations and storage accounts.
  • Assist: Added the ability to specify service tags in NSG rules for Virtual Networks.
  • Assist: Storage accounts will now only be available under private VNETs when a private VNET is configured for the parent account.
  • Assist: We have added messages to the Affiliations, Accounts, Agents, Cost Centers, and Workstation Templates pages when they are empty without any user-added content to help users navigate the UI.
  • Assist: The storage account name is now shown as a field on solution collection repositories.
  • Assist: We have also updated solution collection repositories from verified publishers so that they cannot be deleted. You also cannot delete a solution collection repository that has been promoted to a verified publisher. Please contact support@loomesoftware.com if you would like to remove a verified repository.
  • Assist: We improved the speed of public image retrieval when creating a workstation template.
  • Assist: We have added role based welcome messaging to the home page.
  • Assist: We have improved phrasing in various text and error messages.
  • Assist: We have improved messaging when upgrading a solution collection.
  • Assist: We added a home page for new tenants that have no projects.

Fixes

  • Monitor: Opening a link to Loome Monitor when you aren’t logged in will direct to the correct page after login, and no longer redirects to homepage instead.
  • Monitor: We have fixed an issue where the results page was loaded twice so that it now loads once.
  • Integrate: We ensured that opening a link to Loome Integrate when you aren’t logged in will direct you to the correct page after login, instead of being redirected to the homepage.
  • Integrate: We fixed an issue where the previous encoding selection was missing in the UI, while the value was stored in the database.
  • Integrate: We fixed an issue due to the Daylight Savings Time changeover so that schedules continue to work properly.
  • Integrate: We fixed a bug where the Azure Functions task title did not display correctly.
  • Integrate: We fixed an issue where if a job was waiting on a dependency but set to run, regardless of the outcome the job would run without waiting for the dependent job to finish.
  • Integrate: We have fixed an issue where Sql Retry Logic would not be applied to a connection.
  • Integrate: We fixed an issue where geo data types could not be used.
  • Integrate: Editing a file definition previously did not retain the encoding selection.
  • Assist: We fixed a typo in the data repository error notification message.
  • Assist: We have resolved an issue where you could not set a project budget limit when the parent account had an unlimited budget.
  • Assist: The help link to documentation previously went to the wrong module.
  • Assist: Fixed an issue where there was a 120-character limit on SSH keys for VM access.
  • Assist: Fixed an issue where secure string field types in solution collections were not being hidden from the user in the UX.
  • Assist: We fixed an issue where many agents would cause the agent bar at the bottom to overlap content. It is now a scrollable single line if you have multiple agents.
  • Assist: We have fixed an issue in Safari where you could not erase a search value under publishers and search for a new value.
  • Assist: We fixed an issue where the Contributor role could not create a project and received an error when retrieving the list of data classifications.

Agent

This update requires agent version - 2022.5.10.1

Please update your agent to use new features and improvements.

New Features

  • Monitor: We have added reconciliation support. Read more about reconciliation above.
  • Monitor: We have also added support for exporting to Excel. Read more about ‘Export to Excel’ above.
  • Monitor: We have updated to Microsoft.Data.SqlClient and now support Managed Identity, DNS caching and SQL Retry Logic, as well as other improvements. The Encrypt setting in a SQL Server connection string will now default to encrypt=true rather than encrypt=false.
**Monitor:** Any users who are connecting to **unencrypted databases** will need to add `encrypt=false` to their connection string in Loome Monitor.
  • Monitor: You will now be provided the option to also remove the output tables and views when you delete a rule. When confirming that you would like to delete a rule, you can choose to either only delete the rule and keep the output tables and views by leaving the confirmation field blank, or you can provide the name of the rule to delete both the rule and output tables.
  • Monitor: We have updated the results table to a new grid based system. It includes features such as paging, filtering, sorting, and searching, while it can load 20,000 more records into the table faster than it did before. Read more about the new results table in the ‘New Features’ section above.
  • Monitor: We have added support for the String type custom field.
  • Assist: You can now add a confirmation option when running an action for the Solution Collection you have created, so that the user of the solution collection is prompted if they want to complete the action.
  • Assist: We added the ability to provide a status check function for Solution Collections. For example, it can indicate if the solution collection is ‘running’ or other such statuses.

Improvements

  • Monitor: To learn when new records have been added to a dataset, results will now be ordered by record age and then by the key column(s).
  • Monitor: We have improved the performance of persistent staging to significantly increase the speed of record retrieval.
  • Monitor: We have added support to the agent for SQL connections that would use SQL retry logic. You can specify SQL retry logic provider settings via the connection string.
  • Monitor: A duplicate key column check will be performed as the first steps of running a rule and a friendly error will display to let you know of any problems in this column.
  • Integrate: We have improved the performance of persistent staging to significantly increase the speed of job executions.
  • Integrate: The agent now uses full PowerShell Core rather than PS Core Workspace. This means you can now install and import PowerShell modules.
  • Integrate: We have updated our SQL driver. Encrypt=true; is now the default value in connection strings. Previously, it was Encrypt=false;. If you connect to an unencrypted SQL server, you will need to add Encrypt=false; to your connection string or you will receive errors.
**Integrate:** Any users who are connecting to **unencrypted databases** will need to add `encrypt=false` to their connection string.
  • Integrate: Temporary script files will now be written to the temp folder with the prefix loome-.
  • Integrate: We have added support to the agent for SQL connections that would use SQL retry logic. You can specify SQL retry logic provider settings via the connection string.
  • Assist: We have added a ‘Clear Search’ button in the slicer slide-out when searching for solution collections to refresh any filtered results on the page.
  • Assist: We updated the existing loader in Assist so that it is much faster.

Fixes

  • Monitor: We have fixed an issue where an apostrophe in the key column would cause an error when editing the record.
  • Monitor: We fixed the issue where the results page did not display due to columns with special characters or spaces.
  • Monitor: We have fixed the issue causing rules to fail when the key column was the type ‘uniqueidentifier’.
  • Monitor: We have fixed a bug where it didn’t send communications from a manual reference as it did not include columns with special characters or spaces in a token.
  • Monitor: We also fixed the validation issue that is encountered when toggling between email communication types and saving.
  • Monitor: We fixed the issue where values in a date custom field in the results table were called ‘Invalid date’ instead of ‘Null’. As well as the issue where a rule execution failed when a record status branch was a date column that equals null.
  • Monitor: We fixed an issue where a data field that had a long name would cause the rule execution to fail.
  • Monitor: We fixed an issue where some source columns were not available as a column when creating record status conditions.
  • Monitor: We fixed an issue that caused a reference rule execution to fail if the data source was manual only.
  • Monitor: Previously, you couldn’t use a list value as a record status condition unless it had been attributed to a row, you can now use any list value even if it hasn’t been selected for a record.
  • Monitor: There were more rows inserted into the communications output table than the number of emails that were sent out, it will now insert a number of rows that is equal to the emails sent out.
  • Monitor: The chosen default for a list custom field was not selected as default in the results table.
  • Monitor: Loome Monitor would truncate values as it did not distinguish between some column types and casted columns as varchar(50).
  • Integrate: We have fixed an issue where the agent periodically failed with an error due to encryption keys missing, as the agent wasn’t notified of the ‘TaskExecution’ message.
  • Integrate: We fixed an issue where the credentials of Azure Batch connections were not tested.
  • Integrate: We have resolved an issue that caused the ADF task to occasionally fail when the pipeline status was ‘Queued’. The agent previously caused pipeline statuses other than ‘InProgress’ and ‘Succeeded’, such as ‘Queued’, to be ‘Failed’.
  • Integrate: Previously the connection did not validate as the WS-Trust auth was depreciated. We have reworked the CRM connector to use OAuth instead of a username and password.
  • Integrate: Loome Integrate would previously truncate values as it did not distinguish between column types and casted columns as varchar(50).
  • Integrate: We fixed an issue where debug logs were displayed in the log files when standard level logging was applied in PowerShell tasks.
  • Assist: We have fixed a UI layout issue when creating a workstation template in a small window resolution. We have updated the UI so that it will display correctly when the window is resized.
  • Assist: Loome previously displayed an error of ‘Unexpected character encountered while parsing value: T. Path “, line 0, position 0.’ when deploying a solution collection with an ARM template that includes at least one parameter of the boolean type (i.e., “type”: “bool”). It will now parse the parameter correctly with a Yes/No button on the parameter’s entry form.
  • Assist: Fixed an issue where you could not mount a data repository to a Windows VM when the parent project has a space in the name.
  • Assist: Resolved an issue where subnet creation for Virtual Networks would fail where the project name ends in a non alpha-numeric character.