13th of April, 2023

Service

New Features

Azure Batch improvements

We have added several improvements to the Azure Batch task.

  • You can now use auto scale formulas with Azure Batch Pool by providing a autoscaling_formula parameter on the Azure Batch task. In addition, you can optionally provide a Auto Scale evaluation value by providing the autoscaling_evaluation task parameter.
  • Task parameters are now present on Azure Batch tasks and are added to the top of your Azure Batch script at runtime as environment variables.
  • Azure Batch tasks will now run as pool admin by default.
  • You can now specify a custom image on an Azure Batch Pool Cluster Definition rather than by the task parameter on an Azure Batch task, this will allow you to configure this once per Cluster Definition rather than on each task.

Improvements

  • Alphabetized the top right menu.
  • Improved the error message returned when you run a Monitor Rule task for a rule that no longer exists in Monitor.
  • You can now resize the message column on logging.
  • Users can now download log files in JSON format from the UI while viewing logs.
  • Improved performance of UI and API.
  • Improved security of OAuth login process.
  • Improved performance of logging service.
  • Improved performance of notifications service.
  • Added incremental support to MYOB online instances.
  • Job emails are now part of the notifications system. They will now be sent via email and as in-app notifications, depending on notification settings configured on a job.
    • You can opt out of receiving these via in-app, email, or both from the notification settings button on the notifications slide out.
  • Monitor tasks in Integrate will now store the Project Id and Rule Id rather than the Project Name and Rule Name. This will mean that you can rename projects and rules without breaking Monitor tasks in Integrate.
    • You will need to update to the latest agent for this to work.
    • Old agents will continue to use Project Name and Rule Name successfully.
    • When you update your agent, you need to do one of the following.
      • Edit each task and save it.
      • Let Loome know so they can migrate the data for that tenant in the background (if you have lots of tasks).

Fixes

  • Fixed an issue where the logs page would be intermittently stuck on ‘loading’.
  • Fixed an issue where some users could not switch tenants.
  • Fixed an issue where you could submit a SQL spark task without the required connection.
  • Fixed an issue where some users could not reach the file definitions page.
  • Fixed an issue where a job would be stuck in progress after an agent crash.
  • Fixed an issue where the docker link for agent was pointing to an old instance of the agent.
  • Fixed an issue where log messages appeared twice if the Loome Agent was offline for part of a job execution.

Agent

This update requires agent version - 2023.04.12.1

Please download a new install script and update your agent to use new features, improvements, and fixes.

New Features

  • We have added many new features to Azure Batch tasks. To use these improvements you will need to update your agent. Read more above.

Improvements

  • Improved the error message returned when you run a Monitor Rule task for a rule that no longer exists in Monitor.
  • Updated the XERO connector to support the latest API additions.
  • Added incremental support to MYOB online instances.
  • Added support for the RECORD data type in Google BigQuery, this will support exporting Google Analytics 4 data via Google BigQuery into SQL Server.
  • Monitor tasks in Integrate will now store the Project Id and Rule Id rather than the Project Name and Rule Name. This will mean that you can rename projects and rules without breaking Monitor tasks in Integrate.
    • You will need to update to the latest agent for this to work.
    • Old agents will continue to use Project Name and Rule Name successfully.
    • When you update your agent, you need to do one of the following.
      • Edit each task and save it.
      • Let Loome know so they can migrate the data for that tenant in the background (if you have lots of tasks).
  • Updated how much information is logged to log files on the Loome Agent when it cannot contact the logging server to reduce the disk space used.
  • Added the ability to specify an alternative log file path and/or specify debug level logging by adding the following to the appsettings.json of the agent. These settings will be added to the agent installation wizard in a future release.
    • You can combine the path setting with an attached file share to an Azure Container Instance to send log files to an Azure file share for easy retrieval see documentation.
    //Both Path and Level are optional settings, you need only specify the value you want
    logging: {
        path: 'c:\\logs',
        level: 'Information' //Where level could be Error, Warning, Information, Debug
    }

Fixes

  • Fixed an issue where the Loome Agent might crash while trying to log while offline.
  • Fixed an issue where a job would be stuck in progress after an agent crash.
  • Fixed a gateway timeout issue on the MYOB connector for online instances.
  • Fixed an issue where users could not import data from an on-premises MYOB instance.
  • Fixed an issue where log messages appeared twice if the Loome Agent was offline for part of a job execution.