Getting Started
What’s new in Loome Integrate Online?
Loome Integrate Online incorporates numerous improvements which provide an overall enhanced user experience.
Notable improvements include:
- Implementation of the Agent, to replace the previous back end process, and simplifying installation.
- Creation of Tenants, allowing for multiple instances of Loome Integrate if necessary.
- Easy login via your browser, and zero installation after the Agent script has been run.
- Updated UI: clearer, simpler and easier to navigate.
Perhaps the biggest change is that Loome Integrate Online is Loomes first Software as a Service (SaaS) product. Thanks to the creation of the Loome Integrate Agent, Online can run as a cloud-based SaaS whilst still able to access data sources that are stored on-premises.
Creating a Tenant is your first step of Loome Integrate Online setup, followed by the Agent installation.
The components of Loome Integrate Online
Projects, Jobs and Tasks
Loome Integrate consists of Projects, which contains many Jobs. These Jobs consist of a series of Tasks, which could be any number of the below:
- Execution of a Data Migration
- Execution of a Stored Procedure
- Run a SQL Statement
- Run an OS Command Script
- Run a Python Script
- Run a PowerShell Core script
Projects might belong to separate departments within your business - for example, Accounting, Marketing, Development etc.
Jobs can be named after the things you want to achieve within each Project, and the tasks within Jobs are the technical steps to achieve this goal. Tasks can belong to multiple Jobs, and each can be configured to be dependant on another if necessary. For example, If Task A fails from within Job A, then Job B will not run. You cannot however, run a single Task on its own, all Tasks are run within their parent Job. Jobs can be scheduled to occur in an automated fashion.
The Agent
Think of the agent as the “worker” in Loome Integrate Online – it contains all the code which allows jobs to run. This is to replace the back end process of previous version; we now allow easier data warehousing on multiple servers and hosts, due to the Agent’s ability to be installed on one tenant but across multiple locations.
The Agent runs cross platform, meaning that users can leverage data from sources running on both Windows and Linux. Loome Integrate is now able to provide Big Data connectivity such as Apache Hadoop as a connection type - a feature only possible thanks to the Agent’s Linux Support.
The Agent should be installed on a host which has access to your data sources, such as a server. Think of it as just another program which runs in the background - listening in real time, for job and task executions from Loome Integrate Online in the cloud.
What are the agent benefits?
Easy to Install & Update
- Installing an agent takes only a few minutes and all configuration is handled by the pre-made installation scripts generated by Loome Integrate.
- Updating an agent is just a matter of running the same script you used to install the agent.
Security
- Data never passes through Loome Integrate’s servers, only the agent which is installed on your servers.
- Assign the agent an authenticated user account for password-less authentication with SQL Server data sources.
- All passwords are stored using tenant specific encryption keys and secure, trusted encryption algorithms.
- Future releases will include the ability to source connection data from local sources rather than the Loome Integrate API.
Cross Platform
- The agent supports Windows and Linux platforms for installation whilst also supporting platform specific features such as Windows Powershell commands and Hadoop integration.
- Agents can also run as a Docker container with the image publicly hosted on Docker Hub.
Connections and File Definitions
Connections in Loome Integrate are your data sources - where you can pull data from - for migration tasks. These can be on-premises, such as a SQL server, or any number of cloud-based connections.
Loome Integrate also allows for the migration of data to and from file-based data stores. For example, Loome Integrate treats flat files like any other connection - each file in a directory is essentially a table in a database. Loome Integrate also supports both local files and Azure blob. Loome Integrate is also able to leverage the power of its flat file processor to support Azure Blob Storage Containers as a source and target connection.
Your First Data Migration
Loome Integrate has the ability to replace the traditionally arduous task of staging raw data from the operational system with a quick, code-free process requiring minimal technical expertise. We call it a Data Migration Task.
Loome Integrate continues to minimise the amount of work required to consolidate data from multiple operational or source systems into one central place for querying and reporting.
First we need to create a new Task. To do this click ‘Add Task’ from within your selected Job.
A screen will appear that will ask you to fill in your Task Name, select an Agent and choose your Task Type – in this case, Data Migration.
Tip : Having a ‘healthy’ agent means that Loome Integrate is ready to run tasks without any start up issues. To check an agent’s health at any time, you can view it on the Agent page or you can check for agent notifications in the Notification hub at the top-right of your screen.
Next we need to choose where the source data will be imported from and the target location we will copy into. To do this, select your Source and Target connections, and your Source and Target schemas. Choosing your Source Connection and Schema will define which tables you are able to select for migration in the next step.
This next step displays the tables within the schema that you chose on the last screen. Use the plus button on the right of the screen to select the tables you wish to migrate from the Source Connection.
Tip: If you change your mind, removing a table is as easy as clicking the minus button on the right side of the ‘Target’ section. Once you are happy with the tables you’ve selected, click ‘Submit’ to save the task. You can edit this saved task at any time using the edit button on the Job screen.
Your task is now created and the next step is to run this job. Your Data Migration task will now appear in your ‘Sequenced Tasks’ list, and you will be able to run the task as part of the Job at anytime with the ‘Run Now’ button - located in the top right. Be aware that this will run all Sequenced Tasks as part of the job. Checking the ‘Show Logs after Execution’ box will now take you to a live job execution page.
Loome Integrate will show the logs and status of your job in real time - updating the status icon to a green tick on completion, or a red flame on failure.
