This setup guide goes through how to configure a CI/CD for Loome Intergrate in Github Actions. By the end of this guide, you’ll be able to utilize GitHub Actions’ Repos and Pipelines to move changes from one Tenant to another.
The following are required for this setup guide:
The development tenant is the tenant that has the jobs we will be exporting into Github.
The production tenant is the tenant that we will be importing jobs into from Github Actions. By the end of this guide, the contents of the production tenant should match those in the development tenant.
In Github, CI/CD is achieved through a Workflow Action yaml file. As there is no code to “build” for Loome Integrate tenant imports, most of the work is achieved in the Workflow Action, with the workflow pipeline mainly existing to ensure all the contents of the tenant are provided in a standard format.
In Github, navigate to the Repositor that will contain your Loome Integrate Export and select “Actions” from the menu bar.
We will now go through the process of creating a simple Github Action Workflow file. Begin setup by selecting Simple Workflow for this workflow. This will give a basic template file to start that we will overwrite.
Once on the Actions page, you will need to overwrite the template basic workflow with the following yaml script. You will need to make some modifications to the script to import into your chosen tenant, we will explain these modifications in our next step.
# This is the Loome Integrate Git Integration Workflow file
name: CI
# Controls when the action will run. Triggers the workflow on pushes to master
on:
push:
branches: [ master ]
jobs:
build:
# The type of runner that the job will run on in this case windows:
runs-on: [windows-latest]
steps:
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- uses: actions/checkout@v2
- name: Import Tenant
env:
ClientId: YOUR_CLIENT_ID
IdHostName: YOUR_ID_HOSTNAME
ApiHostName: YOUR_API_HOSTNAME
AgentHostName: YOUR_AGENTHOSTNAME
shell: pwsh
run: |
try {
Add-Type -AssemblyName System.IO.Compression.FileSystem
$ClientId = $env:ClientId;
$IdHostName= $env:IdHostName;
$TokenRequestBody = @{
"grant_type" = "client_credentials";
"client_id" = $ClientId;
"client_secret" = $ClientId;
"scope" = "webApi"
};
$AccessToken = ((Invoke-WebRequest -Uri "https://$IdHostName/connect/token" -Method Post -Body $TokenRequestBody).Content | ConvertFrom-Json).access_token;
$ArtifactOutputDir = $ENV:GITHUB_WORKSPACE;
$ApiHostName = $env:ApiHost;
$BaseUrl = "https://$env:ApiHostName/api/v1/";
$AgentHostname = $env:AgentHostName;
function Upload-Zip ($ZipLocation) {
$AccessHeaders = @{"Authorization" = "bearer $AccessToken" }
$ZipUploadUrl = $BaseUrl + "Import/Zip?agentHostName=$AgentHostName";
try {
$Response = Invoke-RestMethod $ZipUploadUrl -Method 'POST' -Headers $AccessHeaders -InFile $ZipLocation -ContentType "application/zip" -SkipHttpErrorCheck;
}
catch {
throw "an unexpected error occurred";
}
$ResponseMessage = $Response | Out-String;
if ($ResponseMessage.Contains("Import failed"))
{
throw $ResponseMessage;
}
}
$ZipPath = "tenant.zip";
Remove-Item $ZipPath -ErrorAction Ignore;
Compress-Archive -Path $ArtifactOutputDir -DestinationPath $ZipPath;
Upload-Zip -ZipLocation $ZipPath;
}
catch {
Write-Output "Failed Run";
Write-Error $_.Exception.Message;
}
- name: Upload Zipped Tenant
uses: actions/upload-artifact@v1
with:
name: Tenant
path: "tenant.zip"
This script will upload and compress the contents of your tenant as a zip and then upload it directly to Loome Integrate. If anything goes wrong during the import (for example, if new connections were not imported before the import) then the task will fail.
Edit the Script to include the following variables. This step varies slightly depending on the region of Integrate you are using, however there will always be four variables to set:
AgentHostName
ApiHostName
dg-api-au.perspectiveilm.com
dg-api-us.perspectiveilm.com
ClientId
IdHostName
identity-au.perspectiveilm.com
identity-us.perspectiveilm.com
You will now simply need to Start Commit in the top right corner of the page. Add a simple messeage outlining the commit.
Open up Loome Integrate and navigate to your development tenant. To setup Git Syncing you will need to create a new “Git Connection”.
The details of the Git Connection will be the following:
Field | Value |
---|---|
Username | Your Personal Username |
Password | Your Personal Password or Personal Access Token |
Branch | Leave empty (Loome Integrate will create branches for Import/Export) |
Git Repository URL | The URL of your Git Repository (This is the URL you use to clone the repository) |
Create and validate the connection in Integrate, once the connection is validated open up the “Import/Export” settings from the top right hand dropdown.
Open up the “Export to Git” tab, you will need to provide the Git connection that you will use for exporting here. Once you click the save button you will be able to export the contents of your Tenant to your Github repository.
Once this is enabled, you will see a button appear in the top right of the application that says “Sync Changes”. This button will appear any time there are changes available to sync to Git.
In the rare case this button doesn’t appear on the enabling of Git sync, adding an empty job to a project will enable the button as a change will be recorded.
Clicking this button and then confirming the sync from the pop up will export the contents of the tenant to Azure Repos. Once the sync is completed, you will see a notification at the bottom right hand of the page showing the name of the created branch.
Navigating to Azure Repos will show that this branch has been created and your tenant contents have synced.
Before importing jobs into another tenant, you will need to ensure that all connections used by those jobs are also available in both tenants.
Loome Integrate does not include connections in the Import/Export process, there are a few reasons for this:
It is standard practice to not include database connection details and credentials in source control, abiding by this practice ensures that Integrate is correctly handling sensitive data.
There may be different connections used in development and production environments (e.g., production loads may use a cloud data warehouse with more resources available or a different database entirely for some migrations).
Connections that are validated on one tenant with a specific agent may require minor changes to be validated on another tenant, especially if the production tenant has an agent on a different host.
On the development tenant, open up the Import/Export page and select the Import/Export Connections tab.
Clicking the Export Connections button will download a JSON file with all the connections from the development tenant.
On the production tenant, open up the Import/Export page and select the Import/Export Connections tab.
Clicking the Import Connections button will prompt you to provide the connections file downloaded previously. Once you have provided the file, you will be prompted with the number of new connections that have been imported in a notification.
Loome Integrate will only add connections that do not exist in the tenant yet, if a connection with the same name is already present it will not be imported.
Once the connections have been imported, you will need to validate them before you can run jobs that use them.
As mentioned previously, you will need to reconfigure the following as it is not brought over with the import.
Once you have validated the connections, you are ready to set up the CI/CD pipeline for your jobs.
In this setup guide, we have created a full Continuous Integration/Continuous Deployment pipeline for our Integrate tenants. At a high level we support the following flow:
master