When importing from a flat file source, Loome Integrate Online will attempt to detect the maximum string length of each column by reading a set number of rows. With larger files, this may not be enough as the default number of rows to read may not yield the longest value available.
To increase the amount of rows read to get column lengths, modify the value of the FlatFileDetectDataTypeMaxRow setting for your tenant.
Go to the Tenant Settings page.
Modify the value of the FlatFileDetectDataTypeMaxRow.
Increasing the value will mean that Data Migrations may take longer with flat files however it also ensures less of a chance of errors occurring due to string length limits.
Setting the value to 0 will mean that SQL Server targets use VARCHAR(MAX)/NVARCHAR(MAX)
for the column size. This
can result in slower queries however also ensures that there’s no possibility of exceeding the limit.
If you create the target table and configure incremental migration, Loome Integrate Online will not drop and create the target table as part of the migration so you completely control the schema.
This solution does require more work and will often require a SQL Statement task in tandem with the Data Migration.
Please contact us at support@loomesoftware.com with any questions about file definitions.