Azure Data Lake Storage Gen2
Description
A connector used for importing data to and from Azure Data Lake Storage Gen2.
Task Type Support
This connection type supports the following task types:
- Data Migration
Data Migration Support
This connection can be used in a Data Migration.
| Usable as Source | Usable as Target | Usable as Incremental Source | Usable as Incremental Target |
|---|---|---|---|
| ✔ | ✔ | ❌ | ✔ |
Connection String Templates
| Template Name | Description |
|---|---|
| ADLS (Account Key) | A standard connection string for Azure Data Lake Storage Gen2, that requires an Account Access Key. |
| ADLS (System-Assigned Identity) | A connection string that uses a System Assigned Managed Identity, and requires an Account Name, and Authentication will be the Active Directory Default. |
| ADLS (User-Assigned Identity) | A connection string that uses a User Assigned Managed Identity, and requires an Account Name, Authentication of Active Directory Default, and a User Id. |
| OneLake Lakehouse (System-Assigned Identity) | This template is to connect to Microsoft Fabric OneLake Lakehouse that uses System Assigned Managed Identity, and requires an Account Name, Workspace name, Lakehouse name, and Authentication of Active Directory Default. |
| OneLake Lakehouse (User-Assigned Identity) | A connection string for Microsoft Fabric OneLake Lakehouse that uses User Assigned Managed Identity, and requires an Account Name, Workspace name, Lakehouse name, Authentication of Active Directory Default, and your User Id. |
Connection Notes
The connection string for Azure Data Lake Storage Gen2 uses an account access token that you can generate from within the Azure Portal.
Please note that a file definition that uses this connection needs to specify a particular container or a directory in the container as the path. If you specify a container or directory in your path that does not yet exist, it will be created in the Azure Data Lake Storage Connection when the task using it is run.
When exporting into Parquet, if it is not specified in the connection string, Azure Data Lake Storage will default to a LOCAL offset value for any datetime offset columns. You can change the format of the datetime offset using the configuration settings here and adding it to your connection string.
A refresh period for a changes-only data migration is not supported for a data migration to Azure Data Lake Storage. Any data migrations to Azure Data Lake Storage will ignore the refresh period configuration and will display that the refresh period is not supported in the execution logs.
Connecting to Microsoft Fabric OneLake Lakehouse
You can use the Azure Data Lake Storage Gen2 connection type to connect to Microsoft Fabric OneLake Lakehouse.
When creating a connection, choose ‘Azure Data Lake Storage Gen2’ as the connection type.
Add in your connection string or choose a template from the dropdown.
If your agent uses a System Assigned Managed Identity, choose the template ‘OneLake Lakehouse (System-Assigned Identity)’, and validate your connection using an agent with a managed identity.
If your agent uses a User Assigned Managed Identity, choose the template ‘OneLake Lakehouse (User-Assigned Identity)’, or ensure you have added User Id=[Your Client Id] to your connection string. (You can obtain your client Id from Azure.)
Provide your Account Name, Workspace name, Lakehouse name, and User Id (if required) in the connection string.
Choose an agent to validate the connection.
Submit the connection to save and it will validate the connection.