By Daniel Greenwood
Here, we run how Microsoft Fabric allows you to easily ingest your on-premise data using a data gateway. This drastically reduces the complexity of migrating your data to the Cloud and improves your ability to easily glean insights from your data no matter where it currently sits.
Read on to find out how:
As we are aware Microsoft Fabric has been released to the world, albeit in a more preview state. That hasn’t stopped us being asked, how do we get our on-premises data into the Cloud?
Moving to the cloud is a massive step for most companies especially moving our data. This can provide cost implications, downtime and may require special expertise.
Microsoft have provided us with the on-premises data gateway which allows your data to be securely transferred to Microsoft’s Cloud Services. From here, some quick changes inside Fabric and your data can be migrated over ready to utilize all its amazing functions and features!
There are a number of compatible data sources the on-premises gateway securely uploads your data from. SQL servers on-prem can establish connection per database through the gateway. It’s important to ensure the account you authorise for your upload has the sufficient privileges to connect and copy. Other sources, including files such as large excel spreadsheets and SharePoint data, can be moved into Microsoft Fabric.
There are two main types of on-premises data gateways:
On-Premises Data Gateway: This type allows multiple users to connect to various on-premises data sources. It’s suitable for scenarios where multiple individuals need access to multiple data sources. A single installation of this gateway supports all compatible data sources.
Personal Mode On-Premises Data Gateway: This is designed for a single user to connect to data sources and it cannot be shared with others. This gateway is useful when you’re the sole creator of reports and don’t need to share data sources with other users. This provides extended security to ensure only those that require it can copy data from an on-premises source.
Furthermore, there’s a Virtual Network (VNet) Data Gateway which is currently in preview at this time and is currently not yet available. This will permit multiple users to securely connect to data sources protected by virtual networks in Azure Resource Manager. This type of gateway is well-suited for intricate scenarios where you need to keep your data inside your on-prem environment through a VPN to Azure.
Overall, these gateways serve different purposes, allowing users to access on-premises or virtual network secured data sources, depending on the complexity of the scenario and the number of users involved.
Configuration of the gateway is completed on a domain joined windows server VM on your on-prem environment. The server will require to have a connection to all physical or virtual data sources. The application can be downloaded on the Microsoft website free of charge and installed on a compatible server. The application will then take you through a setup wizard for installation and synchronization with Azure’s cloud. Confirmation will be provided on the Data gateway configuration window which will confirm the gateway is in an active state.
An on-premises gateway also can be configured for cluster mode with two gateway connections. Cluster mode enables gateway administrators to prevent the risk of relying on a single point of failure for accessing on-premises data. In cases where the primary gateway becomes inaccessible, data requests are redirected to the secondary gateway that has been added, and this process continues for subsequent gateways.
The data is queried via encryption on the data gateway. It will be using the windows credentials that were configured during the Data gateway installation process.
The data is then transferred through Azure relay which will not traverse the public internet and only communicate to the relay via TLS 1.2. High amounts of data will use an Azure managed service bus and will secure data that is pending transfer on the data gateway in an encrypted state.
As of August 2023, Microsoft Fabric is still in preview and plenty more features will be released before and during general availability. Currently data gateways only provide the ability to import your on-premises data via the dataflow gen 2. From the dataflow, you’re able to manipulate your data through pipelines leading to data outputs such as Power BI.
Vnet integration is on the list of features to be added through the on-premises gateway. This feature will allow you to keep your data within your own network, rather than sending it through Microsoft’s network.
As Microsoft Fabric continues to develop and add further features will be added, we’ll be updating this blog with the latest information. So, be sure to check back for the most up-to-date details!
Interested further or have a question? Connect with us or join us for a 1-hour briefing call to learn more about Microsoft Fabric from Katalyze Data’s experts.