Category Archives: Uncategorized
10. Azure Data Factory – Monitoring – Alerts and Metrics Now, in this chapter, I just want to go through another aspect that is available in the monitoring section of Azure Data Factory, and that’s when it comes on to alerts and metrics. First of all, if you go on to your dashboards here, you… Read More »
7. Azure Synapse – Workload Management Now in this chapter, I just want to go through workload management, which is available as part of your dedicated SQL pool. This is something that we had actually implemented earlier on. So when we were looking at loading data in a SQL pool that time, we had created… Read More »
4. Azure Monitor – Alerts Hi and welcome back. Now in this chapter, I want to go through a special note when it comes to alerts that you can create in Azure Monitor. So if I go on to the monitoring service, so remember, Azure Monitor can be used to gather all of the metrics… Read More »
DP-203 Data Engineering on Microsoft Azure – Monitor and optimize data storage and data processing
1. Best practices for structing files in your data lake Now, in this chapter, I want to give some best practices when it comes to structuring your files. And this is when it comes to building your data lake. So normally when you design a data lake, you might go ahead and create something known… Read More »
DP-203 Data Engineering on Microsoft Azure – Design and Implement Data Security Part 6
17. Lab – Azure Synapse – External Tables Authorization via Azure AD Authentication Now, in this chapter, I want to go through virtual Network service endpoints. Now, what is the purpose of having service endpoints in place? So here in this example, let’s say you have a virtual machine that is part of a virtual… Read More »
DP-203 Data Engineering on Microsoft Azure – Design and Implement Data Security Part 5
14. Lab – Azure Data Lake – Role Based Access Control Now in this chapter I want to explain rolebased access control. Now we know that Azure Active Directory is our identity provider. So here we can create users and the user can actually log into Azure account. Now let’s say that we do define… Read More »
DP-203 Data Engineering on Microsoft Azure – Design and Implement Data Security Part 4
11. Lab – Azure Synapse – Azure AD Authentication – Creating a user Now in the last chapter we had seen how to define a SQL Active Directory admin when it comes to Azure Synapse. Now in this chapter, let’s create a new user in Azure Active Directory and give that user access onto our… Read More »
DP-203 Data Engineering on Microsoft Azure – Design and Implement Data Security Part 3
7. Lab – Azure Synapse – Auditing And welcome back. Now in this chapter I want to go through the auditing feature which is available in Azure Synapse so you can enable the auditing feature for an Aziosql pool in Azure Synapse. This feature can be used to track the database events and write them… Read More »
DP-203 Data Engineering on Microsoft Azure – Design and Implement Data Security Part 2
4. Azure Synapse – Customer Managed Keys Hi, and welcome back. Now, in this chapter, I quickly want to go through the concept of being able to do encryption of your Azure Synapse workspace with the help of your keys defined in your Azure Key vault. So if you want to use Customer Manage keys… Read More »
DP-203 Data Engineering on Microsoft Azure – Design and Implement Data Security
1. Section Introduction Hi and welcome to this section. When it comes to security now, security is a very important aspect and in Azure there are a lot of security aspects that are available for Azure based services. So when it comes to Azure data Lake we’ll be looking at role based access control, we’ll… Read More »