Category Archives: Uncategorized
29. Delta Lake Introduction Hi and welcome back. Now in this chapter I just want to go through Delta Lake when it comes to Azure databricks and we’ll see a couple of examples on Delta Lake in data bricks itself. So with the help of a Delta Lake, you get some more features when it… Read More »
24. Autoscaling a cluster Hi and welcome back. Now, in this chapter, I just want to give some notes when it comes to auto scaling. So I’ve shown you that when you create your cluster in Azure databricks, you have this auto scaling concept. So when creating an Azure databricks cluster, you can specify the… Read More »
22. Lab – Azure Data Lake Storage Credential Passthrough Now in this chapter, I want to go through a scenario wherein you can actually make use of something known as your ad Credential pass-through. So earlier on we had seen that if we wanted to fetch data from a data lake gen two storage account,… Read More »
19. Lab – Getting data from Azure Data Lake – Implementation So in the previous chapter we had created our keyword. We create a key in the keyword. Now it’s time to create something known as a databricks scoped secret. So for that I need to go on to my Databricks resource. So here, here… Read More »
17. Lab – Structured Streaming – Streaming from Azure Event Hubs – Implementation So where did we leave off in the last chapter? So, we had run our notebook to read events from the Event Stream, but we were getting the body of our event in some sort of encoded format. So I have now… Read More »
12. Lab – Filtering on NULL values Now in this chapter, I just want to quickly show you again, how do you filter on null values? Again, something we had seen earlier on when it came to working with Azure Synapse, the Spark Pool. So just one thing that I want to confirm. Let me… Read More »
7. Lab – Reading a CSV file Now, in this chapter, we’ll see how to process our log CSV file. So, one of the most important aspects when it comes on to any service that we have seen. So, from the exam perspective, you should be able to understand how to go through CSV files,… Read More »
4. Lab – Creating a cluster Now in the last chapter, we have gone ahead and create our workspace. In this chapter, we are going to now create a cluster. So the cluster will have the underlying machines that is going to have Spark installed. So as part of the data bricks implementation, you have… Read More »
DP-203 Data Engineering on Microsoft Azure – Design and Develop Data Processing – Azure Databricks
1. What is Azure Databricks Now, before we actually go into Azure data bricks, let me go ahead and first explain the need for Data Bricks itself. So, Databricks is a company that was actually founded by the original creators of Apache Spark. So Data Bricks itself, the service actually makes use of Apache Spark… Read More »
MS-203 Microsoft 365 Messaging – Troubleshoot Mail Flow Problem Part 3
5. Analyze Message Headers with the Remote Connectivity Analyzer Let’s take some time now and talk about a really great web based tool that Microsoft provides us. This tool is called the remote connectivity analyzer, okay. Also known as RCA. We can get to this tool by typing test connectivity Microsoft. com. So we’re going… Read More »