Tuesday, December 21, 2021

Azure Monitor | Data Collection Rule (Use case Scenario)

In my previous post I had discussed about Azure Monitor | Data Collection Rules.

https://hiyusuf.blogspot.com/2021/12/azure-monitor-data-collection-rules.html

In this article we will discuss with one of the use case scenario on implementing Data Collection rule.


Current Scenario:




One of the enterprise having multiple subscriptions across different regions, each subscriptions are linked with its own log Analytics Workspace, which are managed & controlled by IT support team of that region.

This Workspace consist of Logs / counters of VM's , VM scale sets.

Requirement: Now, this enterprise have a centralize team who wants to centrally monitor / needs to view / keep an eye on performance counters for each VM's in each subscriptions from all regions.

Proposed Scenario:












New proposed scenario will be as followed to achieve given requirements: 

- Create a new Log Analytics Workspace per Region (As workspace are region specific).











- Go to Azure Monitor | Data Connection Rule | Create New Rule (For each subscription level)

  • BasicSelect the subscription to manage deployed resources and costs. Use resource groups like folders to organize and manage all of your resources.
  • Resources - Pick a set of machines to collect data from. The Azure Monitor Agent will be automatically installed on these machines.(This will also enable System Assigned Managed Identity on these machines, in addition to existing User Assigned Identities (if any).)
  • Collect & Deliver - Configure which data sources to collect, and where to send the data to.
  • Data Source type - Select which data source type and the data to collect for your resource(s).
  • Destination - Select the destination(s) for where the data will be delivered. ( Destination Type depends on what type of data source selected) example, if data source is selected as windows event logs -- destination type will only show Azure monitor logs & if data source is selected as performance counters -- destination type will show Azure monitor metrics & Azure monitor logs.
  • Select the destination workspace (in one case the centralize workspace).
  • Review & Create.








Thursday, December 9, 2021

Azure Monitor | Data Collection Rules

 Azure Monitor | Data Collection Rules


Data collection rules in Azure Monitor

Data Collection Rules (DCR) define data coming into Azure Monitor and specify where that data should be sent or stored. This article provides an overview of data collection rules including their contents and structure and how you can create and work with them.

Data collection rules currently support the following input sources:

Azure Monitor Agent running on virtual machines, virtual machine scale sets and Azure Arc for servers.

 

Components of a data collection rule

A data collection rule includes the following components.

COMPONENTS OF A DATA COLLECTION RULE

Component

Description

Data sources

Unique source of monitoring data with its own format and method of exposing its data. Examples of a data source include Windows event log, performance counters, and syslog. Each data source matches a particular data source type as described below.

Streams

Unique handle that describes a set of data sources that will be transformed and schematized as one type. Each data source requires one or more streams, and one stream may be used by multiple data sources. All data sources in a stream share a common schema. Use multiple streams for example, when you want to send a particular data source to multiple tables in the same Log Analytics workspace.

Destinations

Set of destinations where the data should be sent. Examples include Log Analytics workspace and Azure Monitor Metrics.

Data flows

Definition of which streams should be sent to which destinations.

Data collection rules are stored regionally, and are available in all public regions where Log Analytics is supported. Government regions and clouds are not currently supported.

The following diagram shows the components of a data collection rule and their relationship:








Create DCR and associations with Azure PowerShell.

https://docs.microsoft.com/en-us/powershell/module/az.monitor/new-azdatacollectionrule?view=azps-7.0.0

Parameters have to be defined based on above PowerShell commands.


Tuesday, December 7, 2021

Azure VM Backup Architecture

 

Azure VM Backup Architecture

Let’s look at the Azure Virtual Machine Architecture.









Diagram: Azure VM backup Architecture, and the process in creating a backup as described in the content.


  • Microsoft has an extension inside of every Azure VM

    • When you configure Backup, it communicates with the Azure Backup Service and associates itself to a policy.

    • It identifies itself to the Azure Backup Service as a VM, and that the service should back it up accordingly.

  • When it’s time for backup per the backup policy, Microsoft sends a command to the Azure Backup extension and then Azure Backup orchestrates a VSS snapshot.

  • Once the snapshot is available it goes to your local VM storage as an instant recovery snapshot which you can quickly recover from as it is in your VM storage.

  • In the background, the snapshot is compared to a snapshot of a previous recovery point and moves only the incremental blocks via HTTPs into the recovery services vault.

  • The recovery services vault has encryption enabled via server-side encryption (SSE), so the backup is encrypted at rest and is protected while in transit.

  • When you secure your data via Azure Disk Encryption you are given keys, key encryption key (KEK) and BitLocker encryption key (BEK), which go to an Azure key vault, and are also backed up via Azure Backup.

    This is important because when you recover, you don’t have to worry about what keys you had when the backup was taken and the keys are restored for you to apply these keys on the recovered data.

Monday, December 6, 2021

Azure SQL Database & Managed Instance Service

 

Azure SQL Database Service

SQL Database is a fully managed Platform as a Service (PaaS) Database Engine that handles most of the database management functions such as upgrading, patching, backups, and monitoring without user involvement. Azure SQL Database is always running on the latest stable version of SQL Server Database Engine and patched OS with 99.99% availability.

With Azure SQL Database, you can create a highly available and high-performance data storage layer for the applications and solutions in Azure. SQL Database can be the right choice for a variety of modern cloud applications because it enables you to process both relational data and non-relational structures, such as graphs, JSON, spatial, and XML.

Deployment models

Azure SQL Database provides the following deployment options for an Azure SQL database:













Diagram of Deployment Options

  • Single database represents a fully managed, isolated database. You might use this option if you have modern cloud applications and microservices that need a single reliable data source. A single database is similar to a contained database in Microsoft SQL Server Database Engine.

  • Managed instance is a fully managed instance of the Microsoft SQL Server Database Engine. It contains a set of databases that can be used together. Use this option for easy migration of on-premises SQL Server databases to the Azure cloud, and for applications that need to use the database features that SQL Server Database Engine provides.

  • Elastic pool is a collection of single databases with a shared set of resources, such as CPU or memory. Single databases can be moved into and out of an elastic pool.

Azure SQL Database Managed Instance

Managed instance is a new deployment option of Azure SQL Database, providing near 100% compatibility with the latest SQL Server on-premises (Enterprise Edition) Database Engine, providing a native virtual network (VNet) implementation that addresses common security concerns, and a business model favorable for on-premises SQL Server customers. The managed instance deployment model allows existing SQL Server customers to lift and shift their on-premises applications to the cloud with minimal application and database changes.

The following diagram outlines key features of managed instances:

Key Features

The managed instance deployment model is designed for customers looking to migrate a large number of apps from on-premises or IaaS, self-built, or ISV provided environment to fully managed PaaS cloud environment, with as low migration effort as possible. Using the fully automated Data Migration Service (DMS) in Azure, customers can lift and shift their on-premises SQL Server to a managed instance that offers compatibility with SQL Server on-premises and complete isolation of customer instances with native VNet support.

The managed instance deployment option aims delivers close to 100% surface area compatibility with the latest on-premises SQL Server version through a staged release plan.

Features and capabilities

Managed instance combines the best features that are available both in Azure SQL Database and SQL Server Database Engine.




Tuesday, October 5, 2021

How to check connectivity to Azure SQL DB?

Sometimes when connection to Azure SQL DB fails and you want to test the network layer to confirm this is not network issue that prevents you from accessing your Azure DB service.

For Azure SQL DB use port 1433

Using PowerShell (Recommended)

Use the following command to confirm you have true for the TcpTestSucceeded

Test-NetConnection <YourServerName>.database.windows.net -Port <TCPPort>


thumbnail image 1 of blog post titled 
	
	
	 
	
	
	
				
		
			
				
						
							How to check connectivity to Azure SQL DB, Managed Instance, Azure Database for MySQL and PostgreSQL


Using PsPing

Download the PsPing tool.


psping <YourServerName>.database.windows.net:<TCPPort>


Further check Azure SQL Connectivity settings reference link: https://docs.microsoft.com/en-us/azure/azure-sql/database/connectivity-settings

Monday, October 4, 2021

Restricting RDP(Remote Desktop) Access to Azure Virtual Machines

By default, every Azure virtual machine has RDP (Remote Desktop Protocol), port 3389 enabled, and allows any RDP connection from any IP in the world. As great as that is, this can be a (huge) security risk.  So, what if we want to change this, and limit who has RDP access to the VM? What if we want only a select range of IPs, or a specific IP to only have access to the VM(s)? For example, your branch office has a static public IP, and we only want RDP access from this IP/location ,change the listening port for RDP port (3389) to a custom port on your VM. How can we achieve this?


Steps to follow:
  • Change the listening port for Remote Desktop on your VM.

When you connect to a computer (either a Windows client or Windows Server) through the Remote Desktop client, the Remote Desktop feature on your computer "hears" the connection request through a defined listening port (3389 by default). You can change that listening port on Windows computers by modifying the registry.

For this example I am using Port 6123 as custom port for RDP.

  1. Start the registry editor. (Type regedit in the Search box.)
  2. Navigate to the following registry subkey: HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Terminal Server\WinStations\RDP-Tcp
  3. Find PortNumber
  4. Click Edit > Modify, and then click Decimal.
  5. Type the new port number 6123, and then click OK.
  6. Close the registry editor.

  • Configure Windows firewall to permit connections to the new port number.
  1. Now, go to control panel > Windows firewall  > Click on Advance Settings >  Inbound Rules > Create a new Rule > Select Port > Select TCP > Specific local ports(6123)> Allow Connection > Select Rule Applies to connected networks > Give new rule a name and description.
  2. Restart your VM.

The next time you connect to this computer by using the Remote Desktop connection, you must type the new port. 



  • Restricting RDP access to your VMs in Azure
Let’s begin, if you go into the property settings of the VM, and select the Networking in Settings > In Inbound Port Rules > and select, “Add inbound port rule

The Inbound Security Rule properties, as follows:

Wait, what do all of these fields mean?

  • Source: The source can by any IP Address, or CIDR Range, or a default-service tag.
  • Source IP Address/CIDR Ranges: Any IP Address, or CIDR Range.
  • Source Service Tag: There are a series of options here, but in short:
    • Load Balancer: Any probes in the Azure Load Balancer
    • Virtual Network: The Virtual Network the VM is connected to
    • Internet: All network traffic in the public virtual network, (including all Azure services, such as Azure Traffic Manager, Storage and SQL)
    • Azure Traffic Manager: Denotes the IP address from where the Azure Load Balancer health probes will originate.
    • Storage.*:  Access to Azure storage services and/or specific Azure regions
    • SQL.*: Access to Azure SQL Database and Warehouse services, and/or specific Azure regions
  • Source Port Ranges: You can use either a range of ports, or use a Wildcard (*) for all ranges.
  • Destination: The source can by any IP Address, or CIDR Range, or the Virtual Network.
  • Destination Port Ranges: You can use either a range of ports, or use a Wildcard (*) for all ranges.
  • Protocol: TCP or UDP, or Any, which includes both TCP and UDP, and ICMP.
  • Action: Allow, or Deny access.
  • Priority: A number between 100-4096. The lowest is 100, and the highest we can input is 4096. Lower the number, higher the priority.
  • Name: The name of the rule. Note, once created, it cannot be changed!

So, these are the values/settings I implemented for the Allow Inbound Rule:

  • Source: IP Addresses
  • Source IP Addresses/CIDR Ranges: xxx.xxx.xxx.xxx(Give Public IP from where you are going to access this VM over RDP)
  • Source Port Ranges: *
  • Destination: Any
  • Destination Port Ranges: 6123
  • Protocol: TCP
  • Action: Allow
  • Priority: 100
  • Name: RDP_6123















Next are the values/settings I implemented for the Deny all RDP Inbound Rule:

  • Source: ServiceTag
  • Source Service Tag: Intenert
  • Source Port Ranges: *
  • Destination: Virtual Network
  • Destination Port Ranges: 3389
  • Protocol: Any
  • Action: Deny
  • Priority: 101
  • Name: Deny-RDP-Access


































The inbound port rules should look something like this now. Why we set Priority the way we did… Well, Rules are checked in the order of priority. Once a rule applies, no more rules are tested for matching.



Once the rules has been submitted, and accepted, and if we try to RDP into the VM we should now only be allowed from the IP Range, or IP & using custom RDP port.



To learn more on Azure NSG (Network Security Groups), visit: https://docs.microsoft.com/en-us/azure/virtual-network/security-overview

Monday, September 27, 2021

Export Microsoft SQL Database to Azure SQL

Export Microsoft SQL Database to Azure SQL


Step by step guide on how to export your database to Azure via Microsoft SQL Server Management Studio, additionally i had came across an error & remedy to fix the issue.

1) First and the simple step is to open SQL Server Management Studio.

2) Find the Database you wish to move to Azure and right click on it.

3) Navigate to ‘Tasks’ then to ‘Deploy to Windows Azure SQL Database’

4) When the first pop up comes up, you will just need to click next.

5) On the next screen you will need to connect to your Azure Server.

Required Details:

  • Server Name
  • Choose ‘SQL Server Authentication’
  • Enter the server Username
  • Enter the server Password

6)   Once connected enter in the New Database name you want to call it. The other setting I left alone, but you may want to change them as you wish. After all is entered you can click next in the bottom right.

7) You will then have a summary page before you press next again to start the transfer. You can then start the transfer!

8) Once transfer is completed you can login to Management Studio with Azure SQL Server credentials or login to Azure portal to view new exported DB.

Error:

One or more unsupported elements were found in the schema used as part of a data package.

Error SQL71564: Error validating element [user]: The element [user] has been orphaned from its login and cannot be deployed.(Microsoft.SqlServer.Dac)

Troubleshoot & Findings: 

Orphaned users in SQL Server occur when a database user is based on a login in the master database, but the login no longer exists in master. This can occur when the login is deleted, or when the database is moved to another server where the login does not exist. This topic describes how to find orphaned users, and remap them to logins.

Source: https://docs.microsoft.com/

Solution:

This means that there is another owner on the Database. To fix this do the following:

Go to the target Database > Expand ‘Security’ > Expand ‘Roles’ > Expand ‘Database Roles’ > right click on ‘db_owner’ > click ‘properties’ as below

Error report would have told you what owners you need to delete. Select the owner and click remove in the bottom right.

Expand your database -> Security -> Schemas > Change Schema Owner

Right click on the schema that is owned by the user you want to delete and change the owner (Properties -> General -> Schema Owner).

Than you will need to delete the ‘Users’ from under the ‘Security’ section.