HybridCloud, Hybrid Cloud Matt Quickenden HybridCloud, Hybrid Cloud Matt Quickenden

HCI Box on a Budget. Leverage Azure Spot & Hyrbrid Use Benefits. Up to 93% savings.

Do you want to take HCI Box for a test drive but dont have $2,681 in the budget? Me either. How about the same box for $178?

This is the price for 730 hours

Following general instructions from jumpstart Azure Arc Jumpstart

once you have the git repo, edit the host.bicep file

...\azure_arc\azure_jumpstart_hcibox\bicep\host\host.bicep

add to the properties for the host virtualMachine the resource vm 'Microsoft.Compute/virtualMachines@2022-03-01'

priority: 'Spot'
    evictionPolicy: 'Deallocate'
    billingProfile: {
        maxPrice: -1
    }

You can review difference regions for either cheaper price per hour or lower eviction rate

0.24393 per hour * 730 hours = $178

If you are elegable for Hybrid Use Benefits through you EA or have licenses you can also enable HUB in the Bicep template under virtual machine properties

licenseType: 'Windows_Server'

Code changes

...
resource vm 'Microsoft.Compute/virtualMachines@2022-03-01' = {
  name: vmName
  location: location
  tags: resourceTags
  properties: {
    licenseType: 'Windows_Server'
    priority: 'Spot'
    evictionPolicy: 'Deallocate'
    billingProfile: {
        maxPrice: -1
    }
...

Good luck, enjoy HCI’ing

Read More
HybridCloud Matt Quickenden HybridCloud Matt Quickenden

The process cannot access the file psconfig.json because it is being used by another process

If you have been having issues deploying Azure Arc Resource Bridge on AKS and running into various issues like the one above for Azure Stack HCI. Microsoft has recently published new guidance. Some pages still have conflicting messaging. I hope this pointer might help someone else.

We do NOT recommend or support running AKS on Azure Stack HCI and Azure Arc Resource Bridge on the same Azure Stack HCI or Windows Server cluster. If you have AKS on Azure Stack HCI installed, run Uninstall-AksHci and start deploying your Azure Arc Resource Bridge from scratch.

Microsoft Learn Reference

https://learn.microsoft.com/en-us/azure/aks/hybrid/troubleshoot-aks-hybrid-preview#issues-with-using-aks-hci-and-azure-arc-resource-bridge

Read More
HybridCloud, Hybrid Cloud, Azure Arc Matt Quickenden HybridCloud, Hybrid Cloud, Azure Arc Matt Quickenden

SQL Counters via Azure Arc

Walk through how to collect SQL counter using Data collection Rule and displaying the results on a workbook an Azure Dashboard.

Microsoft is rapidly expanding the services and features available from the Azure Arc SQL Extension. As this moves into GA and more visibility features get added and importantly, features that allow actions EG controlling backup and even patching using Azure to create a management layer for your SQL Server estate outside of Azure becomes more viable.

Let’s take a brief look at grabbing SQL-specific performance counters to Azure Monitor Metrics.


We need to create a Data Collection Rule (DCR). I am not using Data Collection Endpoints (DCE), but you may want to consider this in your environment.

Select the Arc Resource you want to add

now we need to add some Performance counter

it will add a bunch by default; select None, then Custom to clear all.

Unfortunately, currently, you can not see the SQL counters from this view, so you’ll have to go collect what you want from another source.

You can use Perfmon, unfortunately you cant copy from perfmon.

this query will provide a list

SELECT COUNT(*) FROM sys.dm_os_performance_counters;

Here is a list I have extracted for this DCR

\SQLServer:General Statistics\User Connections
\SQLServer:General Statistics\Logins/sec
\SQLServer:General Statistics\Transactions
\SQLServer:Databases(_Total)\Active Transactions
\SQLServer:SQL Statistics\Batch Requests/sec
\SQLServer:SQL Statistics\Logical Connections
\SQLServer:SQL Statistics\SQL Compilations/sec
\SQLServer:SQL Statistics\SQL Re-Compilations/sec

Paste it in line by line, Add, tick, Delete line, next counter. Adjust the Sample rate as needed. I have used the default 60 seconds. For large SQL servers estates, you might want to consider increasing this depending on your needs.

It’s worth considering building a template for this if you want to do this repeatedly. This resource type has some challenging layers to work through for deployment. Microsoft reference material can be found here.

we pipe these into Azure Monitor Metrics Which is still in preview and for this exercise I will also push the data into a log analytics instance.

as of this writing the metrics for hybrid compute fail to be added to scope of Azure Monitor. For the remainder of this example we will use Log Analytics Workspace for metrics.

Heading over to the Log Analytics Workspace. You can create various Kusto queries to integrate these SQL Metrics

Perf
| where ObjectName == 'SQLServer:General Statistics' and CounterName == 'User Connections'
| project TimeGenerated, Computer, ObjectName, CounterName, CounterValue
| summarize avg(CounterValue) by bin(TimeGenerated,60m) , Computer
| render timechart

You can send this to a Workbook or Dashboard

We can now see the data on a chart in a workbook. By saving the you can give the workbook a name and save it to a resource group

Alternatively, you can publish this to an Azure Dashboard. I am going to use the dashboard I created here Arc SQL Extension - Best Practices Assessment — Crying Cloud

This displays a tile here. You can continue to add tiles and edit the queries. You build workbooks and dashboards to target the specific metrics about your Arc-enabled SQL Servers.

Read More
HybridCloud, Azure Arc Matt Quickenden HybridCloud, Azure Arc Matt Quickenden

Arc SQL Extension - Best Practices Assessment

A look into Azure Arc SQL Extension and how best practices assessment and what it can extract from an Azure Arc enabled SQL Server.

Azure Arc help increase the visibility of your IT estate outside of Azure. Layering on top of the SQL Extension, we can bring a centralized view of your SQL Servers and databases, and now other control features help with management activities.

Let’s take a look at the Best practices assessment (BPA)

First thing to know is “Best practices assessment is only available for SQL Server with Software Assurance, SQL subscription, or with Azure pay-as-you-go billing. Update the license type appropriately. Learn more

If you want to look at you’re overall licenses we can run a query against the Azure Resource Graph. While we are at it lets add it to a dashboard.

resources
| extend SQLversion = properties.version
| extend SQLEdition = properties.edition
| extend licensetype = properties.licenseType
| where type == ("microsoft.azurearcdata/sqlserverinstances")
| project id, name,SQLversion,SQLEdition, licensetype

Note: as of writing this, the property is ‘lincetype’ not ‘licensetype’ this will likely be corrected.

there are some minor edits to the query in this image, added ID and selected ‘formatted results’ so the record is linked.

We can now see the query on the dashboard and see the license types for the servers. We have a ‘Paid’ for sv5-su5-6320018 so we can configure BPA for both instances.

You can change the SQL Server Registration to automatically handle the licensing selection

To enable an assessment pick a Log Analytics Workspace, click “Enable assessment”

The prerequisites are listed here. One to make sure you have completed is.

The SQL Server built-in login NT AUTHORITY\SYSTEM must be the member of SQL Server sysadmin server role for all the SQL Server instances running on the machine.

We can see what resource the deployment creates.

checking the data collection rule

We can see the data source is the local files is checking for CSVs here

C:\Windows\System32\config\systemprofile\AppData\Local\Microsoft SQL Server Extension Agent\Assessment\*.csv

and uploading them into the table SqlAssessment_CL in the LA Workspace instance. We can see the table

Depending on your purpose you may want to consider changing the table settings.

Depending on the size of the servers and the number of databases it might take a while to run. The default schedule is weekly but adjustable.

Once its completes you will be able to explore the results.

You can look at the query that is integrating the SqlAssessment_CL table. The data is being dumped into a single RawData field that needs to be parsed. This query will help you get a jump start on accessing the data.

There is a to explore with the Azure Arc SQL Extension and various ways to use the platform to provide more visibility and control over your SQL servers.

Read More
HybridCloud, MAAS Matt Quickenden HybridCloud, MAAS Matt Quickenden

Creating an Azure Stack HCI Image for MAAS

This blog is a follow on from Creating a MAAS Image Builder Server (Windows Server 2022 example) — Crying Cloud the goal here is to create a MAAS usable Azure Stack HCI image for deployment with MAAS. This is still using the component from cloudbase/windows-imaging-tools: Tools to automate the creation of a Windows image for OpenStack, supporting KVM, Hyper-V, ESXi and more. (github.com) this repo.

The primary issue is if you try to “generalize” an HCI image, if you attempt to use that prepped image, you get the perpetual error “Windows could not finish configuring the system. To attempt to resume configuration, restart the computer”

However, if you don’t “generalize” the HCI base image, the deployment works. The problem with using the existing tools/repo and scripts is creating a MAAS image with this option not to generalize it.


The following content is assuming the layout as described here Creating a MAAS Image Builder Server (Windows Server 2022 example) — Crying Cloud.

Looking into the file C:\BuilderFiles\windows-openstack-imaging-tools\UnattendResources\Logon.ps1 around line 714 we find the command that is calling sysprep.exe

& "$ENV:SystemRoot\System32\Sysprep\Sysprep.exe" `/generalize `/oobe `/shutdown `/unattend:"$unattendedXmlPath"

Without breaking the ability to build other images or have a complete clone of the repo we need to make a few changes. Being able to have a parameter in the ini file makes sense. Create a copy of the ini file C:\BuilderFiles\Scripts\config-Server-HCI-UEFI.ini. I have added a parameter to the [sysprep] section run_sysprep_generalize=False. The differences between the 2022 file and HCI are as follows.

# config-Server-HCI-UEFI key diff to config-Server-2022-UEFI

image_name=Azure Stack HCI SERVERAZURESTACKHCICORE
image_path=C:\BuilderFiles\Images\HCI.10.2022.tgz
custom_scripts_path="C:\BuilderFiles\scripts\HCICS"
unattend_xml_path="UnattendTemplateHCI.xml"

# new parameter in section
[sysprep]
run_sysprep_generalize=False

I have created a copy of unattendTemplate2022.xml and save as unattendTemplateHCI.xml stored here C:\BuilderFiles\windows-openstack-imaging-tools with the following changes.

# Delete the following component from specialize node
#     <settings pass="specialize">

        <component name="Microsoft-Windows-TerminalServices-LocalSessionManager" processorArchitecture="amd64" publicKeyToken="31bf3856ad364e35" language="neutral" versionScope="nonSxS" xmlns:wcm="http://schemas.microsoft.com/WMIConfig/2002/State" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
            <fDenyTSConnections>false</fDenyTSConnections>
        </component>
        <component name="Microsoft-Windows-TerminalServices-RDP-WinStationExtensions" processorArchitecture="amd64" publicKeyToken="31bf3856ad364e35" language="neutral" versionScope="nonSxS" xmlns:wcm="http://schemas.microsoft.com/WMIConfig/2002/State" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
            <UserAuthentication>0</UserAuthentication>
        </component>
        <component name="Networking-MPSSVC-Svc" processorArchitecture="amd64" publicKeyToken="31bf3856ad364e35" language="neutral" versionScope="nonSxS" xmlns:wcm="http://schemas.microsoft.com/WMIConfig/2002/State" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
            <FirewallGroups>
                <FirewallGroup wcm:action="add" wcm:keyValue="RemoteDesktop">
                    <Active>true</Active>
                    <Profile>all</Profile>
                    <Group>@FirewallAPI.dll,-28752</Group>
                </FirewallGroup>
            </FirewallGroups>
        </component>

Next is changes to C:\BuilderFiles\windows-openstack-imaging-tools\UnattendResources\Logon.ps1

# line  575 under 
#     if ($installQemuGuestAgent -and $installQemuGuestAgent -ne 'False') {
#        Install-QemuGuestAgent
#    }

# add the following get-ini parameter call

    try {
        $generalizeWindowsImage = Get-IniFileValue -Path $configIniPath -Section "sysprep" -Key "run_sysprep_generalize" -Default "True"
    } catch{}

then, lets a simple if statement to flip the call

# line 741 under
# Optimize-SparseImage

if ($generalizeWindowsImage -eq "False") {
    & "$ENV:SystemRoot\System32\Sysprep\Sysprep.exe" `/oobe `/shutdown `/unattend:"$unattendedXmlPath"
} else {
    & "$ENV:SystemRoot\System32\Sysprep\Sysprep.exe" `/generalize `/oobe `/shutdown `/unattend:"$unattendedXmlPath"
}

Next, create the C:\BuilderFiles\Scripts\HCICS folder. Which will also copy in the RunBeforeSysprep.ps1 script for HCI I have added an additional script C:\BuilderFiles\Scripts\HCICS\RunBeforeWindowsUpdates.ps1 with a script to install the pre-requisites needed for HCI clusters.

function Write-Log {
    Param($messageToOut)
    add-content -path "c:\build.log" ("{0} - {1}" -f @((Get-Date), $messageToOut))
}

Write-Log "RunBeforeWindowsUpdate.ps1 starting" 

Write-Log "install windows Features"
install-windowsfeature FS-Data-Deduplication,BitLocker,Data-Center-Bridging,Failover-Clustering,NetworkATC,RSAT-AD-PowerShell,RSAT-Hyper-V-Tools,RSAT-Clustering,RSAT-DataCenterBridging-LLDP-Tools

Write-Log "RunBeforeWindowsUpdate.ps1 finished"

And to bring it all together the script to call this Build-HCI.ps1. These can be more normalized but for now, create a separate file with the appropriate parameters.

# Build-HCI.ps1 parameters

Param ( 
    $VerbosePreference = "Continue",
    $ISOImage =  "C:\BuilderFiles\ISOs\AzureStackHCI_20348.587_en-us.iso",
    $ConfigFilePath = "C:\BuilderFiles\Scripts\config-Server-HCI-UEFI.ini",
    $CloudBuildModules = "C:\BuilderFiles\windows-openstack-imaging-tools"
)

Fingers crossed, let’s run the Build-HCI.ps1 script

After deployment we can RDP to the server, as it is named in MAAS

Let’s try the credentials and add the server to Windows Admin Center

I believe there is room to refine this process and automate more of it, but these are the steps I took to get an Azure Stack HCI image with updates, users and features installed and able to deploy directly to bare metal with MAAS

Read More