Deploying Linux VMs on Azure Stack Hub with Enterprise CA Certificates? Here's a Solution!
When deploying Linux VMs on Azure Stack Hub in a corporate environment, you may encounter issues with TLS endpoint certificates signed by an internal Enterprise CA. In this post, we'll explore a technique for importing the root CA certificate into the truststore of your Linux VMs, enabling seamless access to TLS endpoints. We'll also show you how to use Terraform to automate the process, including provisioning a VM, importing the CA certificate, and running a custom script
When is an Azure Arc Server; not an Arc Server? A look at Azure Arc Server Resource types and the lack of feature parity between them
With the recent announcement of the Avanade Solution for Azure Stack that many of us have been working on, I've published an article on the Avanade Insights blog with some additional details and links to more information about the solution. https://www.avanade.com/en/blogs/avanade-insights/cloud-computing/avanade-azure-stack-isnt-like-the-others
I recently ran into an issue where an ExpressRoute had been migrated from Classis (ASM) to the new portal (ARM), however legacy Classic Virtual Networks (VNets) were still in operation. These VNets refused to be deleted by either portal or PowerShell. Disconnecting the old VNet’s Gateway through the Classic portal would show success, but it would stay connected.
There’s no option to disconnect an ASM gateway in the ARM portal, only a delete option. Gave this a shot and predictably, this was the result:
Ok, let’s go to PowerShell and look for that obstinate link. Running Get-AzureDedicatedCircuitLink resulted in the following error:
PS C:\> get-AzureDedicatedCircuitLink -ServiceKey $ServiceKey -VNetName $Vnet
get-AzureDedicatedCircuitLink : InternalError: The server encountered an internal error. Please retry the request.
At line:1 char:1
+ get-AzureDedicatedCircuitLink -ServiceKey xxxxxx-xxxx-xxxx-xxxx-xxx...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : CloseError: (:) [Get-AzureDedicatedCircuitLink], CloudException
+ FullyQualifiedErrorId : Microsoft.WindowsAzure.Commands.ExpressRoute.GetAzureDedicatedCircuitLinkCommand
I couldn’t even find the link. Not only was modifying the circuit an issue, but reads were failing, too.
Turned out to be a simple setting change. When the ExpressRoute was migrated, as there were still Classic VNets, a final step of enabling the circuit for both deployment models was needed. Take a look at the culprit setting here, after running Get-AzureRMExpressRouteCircuit:
"serviceProviderProperties": {
"serviceProviderName": "equinix",
"peeringLocation": "Silicon Valley",
"bandwidthInMbps": 1000
},
"circuitProvisioningState": "Disabled",
"allowClassicOperations": false,
"gatewayManagerEtag": "",
"serviceKey": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
"serviceProviderProvisioningState": "Provisioned"
AllowClassicOperations set to “false” blocks ASM operations from any access, including a simple “get” from the ExpressRoute circuit. Granting access is straightforward:
# Get details of the ExpressRoute circuit
$ckt = Get-AzureRmExpressRouteCircuit -Name "DemoCkt" -ResourceGroupName "DemoRG"
#Set "Allow Classic Operations" to TRUE
$ckt.AllowClassicOperations = $true
More info on this here.
But we still weren’t finished. I could now get a successful response from this:
get-AzureDedicatedCircuitLink -ServiceKey $ServiceKey -VNetName $Vnet
However this still failed:
Remove-AzureDedicatedCircuitLink -ServiceKey $ServiceKey -VNetName $Vnet
So reads worked, but no modify. Ah—I remembered the ARM portal lock feature, and sure enough, a Read-Only lock on the Resource Group was inherited by the ExpressRoute (more about those here). Once the lock was removed, voila, I could remove the stubborn VNets no problem.
# Remove the Circuit Link for the Vnet
Remove-AzureDedicatedCircuitLink -ServiceKey $ServiceKey -VNetName $Vnet
# Disconnect the gateway
Set-AzureVNetGateway -Disconnect –VnetName $Vnet –LocalNetworkSiteName <LocalNetworksitename>
# Delete the gateway
Remove-AzureVNetGateway –VNetName $Vnet
There’s still no command to remove a single Vnet, you have to use the portal (either will work) or you can use PowerShell to edit the NetworkConfig.xml file, then import it.
Once our legacy VNets were cleaned up, I re-enabled the Read-Only lock on the ExpressRoute.
In summary, nothing was “broken”, just an overlooked setting. I would recommend cleaning up your ASM/Classic Vnets before migrating your ExpressRoute, it’s so much easier and cleaner, but if you must leave some legacy virtual networks in place, remember to set the ExpressRoute setting “allowclassicoperations” setting to “True” after the migration is complete.
And don’t forget those pesky ARM Resource Group locks.
One of the advantages of the Azure Resource Management (ARM) deployment model is being able to delegate administration, something we couldn’t do under the Azure Service Management (ASM or RDFE) deployment model (see the previous posts on subscriptions and resource groups). By delegating administrative authority, we can keep the number of subscription admins low and grant access in accordance with the principle of Least Privilege. This post will walk through using Azure role-based access control (RBAC) to achieve all of this.
There are three general roles that apply to all resource types:
In addition to these general roles, there are other built-in roles that are resource-specific. Microsoft has a list of those roles, but because Azure is changing regularly it’s best to use PowerShell to query for the roles:
[powershell]
# list all of the RBAC roles in the subscription (Get-AzureRmRoleDefinition).Name
# see the actions a specific role is allowed to perform (Get-AzureRmRoleDefinition -Name '<Role Name>').Actions
# see the actions a specific role is *not* allowed to perform (Get-AzureRmRoleDefinition -Name '<Role Name>').NotActions
[/powershell]
Azure Active Directory (AD) users and groups can be assigned to any Azure RBAC role. The Azure AD tenant to which the subscription belongs is the source tenant for users and groups.
If you find that the built-in roles aren’t sufficient, you can create custom roles which will be the topic of Part 2.
There is a hierarchy to containers and resources:
Any access granted to a parent container is inherited by all children. For example, an account granted read access at the subscription level can see all resources in the subscription. By the way, there is no concept of denying access in Azure RBAC, so be very careful and deliberate about granting wide-spread access.
Back in the ASM days, to access an Azure subscription one needed to be a subscription admin (or co-admin). Those admins are automatically granted subscription owner access in ARM. However, accounts granted owner access in ARM are not automatically granted subscription co-admin access. If you have resources that you need to manage that are not yet available in ARM you will still need to manage the co-admins list. In either case the recommendation is still the same: keep the number of subscription admins (granted either through Azure RBAC or directly through subscription admins) as low as possible.
Recommendations for a resource group strategy were discussed in a previous post, so I won’t rehash that content. What I do want to talk about is how to implement your strategy.
RGs must be created by an account with subscription owner role. Once the RG is created an RG owner must be assigned to a user who is the actual owner of the RG (owner as in someone who is responsible for the resources that the RG will contain). Once that owner has been established additional roles can be added by the new owner. The advice for RGs is the same as subscriptions: keep the number of owners as low as possible – not everyone who needs access to the RG needs to be an owner of the RG – use Contributor instead.
To keep the administrative overhead as low as possible, use Azure AD groups to manage role membership. Create a group and add user accounts to the group. If you’re syncing your on-premises AD Domain Services (DS) with Azure AD create the group in AD DS and let it sync to Azure AD. Manage these groups using your existing on-premises user and group management process.
Contra much of the advice published on the Internet not every service or application should get its own VNet and not every virtual machine (VM) should get its own storage account. VNets and storage accounts for VM disks (VHDs) are infrastructure resources and need to be managed as such.
Create an RG for the VNets and place all the VNets into that RG. Create a group, name it something descriptive like Network Consumers, and grant it the Reader and Virtual Machine Contributor roles to the VNet RG. Any user account in that group will be able to attach VMs to any of the VNets in the VNet RG.
Create an RG for the VHD storage accounts and place all the VHD storage accounts into that RG. Create a group, name it something descriptive like Storage Consumers, and grant it the Reader and Virtual Machine Contributor roles to the VHD storage account RG. Any user account in that group will be able to use any of the storage accounts in the RG to store a VM’s VHDs.
In some previous posts, we covered some "hacks" to Azure Stack TP1, primarily enabling a customized domain name and exposing to the internet. If you have not noticed yet, the installation has changed greatly. The process is now driven by ECEngine and should be far more indicative of how the final product gets deployed. While the installer has greatly changed, fortunately, the process to expose the stack publicly has only changed in a few minor ways. Without getting too involved in how it works, the installation operates from a series of PowerShell modules and Pester tests tied to a configuration composed from a number XML configuration files. The configuration files support use of variables and parameters to drive most of the PowerShell action. As with TP1, the stack is wired so that the DNS domain name for Active Directory must match the public DNS domain name (think certificates and host headers). This is a much less involved change to TP2, it mostly requires replacing a couple of straggling hard coded entries with variables in some OneNodeConfig.xml files and changing the installer bootstrapper to use it. Once again, I will admonish you that this is wholly unsupported.
There are 6 files that need minor changes, we will start with the XML config files.
Config Files
C:\CloudDeployment\Configuration\Roles\Fabric\IdentityProvider\OneNodeRole.xml Line 11 From
[xml] <IdentityApplication Name="Deployment" ResourceId="https://deploy.azurestack.local/[Deployment_Guid]" DisplayName="Deployment Application" CertPath="{Infrastructure}\ASResourceProvider\Cert\Deployment.IdentityApplication.ClientCertificate.pfx" ConfigPath="{Infrastructure}\ASResourceProvider\Config\Deployment.IdentityApplication.Configuration.json" > </IdentityApplication> [/xml]
To
[xml] <IdentityApplication Name="Deployment" ResourceId="https://deploy.[DOMAINNAMEFQDN]/[Deployment_Guid]" DisplayName="Deployment Application" CertPath="{Infrastructure}\ASResourceProvider\Cert\Deployment.IdentityApplication.ClientCertificate.pfx" ConfigPath="{Infrastructure}\ASResourceProvider\Config\Deployment.IdentityApplication.Configuration.json" > </IdentityApplication> [/xml]
C:\CloudDeployment\Configuration\Roles\Fabric\KeyVault\OneNodeRole.xml Line 12 From
[xml] <IdentityApplication Name="KeyVault" ResourceId="https://vault.azurestack.local/[Deployment_Guid]" DisplayName="AzureStack KeyVault" CertPath="{Infrastructure}\ASResourceProvider\Cert\KeyVault.IdentityApplication.ClientCertificate.pfx" ConfigPath="{Infrastructure}\ASResourceProvider\Config\KeyVault.IdentityApplication.Configuration.json" > <AADPermissions> <ApplicationPermission Name="ReadDirectoryData" /> </AADPermissions> <OAuth2PermissionGrants> <FirstPartyApplication FriendlyName="PowerShell" /> <FirstPartyApplication FriendlyName="VisualStudio" /> <FirstPartyApplication FriendlyName="AzureCLI" /> </OAuth2PermissionGrants> </IdentityApplication> [/xml]
To
[xml] <IdentityApplication Name="KeyVault" ResourceId="https://vault.[DOMAINNAMEFQDN]/[Deployment_Guid]" DisplayName="AzureStack KeyVault" CertPath="{Infrastructure}\ASResourceProvider\Cert\KeyVault.IdentityApplication.ClientCertificate.pfx" ConfigPath="{Infrastructure}\ASResourceProvider\Config\KeyVault.IdentityApplication.Configuration.json" > <AADPermissions> <ApplicationPermission Name="ReadDirectoryData" /> </AADPermissions> <OAuth2PermissionGrants> <FirstPartyApplication FriendlyName="PowerShell" /> <FirstPartyApplication FriendlyName="VisualStudio" /> <FirstPartyApplication FriendlyName="AzureCLI" /> </OAuth2PermissionGrants> </IdentityApplication> [/xml]
Line 26 From
[xml] <AzureKeyVaultSuffix>vault.azurestack.local</AzureKeyVaultSuffix> [/xml]
To
[xml] <AzureKeyVaultSuffix>vault[DOMAINNAMEFQDN]</AzureKeyVaultSuffix> [/xml]
C:\CloudDeployment\Configuration\Roles\Fabric\WAS\OneNodeRole.xml Line(s) 96-97 From
[xml] <IdentityApplication Name="ResourceManager" ResourceId="https://api.azurestack.local/[Deployment_Guid]" HomePage="https://api.azurestack.local/" DisplayName="AzureStack Resource Manager" CertPath="{Infrastructure}\ASResourceProvider\Cert\ResourceManager.IdentityApplication.ClientCertificate.pfx" ConfigPath="{Infrastructure}\ASResourceProvider\Config\ResourceManager.IdentityApplication.Configuration.json" Tags="MicrosoftAzureStack" > [/xml]
To
[xml] <IdentityApplication Name="ResourceManager" ResourceId="https://api.[DOMAINNAMEFQDN]/[Deployment_Guid]" HomePage="https://api.[DOMAINNAMEFQDN]/" DisplayName="AzureStack Resource Manager" CertPath="{Infrastructure}\ASResourceProvider\Cert\ResourceManager.IdentityApplication.ClientCertificate.pfx" ConfigPath="{Infrastructure}\ASResourceProvider\Config\ResourceManager.IdentityApplication.Configuration.json" Tags="MicrosoftAzureStack" > [/xml]
Line(s) 118-120 From
[xml] <IdentityApplication Name="Portal" ResourceId="https://portal.azurestack.local/[Deployment_Guid]" HomePage="https://portal.azurestack.local/" ReplyAddress="https://portal.azurestack.local/" DisplayName="AzureStack Portal" CertPath="{Infrastructure}\ASResourceProvider\Cert\Portal.IdentityApplication.ClientCertificate.pfx" ConfigPath="{Infrastructure}\ASResourceProvider\Config\Portal.IdentityApplication.Configuration.json" > [/xml]
To
[xml] <IdentityApplication Name="Portal" ResourceId="https://portal.[DOMAINNAMEFQDN]/[Deployment_Guid]" HomePage="https://portal.[DOMAINNAMEFQDN]/" ReplyAddress="https://portal.[DOMAINNAMEFQDN]/" DisplayName="AzureStack Portal" CertPath="{Infrastructure}\ASResourceProvider\Cert\Portal.IdentityApplication.ClientCertificate.pfx" ConfigPath="{Infrastructure}\ASResourceProvider\Config\Portal.IdentityApplication.Configuration.json" > [/xml]
Line 129 From
[xml] <ResourceAccessPermissions> <UserImpersonationPermission AppURI="https://api.azurestack.local/[Deployment_Guid]" /> </ResourceAccessPermissions> [/xml]
To
[xml] <ResourceAccessPermissions> <UserImpersonationPermission AppURI="https://api.[DOMAINNAMEFQDN]/[Deployment_Guid]" /> </ResourceAccessPermissions> [/xml]
Line 133 From
[xml] <IdentityApplication Name="Policy" ResourceId="https://policy.azurestack.local/[Deployment_Guid]" DisplayName="AzureStack Policy Service" CertPath="{Infrastructure}\ASResourceProvider\Cert\Policy.IdentityApplication.ClientCertificate.pfx" ConfigPath="{Infrastructure}\ASResourceProvider\Config\Policy.IdentityApplication.Configuration.json" > [/xml]
To
[xml] <IdentityApplication Name="Policy" ResourceId="https://policy.[DOMAINNAMEFQDN]/[Deployment_Guid]" DisplayName="AzureStack Policy Service" CertPath="{Infrastructure}\ASResourceProvider\Cert\Policy.IdentityApplication.ClientCertificate.pfx" ConfigPath="{Infrastructure}\ASResourceProvider\Config\Policy.IdentityApplication.Configuration.json" > [/xml]
Line 142 From
[xml] <IdentityApplication Name="Monitoring" ResourceId="https://monitoring.azurestack.local/[Deployment_Guid]" DisplayName="AzureStack Monitoring Service" CertPath="{Infrastructure}\ASResourceProvider\Cert\Monitoring.IdentityApplication.ClientCertificate.pfx" ConfigPath="{Infrastructure}\ASResourceProvider\Config\Monitoring.IdentityApplication.Configuration.json" > </IdentityApplication> [/xml]
To
[xml] <IdentityApplication Name="Monitoring" ResourceId="https://monitoring.[DOMAINNAMEFQDN]/[Deployment_Guid]" DisplayName="AzureStack Monitoring Service" CertPath="{Infrastructure}\ASResourceProvider\Cert\Monitoring.IdentityApplication.ClientCertificate.pfx" ConfigPath="{Infrastructure}\ASResourceProvider\Config\Monitoring.IdentityApplication.Configuration.json" > </IdentityApplication> [/xml]
C:\CloudDeployment\Configuration\Roles\Fabric\FabricRingServices\XRP\OneNodeRole.xml Line 114 From
[xml] <IdentityApplication Name="Monitoring" ResourceId="https://monitoring.azurestack.local/[Deployment_Guid]" DisplayName="AzureStack Monitoring Service" CertPath="{Infrastructure}\ASResourceProvider\Cert\Monitoring.IdentityApplication.ClientCertificate.pfx" ConfigPath="{Infrastructure}\ASResourceProvider\Config\Monitoring.IdentityApplication.Configuration.json" > </IdentityApplication> [/xml]
To
[xml] <IdentityApplication Name="Monitoring" ResourceId="https://monitoring.[DOMAINNAMEFQDN]/[Deployment_Guid]" DisplayName="AzureStack Monitoring Service" CertPath="{Infrastructure}\ASResourceProvider\Cert\Monitoring.IdentityApplication.ClientCertificate.pfx" ConfigPath="{Infrastructure}\ASResourceProvider\Config\Monitoring.IdentityApplication.Configuration.json" > </IdentityApplication> [/xml]
Scripts
Now we will edit the installation bootstrapping scripts.
We will start by adding two new parameters ($ADDomainName and $DomainNetbiosName) to C:\CloudDeployment\Configuration\New-OneNodeManifest.ps1 and have the manifest generation use them.
[powershell] param ( [Parameter(Mandatory=$true)] [Xml] $InputXml,
[Parameter(Mandatory=$true)] [String] $OutputFile,
[Parameter(Mandatory=$true)] [System.Guid] $DeploymentGuid,
[Parameter(Mandatory=$false)] [String] $Model,
[Parameter(Mandatory=$true)] [String] $HostIPv4Address,
[Parameter(Mandatory=$true)] [String] $HostIPv4DefaultGateway,
[Parameter(Mandatory=$true)] [String] $HostSubnet,
[Parameter(Mandatory=$true)] [bool] $HostUseDhcp,
[Parameter(Mandatory=$true)] [string] $PhysicalMachineMacAddress,
[Parameter(Mandatory=$true)] [String] $HostName,
[Parameter(Mandatory=$true)] [String] $NatIPv4Address,
[Parameter(Mandatory=$true)] [String] $NATIPv4Subnet,
[Parameter(Mandatory=$true)] [String] $NatIPv4DefaultGateway,
[Parameter(Mandatory=$false)] [Int] $PublicVlanId,
[Parameter(Mandatory=$true)] [String] $TimeServer,
[Parameter(Mandatory=$true)] [String] $TimeZone,
[Parameter(Mandatory=$true)] [String[]] $EnvironmentDNS,
[Parameter(Mandatory=$false)] [String] $ADDomainName='azurestack.local',
[Parameter(Mandatory=$false)] [String] $DomainNetbiosName='azurestack',
[Parameter(Mandatory=$true)] [string] $AADDirectoryTenantName,
[Parameter(Mandatory=$true)] [string] $AADDirectoryTenantID,
[Parameter(Mandatory=$true)] [string] $AADAdminSubscriptionOwner,
[Parameter(Mandatory=$true)] [string] $AADClaimsProvider ) $Xml.InnerXml = $Xml.InnerXml.Replace('[PREFIX]', 'MAS') $Xml.InnerXml = $Xml.InnerXml.Replace('[DOMAINNAMEFQDN]', $ADDomainName) $Xml.InnerXml = $Xml.InnerXml.Replace('[DOMAINNAME]', $DomainNetbiosName) [/powershell]
The final edit(s) we need to make are to C:\CloudDeployment\Configuration\InstallAzureStackPOC.ps1. We will start by adding the same parameters to this script.
[powershell] [CmdletBinding(DefaultParameterSetName="DefaultSet")] param ( [Parameter(Mandatory=$false, ParameterSetName="RerunSet")] [Parameter(Mandatory=$true, ParameterSetName="AADSetStaticNAT")] [Parameter(Mandatory=$true, ParameterSetName="DefaultSet")] [SecureString] $AdminPassword,
[Parameter(Mandatory=$false)] [PSCredential] $AADAdminCredential,
[Parameter(Mandatory=$false)] [String] $AdDomainName='azurestack.local',
[Parameter(Mandatory=$false)] [String] $DomainNetbiosName='AzureStack',
[Parameter(Mandatory=$false)] [String] $AADDirectoryTenantName,
[Parameter(Mandatory=$false)] [ValidateSet('Public Azure','Azure - China', 'Azure - US Government')] [String] $AzureEnvironment = 'Public Azure',
[Parameter(Mandatory=$false)] [String[]] $EnvironmentDNS,
[Parameter(Mandatory=$true, ParameterSetName="AADSetStaticNAT")] [String] $NATIPv4Subnet,
[Parameter(Mandatory=$true, ParameterSetName="AADSetStaticNAT")] [String] $NATIPv4Address,
[Parameter(Mandatory=$true, ParameterSetName="AADSetStaticNAT")] [String] $NATIPv4DefaultGateway,
[Parameter(Mandatory=$false)] [Int] $PublicVlanId,
[Parameter(Mandatory=$false)] [string] $TimeServer = 'time.windows.com',
[Parameter(Mandatory=$false, ParameterSetName="RerunSet")] [Switch] $Rerun ) [/powershell]
The next edit will occur at lines 114-115 From
[powershell] $FabricAdminUserName = 'AzureStack\FabricAdmin' $SqlAdminUserName = 'AzureStack\SqlSvc' [/powershell]
To
[powershell] $FabricAdminUserName = "$DomainNetbiosName\FabricAdmin" $SqlAdminUserName = "$DomainNetbiosName\SqlSvc" [/powershell]
Finally we will modify the last statement of the script from line 312 to pass the new parameters.
[powershell] & $PSScriptRoot\New-OneNodeManifest.ps1 -InputXml $xml ` -OutputFile $outputConfigPath ` -Model $model ` -DeploymentGuid $deploymentGuid ` -HostIPv4Address $hostIPv4Address ` -HostIPv4DefaultGateway $hostIPv4Gateway ` -HostSubnet $hostSubnet ` -HostUseDhcp $hostUseDhcp ` -PhysicalMachineMacAddress $physicalMachineMacAddress ` -HostName $hostName ` -NATIPv4Address $NATIPv4Address ` -NATIPv4Subnet $NATIPv4Subnet ` -NATIPv4DefaultGateway $NATIPv4DefaultGateway ` -PublicVlanId $PublicVlanId ` -TimeServer $TimeServer ` -TimeZone $timezone ` -EnvironmentDNS $EnvironmentDNS ` -AADDirectoryTenantName $AADDirectoryTenantName ` -AADDirectoryTenantID $AADDirectoryTenantID ` -AADAdminSubscriptionOwner $AADAdminSubscriptionOwner ` -AADClaimsProvider $AADClaimsProvider ` -ADDomainName $AdDomainName ` -DomainNetbiosName $DomainNetbiosName [/powershell]
NAT Configuration
So, you now have customized the Domain for your one node Azure Stack install and want to get it on the internet. This process is almost identical to TP1 save for two changes. In TP1 there were both BGPVM and NATVM machines; while there is now a single machine MAS-BGPNAT01. The BGPNAT role only exists in the one node (HyperConverged) installation. The other change is the type of Remote Access installation. TP1 also used the "legacy" RRAS for NAT, where all configuration was UI or netsh based. TP2 has transitioned to "modern" Remote Access that is only really manageable through PowerShell. To enable the appropriate NAT mappings we will need to use three PowerShell Cmdlets. Get-NetNat Add-NetNatExternalAddress Add-NetNatStaticMapping I use a script to create all the mappings which takes a simple object, which in our use case is deserialized from JSON. This file is a simple collection of the NAT entries and mappings to be created.
[javascript] { "Portal": { "External": "172.20.40.39", "Ports": [ 80,443,30042,13011,30011,30010,30016,30015,13001,13010,13021,30052,30054,13020,30040,13003,30022,12998,12646,12649,12647,12648,12650,53056,57532,58462,58571,58604,58606,58607,58608,58610,58613,58616,58618,58619,58620,58626,58627,58628,58629,58630,58631,58632,58633,58634,58635,58636,58637,58638,58639,58640,58641,58642,58643,58644,58646,58647,58648,58649,58650,58651,58652,58653,58654,58655,58656,58657,58658,58659,58660,58661,58662,58663,58664,58665,58666,58667,58668,58669,58670,58671,58672,58673,58674,58675,58676,58677,58678,58679,58680,58681,58682,58683,58684,58685,58686,58687,58688,58689,58690,58691,58692,58693,58694,58695,58696,58697,58698,58699,58701 ], "Internal": "192.168.102.5" }, "API": { "External": "172.20.40.38", "Ports": [ 80,443,30042,13011,30011,30010,30016,30015,13001,13010,13021,30052,30054,13020,30040,13003,30022,12998,12646,12649,12647,12648,12650,53056,57532,58462,58571,58604,58606,58607,58608,58610,58613,58616,58618,58619,58620,58626,58627,58628,58629,58630,58631,58632,58633,58634,58635,58636,58637,58638,58639,58640,58641,58642,58643,58644,58646,58647,58648,58649,58650,58651,58652,58653,58654,58655,58656,58657,58658,58659,58660,58661,58662,58663,58664,58665,58666,58667,58668,58669,58670,58671,58672,58673,58674,58675,58676,58677,58678,58679,58680,58681,58682,58683,58684,58685,58686,58687,58688,58689,58690,58691,58692,58693,58694,58695,58696,58697,58698,58699,58701 ], "Internal": "192.168.102.4" }, "DataVault": { "External": "172.20.40.43", "Ports": [80,443], "Internal": "192.168.102.3" }, "CoreDataVault": { "External": "172.20.40.44", "Ports": [80,443], "Internal": "192.168.102.3" }, "Graph": { "External": "172.20.40.40", "Ports": [80,443], "Internal": "192.168.102.8" }, "Extensions": { "External": "172.20.40.41", "Ports": [ 80,443,30042,13011,30011,30010,30016,30015,13001,13010,13021,30052,30054,13020,30040,13003,30022,12998,12646,12649,12647,12648,12650,53056,57532,58462,58571,58604,58606,58607,58608,58610,58613,58616,58618,58619,58620,58626,58627,58628,58629,58630,58631,58632,58633,58634,58635,58636,58637,58638,58639,58640,58641,58642,58643,58644,58646,58647,58648,58649,58650,58651,58652,58653,58654,58655,58656,58657,58658,58659,58660,58661,58662,58663,58664,58665,58666,58667,58668,58669,58670,58671,58672,58673,58674,58675,58676,58677,58678,58679,58680,58681,58682,58683,58684,58685,58686,58687,58688,58689,58690,58691,58692,58693,58694,58695,58696,58697,58698,58699,58701 ], "Internal": "192.168.102.7" }, "Storage": { "External": "172.20.40.42", "Ports": [80,443], "Internal": "192.168.102.6" } } [/javascript]
In the one node TP2 deployment 192.168.102.0 is the subnet for "Public" IP addresses, and if you notice all the VIP's for the stack reside on that subnet. We have 1-to-1 NAT for all the "External" addresses we associate with a given Azure Stack instance.
[powershell] [CmdletBinding()] param ( [Parameter(Mandatory=$true)] [psobject] $NatConfig )
#There's only one could do -Name BGPNAT ... $NatSetup=Get-NetNat
$NatConfigNodeNames=$NatConfig|Get-Member -MemberType NoteProperty|Select-Object -ExpandProperty Name
foreach ($NatConfigNodeName in $NatConfigNodeNames) { Write-Verbose "Configuring NAT for Item $NatConfigNodeName" $ExIp=$NatConfig."$NatConfigNodeName".External $InternalIp=$NatConfig."$NatConfigNodeName".Internal $NatPorts=$NatConfig."$NatConfigNodeName".Ports Write-Verbose "Adding External Address $ExIp" Add-NetNatExternalAddress -NatName $NatSetup.Name -IPAddress $ExIp -PortStart 80 -PortEnd 63356 Write-Verbose "Adding Static Mappings"
foreach ($natport in $NatPorts) { #TCP Write-Verbose "Adding NAT Mapping $($ExIp):$($natport)->$($InternalIp):$($natport)" Add-NetNatStaticMapping -NatName $NatSetup.Name -Protocol TCP ` -ExternalIPAddress $ExIp -InternalIPAddress $InternalIp ` -ExternalPort $natport -InternalPort $NatPort
} } [/powershell]
DNS Records
The final step will be adding the requisite DNS Entries, which have changed slightly as well. In the interest of simplicity assume the final octet of the IP addresses on the 172.20.40.0 subnet have a 1 to 1 NAT mapping to 38.77.x.0 (e.g. 172.20.40.40 –> 38.77.x.40)
| A Record | IP Address |
| api | 38.77.x.38 |
| portal | 38.77.x.39 |
| *.blob | 38.77.x.42 |
| *.table | 38.77.x.42 |
| *.queue | 38.77.x.42 |
| *.vault | 38.77.x.43 |
| data.vaultcore | 38.77.x.44 |
| control.vaultcore | 38.77.x.44 |
| xrp.tenantextensions | 38.77.x.44 |
| compute.adminextensions | 38.77.x.41 |
| network.adminextensions health.adminextensions | 38.77.x.41 |
| storage.adminextensions | 38.77.x.41 |
Connecting to the Stack
You will need to export the root certificate from the CA for your installation for importing on any clients that will access your deployment. Exporting the root certificate is very simple as the host system is joined to the domain which hosts the Azure Stack CA. To export the Root certificate to your desktop run this simple one-liner in the PowerShell console of your Host system (the same command will work from the Console VM).
[powershell] Get-ChildItem -Path Cert:\LocalMachine\Root| ` Where-Object{$_.Subject -like "CN=AzureStackCertificationAuthority*"}| ` Export-Certificate -FilePath "$env:USERPROFILE\Desktop\$($env:USERDOMAIN)RootCA.cer" -Type CERT [/powershell]
The process for importing this certificate on your client will vary depending on the OS version; as such I will avoid giving a scripted method.
Right click the previously exported certificate.
Choose Current User for most use-cases.
Select Browse for the appropriate store.
Select Trusted Root Certificate Authorities
Confirm the Import Prompt
To connect with PowerShell or REST API you will need the deployment GUID. This can be obtained from the host with the following snippet.
[powershell] [xml]$deployinfo = Get-content "C:\CloudDeployment\Config.xml" $deploymentguid = $deployinfo.CustomerConfiguration.Role.Roles.Role.Roles.Role | % {$_.PublicInfo.DeploymentGuid} [/powershell]
This value can then be used to connect to your stack.
[powershell] #Deployment GUID $EnvironmentID='4bc6f444-ff15-4fd7-9bfa-5495891fe876' #The DNS Domain used for the Install $StackDomain="yourazurestack.com" #The AAD Domain Name (e.g. bobsdomain.onmicrosoft.com) $AADDomainName='youraadtenant.com' #The AAD Tenant ID $AADTenantID = "youraadtenant.com" #The Username to be used $AADUserName="username@$AADDomainName" #The Password to be used $AADPassword='P@ssword1'|ConvertTo-SecureString -Force -AsPlainText #The Credential to be used. Alternatively could use Get-Credential $AADCredential=New-Object PSCredential($AADUserName,$AADPassword) #The AAD Application Resource URI $ApiAADResourceID="https://api.$StackDomain/$EnvironmentID" #The ARM Endpoint $StackARMUri="Https://api.$StackDomain/" #The Gallery Endpoint $StackGalleryUri="Https://portal.$($StackDomain):30016/" #The OAuth Redirect Uri $AadAuthUri="https://login.windows.net/$AADTenantID/" #The MS Graph API Endpoint $GraphApiEndpoint="https://graph.windows.net/"
#Add the Azure Stack Environment Add-AzureRmEnvironment -Name "Azure Stack" ` -ActiveDirectoryEndpoint $AadAuthUri ` -ActiveDirectoryServiceEndpointResourceId $ApiAADResourceID ` -ResourceManagerEndpoint $StackARMUri ` -GalleryEndpoint $StackGalleryUri ` -GraphEndpoint $GraphApiEndpoint
#Add the environment to the context using the credential $env = Get-AzureRmEnvironment 'Azure Stack' Add-AzureRmAccount -Environment $env -Credential $AADCredential -Verbose [/powershell]
Note: You will need the a TP2 specific version of the Azure PowerShell for many operations. Enjoy, and stay tuned for more.
