Category Archives: Microsoft Azure

Quickly stop and start your Windows Azure lab

Introduction

I needed  way to easily stop and start my different lab setups in Windows Azure. I don’t want to keep running, and pay for, a set of VMs I use maybe once a month. So here is a PowerShell script to stop and start a set of Azure VMs. One important dimension here is the order in which the VMs start (and possibly; shuts down, depending on your setup). Since all IP addresses are dynamically allocated in Windows Azure I had to make sure that my VMs started in a specific order and that the script executed synchronously. That way whatever IP they had when they were provisioned would most likely be assigned to them when they started. Therefore the script includes code to start and stop VMs in the order you specify them in the input file. It also tries to wait until the current VM is fully started or stopped before proceeding with the next one.

Pre-requisites, input file and usage

The input file is really simple. It is a text file with a Cloud Service name and a VM name on each line, separated by a semi-colon, no header:

<cs name>;<vm name>

The VMs in the file will be started in the order they are listed and stopped in the reverse order!

By default, the script looks for a txt file in its execution directory called Control-AzureVMs.txt. If it is not found PowerShell throws an error. You can override this behavior by specifying your own file with VMs to either start or stop.

You must already have your machine configured to use Windows Azure PowerShell and specify your subscription in the script:

Select-AzureSubscription “<your subscription name here>”

Whether you want to start or stop a set of VMs is controlled by a script parameter. You use either Stop or Start. Optionally you can add your own txt file with VMs after the Start/Stop parameter.

Usage with the default input file:

Control-AzureVMs.ps1 Stop

Control-AzureVMs.ps1 Start

Usage with the custom input file:

Control-AzureVMs.ps1 Stop MySetOfVms.txt

The code

Here is the code. I can think of all sorts of improvements, but I needed this quickly so that will have to wait. Error checking is pretty much non-existent at this point so use at your own risk. I accept no responsibility whatsoever.

https://gist.github.com/morgansimonsen/8529297

UPDATE: With the release of the Windows Azure PowerShell module v0.7.3 you can now create DHCP reservations for your VMs that will persist when the machine is deallocated. Check out Get-AzureStaticVNetIP, Set-AzureStaticVNetIP, Remove-AzureStaticVNetIP and Test-AzureStaticVNetIP.

Bug in Windows Azure PowerShell module v 0.7.0

The latest version of the Windows Azure PowerShell module v 0.7.0 (released 21-10-2013), which features such great improvements as Windows Azure Active Directory logon support, unfortunately has a bug relating to endpoint ACLs.

Any Set-AzureEndpoint or Add-AzureEndpoint command which includes the ACL parameter will seem to succeed, but no ACL changes will be applied to the endpoint. I reverted to the previous release, v 0.6.16.1, and it works correctly.

I submitted an issue on GitHub, hopefully it will be resolved soon.

Troubleshooting ISO file sharing in System Center Virtual Machine Manager 2012 SP1

So I was trying to use sharing of ISO files instead of copying when mounting on virtual machines. The really helpful dialog box in System Center Virtual Machine Manager (SCVMM) 2012 SP1 politely informs you that

image

OK, so what configuration?

I found these steps for VMM 2008: How to Enable Shared ISO Images for Hyper-V Virtual Machines in VMM

In short this is what you have to do:

  • Use a domain account as the VMM service account
  • Grant share and NTFS Read permissions to the VMM library shares that contain ISO files for both the VMM service account and the computer accounts of any Hyper-V hosts that need to access those ISO files
  • Configure constrained delegation for each Hyper-V host to allow delegation of credentials to the VMM library servers
  • Add the Hyper-V hosts to the Windows Autorization Access builtin group

If you use a group to grant access to you Hyper-V hosts to the library share, like I did you have to restart them to update their security tokens. Configuring constrained delegation also requires a restart.

But it wasn’t working! VMM just told med “Update failed” each time I tried to mount an ISO on a VM. This is the error I got:

Error (12700)
VMM cannot complete the host operation on the <Hyper-V host> server because of the error: ‘<VM>’ failed to add device ‘Virtual CD/DVD Disk’. (Virtual machine ID <GUID>)

‘<VM>’: User Account does not have sufficient privilege to open attachment ‘\<VMM Library server>MSSCVMMLibraryISOs<ISO file>’. Error: ‘General access denied error’ (0x80070005). (Virtual machine ID <GUID>)
Unknown error (0x8001)

Recommended Action
Resolve the host issue and then try the operation again.

The Internet didn’t know why it didn’t work either. I know because I asked it!

As far as I cloud see my permissions were correct so I had to figure out if there was another account involved. The best way I know to do that is by auditing object access. So first I enabled auditing of object access on my VMM Library server:

image

Next, to catch all security principals trying to access the files in the VMM Library, I added audit System Access Control List entries for both Authenticated Users and Everyone:

image

After setting this up I tried mounting the ISO again and then opened the Security Event Log. The first event of interest was a logon event (Event ID 4624):

image

The account listed here is the Run As account I use for Hyper-V host management. This was an Audit Success event since the account was allowed to log on to the VMM library server.

Next was an attempt to access a share (Event ID 5140):

image

Again it was the RunAs account this time trying to access the VMM Library share with read permissions (ReadData or ListDirectory). Again, this was an Audit Success event since this was just a request for access. This was followed by something called a detailed file share (Event ID 5145):

image

This was our first Audit Failure event. This event is the result of the File Share request in the previous event. As you can see it was not successful. For the user this will result in an access denied message; exactly what I was seeing in the VMM console.

So apparently the RunAs account is also involved in accessing the share. To test to see if this was the solution I added the RunAs account to the share and NTFS permissions with Read access. Immediately it was successful and the VM could access the ISO file directly from the VMM Library.

Happy that I made it work I was still interested in knowing why the RunAs account was used for this. The answer was to be found in the VMM Add Resource Wizard. This is the wizard used to add fabric resources to VMM, like Hyper-V hosts or clusters. On the Credentials page we see this:

image

So the RunAs account is stored and used to access the hosts discovered with the wizard. The universe was once again in balance.

Listing Windows Azure availability sets

Windows Azure guarantees a 99.95 % uptime SLA, but this is only for multiple instance roles. So, for example, one web server by itself will not be guaranteed 99.95 % uptime, but will rather have “best effort”. Whereas two, or more, in a multi instance role will. So how do you make something multi-instance?

You use something called an availability set. An availability set is spanned across multiple fault and update domains in the datacenter. A fault domain is a collection of resources that are expected to fail together, like a server rack. An update domain is a set of resources that are updated at the same time. An availability set ensures that your instances are spanned across multiple update and fault domains so that no single failure or update can impact all the instances. The point here is that all the instances must be the same and be able to offer the same service to your users. So going back to our example with web servers you have to have at least two identical ones in the same availability set for your service to have a 99.95 % uptime guarantee. It is not correct to say that any VM or instance is guaranteed 99.95 % since it is the service that the instance offers that is guaranteed. Furthermore only Internet facing services have the SLA, since you need to use the Windows Azure load balancer to publish your instances, but more on that another day. Back to availability sets.

You add VMs to availability sets either in the Windows Azure portal, with PowerShell or the REST API. In the portal or PowerShell there is no way to explicitly create an availability set, you always create one as a side effect of configuring a VM. In the portal it looks like this:

image

This is an availability set with only one instance. In the drop down list I can either remove the VM from the availability set or create a new availability set and add the VM to that set.

Note: when you either add or remove a VM from an availability set, it will get a new NIC. This will have the same IP address as before but you will see that its number has been incremented.

In PowerShell you use the Set-AzureAvailabilitySet or New-AzureVMConfig cmdlets. Set-AzureAvailabilitySet updates an existing VM while New-AzureVMConfig lets you specify an availability set when you create the VM.

Something I miss is the ability to list all my availability sets. Since there is no (yet) Get-AzureAvailabilitySet cmdlet you have to extract this information from each of your VMs. Here is a quick one-liner to do that:

https://gist.github.com/morgansimonsen/8039845

This will list the VM names, cloud service names and availability set names (if any) for all your VMs. Feel free to add any kind of formatting or sorting to the output.

A network by any other name…

OK, funny story. A Windows Azure virtual network with the name MDS-vNET1; no problem. A Windows Azure cloud service with a production deployment slot associated with a virtual network with the name MDS-vNet1; no problem. Try to combine those two; problem.

So let me give you a few more details. Everything in this setup was created with PowerShell. I pre-created all the cloud services, affinity groups, storage accounts etc. I created a virtual network XML file and uploaded it with PowerShell too. But the crucial bit came when I created the VMs. On the New-AzureVM cmdlet I specified a vNetName of MDS-vNet1, instead of MDS-vNET1 that was in the network configuration. The command did not give any errors but when I tried to see which subnet a VM was on that information was missing from the Azure portal, and from PowerShell, and from SC App Controller. All I could see in any of those was the incorrect vNetName (MDS-vNet1). The Subnet field was simply missing. If I did an export of the VM with Export-AzureVM, the Subnet section was empty. Needless to say, the VMs were not communicating.

So how to fix it? You cannot change the vNet associated with a deployment once it is up and running. Neither can you move a VM from one subnet to another once it is created. The only time you can select a subnet for a VM is when you create it. In this case I had do to the following:

  1. Export and delete all VMs in the faulty deployment
  2. Delete the cloud service with the faulty deployment
  3. Manually edit all the VM XML files to include the correct subnet names
  4. Re-create the cloud service
  5. Import all the VMs and for the first VM specify the correct vNetName on the New-AzureVM cmdlet.

Problem solved.

As far as I know the only thing case sensitive in Windows Azure are storage account names, which can include only lower case letters and numbers. Not in any other case have I seen an error relating to text case. Strange that it should have such a drastic effect. If you have any experiences similar to mine, please leave a comment.

App Controller and Azure HighMemory SKUs

We are all aware of the lag between the evolution of the Windows Server platform and the System Center suite. It always takes some time from when a new feature in Windows Server is released, before it is supported in the System Center products. Now, it seems, this applies to Windows Azure as well. Consider the following…

Recently I did a Windows Azure project for a customer which included the introduction of System Center App Controller 2012 SP1 in their environment. They were already using System Center Virtual Machine Manager 2012 SP1 to manage their Hyper-V hosts. We specifically wanted to use App Controller to be able to easily migrate VMs from on-premise Hyper-V into Windows Azure IaaS.

To add an Azure subscription to App Controller you need access to the subscription ID (obtained from the Azure portal e.g.) and a management certificate in PFX format, i.e. you need access to both the private and public keys of the certificate. This certificate needs to already be registered in the subscription. I my case adding the two subscriptions the customer used presented no problems. However, when I came back a few weeks later more instances had been created in the subscriptions and now we had the following error in App Controller under the Public Clouds heading:

* Service unavailable

The first I checked was the status of the App Controller Windows Azure Provider (cmazure) service. This Windows service is responsible for communicating with the Windows Azure platform through the REST API. On my server the service was not running. I started the service and refreshed App Controller, but got the same error. Checking the service status again I found that it had crashed!

The first remedy I tried was updating the App Controller installation with the System Center Service Pack 1 Update Rollup 2 (KB2802159). That did not prevent the cmazure service from crashing but now I had a new error in the App Controller console:

* Requested value ‘A6’ was not found.

A6 and A7 are the IDs of the new HighMemory SKUs introduced in Windows Azure back in April 2013. They are higher memory variants of the Large and Extra Large SKUs respectively. My next move was to check if App Controller knew about these newer SKUs. But first I had to determine if one of my subscriptions in Windows Azure were in fact using any A6 or A7 instances, and if so, remove that subscription. To do this I logged on to the portal, and it was indeed the case. One of the subscriptions had two A6 machines.

I deleted that subscription from App Controller so that I could test my theory that the reason for the error was that App Controller did not know about A6 and A7. Here is a screenshot of deploying a new VM to Azure from App Controller:

image

As you can see, the Instance size list does not contain the A6 and A7 SKUs. Mystery solved!

This case was very surprising to me in several respects. First, it seems strange that App Controller keeps its own definition files of what is available in Windows Azure (a platform that is updated every 4-6 weeks) instead for dynamically downloading that information on demand and caching it. Secondly it seems like bad code when the introduction of a new instance SKU actually crashes a Windows service. Surly that exception could easily be handled in the code.

How to manage multiple Windows Azure subscriptions with PowerShell

You can have several Windows Azure subscriptions defined in Windows Azure PowerShell. This lets you easily manage several subscriptions from the same PowerShell console. These are the steps required to add a new Windows Azure Subscription to your Windows Azure PowerShell profile:

  1. Generate a management certificate:
    makecert.exe -pe -n “CN=<certificate subject>” -ss My –r <certificate filename>.cer
  2. Upload management certificate through Windows Azure portal (Settings->Management Certificates):
    image
  3. Copy the subscription ID:
    image
  4. Add subscription to default subscription data file:
    Set-AzureSubscription -SubscriptionName <subscription name> –SubscriptionId <subscription ID> –Certificate <path to cer file> –CurrentStorageAccount <storage account name>

Miscellaneous subscription related commands:

  • List all subscriptions currently defined in Windows Azure PowerShell profile:
    Get-AzureSubscription
  • List current subscription:
    Get-AzureSubscription –Default
  • Set current subscription:
    Set-AzureSubscription -DefaultSubscription <subscription name>
  • Do not specify a default subscription:
    Set-AzureSubscription –NoDefaultSubscription
  • Display currently selected subscription:
    Get-AzureSubscription –Current
  • Set a specific subscription as the active one:
    Select-AzureSubscription –SubscriptionName <subscription>

New PowerShell module for Windows Azure Active Directory

A new version of the PowerShell module for Windows Azure Active Directory is available. This module was previously know as the Microsoft Online PowerShell module. The cmdlets all have the word MSOL in them, and the modules are called MSOnline and MSOnlineExtended. The version is still 1.0.0 as were the previous module. New in this release is support for Windows Server 2012; you can install the module on Windows Server 2012 and also configure the version of ADFS in Windows Server 2012. The documentation has also been updated.

Norwegian content: How to integrate your on-premise Active Directory with Windows Azure Active Directory

I have published a 5 part blog series on the Norwegian Microsoft TechNet Blog, with step by step instructions for setting up integration between your on-premise Window Server Active Directory Directory Service and Windows Azure Active Directory. It covers concepts, single-sign on with ADFS, Directory Synchronization with the DirSync Tool and troubleshooting. So if you speak my native language; head on over:

Morgan

Windows Azure compute cores and you

By default, every Windows Azure subscription has a limit of 20 compute cores. This applies to both VMs (IaaS) and worker roles (PaaS). Fortunately this is a soft limit and you can increase it by submitting a support ticket to the Windows Azure team through the management portal here. The following table lists the number of cores for each available Windows Azure instance:

Compute instance size CPU Cores
Extra Small 1/6 (Shared)
Small 1
Medium 2
Large 4
Extra Large 8

To see how many cores each of your cloud services are using just look in the management portal. You will se something like this:

image

Unfortunately you need to manually add all the cores for each cloud service to get the total number. To work around this limitation you can use PowerShell:

https://gist.github.com/morgansimonsen/8039470

Save the above as a PowerShell script and run it from a prompt with the Windows Azure PowerShell module added. It will return all your VMs, sorted by name, their instance sizes, the number of cores they each use and the total number of cores in use in your subscription.

If you are paying for your Windows Azure subscription with a credit card your credit limit is $15000 by default. This will let you increase to a total of 175 cores in your subscription. If you want more cores and still pay with a credit card you will have to go through a credit check process to increase your credit limit. If you move to an Enterprise Agreement there will be no need for any credit checks to increase your core count beyond 175.

For large deployments the Windows Azure Capacity Planning team will want information from you so that they can allocate the required resources in the regions you specify. Typical questions they will ask you is where (region) you want to deploy, if you can deploy in several regions, if you are creating a new deployment or growing an existing one, which VM sizes you require and other services, like CDN, SQL etc) that you may require.