I thought it might make sense to put a blog article out about this one based on some of the conversations I had with some of you.
The main point here is the fact that a machine you are deploying to in Azure might lure you into a false sense of security based on the tasks you have available to you in VSTS.
Let me expand on what I mean:
Many of my customers now use the Hosted Build Agent offering in VSTS. This has a few big advantages:
- No need to manage machines
- Machines are reimaged and deployed with a known set of tools every time
- It’s – in many cases – a lot cheaper than having your own build farm
- Deploying into Azure is easy, because you are already in an Azure datacentre
On the last one of these four points I like to ask “or is it?”
While “easy” is a great term to market something, it does not actually give us any information about how the deployment process described actually works. To determine whether using hosted agents actually makes things easier, we need to know a bit more about how hosted agents work.
- They are a set of machines in Azure
- They are hosted in the same region as your VSTS account
- They are in their own network area but can use the Azure backbone to access resources in the same region quicker
The last point is both a positive and a negative.
For example: If your build process involves an upload to a storage account, and your solution is located in the same location as your VSTS account, then this will ensure good upload speeds and a quick completion of the deployment step.
If – however – you need to talk to machines that are deployed in a hybrid environment (site-2-site network or ExpressRoute) that is not exposed to the internet, you will be severely limited in terms of the interaction you can establish between the VSTS agent and your machines.
Here are a few examples:
If you use the “Azure File Copy” task to deploy content to a VM, it needs to have a public endpoint, at least at the time of the copy action.
That is because the VSTS agent will upload content to a storage account and then send instructions to the VM to start a download from there
If you use the “PowerShell on Target Machine” task or any other deployment logic that is executed via WinRM, you need to either…
- Allow connections via WinRM from the Azure region that your VSTS account sits in (at least for the time of the deployment)
- Or deploy a dedicated private agent into the VNet where you want to take advantage of WinRM
If you establish any network connections between the agent and the target machine, you need to be extremely careful about the following…
- You should make sure ports/IP white lists do not remain open after deployment (if you cannot avoid opening them in the first place)
- You should make sure you keep up to date with our Azure datacentre IPs
- You can tie the XML white list into a script that allows access temporarily
If you install an agent on a deployment target to use “Deployment Groups” you need to consider that…
- The machine will require an outgoing internet connection (at least with a fairly open whitelist)
- The VSTS agent needs to be installed on each machine you are deploying to
If you end up installing a private agent for network-based deployments (the preferred method), please make sure you consider the following…
- Either provision your agent ad-hoc in an agent phase with an ARM template from an up-to-date image
- Or apply your general patching and machine lifecycle policies (as if it was a standard on-prem machine)
- Do not do this if your machine targets already have public endpoints, or if you are using ARM templates to finish the deployment via DSC or a script extension, while only staging to a storage account from VSTS/li>
In summary, there are a few ways to deploy to VMs in Azure securely from VSTS:
- Consider that most traffic that goes directly to VMs will have to go via a public endpoint
- Use ARM templates and extensions to get around the problem
- This would be the recommended option if you want to use hosted agents
- If you need network-based deployments, consider having a private agent
- Make sure this private agent is maintained to a high standard
Have a think about, what your traffic flow is and make decisions accordingly
Make sure you understand the options on a VSTS task. Some might open NSG routes that you have not considered.
We have open source the task code for first party tasks here: https://github.com/Microsoft/vsts-tasks
If the network situation does not allow for hosted agents to be used sensibly, move to private agents and automate their deployment as much as possible
Be very careful about temporarily opening ports that should not usually be exposed, this should be avoided at all cost.