If you followed my guide on how to publish VSTS extensions to the marketplace using Team Build & Release, you might be asking yourself some of the following questions.

  • How will I test if my extension actually works?
  • What if I ship a buggy extension straight to the marketplace?
  • How do I roll back changes?

Luckily there are a few changes that you can make to the standard pipeline described in my first article that will add an extra layer of quality assurance to your deployment.

Adding a private test deployment

If you have used Release Management in newer versions of TFS or VSTS before you will be familiar with the concept of Environments. By default you have just one for each release, but nothing keeps you from adding another.

The first step is to add a test environment to your release that will be the target for the continuous deployment action.
To start you off you can simply clone the Marketplace release and queue the test environment in front of your marketplace release.

Without changes your test environment will still deploy to your public marketplace extension. We would like to avoid this in this scenario. We would also like to prevent a Marketplace deployment from being kicked off automatically.

Requiring user input for the “Marketplace” (public) deployment

To make the required changes, click “…” on your Marketplace environment and choose “Deployment conditions”. In the window that comes up you can set the deployment conditions as required.

You may want to consider:

  • Automatic deployment after ‘Test’ completes
  • Set a list or group of approvers for the Marketplace release

or:

  • Manual deployment after ‘Test’ completes

In both cases a user has to take an action to continue the deployment process after the test release is successful.

Setting up a test/nightly/QA version of your app

Using Microsoft’s build tools for VSTS extensions, you can easily overwrite some of the key settings in your VSTS/TFS extension.

Here is an example of how I set up my test deployment:

Be sure to:

  • Specify a different extension ID (it has to be unique)
  • Adjust the version look up so that it uses the different extension ID
  • Specify a different extension title (so that it is easier for you to know which one is private and which one is public)
  • Set the visibility to “Private”
  • Set the pricing to “Free”

Start your user testing

Once your deployment has run you will discover a second version of your extension in the VSTS marketplace.

You can use the VSTS release task or the function provided in the marketplace portal to share the private/test version of your extension with your test accounts.

But what if a faulty commit still goes through to the Marketplace?

As all your code is in source control, you can simply revert to the latest known good version, which will kick your automated deployment cycle.
Do not forget to test the reverted extension before deploying the fix to production.

Happy testing!