I got a question over e-mail about Selenium tests the other day and thought it would be a good opportunity to re-familiarize myself with the testing tool.
The person contacting me was trying to run a set of Selenium tests on multiple different browsers using Azure DevOps pipelines. The target was a standard ASP.NET app. Having just written a post trying to make clear how supposedly technical program managers can be, I felt this was a good opportunity to try and prove myself. (at least a little bit)
Here is an explanation for my sample, which you are more than welcome to have a closer look at on GitHub. The sample aims to demonstrate the above scope using Azure DevOps YAML pipelines.
I started by creating a very simple ASP.NET MVC app. I didn’t do much more than literally cloning my GitHub repo and creating a sample app. So far so good:
My first step was to put together an Azure DevOps pipeline that would build and deploy the sample app as a WebApp in Azure.
As a side note: Azure DevOps hosted agents did not always have Selenium web drivers pre-installed. At the time of writing Selenium web drivers are available on hosted agents that also come with Visual Studio, so I’ll be using these for simplicity. When using private agents or agents without the web drivers, you may need to take additional steps to install the drivers either as part of the pipeline or when setting up the agent. Documentation ->
I did this in Azure DevOps and used the most suitable YAML template. You can find a copy of the pipeline in the repo linked at the beginning of this article. The outcome was an Azure WebApp that contained my site.
Next I added a test project for the .NET Framework to my Visual Studio solution. This would be where our Selenium tests would go later on.
I made some changes to the template following the available documentation on multi-stage pipelines.
- Split into a Build & Deploy and a Test stage
- Added a step to upload the Test binaries
- Added a step to download the binaries in the Test stage
- Added a generic MSTest task to run the tests
The outcome was a two stage pipeline that we can use for testing:
Now it was finally time to sprinkle some Selenium goodness over the project. I added a few NuGet packages to my Selenium test package that would enable me to test in both Chrome and Firefox.
I added a really simple test that would simply navigate to the About page and check that we’ve arrived there, leaving some pauses for the navigation in the interactive browser. The code is on GitHub.
I also used a DataTestMethod attribute to define that we’d be running all tests in Chrome and Firefox. To reset the session after each test I used a fresh driver and shut the driver down with a clean up each time.
The next step was to commit and push to Git and let the pipeline run.
We can see two entries in the test summary. MSTest groups our tests first by the test method and then by the data input used. We can therefore see if a test fails on one browser but passes on the other. We also get a breakdown of the runtime on each driver. Firefox took a little longer than Chrome in this test.