I was looking into Microsoft Visual Studio App Center but it looks like it only works for mobile applications.
I want to set up an automated testing pipeline using Selenium that works like this:
- Developers work on a web app locally and push code to our internal GitLab server.
- An on-site service monitors the repository and when it sees new commits:
- Downloads the latest code.
- Builds the project for production.
- Executes Node.js unit tests.
- Publishes to our internal testing environment.
- Runs Selenium browser tests on multiple devices like Windows desktops, mobile phones, and tablets using different browsers like Chrome, Firefox, and Safari.
- Takes screenshots and compares them with baseline images from earlier test runs.
- Displays all test outcomes.
- If everything passes, asks for approval to push to live site.
- Deploys to production server.
The browser testing needs to work with internal URLs that aren’t accessible from the internet, similar to how BrowserStack Local works.
Does Microsoft have on-premises solutions that can handle this type of workflow? What products should I be looking at?
We switched from Jenkins to Azure DevOPS Server two years ago for this exact reason. Self-hosted agents are perfect for internal testing - everything stays inside your network. Here’s what others haven’t mentioned: set up multiple agent pools with different capabilities for cross-browser testing. We run dedicated Windows agents for Chrome/Firefox/Edge and separate macOS agents for Safari. Screenshot comparison isn’t built-in, but we added custom PowerShell scripts that use ImageMagick for pixel-perfect comparisons. Release pipelines handle approvals really well - you can create different approval chains for staging vs production. Installation was pretty straightforward on our Windows Server setup.
totally! Azure DevOps Server is perfect for what u need. it’s designed for on-prem setups, doing all the stuff like Git, builds, and Selenium tests. works great with internal URLs since it’s within ur network.
Azure DevOps Server is your best bet here. I’ve set up similar pipelines for internal web apps and it handles all this stuff. The big win is it runs on your own infrastructure, so hitting internal URLs for testing is easy - no tunneling headaches. You can set up build agents on your machines to run Selenium tests across different browsers and devices. Visual comparison testing needs extra tools like Percy or Applitools, but the main CI/CD stuff like automated deployments and approval gates comes built-in. Learning curve isn’t bad if you already know Git workflows.