.net, C# tip, UWP, Visual Studio

Testing your Windows App with Appium in Windows 10 and Visual Studio 2015

At Connect(); // 2016, Scott Hanselman’s keynote include a short description of a tool called Appium (presented by Stacey Doerr). This tool allows you to create and automate UI tests for Windows Apps – not just UWP apps, but basically any app which runs on your Windows machine. Automated UI testing is definitely something that I’ve missed when moving from web development to UWP development, so I was quite excited to find out there’s a project would help fill this gap.

As is often the case, getting started with new things is tricky –  when I follow present instructions from Microsoft, I found some errors occurred. That’s likely to be caused by my development machine set up – but you might hit the same issue. In this post, I’ll describe the process I followed to get Appium working, and I’ll also document the error messages I found on the way.

I hope that this blog post becomes irrelevant soon and that this isn’t an issue affecting a lot of people.

Installing and Troubleshooting Appium

Step 1 – Install Node.js

Install Node.js from here.

Step 2 – Open a PowerShell prompt as Administrator, and install Appium

From an elevated PowerShell prompt, run the command:

npm install –g appium

When I ran this command, the following warnings were printed to the screen – however I don’t think they’re anything to worry about:

npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@^1.0.12(node_modules\appium\node_modules\fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.0.15: wanted {"os":"darwin","arch":"any"} (current: {"os":"win32","arch":"x64"})

 Step 3 – From an elevated PowerShell prompt, run Appium

From an elevated PowerShell prompt, run the command:

appium

After a few seconds, the following text is printed to the screen.

Welcome to Appium v1.6.0
Appium REST http interface listener started on 0.0.0.0:4723

At this point I tried to run the tests in the Sample Calculator App provided by Appium on GitHub – found here. I used Visual Studio to run these tests, but found all 5 tests failed, and the following error was printed to the PowerShell prompt.

[Appium] Creating new WindowsDriver session
[Appium] Capabilities:
[Appium]   app: 'Microsoft.WindowsCalculator_8wekyb3d8bbwe!App'
[Appium]   platformName: 'Windows'
[Appium]   deviceName: 'WindowsPC'
[BaseDriver] The following capabilities were provided, but are not recognized by appium: app.
[BaseDriver] Session created with session id: dcfce8e7-9615-4da1-afc5-9fa2097673ed
[WinAppDriver] Verifying WinAppDriver is installed with correct checksum
[debug] [WinAppDriver] Deleting WinAppDriver session
[MJSONWP] Encountered internal error running command: 
	Error: Could not verify WinAppDriver install; re-run install
    at WinAppDriver.start$ (lib/winappdriver.js:35:13)
    at tryCatch (C:\Users\Jeremy\AppData\Roaming\npm\node_modules\appium\node_modules\babel-runtime\regenerator\runtime.js:67:40)
    at GeneratorFunctionPrototype.invoke [as _invoke] (C:\Users\Jeremy\AppData\Roaming\npm\node_modules\appium\node_modules\babel-runtime\regenerator\runtime.js:315:22)
    at GeneratorFunctionPrototype.prototype.(anonymous function) [as next] (C:\Users\Jeremy\AppData\Roaming\npm\node_modules\appium\node_modules\babel-runtime\regenerator\runtime.js:100:21)
    at GeneratorFunctionPrototype.invoke (C:\Users\Jeremy\AppData\Roaming\npm\node_modules\appium\node_modules\babel-runtime\regenerator\runtime.js:136:37)

For some reason on my machine, the WinAppDriver has not been installed correctly during the installation of Appium.

Step 4 – Manually install v0.5-beta of the WinAppDriver

This is pretty easy to fix – we can just grab the WinAppDriver installer from its GitHub site. But for version 1.6.0 of Appium, I found that it was important to select the correct version of WinAppDriver – specifically v0.5-beta, released on September 16 2016. Higher versions did not work for me with Appium v1.6.0.

Step 5 – Restart Appium from an elevated PowerShell prompt

Installed WinAppDriver v0.5-beta was a pretty simple process, I just double clicked on the file and selected all the default options. Then I repeated Step 3 and restarted Appium from the elevated PowerShell prompt. Again, after a few seconds, the same message appeared.

Welcome to Appium v1.6.0
Appium REST http interface listener started on 0.0.0.0:4723

This time, when I ran the tests for the Sample Calculator App from GitHub they all passed. Also, the PowerShell prompt showed no errors – instead of saying that it couldn’t verify the WinAppDriver install, I got the message below:

[WinAppDriver] Verifying WinAppDriver is installed with correct checksum
[debug] [WinAppDriver] WinAppDriver changed state to 'starting'
[WinAppDriver] Killing any old WinAppDrivers, running: FOR /F "usebackq tokens=5" %a in (`netstat -nao ^| findstr /R /C:"4823 "`) do (FOR /F "usebackq" %b in (`TASKLIST /FI "PID eq %a" ^| findstr /I winappdriver.exe`) do (IF NOT %b=="" TASKKILL /F /PID %a))
[WinAppDriver] No old WinAppDrivers seemed to exist
[WinAppDriver] Spawning winappdriver with: undefined 4823/wd/hub
[WinAppDriver] [STDOUT] Windows Application Driver Beta listening for requests at: http://127.0.0.1:4823/wd/hub
[debug] [WinAppDriver] WinAppDriver changed state to 'online'

I was able to see the standard Windows Calculator appear, and a series of automated UI tests were carried out on the app.

How do I get automation information for these apps?

When you look at the Sample Calculator App and the basic scenarios for testing, you’ll see some code with some strange constant values – such as the in the snippet below.

DesiredCapabilities appCapabilities = new DesiredCapabilities();
appCapabilities.SetCapability("app""Microsoft.WindowsCalculator_8wekyb3d8bbwe!App");
appCapabilities.SetCapability("platformName""Windows");
appCapabilities.SetCapability("deviceName""WindowsPC");
CalculatorSession = new RemoteWebDriver(new Uri(WindowsApplicationDriverUrl), appCapabilities);
Assert.IsNotNull(CalculatorSession);
CalculatorSession.Manage().Timeouts().ImplicitlyWait(TimeSpan.FromSeconds(2));
 
// Make sure we're in standard mode
CalculatorSession.FindElementByXPath("//Button[starts-with(@Name, \"Menu\")]").Click();
OriginalCalculatorMode = CalculatorSession.FindElementByXPath("//List[@AutomationId=\"FlyoutNav\"]//ListItem[@IsSelected=\"True\"]").Text;
CalculatorSession.FindElementByXPath("//ListItem[@Name=\"Standard Calculator\"]").Click();

The code above shows that the test looks for an app with identifier:

“Microsoft.WindowsCalculator_8wekyb3d8bbwe!App”

It’s obvious this is for the Microsoft Windows Calculator app – but most of us won’t recognise the strange looking code appended at the end of this string. This is the application’s automation identifier.

In order to locate this identifier, start the standard Calculator application from within Windows (open a Run prompt and enter “Calc”).

There’s a tool shipped with the Visual Studio 2015 called “Inspect” – it should normally be available at the location:

C:\Program Files (x86)\Windows Kits\10\bin\x86

Start Inspect.exe from the directory specified above. When you run the Inspect application, you’ll get a huge amount of information about the objects currently being managed by Windows 10 – when you drill into the tree view on the left side of the screen to see running applications, you can select “Calculator”, and on the right hand side a value for “AutomationId” will be shown – I’ve highlighted it in red below.

inspect

The other items – menus, buttons, and display elements – can also be obtained from this view when you select the corresponding menu, button or display elements – a particularly useful property is “Legacy|Accessible:Name” when identifying elements using the FindElementByXPath method.

Conclusion

I hope this post is useful for anyone interested in automating UI tests for Windows App, and particularly if you’re having problems getting Appium to work. There’s some really useful sample apps on GitHub from Appium – I found coding for Windows Apps to be a bit confusing to start off with, but with a few key bits of information – like using the Inspect tool  – you can start to tie together how the sample apps were written and how they work. This should get you up and running with your own automated UI test code. I’m excited about the opportunities this tool gives me to improve the quality of my applications – I hope this post helps you get started too.