Jeremy Lindsay

.net, Cake, Continuous Integration, Raspberry Pi 3, UWP

Deploy a UWP application to a Windows 10 device from the command line with Cake

I’ve wanted to improve my continuous integration process for building, testing and deploying UWP applications for a while. For these UWP apps, I’ve been tied to using VS2017 for build and deploy operations – and VS2017 is great, but I’ve felt restricted by the ‘point and click’ nature of these operations in VS2017.

Running automated tests for any .NET project is well documented, but until relatively recently I’ve not had a really good way to use a command line to:

  • build my UWP project and solution,
  • run tests for the solution,
  • build an .appxbundle file if the tests pass, and
  • and deploy the appxbundle to my Windows 10 device.

Trying to find out what’s going on under the hood is the kind of challenge that’s catnip for me, and this is my chance to share what I’ve learned with the community.

This is part of a series of posts about building and deploying different types of C# applications to different operating systems.

If you follow this blog, you’ll probably know I normally deploy apps to my Raspberry Pi 3, but the principles in here could be applied to other Windows 10 devices, such as an Xbox or Windows Phone.

Step 1 – Create the demo UWP and test projects.

I’ll keep the description of this bit quick – I’ll just use the UWP template in Visual Studio 2017 – it’s only a blank white screen but that’s ok for this demonstration.

screenshot.1500076564

I’ve also created an empty unit test project – again the function isn’t important for this demonstration, we just need a project with a runnable unit test.

screenshot.1500076646

I’ve written a simple dummy ‘test’, shown below – this is just created for the purposes of demonstrating how Cake can run a Unit Test project written using MSTest:

using Microsoft.VisualStudio.TestTools.UnitTesting;
 
namespace UnitTestProject2
{
    [TestClass]
    public class UnitTest1
    {
        [TestMethod]
        public void TestMethod1()
        {
            Assert.IsTrue(true);
        }
    }
}

Step 2: Let’s build our project and run the tests using Cake

Open a powershell prompt (I use the package manager console in VS2017) and navigate to the UWP project folder. Now get the Cake bootstrapper script and example Cake build file using the commands below:

Invoke-WebRequest http://cakebuild.net/download/bootstrapper/windows -OutFile build.ps1

Invoke-WebRequest https://raw.githubusercontent.com/cake-build/example/master/build.cake -OutFile build.cake

I edited the build.cake file to have the text below – this script cleans the binaries, restores the NuGet packages for the projects, builds them and runs the MSTests we created.

#tool nuget:?package=NUnit.ConsoleRunner&version=3.4.0
//////////////////////////////////////////////////////////////////////
// ARGUMENTS
//////////////////////////////////////////////////////////////////////

var target = Argument("target", "Default");
var configuration = Argument("configuration", "Release");

//////////////////////////////////////////////////////////////////////
// PREPARATION
//////////////////////////////////////////////////////////////////////

// Define directories.
var buildDir = Directory("./App3/bin") + Directory(configuration);

//////////////////////////////////////////////////////////////////////
// TASKS
//////////////////////////////////////////////////////////////////////

Task("Clean")
    .Does(() =>
{
    CleanDirectory(buildDir);
});

Task("Restore-NuGet-Packages")
    .IsDependentOn("Clean")
    .Does(() =>
{
    NuGetRestore("../App3.sln");
});

Task("Build")
    .IsDependentOn("Restore-NuGet-Packages")
    .Does(() =>
{
    if(IsRunningOnWindows())
    {
      // Use MSBuild
      MSBuild("../App3.sln", settings =>
        settings.SetConfiguration(configuration));
    }
    else
    {
      // Use XBuild
      XBuild("../App3.sln", settings =>
        settings.SetConfiguration(configuration));
    }
});

Task("Run-Unit-Tests")
    .IsDependentOn("Build")
    .Does(() =>
{
    MSTest("../**/bin/" + configuration + "/UnitTestProject2.dll");
});

//////////////////////////////////////////////////////////////////////
// TASK TARGETS
//////////////////////////////////////////////////////////////////////

Task("Default")
    .IsDependentOn("Run-Unit-Tests");

//////////////////////////////////////////////////////////////////////
// EXECUTION
//////////////////////////////////////////////////////////////////////

RunTarget(target);

Cake’s built in benchmarking shows the order in which the tasks are executed

Task Duration 
--------------------------------------------------
Clean                  00:00:00.0124995 
Restore-NuGet-Packages 00:00:03.5300892 
Build                  00:00:00.8472346 
Run-Unit-Tests         00:00:01.4200992 
Default                00:00:00.0016743 
--------------------------------------------------
Total:                 00:00:05.8115968

And obviously if any of these steps had failed (for example if a test failed), execution would stop at this point.

Step 3: Building an AppxBundle in Cake

If I want to build an appxbundle for a UWP project from the command line, I’d run the code below:

MSBuild ..\App3\App3.csproj /p:AppxBundle=Always /p:AppxBundlePlatforms="x86|arm" /Verbosity:minimal

There’s four arguments have told MSBuild about:

  • The location of the csproj file that I want to target
  • I want to build the AppxBundle
  • I want to target x86 and ARM platforms (ARM doesn’t work on its own)
  • And that I want to minimise the verbosity of the output logs.

I could use StartProcess to get Cake to run MSBuild in a task, but Cake already has methods for MSBuild (and many of its parameters) baked in. For those parameters which Cake doesn’t know about, it’s very easy to use the WithProperty fluent method to add the argument’s parameter and value. The code below shows how I can implement the command to build the AppxBundle in Cake’s C# syntax.

var applicationProjectFile = @"../App3/App3.csproj";
 
// ...

MSBuild(applicationProjectFile, new MSBuildSettings
    {
        Verbosity = Verbosity.Minimal
    }
    .WithProperty("AppxBundle", "Always")
    .WithProperty("AppxBundlePlatforms", "x86|arm")
);

After this code runs in a task, an AppxBundle is generated in a folder in the project with the path:

AppPackages\App3_1.0.0.0_Debug_Test\App3_1.0.0.0_x86_arm_Debug.appxbundle

The path and file name isn’t massively readable, and is also likely to change, so I wrote a short method to search the project directories and return the path of the first AppxBundle found.

private string FindFirstAppxBundlePath()
{
    var files = System.IO.Directory.GetFiles(@"..\", @"*.appxbundle", SearchOption.AllDirectories);
    
    if (files.Count() > 0)
    {
        return files[0];
    }
    else
    {
        throw new System.Exception("No appxbundle found");
    }
}

Now that I have the path to the AppxBundle, I’m ready to deploy it to my Windows device.

Step 4: Deploying the AppxBundle

Microsoft have provided a command line tool in the Windows 10 SDK for deploying AppxBundles – this tool is called WinAppDeployCmd. The syntax used to deploy an AppxBundle is:

WinAppDeployCmd install -file "\MyApp.appxbundle" -ip 192.168.0.1

It’s very straightforward to use a command line tool with Cake – I’ve blogged about this before and how to use StartProcess to call an executable which Cake’s context is aware about.

But what about command line tools which Cake doesn’t know about? It turns out that it’s easy to register tools within Cake’s context – you just need to know the path to the tool, and the code below shows how to add the UWP app deployment tool to the context:

Setup(context => {
    context.Tools.RegisterFile(@"C:\Program Files (x86)\Windows Kits\10\bin\x86\WinAppDeployCmd.exe");
});

If you don’t have this tool on your development machine or CI box, it might be because you don’t have the Windows 10 SDK installed.

So with this tool in Cake’s context, it’s very simple to create a dedicated task and pull the details of this tool out of context for use with StartProcess, as shown below.

Task("Deploy-Appxbundle")
	.IsDependentOn("Build-Appxbundle")
	.Does(() =>
{
    FilePath deployTool = Context.Tools.Resolve("WinAppDeployCmd.exe");
 
    Information(appxBundlePath);
 
    var processSuccessCode = StartProcess(deployTool, new ProcessSettings {
        Arguments = new ProcessArgumentBuilder()
            .Append(@"install")
            .Append(@"-file")
            .Append(appxBundlePath)
            .Append(@"-ip")
            .Append(raspberryPiIpAddress)
        });
 
    if (processSuccessCode != 0)
    {
        throw new Exception("Deploy-Appxbundle: UWP application was not successfully deployed");
    }
});

And now we can run our Cake script to automatically build and deploy the UWP application – I’ve pasted the benchmarking statistics from Cake below.

Task                     Duration
--------------------------------------------------
Clean                    00:00:00.0821960
Restore-NuGet-Packages   00:00:09.7173174
Build                    00:00:01.5771689
Run-Unit-Tests           00:00:03.2204312
Build-Appxbundle         00:01:09.6506712
Deploy-Appxbundle        00:02:13.8439852
--------------------------------------------------
Total:                   00:03:38.0917699

And to prove it was actually deployed, here’s a screenshot of the list of apps on my Raspberry Pi (from the device portal) before running the script…

screenshot.1500907026

…and here’s one from after – you can see the UWP app was successfully deployed.

screenshot.1500907690

I’ve uploaded my project’s build.cake file into a public gist – you can copy this and chan-ge it to suit your particular project (I haven’t uploaded a full UWP project because sometimes people have issues with the *.pfx file).

Wrapping up

I’ve found it’s possible to build and deploy a UWP app using the command line, and beyond that it’s possible to integrate the build and deployment process into a Cake script. So even though I still create my application in VS2017 – and I’ll probably keep using VS2017 – it means that I have a much more structured and automated integration process.


About me: I regularly post about .NET – if you’re interested, please follow me on Twitter, or have a look at my previous posts here. Thanks!

.net core, Cake, Raspberry Pi 3

Running a .NET Core 2 app on Raspbian Jessie, and deploying to the Pi with Cake

I’ve managed to get .NET Core apps running on Windows 10 IoT Core, and on Ubuntu 16.04 (and also Ubuntu MATE), but until recently I’d never tried with Raspbian. I’ve read a few posts from people saying that they couldn’t get it to work, and a couple of nights ago I decided to bite the bullet and give it a go.

There’s a good post here on how to get the .NET Core runtime on a Raspberry Pi running Raspbian, and as always, there are a few tricks to getting things running from scratch. To augment this, I’ve created a “Hello World” application template for .NET Core Raspberry Pi on NuGet which I think will make things easier for the community.

At a very high level, the steps to getting a .NET Core 2 app on Raspbian are:

  • Install Raspbian onto an SD card and insert into your Raspberry Pi 3.
  • Set up SSH and test that you can log into your Raspberry Pi from your development machine.
  • Install .NET Core 2 onto the Raspberry Pi.
  • On your development machine, install the Raspberry Pi C# template for dotnet core.
  • Create a new console application using this template.
  • Deploy this application to your Pi running Raspbian using Cake.

I’ll run through each of these steps below.

Install Raspbian onto an SD card and insert into your Raspberry Pi 3

There are already great explanations of how to install the Raspbian OS onto a Raspberry Pi 3 – many people who have a Pi know how to do this already and I don’t really want to just repeat a well understood process here – so I’ve just put some useful links below:

Set up SSH and test that you can log into your Raspberry Pi from your development machine

Once you’ve set up Raspbian and booted your Pi to the desktop, you’ll need to allow SSH connections. These aren’t enabled by default but it’s very easy to configure this.

First open the main Raspberry Pi menu on your desktop as shown in the image below, and open the Preferences sub-menu to reveal the “Raspberry Pi Configuration” option.

2017-07-21-102129_1824x984_scrot

Open the “Raspberry Pi Configuration” option screen, and click on the “Interfaces” tab. There’s lots of useful settings here, but the one we want to enable is SSH – click on the “Enabled” radio button as shown below, and then click on OK. SSH is now enabled on your Pi.

2017-07-21-102157_1824x984_scrot

You’ll need to find the IP address of your Raspberry Pi – I think the easiest way is to open a terminal on your Pi and type:

hostname -I

This tells me that my Pi has the IP address 192.168.1.111.

Now we need to check you can log in from your development machine. I personally find it’s easiest to use PuTTY to do this. I’ve blogged about installing PuTTY before so I won’t repeat it all here – but a few tips are:

This bit is really important: When you install PuTTY using the installer, it ships with a couple of other programs – pscp.exe and plink.exe, which live in the same directory as putty.exe. You’ll need both of these to deploy the code to the Pi. Pscp.exe allows you to copy files from Windows to Linux, and plink allows you to remotely change permissions on the files you deploy.

It also makes life easier to add the putty installation directory to your machine’s path. If Cake doesn’t know where pscp and plink are on your machine, then you’ll probably see and error about unknown executables during deployment.

So open PuTTY, enter the Pi’s IP address and select the “Connection Type” to be SSH, as shown below:

screenshot.1500803365

When you click Open, a command prompt should open where you can type the username and password for the Pi 3.

screenshot.1500805617

The default username and password combo for Raspbian is “pi” and “raspberry“, and you should change the default password as soon as possible.

Install .NET Core 2 onto the Raspberry Pi

There’s a straightforward set of commands that you can run through PuTTY to install .NET Core 2 onto your Pi running Raspbian – I’ve written them below:

# Update the Raspbian Jessie install
sudo apt-get -y update

# Install the packages necessary for .NET Core
sudo apt-get -y install libunwind8 gettext

# Download the nightly binaries for .NET Core 2
wget https://dotnetcli.blob.core.windows.net/dotnet/Runtime/release/2.0.0/dotnet-runtime-latest-linux-arm.tar.gz

# Create a folder to hold the .NET Core 2 installation
sudo mkdir /opt/dotnet

# Unzip the dotnet zip into the dotnet installation folder
sudo tar -xvf dotnet-runtime-latest-linux-arm.tar.gz -C /opt/dotnet

# set up a symbolic link to a directory on the path so we can call dotnet
sudo ln -s /opt/dotnet/dotnet /usr/local/bin

Now you can test this install by running the dotnet –info command to see the version installed on Raspbian.

screenshot.1500810837

On your development machine, install the Raspberry Pi C# template

Now that we have .NET Core 2 installed on our Raspbian, we can go back to our development machine to create an application to run on the Pi.

First, install the template for creating Raspberry Pi applications

 dotnet new -i RaspberryPi.Template::*

This will create a new template available to dotnet core – you can list them all with the command:

dotnet new --list

And in the screenshot below, you can see there is now a new template called “Empty .NET Core IoT Project”, highlighted in red below.

screenshot.1500806357

Create a new console application using this template

It’s really easy to create a new console application now – just run the command below (obviously my application is called “HelloRaspbian”, but yours could be something different):

dotnet new coreiot -n HelloRaspbian

When you browse to this new application folder using your preferred development tool (mine is VSCode), you’ll see some files – we need to make a couple of changes.

First, run the command below to pull down the latest Cake build PowerShell file:

Invoke-WebRequest http://cakebuild.net/download/bootstrapper/windows -OutFile build.ps1

This command is also in the README.txt file which comes packaged with the application.

Now, open the build.cake file and you’ll see some defaults at the top of the file:

///////////////////////////////////////////////////////////////////////
// ARGUMENTS (WITH DEFAULT PARAMETERS FOR LINUX (Ubuntu 16.04, Raspbian Jessie, etc)
///////////////////////////////////////////////////////////////////////
var runtime = Argument("runtime", "linux-arm");
var destinationIp = Argument("destinationPi", "<>");
var destinationDirectory = Argument("destinationDirectory", @"<>");
var username = Argument("username", "<>");
var executableName = Argument("executableName", "HelloRaspbian");

Replaced those placeholders with the correct environment variables – I’ve shown my own settings below:

///////////////////////////////////////////////////////////////////////
// ARGUMENTS (WITH DEFAULT PARAMETERS FOR LINUX (Ubuntu 16.04, Raspbian Jessie, etc)
///////////////////////////////////////////////////////////////////////
var runtime = Argument("runtime", "linux-arm");
var destinationIp = Argument("destinationPi", "192.168.1.111");
var destinationDirectory = Argument("destinationDirectory", @"/home/pi/DotNetConsoleApps/RaspbianTest");
var username = Argument("username", "pi");
var executableName = Argument("executableName", "HelloRaspbian");

I’ve created a folder on the Pi to deploy my application to, using the command below at the PuTTY SSH prompt at my home directory (/home/pi/).

mkdir -p DotNetConsoleApps/RaspbianTest

Deploy this application to your Pi running Raspbian using Cake

Once I’ve replaced the placeholders in my Cake file, the only thing left to do is run the build.ps1 file from a PowerShell prompt.

screenshot.1500811107

To test this, go back to the PuTTY SSH prompt and navigate to your home directory and run:

./DotNetConsoleApps/RaspbianTest/HelloRaspbian

And you’ll get a text output saying “Hello Internet of Things!”

screenshot.1500810451

Wrapping up

I hope this post is useful to anyone trying to get a C# console application running on Raspbian. I think Raspbian is the default OS for Raspberry Pi users, so this should open up many development opportunities. My Raspberry Pi template makes creating the default console application easier, and Cake is a brilliant way to orchestrate the deployment process (rather than dragging and dropping files using tools like WinSCP, and having to change file permission manually). I’ll be blogging more on the future on deploying IoT applications to this platform.

I’ve written a few posts now about how to deploy C# Raspberry Pi applications to Windows 10 IoT Core, Ubuntu, and Raspbian (all using Cake as the orchestration tool) – next time I’ll write about how to use Cake to automatically build a UWP AppxBundle and deploy that AppxBundle to Windows 10 IoT Core.


About me: I regularly post about .NET – if you’re interested, please follow me on Twitter, or have a look at my previous posts here. Thanks!

.net, Cake, Performance

Calling a custom executable from Cake using StartProcess and ProcessSettings

I’ve previously written about how I’ve used Cake to orchestrate my build and deployment processes, and write the code for these processes in C# rather than PowerShell. This time I’ll write about how I’ve improved the speed of my deployment process by using custom tools which aren’t yet built into Cake.

Some background about my deployment process

A common part of a deployment process is copying files repetitively from a source to a destination, and Cake provides a good way to do this – the CopyFiles static method.

To use this, we just need to specify the source directory, the remote destination directory, and plug these in as parameters. I’ve written some sample code below showing how a “Deploy” task might move an application from a “publish” directory to a remote machine.

Task("Deploy")
    .Does(() =>
    {
	// The files I want to copy are in the publish directory - I use the
	// wildcard character to specify that I want to copy everything
	var source = @".\publish\*";
 
	// The destination is on my local network and accessible on the path below
	var destination = @"\\192.168.1.125\c$\ConsoleApps\DeployedApplication";
 
	// Now just plug in the source, destination
	// The boolean parameter ensures we preserve the folder structure
	CopyFiles(source, destination, true);
    });

This works well, but it also copies across every file, every time – it doesn’t matter whether the file has changed or not – and this the slowest part of my deployment process.  I would prefer to mirror my source and destination files, and only copy across files that have changed. This would speed up deployments across my local network.

Using RoboCopy to mirror directory structures

Microsoft have created a command line file copy utility which allows me to copy or mirror directory structures called RoboCopy (Robust File Copy) – it’ll only copy the files/directories that have changed, and this sounds like exactly what I need.

In copying files from source to destination, I’ve chosen the word “mirror” because RoboCopy needs to be passed a switch “/MIR”, which is short for mirror. From Microsoft’s TechNet documentation:

/MIR is an option to ROBOCOPY where you mirror a directory tree with all the subfolders including the empty directories and you purge files and folders on the destination server that no longer exists in source.

The command I’d need to mirror files has the format:

robocopy /MIR source_directory destination_directory

And to copy from my source directory

".\publish\"

to the destination on the C drive of a machine with IP address 192.168.1.125:

"\ConsoleApps\DeployedApplication"

I just need to plug these values as arguments to the robocopy executable, as shown below:

robocopy /MIR ".\publish\" "\\192.168.1.125\c$\ConsoleApps\DeployedApplication"

But how can I use RoboCopy with Cake?

Turns out it’s quite easy with a few things in Cake which can help us.

  • We can use the StartProcess method – I can pass in executable that I want to run (i.e. robocopy.exe), and I can also pass in the arguments for this executable.

I don’t need to pass in the full path to the executable because robocopy.exe is in a folder which is in my machine’s path, i.e. C:\Windows\System32

  • I can also clean up my code a little by keeping this code in its own method in the Cake.build file.
private void MirrorFiles(string source, string destination)
{
    StartProcess("robocopy", new ProcessSettings {
        Arguments = new ProcessArgumentBuilder()
            .Append(@"/MIR")
            .Append(source)
            .Append(destination)
        }
    );
}

Now I can adjust the Task shown previously (i.e. “Deploy”) to use this method instead:

Task("Deploy")
    .Does(() =>
    {
	// The files I want to copy are in the publish directory
	var source = @".\publish\";
 
	// The destination is on my local network and accessible on the path below
	var destination = @"\\192.168.1.125\c$\ConsoleApps\DeployedApplication";
 
	// Now just plug in the source, destination
	MirrorFiles(source, destination);
    }

What practical difference does this make?

In my application’s Cake build script, there’s very little difference – a new method to mirror files, and a slight change to a task to copy (or mirror) files across a network.

But the real advantage comes when we look at the time taken to deploy my application.

Another nice feature of Cake is that it writes out the time taken by each task which helps with performance benchmarking.

I’ve pasted the timings for each stage of my original deployment process below for when I just copy files instead of using robocopy:

Task                  Duration 
--------------------------------------------------
Clean                 00:00:00.2378497 
Restore               00:00:03.9107662 
Build                 00:00:05.2128133 
Publish               00:00:08.0715728 
Deploy                00:00:43.1527382 
Default               00:00:00.0021827 
--------------------------------------------------
Total:                00:01:00.5879229

Notice it took 43 seconds to deploy my application’s files from source to destination – approximately 75% of the total time. And everytime I change my application and re-deploy, the time taken to carry out this operation is approximately the same, copying across files that have changed and also those that haven’t changed.

Let’s change the script to mirror files using robocopy (i.e. only copy across files that have changed since the last build) rather than just copying all files – I’ve pasted the new timings below:

Task                  Duration 
--------------------------------------------------
Clean                 00:00:00.2661543 
Restore               00:00:02.7529030 
Build                 00:00:04.7478403 
Publish               00:00:06.3981560 
Deploy                00:00:00.6685282 
Default               00:00:00.0033186 
--------------------------------------------------
Total:                00:00:14.8369004

It has gone from copying every file in 43 seconds to just copying 5 files that changed in 0.66 seconds – this is a huge difference for me, making it much quicker and more convenient for me to make an application, change, build and deploy to my test rig.

Wrapping up

In this post I wanted to share with the community how flexible Cake is by demonstrating how I’ve used Robocopy to speed my up deployments.

  • I’ve been able to switch out Cake’s built in copying feature, and instead use a local executable (that isn’t a core part of Cake or an addin) by passing it to the StartProcess method.
  • I’ve been able to write a private method in my C# Cake.build script to keep the code clean.
  • Finally I’ve been able to use Cake’s default output to benchmark performance before and after my change.

Being able to extend the core features in Cake with StartProcess is really powerful – it’s not quite as re-useable as building a dedicated add-in, but it can still allow us to quickly customise and optimise build scripts.


About me: I regularly post about .NET – if you’re interested, please follow me on Twitter, or have a look at my previous posts here. Thanks!