I was once asked how to build and maintain an open source project. A successful open source project not only solves a particular problem well, but also follows best practices in order to have a high code quality.
Last year December, I published my open source project, go-onedrive, a Golang client library for accessing the Microsoft OneDrive REST API.
Last month, go-onedrive successfully received 28 stars, which is the highest number of stars I received on GitHub. Also, there are new contributors joining the project. Meanwhile, I am also honoured to have received code review feedback about the go-onedrive project from Brian Ketelsen, Principal Cloud Advocate in Microsoft. Even though it’s not a great achievement, but I’m still satisfied because it’s the first time my projects on GitHub received so many eyeballs.
Hence, in this article, I will talk about maintaining this open source project, go-onedrive.
Changes Based on Feedback Received
Not all contributors will submit a pull request for their feedback. They may just submit an issue on GitHub and we have to work on that. Sometimes, they may simply message us to let us know their thoughts. Hence, we need to make the changes according to the valid feedback and comments.
Brian Ketelsen commented about the Golang file naming in the project. According to Brian, Golang source file names should be all lower case (same with package names and directory names) because some file systems are case-preserving. Hence, there will be cases where both myFile.go and myfile.go can be two different files in the same directory.
I am glad to have two contributors, @nullaus and @ti55987, joining from another part of the world. They not only provided great improvement to the library by introducing new functionalities, but also followed the existing coding style in the project. So, I’d like to take this opportunity to thank them.
As the owner of the library, I’m aware that how contributors communicate can have a significant impact on the success of the Pull Request. I respect the time taken by the contributors to improve the codes. Hence, if sometimes they’re not able to correct the Pull Request based on the agreement, then I will try to make the changes on their behalf.
In the most recent change, one of the contributors submitted a new service for Shared Link in OneDrive. I noticed that hardcoded strings are used in the new code. This makes the client to be able to send any valid string and there will be no errors. Hence, I changed the code to use proper enums.
Publish Package with VS Code
Once all the necessary changes are done, it’s time to publish a new package for the users on Pkg.go.dev.
Firstly, we use the following command to tidy up the dependencies in go-onedrive library. This removes any dependencies the module might have accumulated that are no longer necessary.
go mod tidy
Now we can proceed to publish our module by first creating a Release of it on the GitHub. To do so, we open the Command Palette (Ctrl + Shift + P) and choose Git: Create Tag.
Finally, we need to push the tag to the remote server. To do so, we simply open the Command Palette again and choose Git: Push (Follow Tags). Then we shall see our new package now available on GitHub Releases page, as shown below.
Now, how to we update the Pkg.go.dev? Now if we navigate to the v.1.1.0 web page on Pkg.go.dev, we will be told that it cannot yet be found, as shown below.
This is because new packages are added to Pkg.go.dev through the scheduled monitoring of the Go Module Index. Hence, if we don’t see our packages on the website, we can manually make a request to fetch our packages by following the instructions here. The following screenshot shows how I make a request to proxy.golang.org, where the data of the Pkg.go.dev is downloaded from. In my case, after doing that, in less than 10 minutes, the package will be made available on the Pkg.go.dev website.
With this new release, the users can simply update to use the latest version of go-onedrive in the go.mod file of their projects. After running the go mod tidy command, then they are good to go to use the new go-onedrive library.
Software testing makes sure that the software is doing exactly what it is supposed to do and it also points out all the problems and errors found in the software. Hence, involving testing as early as possible and as frequent as possible is a key to build a quality software which will be accepted by the customers or the clients.
There are many topics I’d love to cover about testing. However, in this article, I will only focus on my recent learning about setting up automated GUI testing for my UWP program on Windows 10.
Appium
One of the key challenges in testing UWP app is to do the GUI testing. In the early stage, it’s possible to do that manually by clicking around the app. However, as the app grows larger, testing it manually is very time consuming. After sharing my thoughts with my senior Riza Marhaban, he introduced me a test automation framework called Appium.
Together with Windows App Driver which enables Appium by using new APIs added in Windows 10 Anniversary Edition, we can use them to do GUI test automation on Windows apps. The following video demonstrates the results of GUI testing with Appium in my demo project Lunar.Paint.Uwp.
Here, I will list down those great tutorials about automated GUI testing of UWP apps using Appium which are ranked top in Google Search:
Some of them were written about four years ago when Barack Obama was still the President of the USA. In addition, none of them continues the story with DevOps. Hence, my article here will talk about GUI testing with Appium from the beginning of a new UWP project until it gets built on Azure DevOps.
Getting Started with Windows Template Studio for UWP
Here, I will start a new UWP project using Windows Template Studio.
There is one section in the project configuration called Testing, as shown in the screenshot above. In order to use Appium, we need to add the testing with Win App Driver feature. After that, we shall see a Test Project suffixed with “Tests.WinAppDriver” being added.
By default, the test project has already come with necessary NuGet packages, such as Appium.WebDriver and MSTest.
Writing GUI Test Cases: Setup
The test project comes with a file called BasicTest.cs. In the file, there are two important variables, i.e. WindowsApplicationDriverUrl and AppToLaunch.
The WindowsApplicationDriverUrl is pointing to the server of WinAppDriver which we will install later. Normally we don’t need to change it as the default value will be “http://127.0.0.1:4723”.
The AppToLaunch variable is the one we need to change. Here, we need to replace the part before “!App” with the Package Family Name, which can be found in the Packaging tab of the UWP app manifest, as shown in the screenshot below.
Take note that there is a line of comment right above the AppToLaunch variable. It says, “The app must also be installed (or launched for debugging) for WinAppDriver to be able to launch it.” This is a very important line. It means when we are testing locally, we need to make sure the latest of our UWP app is deployed locally. Also, it means that the UWP app needs to be available on the Build Agent which we will talk about in later part of this article.
To access this button, in the test code, we can do like the following.
var welcomeScreenEnterButton = AppSession.FindElementByAccessibilityId("WelcomeScreenEnterButton");
Of course, if we want to have an AccessibilityId which is different from the Name of the XAML control (or the XAML control doesn’t have a Name), then we can specify the AccessibilityId in the XAML directly as follows.
Then to access this button, in the test code, we need to use EnterButton instead.
var welcomeScreenEnterButton = AppSession.FindElementByAccessibilityId("EnterButton");
Writing GUI Test Cases: AccessibilityName
The method above works well with XAML controls which are having simple text as the content. If the content property is not string, for example if the XAML control is a Grid that consists of many other XAML controls or the XAML control is a custom user control, then Appium will fail to detect the control with the AccessibilityId with the following exception message “OpenQA.Selenium.WebDriverException: An element could not be located on the page using the given search parameters”.
Thanks to GeorgiG from UltraPay, there is a solution to this problem. As GeorgiG pointed out in his post on Stack Overflow, the workaround is to overwrite the AutomationProperties.Name with a non-empty string value, such as “=”.
Hence, in my demo project, I have the following code for a Grid.
Then in the test cases, I can easily access the Grid with the following code.
var welcomeScreen = AppSession.FindElementByAccessibilityId("WelcomeScreen");
Writing GUI Test Cases: Inspect Tool
The methods listed out above work fine for the XAML controls in our program. How about for the prompt? For example, when user clicks on the “Open” button and an “Open” window is prompted. How do we instruct Appium to react to that?
Next, we can mouse over the Open prompt to find out the AccessibilityId of the GUI element that we need to access. For example, the AccessibilityId of the area where we key in the file name is 1148, as shown in the screenshot below.
This explains why in the test cases, we have the following code to access it.
var openFileText = AppSession.FindElementByAccessibilityId("1148");
Now, we have done our software test locally. How do we make the testing to be part of our build pipeline on Azure DevOps?
This turns out to be quite a complicated setup. Here, I setup the build pipeline based on the .NET Desktop pipeline in the template.
Next, we need to make sure the pipeline is building our solution with VS2019 on Windows 10 at least. Otherwise, we will receive the error “Error CS0234: The type or namespace name ‘ApplicationModel’ does not exist in the namespace ‘Windows’ (are you missing an assembly reference?)” in the build pipeline.
Now, if we queue our pipeline again, we will encounter a new error which states that “Error APPX0104: Certificate file ‘xxxxxx.pfx’ not found.” This is because for UWP app, we need to package our app with a cert. However, by default, the cert will not be committed to the Git repository. Hence, there is no such cert in the build pipeline.
To solve this problem, we need to first head to the Library of the Pipelines and add the following Variable Group.
Take note that, the required cert is now still not yet available on the pipeline. Hence, we need to upload the cert as one of the Secured Files in the Library as well.
So, how to we move this cert from the Library to the build pipeline? We need the following task.
This is not enough because the task will only copy the cert to a temporary storage on the build agent. However, when the agent tries to build, it will still be searching for the cert in the project folder of our UWP app, i.e. Lunar.Paint.Uwp.
Hence, as shown in the screenshot above, we have two more powershell script tasks to do a little more work.
With these three new tasks, the build task should be executed correctly.
Here, my BuildPlatform is x64 and the BuildConfiguration is set to release. Also in the MSBuild Arguments, I specify the PackageCertificatePassword because otherwise it will throw an error in the build process saying “[error]C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\MSBuild\Microsoft\VisualStudio\v16.0\AppxPackage\Microsoft.AppXPackage.Targets(828,5): Error : Certificate could not be opened: Lunar.Paint.Uwp_TemporaryKey.pfx.”
Introduction of WinAppDriver to the Build Pipeline
Okay, so how do we run the test cases above on Azure DevOps?
Actually, it only requires the following five steps as highlighted in the following screenshot.
Firstly, we need to start the WinAppDriver.
Secondly, we need to introduce two tasks after it to execute some PowerShell scripts. Before showing what they are doing, we need to recall one thing.
Remember the one line of comment above the AppToLaunch variable in our test project? It says, “The app must also be installed (or launched for debugging) for WinAppDriver to be able to launch it.” Hence, we must install the UWP app using the files in AppxPackages generated by the Build task. This is what the two Powershell tasks are doing.
The first Powershell task is to import the cert to the store.
The second task, as shown in the following screenshot, is to install the UWP app using Add-AppDevPackage.ps1. Take note that here we need to do SilentContinue else it will wait for user interaction and cause the pipeline to be stuck.
At the point of writing this article, the Windows Template Studio automatically sets the Targeting of the UWP app to be “Windows 10, version 2004 (10.0; Build 19041)”. However, the Azure DevOps pipeline is still not yet updated to Windows 10 v2004, so we should lower the Target Version to be v1903 and minimum version to be v1809 in order to have the project built successfully on the Azure DevOps pipeline.
Thirdly, we will need the test with VsTest. This task exists in the default template and nothing needs to be changed here.
Fourthly, we need to stop the WinAppDriver.
That’s all. Now when the Build Pipeline is triggered, we can see the GUI test cases are being run as well.
In addition, Azure DevOps will also give us a nice test report for each of our builds, as shown in the following the screenshot.
Conclusion: To Be Continued
Well, this is actually just the beginning of testing journey. I will continue to learn more about software testing especially in the DevOps part and share with you all in the future.
Feel free to leave a comment here to share with other readers and me about your thoughts. Thank you!
I thus decided to document down the steps on how I approach this problem to help the developers out there who are facing the same issue.
Step 1: Setup the UWP Class Library Project
In this post, the new project we create is called “RedButton” which is meant to provide red button in different style. Yup, it’s important to make our demo as simple as possible so that we can focus on the key thing, i.e. generating the NuGet package.
Before we proceed with the new project, we need to configure the project Build properties, as shown in the following screenshot, to enable the XML Documentation file. This will make a XML file generated in the output folder which we need to use later.
Now, we can proceed to add a new user control, called SimpleButton.xaml.
So this marks the end of the steps where we create an UWP user control where we need to package it with NuGet.
Step 2: Install NuGet.exe
Before we proceed, please make sure we have nuget installed. To verify that, just run the following command in the PowerShell.
> nuget
If it is installed, it will show something as follows.
If it is not installed, please download the latest recommended nuget.exe from the NuGet website. After that, add the path the the folder containing the nuget.exe file in the PATH environment variable.
Now we need to navigate in PowerShell to the project root folder, i.e. the folder containing RedButton.csproj. Then, we need to key in the following command to run it.
nuget spec
If the command is successfully executed, there will be a message saying “Created ‘RedButton.nuspec’ successfully.”
Now, we can open the RedButton.nuspec in Visual Studio. Take note that the file itself is not yet included in the solution. So we need to make sure we have enabled the “Show All Files” in the Solution Explorer to see the NuSpec file.
In order to automate the publish of our package to the NuGet, we will need to implement Continuous Integration. Here, the tools that we will be using are GitHub and Azure DevOps.
After committing our codes to GitHub, we will proceed to setup the Azure DevOps pipeline.
Firstly, we can make use of the UWP build template available on the Azure DevOps.
At the time I am writing this port, there are 5 tasks in the agent job:
Use NuGet 4.4.1;
NuGet restore **\*.sln;
Build solution **\*.sln;
Publish artifact: drop;
Deploy to Visual Studio App Center.
Take note that the NuGet version by default is 4.4.1, which is rather old and new things like <license> element in our NuSpec file will not be accepted. Hence, to solve this problem, we can refer to the list of available NuGet version at https://dist.nuget.org/tools.json.
At the time this post is written in April 2020, the latest released and blessed NuGet version is 5.5.1. So we will change it to 5.5.1. Please update it to any other latest number according to your needs and the time you read this post.
After that, for the second task, we need to update its “Path to solution, packages.config, or project.json” to be pointing at “RedButton\RedButton.csproj”.
Similarly, for the “Solution” field in the third task, we also need to point it to the “RedButton\RedButton.csproj”. Previously I pointed it to the RedButton folder which contains the .sln file, it will not work even though it is asking for “Solution”.
On the third task, we also need to update the “Visual Studio Version” to be “Visual Studio 2019” (or any other suitable VS for our UWP app). It seems to be not working when I was using VS2017. After that, I also updated the field “Configuration” to Release because by default it’s set to Debug and publishing Debug mode to public is not a good idea. I have also enabled “Clean” build to avoid incremental build which is not useful in my case. Finally, I changed the MSBuild Architecture to use MSBuild x64. The update of the third task is reflected on the screenshot below.
For the forth task, similarly, we also set its “Path to publish” to “RedButton”. Ah-ha, this time we are using the solution folder itself. By right, this fourth task is not relevant if we just publish our UWP class library to a NuGet server. I still keep it and set its path to publish to be the solution so that later I can view the build results of previous tasks by downloading it from the Artifact of the build.
I’d recommend to have this step because sometimes your built output folder structure may not be the same as what I have here depends on how you structure your project. Hence, based on the output folder, you many need to make some adjustments to the paths used in the Azure DevOps.
By default, the fifth task is disabled. Since we are also not going to upload our UWP app to VS App Center, so we will not work on that fifth task. Instead, we are going to add three new tasks.
Firstly, we will introduce the NuGet pack task as the sixth task. The task in the template is by default called “NuGet restore” but we can change the command from “restore” to “pack” after adding the task, as shown in the following screenshot.
There is one more important information that we need to provide for NuGet packaging. It’s the version of our package. We can either do it manually or automatically. It’s better to automate the versioning else we may screw it up anytime down the road.
There are several ways to do auto versioning. Here, we will go with the “Date and Time” method, as shown in the screenshot below.
This way of versioning will append datetime at the end of our version automatically. Doing so allows us to quickly test the release on the NuGet server instead of spending additional time on updating the version number. Of course, doing so means that the releases will be categorized as pre-released which users cannot see on Visual Studio unless they check the “Include prerelease” checkbox.
Secondly, if you are also curious about the package generated by the sixth task above, you can add a task similar to the fourth task, i.e. publish the package as artifact for download later. Here, the “Path to publish” will be “$(Build.ArtifactStagingDirectory)”.
Since a NuGet package is just a zipped file, we can change its extension from .nupkg to .zip to view its content on Windows. I did the similar on MacOS but it didn’t work, so I guess it is possible on Windows only.
Thirdly, we need to introduce the NuGet push task after the task above to be the eighth task. Here, we need to set its “Path to NuGet package(s) to publish” to “$(Build.ArtifactStagingDirectory)/*.nupkg”.
Then, we need to specify that we will publish our package to the nuget.org server which is an external NuGet server. By clicking on the “+ New” button, we can then see the following popup.
With this NuGet push task setup successfully, we can proceed to save and run this pipeline.
After the tasks are all executed smoothly and successfully, we shall see our pre-released NuGet package available on the nuget.org website. Note that it requires an amount of time to do package validating before public can use the new package.
This is not a happy ending yet. In fact, if we try this NuGet package, we will see the following error which states that it “cannot locate resource from ‘ms-appx:///RedButton/SimpleButton.xaml’.”
To do so, we have to introduce a new task right after the third task, which is to copy the XBF file from obj folder to the Release folder in the bin folder, as shown in the following screenshot.
Step 5: Targeting Framework
Before we make our NuGet package to work, we need to specify the framework it is targeting at. To do so, we need to introduce the <files> to our NuSpec.
So, the NuSpec should look something as follows now.
Now with this, we can use our prerelease version of our UWP control in another UWP project through NuGet.
Step 6: Platform Release Issues
There will be time which requires us to specify the Platform to be, for example, x64 in the third task of the Azure DevOps pipeline above. That will result in putting the Release folder in both obj and bin to be moved to obj\x64 and bin\x64, respectively. This will undoubtedly make the entire build pipeline fails.
Hence we need to update the paths in the Copy File task (the fourth task) and add another Copy File task to move the Release folder back to be directly under the bin directory. Without doing this, the nuget pack task will fail as well.
Step 7: Dependencies
If our control relies on the other NuGet packages, for example Telerik.UI.for.UniversalWindowsPlatform, then we have to include them too inside the <metadata> in the NuSpec, as shown below.
Okay, after we are happy with the prerelease of our NuGet package, we can officially release our package on the NuGet server. To do so, simply turn off the automatic package versioning on Azure DevOps, as shown in the screenshot below.
With this step, now when we run the pipeline again, it will generate a new release of the package without the prerelease label. The version number will follow the version we provide in the NuSpec file.
Journey: 3 Days 3 Nights
The motivation of this project comes from a problem I encounter at workplace because our UWP class library could not be used whenever we consumed it as a NuGet package. This was also the time when Google and StackOverflow didn’t have proper answers on this.
Hence, it took me 1 working day and 2 days during weekend to research and come up with the steps above. Hopefully with my post, people around the world can easily pickup this skill without wasting too much effort and time.
Finally, I’d like to thank my senior Riza Marhaban for encouraging me in this tough period. Step 7 above is actually his idea as well. In addition, I have friend encouraging me online too in this tough Covid-19 lockdown. Thanks to all of them, I manage to learn something new in this weekend.
Learning about containers is essentially a huge topic but for beginners, there needs to be something small to help them get started. Hence, in this article, we will focus only on the key concepts of containers and the steps to containerize the program and deploy it to Azure Web App.
As explained in the book “How to Containerize Your Go Code“, containers isolates an application so that container thinks it’s running on its own private machine. So, a container is similar to a VM but it uses the OS kernel on the host rather than having its own.
The Dockerfile starts with a FROM command that specifies the starting point for the image to build. For our project, we don’t have any dependencies, so we can start from scratch. So what is scratch? Scratch is basically a special Docker image that is empty (0B). That means there will be nothing else in our container later aside from what we put in with the rest of the Dockerfile.
The reason why we build from scratch is because not only we can have a smaller image to build later, but also our container will have smaller attack surface. This is because the less code there is within our container, the less likely it is to include a vulnerability.
The EXPOSE 80 command is telling Docker that we need to open the port 80 because the web server is listening on port 80. Hence, in order to access our program from outside the container through HTTP, we need to define it in the Dockerfile that we need the port 80 to be always opened.
The next three COPY commands are basically copying firstly the GoLab executable into the root directory of the container and secondly the two directories, public and templates into the container. Without the HTML, CSS, and JavaScript, our web app will not work.
Now you may wonder why the first COPY command says GoLab instead of GoLab.exe. We shall discuss it later in this article.
After that, we use ENV command to set the environment variables that we will be using in the app.
Finally we have the line CMD [“/GoLab”] to directs the container as to which command to execute when the container is run.
Since the container is not a Windows container, the code that runs inside the container thus needs to be a Linux binary. Fortunately, this is really simple to obtain with the cross-compilation support in Go using the following command.
$ $env:GOOS = "linux" $ go build -o GoLab .
Thus, in the Dockerfile, we use GoLab file instead of GoLab.exe.
We can now proceed to build the container image with the following command (Take note of the dot in the end of line).
$ docker image build -t chunlindocker/golab:v1 .
The -t flag is for us to specify the name and tag of the container. In this case, I call it chunlindocker/golab:v1 where chunlindocker is the Docker ID of my Docker Hub. Naming in such a way later helps me to push it to a registry, i.e. the Docker Hub.
If we want to build the image with another dockerfile, for example Dockerfile.development, we can do it as follows.
Once the docker image is built, we can see it listen when we perform the list command as shown in the screenshot below.
Now the container image is “deployable”. That means we can run it anywhere with a running docker engine. Since our laptop has Docker installed, so we can proceed to run it locally with the following command.
$ docker container run -P chunlindocker/golab:v1
If you run the command above in the Terminal window inside VS Code, you will see that the command line is “stuck”. This is because the container is already running on local machine. So what we need to do is just open another terminal window and view all the running containers.
To help humans, Docker auto generates a random name with two words and assigns it to the container. We can see that the container we created is given a random name “nifty_elgama”, lol. So now our container has a “human” name to call. If you want to remove the container later, you not only need to Ctrl+C to stop it, but to totally remove it, you need to use the rm command as follows.
$ docker container rm nifty_elgama
The PORTS column shown in the screenshot is important because it tells us how ports exposed on the container can be accessed from the host. So to test it locally, we shall visit http://localhost:32768.
So our next step is to upload it to a container registry so that later it can be pulled onto any machines, including Azure Web Apps, that will run it. To do so, we do push the image we built above to Docker Hub with the following command.
$ docker push chunlindocker/golab:v1
So, now how do we deploy the container to Azure?
Firstly, we need to create a Web App for Containers on the Azure Portal, as shown in the screenshot below.
The last item in the configuration is the “Configure Container”. Clicking on that, we will be brought to the following screen where we can then specify the container image we want to use and pull it from Docker Hub.
You can of course deploy a private container from Docker Hub by choosing “Private” as Repository Access. Then Azure Portal will prompt you for Docker Hub login credential for it to pull image from Docker Hub.
Once the App Service is created, we can proceed to read the Logs under “Container Settings”. Then we can see the container initializing process.
After that we can proceed to fill up the Application Settings with the environment variables we have in the web application and then we are good to go.