TCP Listener on Microsoft Azure with Service Fabric

azure-service-fabric-load-balancer.png

Getting TCP listener to run on Microsoft Azure is always an interesting topic to work on. Previously, I did one experimental TCP listener on Azure Cloud Service and it works quite well.

Today, I’d like to share with you my another experiment which is hosting a TCP Listener on Microsoft Azure with Service Fabric.

Step 0: Installing Service Fabric SDK

Most of the time, it’s better to run the Visual Studio 2017 in Administrator mode otherwise debugging and deployment of Service Fabric applications may have errors.

Before we can start a new Service Fabric application project on Visual Studio, we first need to make sure Service Fabric SDK is installed.

service-fabric-sdk-must-be-installed.png
Visual Studio will prompt us to install Service Fabric SDK.

The template that I use is Stateless Service under .NET Core 2.0. This project template is to create a stateless reliable service with .NET Core.

Step 1: Add TCP Endpoint

In the ServiceManifest.xml of the PackageRoot folder of the application project, we need to specify an endpoint that our TCP Listener will be listening to. In my case, I am using port 9005. So I need to add an endpoint as shown below in the ServiceManifest.xml.

<Endpoint Name="TcpEndpoint" Protocol="tcp" Port="9005"/>

Step 2: Create Listeners

In the StatelessService class, there is a CreateServiceInstanceListeners method that we can override to create TCP listeners with the following codes.

protected override IEnumerable CreateServiceInstanceListeners()
{
    var endpoints = Context.CodePackageActivationContext.GetEndpoints()
        .Where(endpoint => endpoint.Protocol == EndpointProtocol.Tcp)
        .Select(endpoint => endpoint.Name);

    return endpoints.Select(endpoint => new ServiceInstanceListener(
        serviceContext => new TcpCommunicationListener(serviceContext, ServiceEventSource.Current, endpoint), endpoint));
}

Then in the RunAsync method, which is the main entry point for our service instance, we can simply include the code for TCP Listener to receive and send message to the clients.runasync.png

Step 3: Create Service Fabric Cluster

 

There are a few simple steps for us to follow in order to create a new Service Fabric cluster on Microsoft Azure.

Firstly, we need to specify some basic information, such as cluster name, OS, and default VM credentials.

service-fabric-step1-configure-basic-settings.png
Configure basic settings for a new Azure Service Fabric cluster.

Secondly, we need to define Node Types. Node types can be seen as equivalent to the roles in Cloud Service. Node types define the VM sizes, the number of VMs, and their properties. Every node type that is defined in a Service Fabric cluster maps to a virtual machine scale set.

We can start with only one node type. The portal will then prompt us to select one VM size. By default, it only shows three recommended sizes. If you would like to find out more other specs with lower price, please click on “View All”.

I once use A0 (which coasted USD 14.88) for experimental purpose. However, it turns out that the newly created service fabric cluster is totally not connectable with a status saying “Upgrade service unreachable”. The funny thing is that the status was only shown after everything in the resource group is setup successfully which strangely took about one hour plus to finish. So I wasted about one hour for that. Hence, please use at least the recommended size for the VM.

service-fabric-step2-configure-cluster.png
We need to specify the VM spec for each of the node type.

A very interesting point to take note is that, there is a checkbox for us to configure advance settings for node type, as shown in the following screenshot. The default values here will affect things such as the Service Fabric dashboard URL we use later. It’s fine to leave them as default.

service-fabric-step2-configure-cluster-advance-settings.png
Default values in the advanced settings of node type.

Thirdly, we need to configure the security settings by specifying which Key Vault to use. If you don’t have any suitable key vault, then it will take about one minute to create a new key vault for you. After the new key vault is created, you may be prompted with an error stopping you to proceed, as shown in the following screenshot.

service-fabric-step3-configure-security-settings-error.png
New key vault created here by default is not enabled for deployment.

To fix the error, we first need to visit the Key Vaults page. After that, we need to find out the key vault we just created above. Then we proceed to tick the corresponding checkbox to enable the key vault access to Azure VM for deployment, as shown in the following screenshot.

service-fabric-step3-configure-key-vault.png
Enable it so that Azure VM can retrieve certificates stored as secret from the key vault.

Now, if we got back to the Step 3 of the service fabric cluster setup, we can get rid of the error message by re-selecting the key vault. After keying a certificate name, we will need to wait for 30 seconds for validation. Then we will be given a link to download our certificate for later use.

service-fabric-step4-download-cert.png
Let’s download the cert from here!

This marks the end of our service fabric cluster setup. What we need to do is just to click on the “Create” button.

The creation process took about 40 minutes to complete. It actually went through many stages which are better described in the article “Azure Service Fabric Cluster – Deployment Issues”, written by Cosmin Muscalu.

Step 4: Publish App from Visual Studio

After the service fabric cluster is done, we can proceed to publish our application to it.

In the Solution Explorer, we simply need to right-click on the Service Fabric project and choose Publish, as shown in the following image.

solution-explorer
Solution Explorer

A window will popup and prompt us that the Connection Endpoint is not valid, as shown below.

cannot-publish-to-server.png
Failed to connect to server and thus we cannot publish the app to Azure.

Now, according to the article from the link “How to configure secure connection”, we have to install the certificate that we downloaded earlier from Azure Portal in Step 2.

Since there is no password for the pfx file, we simply need to accept all default settings while importing the certificate.

Now if we go back to the Publish window, we will see a green tick icon appearing at the side of the Connection Endpoint. Now, we are good to proceed to do a publish. The deployment of a simple TCP Listener normally takes less than one minute to finish.

Step 5: Open Port Access

After the deployment is done, we need to open up the 9005 port that we specified above in Step 2. To do so, we need to visit the Load Balancer used by the service fabric cluster to add a new rule for the port 9005 to be accessible from public.

add-load-balancing-rule.png
Add a new load balancing rule for the service fabric.

The process of adding a new rule normally takes about three minutes to complete.

Please take note that we need to note down the Public IP Address of our load balancer as well.

load-balancer-public-ip-address.png
The Public IP Address of a load balancer can be found in its Overview panel.

Step 6: Open Up Service Fabric Explorer

Finally, we need to open up the Explorer for our service fabric cluster. To do so, we can retrieve the dashboard URL in the Overview panel of the service fabric cluster.

service-fabric-admin-dashboard.png
The Service Fabric Explorer URL is here.

To access the Explorer, we first need to select a certificate that we downloaded earlier to authenticate ourselves to the Explorer, as shown in the screenshot below.

select-certificate.png
Selecting a certificate on Google Chrome.

Step 7: Communicate with TCP Listener

Now, if we build a simple TCP client to talk to the server at the IP address of the load balancer that we noted down earlier, we can easily send and receive response from the server, as shown in the screenshot below.

tcp-client.png
Hooray, we receive the response from the application on Azure Service Fabric!

So yup, that’s all for a very simple TCP Listener which is hosted on Microsoft Azure.

I will continue to research more about this topic with my teammates so that I can find out more about this cool technology.

Protecting Web API with User Password

identity-server

In my previous post, I shared about the way to connect Android app with IdentityServer4 using AppAuth for Android. However, that way will popup a login page on a web browser on phone when users are trying to login to our app. This may not be what the business people want. Sometimes, they are looking for a customized native login page on the app itself.

To do so, we can continue to make use of IdentityServer4.

IdentityServer4 has a grant which is called Resource Owner Password Grant. It allows a client to send username and password to the token service and get an access token back that represents that user. Generally speaking, it is not really recommended to use the AppAuth way. However, since the mobile app is built by our own team, so using the resource owner password grant is okay.

Identity Server Setup: Adding New API Resource

In this setup, I will be using in-memory configuration.

As a start, I need to introduce a new ApiResource with the following codes in the Startup.cs of our IdentityServer project.

var availableResources = new List<ApiResource>();
...
availableResources.Add(new ApiResource("mobile-app-api", "Mobile App API Main Scope"));
...
services.AddIdentityServer()
    ...
    .AddInMemoryApiResources(availableResources)
    .AddInMemoryClients(new ClientStore(Configuration).GetClients())
    .AddAspNetIdentity<ApplicationUser>();

Identity Server Setup: Defining New Client

As the code above shows, there is a ClientStore that we need to add a new client to with the following codes.

public class ClientStore : IClientStore
{
    ...

    public IEnumerable<Client> GetClients()
    {
        var availableClients = new List<Client>();
        
        ...
        
        availableClients.Add(new Client
        {
            ClientId = "mobile-app-api",
            ClientName = "Mobile App APIs",
            AllowedGrantTypes = GrantTypes.ResourceOwnerPassword,
            ClientSecrets = { new Secret(Configuration["MobileAppApi:ClientSecret"].Sha256()) },
            AllowedScopes = { "mobile-app-api" }
        });

        return availableClients;
    }
}

Configuring Services in Web API

In the Startup.cs of our Web API project, we need to update it as follows.

public void ConfigureServices(IServiceCollection services)
{
    services.AddMvc();

    services.AddAuthorization();

    services.AddAuthentication("Bearer")
    .AddIdentityServerAuthentication(options =>
    {
        options.Authority = "<URL of the identity server>";
        options.RequireHttpsMetadata = true;
        options.ApiName = "mobile-app-api";
    });

    services.Configure<MvcOptions>(options =>
    {
        options.Filters.Add(new RequireHttpsAttribute());
    });
}

Configuring HTTP Request Pipeline in Web API

Besides the step above, we also need to make sure the following one line “app.UseAuthentication()” in the Startup.cs. Without this, we cannot make the authentication and authorization to work in our Web API project.

public void Configure(IApplicationBuilder app, IHostingEnvironment env)
{
    ...
    app.UseAuthentication();
    app.UseMvc();
}

Receiving Username and Password to Return Access Token

We also need to add a new controller to receive username and password which will in return tell the mobile app whether the login of the user is successful or not. If the user is logged in successfully, then an access token will be returned.

[Route("api/[controller]")]
public class AuthenticateController : Controller
{
    ...
    [HttpPost]
    [Route("login")]
    public async Task<ActionResult> Login([FromBody] string userName, string password)
    {
        var disco = await DiscoveryClient.GetAsync("<URL of the identity server>");
        var tokenClient = new TokenClient(disco.TokenEndpoint, "mobile-app-api", Configuration["MobileAppApi:ClientSecret"]);
        var tokenResponse = await tokenClient.RequestResourceOwnerPasswordAsync(userName, password, "mobile-app-api");

        if (tokenResponse.IsError)
        {
            return Unauthorized();
        }

        return new JsonResult(tokenResponse.Json);
    }
    ...
}

Securing our APIs

We can now proceed to protect our Web APIs with [Authorize] attribute. In the code below, I also try to return the available claims via the API. The claims will tell the Web API who is logging in and calling the API now via the mobile app.

[HttpGet]
[Authorize]
public IEnumerable<string> Get()
{
    var claimTypesAndValues = new List<string>();

    foreach (var claim in User.Claims)
    {
        claimTypesAndValues.Add($"{ claim.Type }: { claim.Value }");
    }

    return claimTypesAndValues.ToArray();
}

Conclusion

This project took me two days to find out how to make the authentication works because I misunderstand how IdentityServer4 works in this case. Hence, it is always important to fully understand the things on your hands before working on them.

do-not-give-up.png
Do not give up! (Source: A Good Librarian Like a Good Shepherd)

Reference

Connecting Android App with IdentityServer4

android-identity-server-appauth.png

For those ASP .NET web developers, Identity Server should be quite familiar to them especially they are looking for SSO solution.

After successfully integrating Identity Server in our ASP .NET Core MVC web applications, it is now time for us to research about how our mobile app can be integrating with IdentityServer4 too.

Background

We have two types of users. The admin will be logging in to the system via our web application. The normal staff will log in to the system via mobile app. Different sets of features are provided for both web and mobile apps.

Setting up Client on Identity Server

To begin, we need to add new client to the MemoryClients of Identity Server.

According to the sample code done by Hadi Dbouk, we setup the new client as shown in the following code.

using IdentityServer4.Models;
...
public class ClientStore : IClientStore {
    ...
    var availableClients = new List();
    ...
    availableClients.Add(new Client 
    {
        ClientId = "my-awesome-app",
        ClientName = "My Awesome App",
        AllowedGrantTypes = GrantTypes.Code,
        RequirePkce = true,
        RequireConsent = false,
        ClientSecrets = 
        {
            new Secret("my-secret".Sha256())
        },
        RefreshTokenUsage = TokenUsage.ReUse,
        RedirectUris = { "gclprojects.chunlin.myapp:/oauth2callback" },
        AllowedScopes = 
        {
            StandardScopes.OpenId,
            StandardScopes.Profile,
            StandardScopes.Email,
            StandardScopes.OfflineAccess
        },
        AllowOfflineAccess = true
    }
    );
}

For mobile apps, there are two grant types recommended, i.e. Authorization Code and Hybrid. However, as when this post is written, the support of Hybrid is still not mature in AppAuth for Android, so we decided to use GrantTypes.Code instead.

However, OAuth2.0 clients using authorization codes can be attacked. In the attack, the authorization code returned from an authorization endpoint is intercepted within a communication path that is not protected by TLS. To mitigate the attack, PKCE (Proof Key for Code Exchange) is required.

We don’t have consent screen for our apps, so we set RequireConsent to false.

For the RefreshTokenUsage, there are two possible values, i.e. ReUse and OneTime. The only difference is that ReUse will make the refresh token handle to stay the same when refreshing tokens. OneTime will update the refresh token handle once the tokens are refreshed.

Once the authorization flow is completed, users will be redirected to a URI. As documented in AppAuth for Android Readme, custom scheme based redirect URI (i.e. those of form “my.scheme:/path”) should be used for the authorization redirect because it is the most widely supported across many Android versions.

By setting AllowOfflineAccess to be true and give the client access to the offline_access scope, we allow requesting refresh tokens for long lived API access.

Android Setup: Installation of AppAuth

The version of AppAuth for Android is v0.7.0 at the point of time this post is written. To install it for our app, we first need to set it in build.gradle (Module: app).

apply plugin: 'com.android.application'

android {
    ...    defaultConfig {
        ...
        minSdkVersion 21
        targetSdkVersion 26
        ...
        manifestPlaceholders = [
            'appAuthRedirectScheme': 'gclprojects.chunlin.myapp'
        ]
    }
    ...
}

dependencies {
    ...
    compile 'com.android.support:appcompat-v7:26.+'
    compile 'com.android.support:design:26.+'
    compile "com.android.support:customtabs:26.0.0-alpha1"
    compile 'net.openid:appauth:0.7.0'
    ...
}

 

appauth-code-flow.png
AppAuth for Android authorization code flow. (Reference: The proper way to use OAuth in a native app.)

Android Setup: Updating Manifest

In the AndroidManifest.xml, we need to add the redirect URI to the RedirectUriReceiverActivity, as shown in the following code.

<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
    package="gclprojects.chunlin.myapp">
    ...
    <application...>
        <activity
            android:name="net.openid.appauth.RedirectUriReceiverActivity"
            android:theme="@style/Theme.AppCompact.NoActionBar">
        <intent-filter>
            <action android:name="android.intent.action.VIEW"/>

            <category android:name="android.intent.category.DEFAULT"/>
            <category android:name="android.intent.category.BROWSABLE"/>

            <data android:scheme="gclprojects.chunlin.myapp"/>
        </intent-filter>
    </application>
    ...
</manifest>

Android Setup: Authorizing Users

On the Android app, we will have one “Login” button.

<Button
    android:onClick="Login"
    android:layout_width="wrap_content"
    android:layout_height="wrap_content"
    android:text="Login"
    android:layout_centerInParent="true"/>

By clicking on it, the authorization steps will begin.

public void Login(View view) {
    AuthManager authManager = AuthManager.getInstance(this);
    AuthorizationService authService = authManager.getAuthService();

    AuthorizationRequest.Builder authRequestBuilder = new AuthorizationRequest
            .Builder(
            authManager.getAuthConfig(),
            "my-awesome-app",
            "code",
            Uri.parse("gclprojects.chunlin.myapp:/oauth2callback"))
            .setScope("openid profile email offline_access");

    String codeVerifier = CodeVerifierUtil.generateRandomCodeVerifier();
    SharedPreferencesRepository sharedPreferencesRepository = new SharedPreferencesRepository(this);
    sharedPreferencesRepository.saveCodeVerifier(codeVerifier);

    authRequestBuilder.setCodeVerifier(codeVerifier);

    AuthorizationRequest authRequest = authRequestBuilder.build();

    Intent authIntent = new Intent(this, LoginAuthActivity.class);
    PendingIntent pendingIntent = PendingIntent.getActivity(this, authRequest.hashCode(), authIntent, 0);

    authService.performAuthorizationRequest(
            authRequest,
            pendingIntent);
}

The code above uses some other classes and interact with other activity. I won’t talk about them here because the codes can be found on my Github repository which is forked from Hadi Dbouk’s.

Android Setup: Post Authorization and Refresh Token

According to the code in the LoginAuthActivity.java, if the login fails, the user will be brought back to the Login Activity. However, if it succeeds, the user can then reach to another activities in the app which require user to login first. We can also then get Access Token, Refresh Token, and ID Token from authManager. With the Access Token, we then can access our backend APIs.

Since access tokens have finite lifetimes, refresh tokens allow requesting new access tokens without user interaction. In order to have the client to request Refresh Token, we need to authorize it by setting AllowOfflineAccess to true. When we make a request to our APIs, we need to check if the Access Token is expired, if it is so, we need to make a new request with the Refresh Token to the IdentityServer to have a new Access Token.

The way how we can retrieve new Access Token with a Refresh Token in AppAuth is shown in the TokenTimer class in TokenService.java using createTokenRefreshRequest.

private class TokenTimer extends TimerTask {
    ...

    @Override
    public void run() {

        if(MyApp.Token == null)
            return;

        final AuthManager authManager = AuthManager.getInstance(TokenService.this);

        final AuthState authState = authManager.getAuthState();


        if(authState.getNeedsTokenRefresh()) {
            //Get New Token

            ClientSecretPost clientSecretPost = new ClientSecretPost("driver008!");
            final TokenRequest request = authState.createTokenRefreshRequest();
            final AuthorizationService authService = authManager.getAuthService();

            authService.performTokenRequest(request, clientSecretPost, new AuthorizationService.TokenResponseCallback() {
                @Override
                public void onTokenRequestCompleted(@Nullable TokenResponse response, @Nullable AuthorizationException ex) {
                    if(ex != null){
                        ex.printStackTrace();
                        return;
                    }
                    authManager.updateAuthState(response,ex);
                    MyApp.Token = authState.getIdToken();
                }
            });

        }

    }
}

Conclusion

Yup, that’s all for integrating the Identity Server in an Android App to provide a seamless login experience to our users. If you find any mistake in this article, kindly let me know in the comment section. Thanks in advance!

References

 

Setup Ubuntu Server at Tokyo and Transform it to Desktop with RDP Installed

vultr-ubuntu-xrdp-xfce

While waiting for lunch, it’s nice to do some warmups. Setting a server overseas seems a pretty cool warmup to do for developers, right? Recently, my friend recommended me to try out Vultr which provide cloud servers. So today, I’m going to share how I deploy a Ubuntu server which is located in Tokyo, a city far away from where I am now.

Step 1: Choosing Server Location

Vultr is currently available in many cities in popular countries such as Japan, Singapore, Germany, United States, Australia, etc.

server-location.png

Step 2: Choosing Server Type and Size

Subsequently, we will be asked to select the type and size for the server. Here, I choose 60 GB SSD server with Ubuntu 16.04 x64 installed. I tried with Ubuntu 17.10 x64 before but I couldn’t successfully RDP into it. Then the latest Ubuntu 18.04 x64 is not yet tried by me. So ya, we will stick to using Ubuntu 16.04 x64 in this article.

vultr-ubuntu-pricing.png

Step 3: Uploading SSH Key

Vultr is friendly to provide us a tutorial about generating SSH Keys on Windows and Linux.

The steps for creating SSH key on Windows with PuTTYgen is as follows.

Firstly, we need to click on the “Generate” button on PuTTYgen.

generate-key-pair.png

Secondly, once the Public Key is generated, we need to enter a key passphrase for additional security.

Thirdly, we click on the “Save Private Key” button to save the private key on somewhere safe.

Fourthly, we copy all of the text in the Public Key field and paste it to the textbox in Vultr under the “Add SSH Key” section.

adding-ssh-key.png

Step 4: Naming and Deployment

Before we can deploy the server, we need to key in the hostname for the new server.

After we have done that, then we can instruct Vultr to deploy the server by clicking on the “Deploy Now” button at the bottom of the page.

Within 5 minutes, the server should finish installing and booting up.

Step 5: Getting IP Address, Username, and Password

In order to get the user credential to access the server, we need to click on the “Server Details” to view the IP address, username, and password.

Step 6: Updating Root Password

The default password is not user-friendly. Hence, once we login to the server via PuTTY, we need to immediately update the root password using the command below for our own good.

# passwd

Step 7: Installing Ubuntu Desktop

Firstly, let’s do some updating for the packages via the following commands.

# sudo apt-get update
# sudo apt-get upgrade

This will take about 2 minutes to finish.

Then we can proceed to install the default desktop using the following command.

# sudo apt-get install ubuntu-desktop

This will take about 4 minutes to finish. Take note that at this point of time Unity will be the desktop environment.

After that, we update the packages again.

# sudo apt-get update

Step 8: Installing Text Editor

We are going to change some configurations later, so we will need to use a text editor. Here, I’ll use the Nano Text Editor by installing it first.

# sudo apt-get install nano

Step 9: Installing xrdp

xrdp is an open source Remote Desktop Protocol (RDP) server which provides a graphical login to remote machines. This helps us to connect to the server using Microsoft Remote Desktop Client.

sudo apt-get install xrdp

Step 10: Changing to Use Xfce Desktop Environment

We will then proceed to install Xfce which is a lightweight desktop environment for UNIX-like operating systems.

sudo apt-get install xfce4

After it is installed successfully, please run the following command. This is to tell the Ubuntu server to know that Xfce has been chosen to replace Unity as desktop environment.

echo xfce4-session >~/.xsession

Step 11: Inspect xrdp Settings

We need to configure the xrdp settings by editing the startwm.sh in Nano Text Editor.

nano /etc/xrdp/startwm.sh

We need to edit the file by changing entire of the file content to be as follows.

if [ -r /etc/default/locale ]; then
 . /etc/default.locale
 export LANG LANGUAGE
fi

startxfce4

Then we need to restart xrdp.

# sudo service xrdp restart

After that, we restart the server.

# reboot now

Step 12: Connecting with Remote Desktop Client

After the server has been restarted, we can access the server with Windows Remote Desktop Client.

rdp

At this point of time, some of you may encounter error when logging in via RDP. The error will be saying things as follows.

Connecting to sesman IP 127.0.0.1 port 3350
sesman connect ok
sending login info to session manager, please wait...
xrdp_mm_process_login_response:login successful for display
started connecting
connecting to 127.0.0.1 5910
error-problem connecting
problem-connecting.png
Problem of connecting via xrdp.

As pointed out in one of the discussion threads on Ask Ubuntu, the problem seems to be xrdp, vnc4server, and tightvncserver are installed in the wrong order. So in order to fix that, we just need to remove them and re-install them in a correct order with the following set of commands.

# sudo apt-get remove xrdp vnc4server tightvncserver
# sudo apt-get install tightvncserver
# sudo apt-get install xrdp
# sudo service xrdp restart

After the server is restarted, we should have no problem accessing our server via RDP client on Windows.

success.png

References