Monthly Archives: May 2020

Debugging GitHub Actions

I’ve recently been playing with GitHub actions. Having been around the block a few times, I’ve seen a fair few methods of building and deploying software, and of those, a fair few that are automated in some way. Oddly, debugging these things seems to be in the same place it was around 10 years ago: you see an error, try to work out what caused it, fix it, and run the build / deploy, rinse and repeat.

In some respects, this process may have actually become harder to debug since the days of TFS (at least with TFS, you could connect to the server and see why the software wasn’t building).

Anyway, onto GitHub actions

I’ve been trying to set-up an automated CI/CD pipeline for a new utility that I’ve been playing with.

After I’d configured a basic build, I started getting the following error:

MSBUILD : error MSB1003: Specify a project or solution file. The current working directory does not contain a project or solution file.

So, I thought future me (and perhaps one or two other people) may like to see the process that I went through to resolve (or at least to diagnose) this.

1. Git Bash

Your build is very likely trying to run on a Linux platform. If you have a look at your build file, it will tell you (or, more accurately, you will tell it) where it’s building:

So, the first step is to load up bash and manually type in the commands that the build is executing, one at a time. Again, these are all in the build file:

Make sure that you do this from the root directory, in case your problem relates to the path.

2. Add Debug Steps

Assuming that you’ve done step one and everything is working, the next stage is to start adding some logging. I strongly suspect that someone reading this will tell me there’s an easier way to go about this (please tell me there’s an easier way to go about this!) but this is how I went about adding tracing:

steps:	
	- uses: actions/[email protected]
	- name: Setup .NET Core
	uses: actions/[email protected]
	with:
	dotnet-version: 3.1.101
	- name: where are we
	run: pwd
	- name: list some key files
	run: ls -lrt
	- name: try a different die
	run: ls -lrt websitemeta
	- name: Install dependencies
	run: dotnet restore ./websitemeta/

As you can see, I was working under the assumption that my build was failing because the directory paths were incorrect. In fact, the paths were fine:

With the logging stage, there’s two ways to look at this
1. You’re closely following the scientific method of establishing a hypothesis and testing it; or
2. You’re blindly logging as much information as you can to try and extract a clue.

You’d be surprised how quickly 1 turns into 2!

3. Remember the platform that you’re running on

Okay, so in Step 1, I stated that you should try running the build in bash; but remember that, despite the Unix like interface, you’re still on Windows. As you can see from my build file, my build is on Ubuntu. In my particular case, this held the key – in fact, my build was failing because I’d used the wrong case for the directory path.

This is also true for your tests; for example, if you’ve hard-coded a directory path like this in your test:

string path = "tmp\myfile.txt";

It will fail, because in Unix, it would be:

string path = "tmp/myfile.txt";

Deploying to NuGet Manually and Using GitHub Actions

I’ve been playing with GitHub Actions in order to create some kind of basic pipeline for a little NuGet library that I created. In this post, I’ll describe the manual process, and then one option for automating that process.

Manual process

The manual process is actually not too bad. I know that CI/CD is the latest TDD (that is, the thing that everyone wants to agree is a good thing); and I’m not saying it’s a bad thing – I am, however, saying that if you have a limited amount of time, and no product to deploy, your time may be better spent creating one, rather than creating a deployment pipeline for something that doesn’t exist. Having a deployment process that’s written down in a text file has its downsides, but people will first judge your software by it’s quality, not by the method it arrived on the server.

Having said that, one thing that automating this whole process does give you, is a guarantee that all the tests are passing before it gets published.

The first step in deploying a NuGet package is to configure the properties of the project. You can have VS generate the NuGet package for you (Generate NuGet package on build), or you can do so yourself. Either way, you’ll need to update the version number:

Assuming that you didn’t ask VS to generate on build, your next step is to switch to the Release profile, build and the “Pack” the project:

Pack will produce a .nupkg file, which you’ll find in your bin folder:

This assumes that you’ve previously created the NuGet package in NuGet; if you have then log-in to nuget.org and select to upload a new package:

Point it to your lastest.nupkg, and the package is published.

That whole process takes around 2 – 3 minutes, maybe less.

GitHub Actions

So, we’ve got a relatively smooth manual deployment pipeline, and I had a bit of a diatribe at the start of this post about how great manual deployments are. You may be wondering why I’m trying to automate this. In fact, there are two big advantages to an automated deployment (apart from the fact that I just wanted to play with GitHub actions!):

1. You’re guaranteed to have your tests pass before deployment.
2. There’s a record of the deployment, and what was deployed.

Specifically, GitHub actions has the following advantages over most other deployment pipelines:

1. It’s free.

Setting up a New Pipeline

To start with, navigate to the Actions tab under your repository in GitHub:

Here, you’re presented with a list of pre-built workflows – some relate more to what you may be doing than others. Basically, GitHub uses a language analyser to try and work out what language you’re using for your project and gives you options based on that.

The default workflow does a build, and then runs the tests, so let’s set that one up first:

You’re then taken to a YAML editor (unfortunately, YAML seems to be the language of choice for GitHub Actions):

Let’s make a mental note of the “marketplace” on the right hand side, but for now, let’s just save this, and see what happens.

We can now test this by navigating to Actions, and generating a push:

If your build fails (like mine) you can have a look in the logs:

You can drill into each section, and see the error; for example:

To fix the issues, you can edit the build directly in GitHub, and then re-try the build. I’m planning a follow-up post to this on some techniques for debugging these things.

Once that’s working, we want to publish to NuGet. Let’s have a look if there’s anything already out there:

The Marketplace has hundreds of pre-built worfklows that you can just incorporate into your own. Simply copy the supplied YAML. For my project, it looks like this:

name: .NET Core	
	on:
	push:
	branches: [ master ]
	pull_request:
	branches: [ master ]
	jobs:
	build:
	runs-on: ubuntu-latest
	steps:
	- uses: actions/[email protected]
	- name: Setup .NET Core
	uses: actions/[email protected]
	with:
	dotnet-version: 3.1.101
	- name: Install dependencies
	run: dotnet restore ./WebSiteMeta/
	- name: Build
	run: dotnet build --configuration Release --no-restore ./WebSiteMeta
	- name: Test
	run: dotnet test --no-restore --verbosity normal ./WebSiteMeta
	- name: Publish NuGet
	uses: rohith/[email protected]
	with:
	# Filepath of the project to be packaged, relative to root of repository
	PROJECT_FILE_PATH: WebSiteMeta/WebSiteMeta.Scraper/WebSiteMeta.Scraper.csproj
	# NuGet package id to check against version changes, defaults to project name
	#PACKAGE_NAME: # optional
	# Filepath containing version info, relative to root of repository
	#VERSION_FILE_PATH: # optional
	# Regex pattern to extract version info in a capturing group
	#VERSION_REGEX: # optional, default is <Version>(.*)<\/Version>
	# Static version, useful for external providers like Nerdbank.GitVersioning
	#VERSION_STATIC: # optional
	# Flag to enable / disable git tagging
	TAG_COMMIT: false # optional, default is true
	# Format of the git tag, `[*]` gets replaced with version
	#TAG_FORMAT: # optional, default is v*
	# API key for the NuGet feed
	NUGET_KEY: ${{secrets.NUGET_API_KEY}} # optional

If you’re interested how the secret was generated, then have a look at this earlier post.

GitHub Secrets

I’ve recently been playing around with GitHub Actions. The reason being that I created this little NuGet package, because I needed such a utility for a web site that I’m working on.

I’ll go into how I set-up a basic CI/CD pipeline in a later post; but after playing with GitHub actions for a short period of time, it because clear that the first thing you need to understand is how secrets work. Otherwise you may, for example, generate a GitHub API key, and check that into your source control; and then get a rather harshly worded e-mail from GitHub telling you to stop doing that.

It turns out that storing secrets in GitHub is very easy. First, visit your repository; and, under the Settings tab, you’ll find Secrets:

Here, you just select Add Secret. Give the secret a name and tell it what the secret is:

Finally, to use your secret inside a workflow, you use the following format:

${{secrets.SECRET_NAME}}

References

https://help.github.com/en/actions/configuring-and-managing-workflows/creating-and-storing-encrypted-secrets

Asp.Net Policy Authorization Based on a DB Field on the User Table

If you read this, you’ll learn how to create authorisation, based on a policy. Specifically, they use the example of the user’s age, and create a restriction to say that only users over the age of 21 can access a resource.

The age was something that they got from a claim, stored against the user. But what if your requirement is a little more complex? For example, what if you have a situation, such as a popular question and answer site that you may recognise, where you are trying to restrict access to a resource based on something that can change during the user session. In this post, I’m covering how you can follow the same policy structure, but force the program to go back to the DB to get details about the user, each time they try to access the restricted resource.

As a background, the particular use case here is that a user of my site can approve something, but only where they have a sufficient rating.

Basic Set-up

The first thing that you’ll need is a custom Authorization Handler. For my project, I’ve created a sub directory called Authorization (I’ve even spelled it wrong to be consistent):

The Authorization Handler inherits from an abstract class, which forces you to override a single method: HandleRequirementAsync. Essentially, you tell the handler about your requirement (we’ll come to the requirement next), and then it passes this back to you in the method; let’s have a look at some code:

    public class ApproverAuthHandler : AuthorizationHandler<ApproverAuthRequirement>
    {
        protected override Task HandleRequirementAsync(
            AuthorizationHandlerContext context, 
            ApproverAuthRequirement requirement)
        {

The Requirement is just a class to hold the relevant data that you need. In our example, the requirement would hold the user’s rating; here’s the requirement code:

    public class ApproverAuthRequirement : IAuthorizationRequirement
    {
        public int UserRating { get; set; }

        public ApproverAuthRequirement(int userRating)
        {
            UserRating = userRating;
        }
    }

IAuthorizationRequirment is what is known as a marker interface. To be honest, I hadn’t come across the term before, but I have used the pattern before. Essentially, this is an empty interface: its purpose it to allow you to pass the class type as a strongly typed interface; but in reality, the class can me anything.

Anyway, let’s get back to the Authorization Handler. Inside HandlerRequirementAsync, you can do anything you choose; should your check be successful, you call context.Succeed(requirement), otherwise, do nothing. In the above linked code, they check a claim against the requirement; we’re going to just get some information from the DB, and check that against the requirement:

        protected override Task HandleRequirementAsync(
            AuthorizationHandlerContext context, 
            ApproverAuthRequirement requirement)
        {
            int? userRating = _userService.GetSubjectRating(context.User);            
            if (!userRating.HasValue)
            {
                return Task.CompletedTask;
            }

            if (userRating >= requirement.UserRating)
            {
                context.Succeed(requirement);
            }

            return Task.CompletedTask;
        }

User service is simply a service that calls into a repository, and returns the user rating.

Unit Testing

All well and good, and this looks eminently testable. In fact, it is: you simply mock out the service, and the test looks like this:

            // Arrange
            var requirements = new[] { new ApproverAuthRequirement(100) };            
            var user = new ClaimsPrincipal(
                        new ClaimsIdentity(
                            new Claim[] { },
                            "Basic")
                        );

            var userService = Substitute.For<IUserService>();
            userService.GetSubjectRating(Arg.Any<IPrincipal>()).Returns(100);

            var context = new AuthorizationHandlerContext(requirements, user, null);
            var sut = new ApproverAuthHandler(userService);

            // Act
            await sut.HandleAsync(context);

            // Assert
            Assert.True(context.HasSucceeded);

No doubt, you could check for the exact service principal under test, although you’re testing the Authorization Handler, so I think this is sufficient.

Resource Based Handler

In this article, there is a similar process discussed; it warrants some further investigation on my part, but I don’t currently see any real difference between my implementation, and this abstraction (although, admittedly, this version looks easier to test).

References

https://docs.microsoft.com/en-us/aspnet/core/security/authorization/policies?view=aspnetcore-3.1

https://stackoverflow.com/questions/51272610/unit-test-authorizationhandler

https://docs.microsoft.com/en-us/aspnet/core/security/authorization/resourcebased?view=aspnetcore-3.1

Force React Components to sit Side-by-Side

For anyone that knows me, they’ll know that UI and UX is not exactly my strong suit. If I had my way, we’d all just use console applications like this:

>placeorder /productcode coffee /quantity 1

Unfortunately, the trend seems to have gone a different way, and now we have CSS. CSS is a brilliant idea; however, you need to be able to visualise what you want your stuff to look like first.

Anyway, onto this post. If you know anything about CSS or Bootstrap, then you’ve probably already read everything that could interest you about this post!

In my latest project, I have a search box, and I wanted to line up the controls on the screen like this:

Search         [Search Text]        [Search Button]

I’m using React, so the original code looked like this (more or less):

<label>Search</label>
<input type='text'/>
<SimpleButton buttonAction={props.searchAction} buttonLabel="Search" />            

And it rendered like this:

Search         [Search Text]        
[Search Button]

My first gambit was to define a CSS style (so I can now put “Front End Developer” on my CV):

.rowLine {
    display:flex; 
    flex-direction:row;
}

And I changed the HTML to look like this:

<div className='rowLine'>
    <label>Search</label>
    <input type='text'/>
    <SimpleButton buttonAction={props.searchAction} buttonLabel="Search" />            
</div>

That worked, and I copied most of it from here. So now I’ve updated my CV to “Senior Front End Developer”.

It then occurred to me that, as good as this looks, there’s probably something in Bootstrap, and if there is, then my web page can look like the rest of the internet. It turns out I was right:

<div className="form-group row">
    <label htmlFor="searchText" className="col-sm-2 col-form-label">Search</label>
    <div className="col-sm-8">
        <input id="searchText" className="form-control" type='text'
                    placeholder="e.g. Goats" />
    </div>
    <div className="col-sm-2">
        <SimpleButton buttonAction={props.searchAction} buttonLabel="Search" />            
    </div>
</div>        

You may notice that React has its own “for” property, called “htmlFor”.

Disclaimer / Apology

As usual, please take everything you read in my blog with a healthy dose of salt. If you are a front end developer then please take solace in the fact that, despite me being facetious, I really need frameworks like Bootstrap, because I would never think to change the colour of a button, or to align the sizes.

This was all so much simpler in the days of Turbo Pascal / Turbo C, where you would draw your buttons using the ANSI character set!

References

https://stackoverflow.com/questions/39702130/line-two-divs-side-by-side-with-css-and-react

https://stackoverflow.com/questions/39187722/error-ts2339-property-for-does-not-exist-on-type-htmlpropshtmllabelelement

https://www.tutorialrepublic.com/twitter-bootstrap-tutorial/bootstrap-forms.php

Using XMLHttpRequest from Javascript to Call a .Net Core Api

Imagine you have the following API endpoint:

https://localhost:1234/doStuff

The controller method for this might look like this:

[HttpPost]
public IActionResult DoStuff([FromBody]Data data)

If, for example, you create the sample .Net Core React app, you’ll see it getting the data using a construct like this:

const response = await fetch('doStuff');

This works fine if you’re retrieving data, but if you want to send some data through to the endpoint body, XMLHttpRequest allows you to send data in the body like this:

const xhr = new XMLHttpRequest();

xhr.open('POST', 'doStuff')
xhr.setRequestHeader('Content-Type', 'application/json');
xhr.send(JSON.stringify({ 
  data1: this.state.data1,
  data2: this.state.data2
}));

Unsupported Media Type

If, in the above code, you were to use:

const xhr = new XMLHttpRequest();

xhr.open('POST', 'doStuff')
//xhr.setRequestHeader('Content-Type', 'application/json');
xhr.send(JSON.stringify({ 
  data1: this.state.data1,
  data2: this.state.data2
}));

You would be sending form data through to the controller. The call would not work, and you would get the following error from F12:

{"type":"https://tools.ietf.org/html/rfc7231#section-6.5.13","title":"Unsupported Media Type","status":415,"traceId":"|c86a375c-41ba2fd85652022a."}

There are two ways around this: the first, you’ve seen above, you set the content type; but you can also change the controller method to look like this:

[HttpPost]
public IActionResult DoStuff([FromForm]Data data)

References

https://stackoverflow.com/questions/39519246/make-xmlhttprequest-post-using-json

https://attacomsian.com/blog/http-requests-xhr