Tag Archives: System.Text.Json

Manually Parsing a Json String Using System.Text.Json

In this post from 2016 I gave some details as to how you could manually parse a JSON string using Newtonsoft JSON.NET. What I don’t think I covered in that post, was why you may wish to do that, nor did I cover how you could do that using System.Text.Json (although since the library was only introduced in .Net Core 3 – circa 2019, that part couldn’t really be helped!)

Why?

Let’s start with why you would want to parse a JSON string manually – I mean, the serialisation functions in, pretty much, any JSON library are really good.

The problem is coupling: by using serialisation, you’re coupling your data to a given shape, and very tightly coupling it to that shape, too. So much so, that a common pattern if you’re passing data between two services, and using serialisation, it to share the model class between the services. This sounds quite innocuous at first glance, but let’s consider a few factors (I’m assuming we’re talking exclusively about .Net, but I imagine the points are valid outside of that, too):

1. Why are you serialising and de-serialising the data to begin with?
2. Single Responsibility Principle.
3. Microservices.

Let’s start with (1). Your answer may be different, but typically, if you’re passing data as a string, it’s because you’re trying to remove a dependency to a given complex type. After all, a string can be passed anywhere: an API call, a message broker, even held in a database.

What has this got to do with the SRP (2)? Well, the SRP is typically used to describe the reason that a module has to change (admittedly it is slightly mis-named). Let’s see how the two modules may interact:

Now, let’s look at the interaction with a common module:

As you can see, they both have a dependency on a single external (external to the service) dependency. If the CustomerModel changes, then both services may also need to change, but they also need to change for alterations for business rules that relate to the module itself: so they now have two reasons to change.

Of course, you don’t have to have a common dependency like this; you could structure your system like this:

However, you don’t solve your problem – in fact, you arguably make it worse: if you change CustomerModel referenced by Service 1 you potentially break Service 2, so you now need to change CustomerModel referenced by Service 2, and Service 2!

Just to clarify what I’m saying here: there may be valid reasons for both of these designs – but if you use them, then be aware that you’re coupling the services to each other; which brings us to point (3): if you’re trying to create a Service Oriented Architecture of any description, then this kind of coupling may hinder your efforts.

The Solution

A quick caveat here: whatever you do in a system, the parts of that system will be coupled to some extent. For example, if you have built a Microservice Architecture where your system is running a nuclear reactor, and then you decide to change one of the services from monitoring the cooling tower to, instead, mine bit-coins, you’re going to find that there is an element of coupling. Some of that will be business coupling (i.e. the cooling tower may overheat), but some will be technical – a downstream service may depend on the monitoring service to assert that something is safe.

Apologies, I know absolutely nothing about nuclear reactors; or, for that matter, about mining bit-coin!

All that said, if you manually parse the JSON that’s received, you remove some dependency on the shape of the data.

The following reads a JSON document, and iterates through an array:

            using var doc = JsonDocument.Parse(json);
            var element = doc.RootElement;

            foreach (var eachElement in element.EnumerateArray())
            {
                string name = eachElement.GetProperty("Name").GetString();
                decimal someFigure = eachElement.GetProperty("SomeFigure").GetDecimal();

                if (someFigure > 500)
                {
                    Console.WriteLine($"{name} has more than 500!");
                }
            }

As you can see, if the property name of SomeFigure changed, the code would break; however, there may be a dozen more fields in each element that could change, and we wouldn’t care.

My object won’t deserialise using System.Text.Json

I imagine this is a very common scenario, so I thought it worth documenting.

Let’s take the following controller method in a bog standard Asp.Net Core API:

        public async Task<DataContainer> Get()
        {
            var data = Enumerable.Range(1, 5).Select(index => new MyData
            {
                Value1 = index,
                Value2 = "some other value"
            })
            .ToList();

            return new DataContainer()
            {
                Data = data
            };
        }

Here’s a method that consumes it:

            var client = _clientFactory.CreateClient();
            var result = await client.GetAsync($"https://localhost:22222/endpoint");

            using var responseStream = await result.Content.ReadAsStreamAsync();
            var data = await JsonSerializer.DeserializeAsync<DataContainer>(responseStream);

So, you’ve expect this to work, right? It makes no difference what DataContainer looks like, because you’re serialising and deserialising using the same technology. In fact, it just returns a correctly structured object, but every value will be null.

I’ll tell you what does work, is this:

            var client = _clientFactory.CreateClient();
            var result = await client.GetAsync($"https://localhost:22222/endpoint");

            var responseString = await result.Content.ReadAsStringAsync();
            var data = Newtonsoft.Json.JsonConvert.DeserializeObject<DataContainer>(responseString);             

If you look at the JSON, it looks kosher enough (again, the exact structure is irrelevant). So, what’s the issue?

Interestingly, when the JSON is serialised, it is serialised into camelCase; so if you have a variable such as MyData, it will get serialised as myData. Newtonsoft deals with this, because it’s a common use case in .Net; however, System.Text.Json.Deserialize* assumes that the casing will match the object!

This is odd considering it changes it on the way out!!

Okay, so what’s the fix?

You just tell Deserialize* to use camelCase:

            var client = _clientFactory.CreateClient();
            var result = await client.GetAsync($"https://localhost:22222/endpoint");

            using var responseStream = await result.Content.ReadAsStreamAsync();

            var options = new JsonSerializerOptions()
            {
                PropertyNamingPolicy = JsonNamingPolicy.CamelCase
            };
            var data = await JsonSerializer.DeserializeAsync<DataContainer>(responseStream, options);

Annoying, eh?

References

https://docs.microsoft.com/en-us/dotnet/standard/serialization/system-text-json-migrate-from-newtonsoft-how-to#table-of-differences-between-newtonsoftjson-and-systemtextjson

https://docs.microsoft.com/en-us/dotnet/standard/serialization/system-text-json-how-to#use-camel-case-for-all-json-property-names