Daniel Little Dev

Don't Ignore Your Functions

Mon, 05 Oct 2020 20:00:00 GMT

Ignoring return values can often be dangerous in subtle ways but you may be so used to doing it that you don't even notice anymore. It's likely that at some point you have run into an issue caused by forgetting to use the return value of a function. It's even more likely you throw away the return values of functions without thinking too much about it. Most of the time there's no real impact of doing so, and if there is it's certainly not immediate. But if you find yourself doing this often it is important to ask why. Presumably, the function is returning this value for a reason? Is there something wrong with the design of the API, or are you missing something important?

Adding to a date doesn't change it

Have you ever seen code like this?

var date = new DateTime(2000, 01, 01);

date.AddYears(1);
Console.WriteLine($"{date.ToShortDateString()}");

It looks pretty straightforward, but there's a bug in this code. If you're familiar with the dotnet DateTime API it might be an obvious one but it can be subtle and confusing if you've never used it before.

The issue is that when you call AddYears it doesn't modify the date, instead it returns a brand new date. Therefore when WriteLine is called the value will still say 2000/01/01 instead of 2001/01/01 like you may have expected.

To get this to work correctly you'd have to capture the new date by assigning the new value to the date variable, like so.

var date = new DateTime(2000, 01, 01);

date = date.AddYears(1);
Console.WriteLine($"{date.ToShortDateString()}");

So why does AddYears return a new date object instead of adding a year to the existing date? It does this because date is an immutable type. This means that once you create a date you can never modify or change it, in any way. If you need a different date you'll always need to create a new one.

Immutability itself is a very useful technique because it can help manage complexity due to limiting the possibilities you have to consider; there is only one way to change the date, replace it. However, issues like the example above can be hard to find if you're not looking for them. Wouldn't it be great if the C# compiler could detect issues like this and prevent you from making the mistake in the first place!

Async and you forgot to await

Let's look at a different problem for a moment. Say you had an async function which calls out to two async functions but you forget to await one of them.

async Task<ActionResult<Resource>> GetResource(string id) {

	AuditRequestAsync(id); // Forgot to await this, oops

	return await LoadResourceAsync(id);
}

In this case, you'll get a warning from the C# compiler telling you that there might be a missing await. This is fantastic because the majority of the time this is almost certainly a bug. However, depending on how closely you monitor your warning, it's still possible to compile the app with the default compiler options. But what happens if we have the same function but without the async keyword, like so.

Task<ActionResult<Resource>> GetResource(string id) {

	AuditRequestAsync(id);

	return LoadResourceAsync(id);
}

Semantically this function is exactly the same, it also suffers from the same bug but this time there's not even a warning. I've found this kind of issue to more common than it appears because a function can start off by only returning a task without needing the async keyword. In the example above if the AuditRequestAsync function was added later on, or by a different author, they could forget to add the async keyword and the program will still happily compile.

To make matters worst, the function might work in some cases, but fail in others. The AuditRequestAsync function will still run, but without await there is no guarantee the caller will still be around when it finishes. In some cases you might get an error regarding multiple active result sets if they both make database calls. In others, you might not notice anything is wrong at all. Issues like these can often lie dormant until other changes trigger them, or result in indeterministic (or at the very least non obvious) behaviour making them extremely hard to track down and fix.

Implicitly ignoring functions is dangerous

What these examples have in common is that the value returned by a function (AddYears and AuditResourceRequestAsync) was implicitly ignored, resulting in a bug. If the compiler had issued a warning or an error indicating that the value was unused or implicitly ignored these issues could have been caught earlier or prevented entirely.

There are also many more scenarios that suffer from this problem. For example using APIs like LINQ, Reactive Extensions, using Structs, and all immutable types, are cases where forgetting to use the value is almost certainly a bug. Even regular functions, particularly those that return values without side effects would benefit from making it obvious that a return value was ignored or forgotten.

Explicitly ignoring functions is safer

Instead of implicitly throwing away unused values, if we don't want to use a value we should explicitly discard it.

To help catch unused values you could use a custom analyzer to create a warning for all unused values, not just a missing await inside an async function. There are no analyzers that do this yet, however, there are a few similar existing analyzers for some of these cases, such as async await.

Once there are warnings for unused return values it becomes clearer that an unused value is either a bug or something that can be explicitly discarded.

A task, for example, would result in a warning about a possible missing await. If you do want to ignore the task then you can use a standalone discard to let others know you don't care about it.

_ = Task.Run(() => { ... }) // Explicitly discard the result

This makes it clear that you made a conscious decision not to use the value. It shows other developers that a decision was made not to use the value as opposed to forgetting to use it.

When someone else reads that code it is the difference between:

Did they forget to use the return value? I'll need to investigate, versus...
They've chosen to ignore the return value. I can move on.

Making things more explicit will prevent bugs and save you and others a lot of time.

Would this really work

There's a lot of code out there today which was written without explicit discards in mind. And I don't expect the world of C# to change drastically overnight. Nevertheless, the aim of this article is to get more you interested in an analyzer that warns developers about unused return values in the same way your editor warns you about an unused variable today.

You may still be wondering if so much C# code was written without explicit ignores in mind, would this be even practical to do in C#? Recently I've been doing a lot of dotnet development using F# which does emit compiler warning if a function return value is not used. So I can say that even with the dotnet as it exists today, I have been pleasantly surprised. I didn't need to discard half as many values as I thought I would.

The large majority of code didn't need a single discard. There were only a few cases where I needed to use a discard, for example discarding the value at the end of a mutable fluent API.

In that case, I would use an extension method to explicitly "cap off" the expression.

builder
    .RegisterType<HttpClient>().AsSelf()
    .InstancePerLifetimeScope()
    .Ignore() // <- Takes a type <T> and returns a void

I did however still run into at least one bug where I explicitly discarded a value that I shouldn't have. In the end, I found that even explicitly discarding values was something that should be done sparingly. Whenever I discarded a return value I found myself rethinking the code instead.

I was more likely to use or log a status code or return code.

var response = await httpClient.PostAsync(url, null);
log.Information("Responded with {StatusCode}", response.StatusCode);

If I wanted to run a task in the background I kept the task in a Task service and it simplified error handling for all background tasks.

BackgroundService.Register(async () => {...})

// Instead of

_ = Task.Run(async () => {...}) // Hope you handled the exceptions in here

If I used the builder pattern or a fluent API I considered using an immutable API and returning the value instead of a using a mutable one. For example using LINQ vs using the List API.

public IEnumerable<Out> GetResults(IEnumerable<In> items) => items
	.Where(x => ...)
	**.Select(x => ...)
	.OrderBy(x => ...)

I'm also cautious of functions that don't return values but that's another story.

I'm interested but maybe not convinced

Whenever I first stumble across a new concept or technique I find that I need to play with it for a while to start to get a good feel for how it works. Have a go at building an analyzer and see where the warnings are. Try writing a program from scratch with the warnings on. Turn on warnings as errors so you aren't tempted to take shortcuts and follow where the analyzer takes you.

But most importantly, if you find yourself discarding values without using them, ask yourself why.

Type-Safe Hypermedia Controls

Sun, 12 Jul 2020 08:00:00 GMT

Hypermedia Controls in REST provide links and actions with each response pointing to related resources. This concept is a powerful tool that enables the server to remain in control of links and access. If you're not sure what I mean by this, take a moment to read my previous article on my approach regarding Practical Hypermedia Controls.

Enter Typescript. Adding types to JavaScript allows us to write safer code, so naturally I wanted to figure out how to make a Hypermedia Controls type-safe. Typically when writing a type-safe API you might start out with something like this.

const api = (fetch: Fetch) => ({
  getAccount: async (id: string) : AccountResource => {
    const response = await fetch(`/account/${id}`, jsonGetOptions);
    return response.json() as AccountResource;
  }
  updateAccount: async (account: AccountResource) : AccountResource => {
    const response = await fetch(`/account/${account.id}`, jsonPutOptionsWith(account));
    return response.json() as AccountResource;
  }
})

In this example, the Request and Response types are strongly typed but the URL is hardcoded and you can't tell when you'd have access or who has access to this endpoint. Using Hypermedia Controls can provide that missing context. Links and actions are related to a resource, and the server can dictate if or when the various links and actions are available.

In order to see how to combine these two concepts, let's look at what the usage of type-safe Hypermedia controls would look like.

// Fetch a root resource
const account = await fetchAccount(url)
const accountHypermedia = accountHypermedia(account)

const accountUpdate = { ...account, email: "daniellittle@elsewhere" }

// Invoke the Update action
const updatedAccount = await accountHypermedia.actions.update.call(updatedAccount) // <- Typechecked!

In this example the account email address is being updated and then the Hypermedia Controls are used to update the resource. Fetching the account resource contains the raw Hypermedia Controls but passing the resource into the accountHypermedia function transforms the hypermedia into a type safe API style object. There are a few interesting things happening here, so let's peel back the covers.

Types for Links and Actions

First, Let's take a look at what the AccountResource type looks like.

export type AccountResource = {
  id: Identifier
  username: string
  name: string
  email: string

  _links: {
    self: Relationship
  }
  _actions: {
    activate?: Relationship
    deactivate?: Relationship
    update: Relationship
  }
}

The Hypermedia Controls are statically defined inside of the _links and _actions properties which contain all the well known relationships. Looking at these relationships we can see the resource always contains a self link and an update action but optionally contains activate and deactivate. It's important to note that the types are only coupled to the names of the relationships (rels) and their optionality. The server still controls the URLs and the presence of optional relationships.

Be cautious of making the types for the Hypermedia Controls too dynamic. On my first attempt at adding types for hypermedia, I used a more general dictionary type for links and actions. My thinking was that this would more accurately model the changing and dynamic hypermedia relationships a resource would have.

type Relationships = { [rel: string]: Relationship | unknown }

This assumption quickly turned out to be false and worked against the goal and benefits of strong typing. The relationships were not as dynamic as I had originally assumed. Links and actions don't change frequently, so you can safely define them as part of the type. Another downside was that you can't easily see what relationships are contextual and which ones are always present.

Hypermedia is often taken a bit too far and is often associated with machine-readable metadata or form builders. My advice here is to avoid designing your types for general hypermedia clients. Instead, think of these types as representing a well defined and static contract between the client and the server.

All links and actions use the Relationship type which represents the relationship and its location. A relationship can be either a simple URL or a contain extra info such as the Method or Title.

export type Href = string
export type DetailedRelationship = {
  href: Href
  method?: Method
  title?: string
}
export type Relationship =
  | Href
  | DetailedRelationship

I usually use the DetailedRelationship type but sometimes it's conventient to only provide the URL for links, which typically use the GET verb.

Contextual Relationships

In the AccountResource above you can see there are three potential actions. The update action is always available but activate and deactivate are optional so the client only has to check for the presence of the optional relationships. The server can then decide when these optional actions are available, enabling the actions for the client based on the state of the resource.

const account = fetchAccount(url)
const accountHypermedia = accountHypermedia(account)

if (accountHypermedia.deactivate) {
  // The account can be deactivated!

  await accountHypermedia.deactivate.call() // <- Also Typechecked, no request payload is needed!
}

In this sample, deactivate has to be null checked before it can be used. The call function also knows that deactivate takes no payload and what the return type is.

Creating a Hypermedia Model

Next, let's look into the accountHypermedia function, which does the heavy lifting of transforming the resource with hypermedia into a typed hypermedia model containing all the links and actions. To make the conversion easier I've also written a function createHypermediaModel which helps to create the API for a resource.

type none = void // Used when a Request requires no payload (function <T>(arg: T) would need no arguments)

const accountHypermedia = createHypermediaModel((resource: AccountResource, resolve) => ({
  links: {
    self: resolve<none, AccountResource>(resource._links.self)
  },
  actions: {
    deactivate: resolve<none, AccountResource>(resource._actions.deactivate),
    update: resolve<none, AccountResource>(resource._actions.update)
  }
}))

You can view this code as a mapping from the resource to a set of, ready to use, functions. The resolve function takes the relationship and returns an object containing a strongly typed call function as well as the href and title if one was provided.

resolve<none, AccountResource>(resource._links.self)

Note: In Typescript, you are able to pass through a generic function as a parameter. The resolve parameter makes use of this to compose (an instance of) fetch and the request/response types.

The ResolvedRelationship makes it convenient to access the href and other metadata if you only have access to the hypermedia model.

export type ResolvedRelationship<Request, Response> = {
  call: (request: Request) => Promise<Response>
  href: string
  title: string | undefined
}

I use href from the ResolvedRelationship to follow links to different pages by changing the URL. This means exposing the Method isn't nessesary as they are always GET requests.

Multiple Resources

The createHypermediaModel function focuses on creating a hypermedia model for a single resource. In order to create a model for an entire API you can use a createApi function to create a single object composing the sub-APIs for each individual resource.

export function createApi(fetch: Fetch) {
  const resolve = createResolver(fetch)

  return {
    getAccount: (url: string) => resolve<none, AccountResource>(url).call(),
    accounts: accountHypermedia(resolve),

    // More models go here!
  }
}

That covers all the main pieces of using createHypermediaModel to build a type-safe hypermedia API. Please let me know if you liked this approach as I'm considering wrapping this up into an npm package. However, I've glossed over the detail of how createHypermediaModel works. It's mostly the glue and pluming but there are a few interesting parts. Feel free to read the apendix below if you'd like to dig deeper under the covers.

That's all I have for now and as always thanks for reading!

Apendix: Deeper into the Code

Here is the bulk of the code, feel free to skim over it and jump to the alaysis at the bottom.

export type JsonFetch = <Request, Response>(method: Method, url: string, data?: Request) => Promise<Response>
export type Resolver = <Request, Response>(relationship: Relationship) => ResolvedRelationship<Request, Response>

export const getHref = (rel: Relationship) => (typeof rel === "string" ? rel : rel.href)
export const getTitle = (rel: Relationship) => (typeof rel === "string" ? undefined : rel.title)

export const createResolver = (fetch: JsonFetch) => <Request, Response>(
  relationship: Relationship
): ResolvedRelationship<Request, Response> => {

  const apiCall = async (request: Request) => {
    const rel: { href: Href; method: Method; name?: string } =
      typeof relationship === "string"
        ? {
            href: relationship,
            method: "get"
          }
        : {
            ...relationship,
            method: relationship.method || "get"
          }
    const response = await fetch<Request, Response>(rel.method, rel.href, request)
    return response as Response
  }

  return {
    call: apiCall,
    href: getHref(relationship),
    title: getTitle(relationship)
  }
}

export const createHypermediaModel = <Resource, T>(
  builder: (resource: Resource, resolver: Resolver) => T
) => (resolver: Resolver) => (resource: Resource) => builder(resource, resolver)

The code is written in a functional programming style and functions declared before they are used. Therefore it is usually easier to look at functions starting from the bottom and going up.

The first function is, therefore, ceateHypermediaModel, which uses a bit of currying so the resolver and resource can be provided at different times. Dependencies such as Fetch and the Resolver are threaded through the call stack so no global references are needed.

The other main function is createResolver which constructs the ResolvedRelationship. Its main job is to wrap up the call to fetch using the given relationship and the request/response types.

Practical Hypermedia Controls

Tue, 07 Jul 2020 08:00:00 GMT

A lot has been written about REST but less so when it comes to Hypermedia Controls. I haven't seen too many Hypermedia based APIs out in the wild. I theorize that there are two main reasons for this. First, it is something many people haven't been exposed to, and second, it requires a little more up-front effort in order to get the ball rolling. However, I believe it's an incredibly useful pattern and it's easier to get started than you might think. This post aims to help out with that first problem, exposure. We'll take a look at what Hypermedia Controls are and why they're useful.

Whatever the reason for the rarity of Hypermedia Controls, what you might commonly see in a REST API instead, is something like this.

{
  "id": "0001"
  "username": "daniellittledev"
  "name": "Daniel Little"
  "email": "daniellittle@somewhere"
  "canUpdate": true
  "canDeactivate": false
}

This JSON object represents a user account. In this system, there are a few simple rules. Only the owner of the resource can update it and only admins can deactivate or activate accounts. This resource uses boolean flags to inform the client what state the object is in and what actions it can perform.

However, there are some problems with this model.

We can't tell if the current user should be able to activate or deactivate the account.
We can't tell if the account can be activated, we have to assume it can be because we can't deactivate it.
We don't know how to update or deactivate an account, we'd need more external information like the URL.

Wouldn't it be nice if all these things were a bit easier to figure out? I think so too!

Enter Hypermedia Controls

Hypermedia Controls in REST APIs are all about adding links and actions to a resource instead of needing to interoperate what links and actions are available based on state. Examples of state are boolean flags like above or a status value like ready or pending. Here's what the same resource would look like if we added Hypermedia Controls instead.

{
  "id": "0001",
  "username": "daniellittledev",
  "name": "Daniel Little",
  "email": "daniellittle@somewhere",

  "links": { ... }
  "actions": {
    "update": { "href": "/account/0001", "method": "put"}
    "activate": { "href": "/account/0001/activate", "method": "post" }
  }
}

Now we can clearly see which actions are available. There are two available actions, update and activate, along with the URL and verb needed to make the calls. Instead of the client needing to infer this from the data, the server directly dictates what actions are available. We don't need to assume the client can activate the account if canDeactivate is false. There's also no danger of adding conflicting flags. The client can directly show or enable buttons or links directly based on what Hypermedia Controls are present.

Let's look at our previous problems again, now that we're using Hypermedia.

There is no action for deactivating the account; so the user can't see or perform that action.
There is an action to activate the account; so the user should see and perform that action.
We know the URL and method, so the client doesn't need hardcode or build-up any URLs.

Much better! This also helps out immensely when debugging or looking at the REST API directly. There's no need for all that extra context to be extracted from the client because it's all there in the resource.

Also, note that to perform an update you would simply modify the resource as desired and send it right back. You can optionally remove the links and actions, but otherwise, they're typically ignored.

All this makes the API much easier to reason about and use. But the key benefit is the client no longer needs to interpret any state to figure out what is going on. We're able to keep the knowledge, of who can do what and when, on the server without having to duplicate any logic into the client. And while this example is a fairly trivial one, a few more flags like the ones above or worse, enum states where the client has to decide what buttons are available based on states like Pending, InProgress or Completed, can easily increase complexity and the burden of keeping everything in sync.

More Control

We can also do more than just make links and actions available or unavailable. We could also show that an action exists but isn't available. On the UI this could manifest as a visible but disabled button.

"actions": {
  "update": null // or { disabled: true }
}

If you'd like things to be a bit more dynamic or server controlled you can also include a title for an action which could be used as the label for a link or button.

"actions": {
  "update": { "href": "/account/0001", "method": "put", "title": "Update Account" }
}

It's really up to you to choose how much information is provided to the client. However, I'd be cautious of adding too much. You might be tempted to include other metadata like field types or validation rules. But the main goal here is to keep duplicate logic out of the client and keep the server in control of application state. Decoupling too much tends to lead to the Inner Platform Effect.

Client-Side Routing

There's one unexpected question I came across while implementing a Hypermedia based API. Which is how they affect client-side routing. I've found keeping API URLs in sync with the browser URLs to be a good approach. Or to be more accurate, I believe they should be the same URL. If you're coming from a non-hypermedia based API your URLs will most likely have diverged so I'll demonstrate what this looks like in practice.

The root hypermedia resource is the starting point for the API, it provides all the top level links you'd need to get started.

// Request:
GET "/"

// Response:
{
  "links": {
    "my-account": "/account/0001",
    "accounts": "/account"
  }
}

Following a link such as my-account (which the client knows by name) would update the browsers route to match, eg. /account/0001. The client-side routing rules mirror the server-side rules to render the relevant components but simply forward the URL directly to the API to fetch a resource. Note that the value here is that the client doesn't have to hardcode, reconstruct or build any URLs.

// Request:
GET "/account/0001"

// Response:
{
  "links": {
    "self": "/account/0001",
    "home": "/"
  }
}

Actions are a little different and don't usually represent a route change. I recommend using a get-post-redirect style pattern so that a link (self) or a redirect is used to determine the next URL for the client.

Try it out

Next time you're building a form with a few actions especially if those actions change under different conditions or states consider using Hypermedia Controls. I hope you enjoyed this article, and if so make sure to keep an eye out for my next post on Type Safe Hypermedia with Typescript.

Power Query - The M Language

Mon, 22 Jun 2020 10:40:00 GMT

Recently I had the opportunity to write a Custom Connector for Power BI and come across something I didn't expect. To build a Connector you use Power Query also known as the M formula language. Power Query was designed to build data transformation pipelines, focusing on repeatable importing and manipulations of tables.

A core capability of Power Query is to filter and combine, that is, to mash-up data from one or more of a rich collection of supported data sources.

But what I didn't expect is that Power Query is a lazy functional programming language. The documentation describes it as "a functional, case sensitive language similar to F#". So not only does Power BI include a programming language it's also a functional programming language! Although if I had to describe it, I'd say it's more of a simplified cross between JavaScript and Haskell. Curious? Well then, let's take a quick look at what Power Query looks like, and some of its features.

The first thing you might notice about Power Query is that it's almost entirely expression-based. Every function is comprised of a single expression.

Function = () => "Hello"

Function() // returns "Hello"

Here we have a global variable called Function which contains a function which returns "hello". All functions are lambdas.

One of the most common building blocks is the let expression, which lets you build up more complex functions using variables.

let
   hello = "Hello",  
   world = "World"
in   
   hello & " " & world & "!"

If you're not familiar with the let syntax it's quite common in expression-based functional languages. A let expression allows you to define and bind other expressions for use in the in clause. You can read it like, let these "names" be bound to these "values" in this "expression".

Let's look at a more complex function. The function below makes a web request to a given URL and returns the result as JSON.

Fetch = (url as text) as record =>
    let
        accessToken = Extension.CurrentCredential()[access_token],
        authHeader = "Bearer " & accessToken,
        response =
            Web.Contents(url, [
                ManualCredentials = true,
                Headers = [ Authorization = authHeader]
            ])
    in
        Json.Document(response);

There are a few things going on in this function but one of the interesting things to point out are the two main types you see in Power Query, apart from tables of course. These are lists and records, and the interesting bit is that the symbols are opposite to what you might expect if coming from most other languages.

Lists are defined using braces {} and records are defined using square brackets [], the exact opposite of JavaScript or FSharp.

list = { 1, 2, 3}

record = [
    Property = true,
    AnotherProperty = "Hello"
]

Accessing a property is also a little different and also uses [].

value = record[Property]

In the Fetch function above you can see the access_token property being used via the credentials record.

Extension.CurrentCredential()[access_token]

Briefly, on types, the language is dynamic with optional types and can also be extended with custom metadata. Below the function starts without any types and you can see how the code changes as I add more type information.

(url) => ...expression

(url as text) => ...expression

(url as text) as record => ...expression

(url as text) as record meta [ Documentation.Name = "My Record Type" ] => ...expression

Now, I've saved the best until last. One of the most powerful features of Power Query is that it's lazy. Take for example the following code.

let
    pets = {
        Fetch("https://api.service.com/dogs"),
        Fetch("https://api.service.com/cats")
    }
in
    pets{0}

In this expression, we have a list of pets which contains the result of some API calls to a web service. However, only the first element of the array is ever accessed and because Power Query is lazy the second fetch will never actually be evaluated. In a custom connector, this could be used to access a paged rest API in order to import a table. And because evaluation is lazy using the standard take function will only fetch the pages that are needed are fetched.

It's exciting to see functional programming, especially a language as interesting as Power Query, popping up in the mainstream. I had a blast learning about Power Query and I learnt so much along the way. If you're interested in trying out a functional first language, whatever it may be, I encourage you, now is the time to check one out!

Environment Settings for Build Once Packages in React

Tue, 14 Jan 2020 17:00:00 GMT

Let's look at some questions:

How do you specify which API to use for an environment?
How do you configure the environment settings of frontend apps for each environment?
How to you achieve: build once, deploy everywhere packages for a frontend app?

Most frontend applications currently take the approach of building a package for each environment. This approach is reasonably well supported by Webpack and it is a common approach. There's also a lot of workarounds such as sniffing the URL, injecting extra scripts or using a find and replace script on the bundle. These approaches can provide a single package but are often custom solutions for individual applications.

I think there are two main reasons for this, the first, is that when you create-react-app out of the box there is no built-in support for build once, deploy everywhere. You have to fend for yourself. But second, and possibly, more importantly, it is more difficult to set up runtime style environment settings needed for build once, deploy everywhere packages.

Build Once, Deploy Everywhere

As this approach is less common, let's look at what exactly this is, why it's important and some of the benefits of this approach. The build once, deploy everywhere pattern refers to the building a single deployment artefact which can be deployed to one or many different environments.

Shorter build times

Building packages is expensive, building a package once saves quite a bit of time and compute. If your build takes 40 seconds, and you have two environments, the total build time is effectively doubled to over a minute. Fast builds and deployment times are also important for low deployment lead times and short development cycles. Today it's also common to have you build/ci agent in the cloud where we'll typically pay per minute or per agent for parallel builds. Here short build times have an even greater impact on development and costs.

Exact same code

By building the package once and using that same package in every environment, you can guarantee they'll all be running the exact same code. There are multiple reasons why you might end up with slight unexpected variations such as non-deterministic compilers, resulting in different optimisations. Dependencies could change, for example, if the build step runs an auto package restore or the build script relies on an external service. There may also be bugs in the tooling which can be unforeseen. By building the package once, there are fewer opportunities for things to go wrong, which means less risk.

Simplified deployment pipeline

Modern deployment pipelines such as those in products such as Octopus Deploy or Azure DevOps support environment based deployments, deployment promotion and variable substitution. These tools assume a single artifact or package which can then be deployed to an environment. Variable substitution is applied at deployment time after the package has been created, typically as a final configuration step during deployment. Taking advantage of these features results in a simplified deployment pipeline. The tooling can often assist with promoting to the next environment or marking the package as containing defects, managing the lifecycle of the package for us.

Loading Environment Settings in React

In order to load the settings, I've gone with the approach of using a single file to keep loading times as low as possible.

There is a single settings.json file which contains the environment you want to load settings for, as well as blocks containing the default and environment settings. In the example below the environment is development which is what I'd also check into version control.

// settings.json
{
  "environment": "development",
  "default": {
    "api": "defaultapi.com",
    "siteName": "Example Site"
  },
  "development": {
    "api": "developmentapi.com",
    "banner": "You are in the Development Environment"
  }
}

On deployment, environment property is replaced with the appropriate environment name. Settings can also be added or modified. Then, when the application is run the application, I'll get the combination of (using spread internally) the default and matching environment blocks.

To tie it all together, I'm using a component called AppSettingsLoader which loads the settings given the URL of a settings file. It also takes three other props to determine what to show while the settings are being loaded or unavailable, what to render once the settings are available, and optionally an error prop.

import AppSettingsLoader from "react-environment-settings";
import settingsAssetUrl from "./settings.json.txt";

interface Settings {
  api: string;
  banner: string;
}

<AppSettingsLoader<Settings>
  settingsUrl={settingsAssetUrl}
  loading={() => <div>Loading settings...</div>}
  ready={s => <pre>{JSON.stringify(s, null, 2)}</pre>}
/>

The AppSettingsLoader is also a generic component; in this example, you can also see the Settings interface used to specify the type of the combined settings. Therefore, the final product of the Settings for all environments need to match a common interface.

<AppSettingsLoader<Settings> ... />

In order to load the settings themselves, there were a number of issues I had to work through. I wanted to keep the settings file separate to the application bundle, but json files are bundled if you import them. I also wanted to avoid caching issues so I couldn't just move the file into the public folder and fetch it from there.

In order to load the settings file with Typescript and Webpack I was able to use the Importing Other Assets functionality which lets you import a file, in this case, a txt file. Importing the file will cause it to be processed giving it a unique filename, but unlike a json import the file contents are unmodified and the file is stored in the media folder.

> build
  > static
    > css
    > js
    > media
      > settings.json.4355cbd.txt

Above is an example of where you can locate the file after a build. As you can see the settings file is easy to locate, having a similar but also cache busting filename. This means it's easy to locate the settings file and perform environment name and variable substitution.

Importing the file in Typescript also requires a module declaration.

// global.d.ts
declare module '*.txt' {
  const content: string;
  export default content;
}

And when the file is imported, you'd receive the path to the file which can be passed to the AppSettingsLoader and loaded via a fetch.

import settingsAssetUrl from "./settings.json.txt";

You could also provide a Url from another source if needed.

Finally, by using AppSettingsLoader at the root of an application, you're then able to breifly delay running the application until the settings have been loaded and are available.

<AppSettingsLoader<Settings>
  settingsUrl={settingsAssetUrl}
  loading={() => <div>Loading...</div>}
  ready={s => <App appSettings={s} />}
/>

Installation and Code

If you'd like to try it out, you can install this package.

npm i react-environment-settings

I won't go into all the code, but you can find the full project on GitHub daniellittledev/react-environment-settings. Originally I was only planning on sharing my approach with a blog post, but it just wouldn't be complete without a package on npm.

I've tried to keep the code relatively simple, so I'll just share a few key parts.

The bulk of the logic is inside two main files, loadSettings.ts which loads the merges the settings, and index.ts which contains the component itself.

Below is the main load settings function. Here I'd like to highlight in terms of error handling, fetch and loading issues are both caught at the top level of this function.

export async function loadSettings<T>(
  settingsUrl: string
) {
  try {
    const settings = await getSettings(settingsUrl);
    const allSettings = getSelectedSettings(settings);
    const mergedSettings = mergeSettings(allSettings) as unknown;
    return success<T>(mergedSettings as T);
  } catch (ex) {
    return error<T>(ex);
  }
};

This error is then directly accessible using the error prop on the AppSettingsLoader component.

switch (settings.type) {
  ...
  case "Error":
    return props.error ? (
      props.error(settings.error)
    ) : (
      <div>Error loading settings</div>
    );
}

So you have full control if something goes wrong.

Wrapping up

Good luck out there with your settings loading journey. If you're trying to get a new project up and running or updating an existing project, I hope this post has helped you.

Practising Continuous Deployment

Mon, 21 Oct 2019 17:00:00 GMT

I've long been an advocate of Continuous Deployment, but it's always felt somewhat out of reach. I've come close with projects that have had decent test coverage and automated deployments, but there was always something missing. There was always still a button you needed to click to promote from test into production. Often unreleased code could build-up, resulting in a bigger riskier release or more manual testing.

Continuous Deployment can be difficult to adopt because you need to build reliability and confidence into your build and deployment pipelines. But more difficult is the culture shift, it's a different way of working and you can't just slap some tests on it and declare Continuous Deployment. Buy-in is important and so is process, because while Continuous Deployment is positively self-reinforcing, not practising it is negatively reinforcing. You need critical mass, and you need to jump the gap quickly. So while I've come close, it remained difficult to bridge the gap into the magical world of Continuous Deployment.

However, during my time with the wonderful folks at Stacktrace, we practised Continuous Delivery amongst a team of eight developers. To date, it's that only team I've joined that was already practising Continuous Delivery. It was everything I hoped it would be. And since then I've brought a lot of what I learnt with me, continuing to put Continuous Delivery into practice.

So I'd like to tell you about the strategies we used.

Shared code ownership

Everyone was equally across most areas of the codebase, and everyone felt comfortable working on any feature or change. We all paid attention to quality and avoided cutting corners. This made everything easier; you can take for granted cohesion, consistency and tests to cover your back. We also made sure to keep it that way.

Always production-ready

All commits are production-ready, you know that each push could potentially make it to production. If code isn't ready to go live, then it's either not wired up to the UI, silently running beside an existing version or behind a feature toggle. This ensures each release is small and all code is continuously integrated, so everyone sees the latest code even if it's not finished.

Features also need to be backwards compatibility typically through parallel implementations or on-the-fly database migrations.

Short-lived branches

Branches are kept small and short-lived typically only lasting for 1-2 days. They're also only used for Pull Requests and usually don't represent a full or complete feature. As long as they're production-ready they're good to go.

Rebasing onto master

Master is always moving forward and in order to get value out of continuous integration, you need the latest code on your machine. We frequently rebased development branches onto master. To make this even easier, I'll use a function like fmar, which stands for fetch master and rebase.

function fmar {
  git fetch
  git rebase origin/master
}

Quick Code Reviews

All code is merged into master from a PR, and each PR must be reviewed by someone else. Sometimes we'd pick someone familiar with the change and sometimes we'd pick someone different to share things around.

We also treated code reviews as a priority. Initially it's the responsibility of the author to find someone to review the PR. Once you agreed to review a PR, it was then up to you to review it promptly or hand the PR back if something unexpected came up.

We also tried to as much as possible give feedback as advice. PRs were almost always approved, instead of blocked, but most feedback was actioned regardless. Sometimes in a follow-up PR instead of delaying an existing PR.

Clean history

The key to quick PRs is code and commits that are easy to read and easy to follow. We would regularly rebase the commits in a PR so commits would tell a story or make a change easier to follow. And because branches were short-lived force pushing isn't a problem. We'd also use git pushfl or git push --force-with-lease for added safety, which aborts if anyone else has pushed to that branch.

Automated Code Formatters

Another tool we used to keep PRs quick was automated code formatting. This ensures that all code is consistent and that no one needs to waste time on style discussions in a PR.

Robust Test Suite

A good test suite is invaluable for continuous integration and deployment. Many changes can be verified via the tests without needing to run the application and perform lots of slower manual testing. Every change is backed by tests, and the tests are first-class code which is also actively maintained and improved.

Fully Automated deployments

Setting up fully Automated deployments is a key aspect because it reinforces many of the concepts above. Completing a PR doesn't just sit in test until it's ready. Keeping commits production-ready is more than just an ideal. The PR will go all the way to production, and it needs to be production-ready.

This is also where a robust test suite really becomes indispensable. Automated builds always run before PRs can be integrated, and deployments are exercised by deploying to a test environment before they're deployed to production.

Testing in Prod

The last point I want to mention is testing in production. Having a test tenancy in the production environment allows for the testing and review of new functionality and changes that have not yet been enabled for all users. This was invaluable to exercise the system from end to end with real data and with all the real dependencies.

A Reflection

I found it an incredibly valuable experience to practice continuous deployment, but even more importantly, the practises that support it, make it possible and sustain it.

Joining a team and learning through immersing myself in a culture that was already practising helped to bring these strategies into focus for me.

I find that sometimes, it is hard to do things one step at a time because they can all seem to depend on each other. In the case of Continuous Deployment I still think that's true to some extent. There is a gap you have to jump to truly adopt Continuous Deployment. But I feel I have a much better idea of what makes Continuous Deployment successful.

I'd also love to hear about what makes Continuous Deployment work for you. If you think I've missed something or one thing really stands out for you tag me on twitter @daniellittledev.

How to run Expecto with dotnet test

Mon, 07 Oct 2019 19:00:00 GMT

Expecto is a fantastic test framework and test runner for FSharp. But contrary to the name, the thing I like most about Expecto is that it contains so little magic. Your test project is a simple Console applications and to run you tests you just run the app. No Visual Studio plugins required.

dotnet run

However, recently I wanted to also run my tests using dotnet test to fit into a existing test suit and build script. By default Expecto tests won't be discovered by dotnet test but the good news is it's only two packages and an attribute away from working with both run and test.

I started with an application that used no auto test discovery opting for explicit test registration like the following.

let allTests =
    testList "all-tests" [
        BuilderTests.tests
        Convention.test
        Domain.tests
    ]
    
[<EntryPoint>]
let main argv =
    printfn "Running tests!"   
    runTestsWithArgs config argv allTests

We'll come back to this later, first we need to install some packages.

In order to set up tests with dotnet core we need to add the Test SDK to the project file.

<PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.3.0" />

However this will generate an entry point for your application and because we already have one we'll need to disable that by setting GenerateProgramFile to false in the project file.

<PropertyGroup>
    <GenerateProgramFile>false</GenerateProgramFile>
</PropertyGroup>

After that we'll need to add the package YoloDev.Expecto.TestSdk in order dotnet test to discover Expecto tests.

<PackageReference Include="YoloDev.Expecto.TestSdk" Version="0.8.0" />

This package exposes Expecto Test Attributes for discovery via the Microsoft Test SDK.

Finally, we need to make one change to the Program.fs file, adding the Tests attribute to the tests so they can be discovered.

[<Tests>]
let allTests =
    testList "all-tests" [
        BuilderTests.tests
        Convention.test
        Domain.tests
    ]
    
[<EntryPoint>]
let main argv =
    printfn "Running tests!"   
    runTestsWithArgs config argv allTests

Now when we run dotnet test we see the following:

> dotnet test
Test run for App.Tests.dll(.NETCoreApp,Version=v2.2)
Microsoft (R) Test Execution Command Line Tool Version 16.3.0
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...

A total of 1 test files matched the specified pattern.

Test Run Successful.
Total tests: 31
     Passed: 31
 Total time: 1.3619 Seconds

Success!

Reliable Messaging over a Service Bus

Thu, 16 May 2019 18:00:00 GMT

When working with a traditional database inside of a monolith application, we don't usually need to think too much about how transactions work. We can rely on a transaction to either roll back all the changes or commit them. Either way, our model is always consistent.

More recently, many applications have become distributed in one way or another. Many use a Service Bus or a REST API to communicate with other external systems. Communication between these systems is not transactional, so they won't be able to share a transaction and rollback if there are failures. So we'll need a method of reliably sending messages from one system to another. In this post, we'll be looking at how to implement reliable messaging between two applications using a Service Bus.

What's the problem?

When sending messages over a Service Bus there are a number of failure points to consider. If there is a failure at any of these points then information could be lost, state could be corrupted or the process may stall.

We need to do three things when processing a message

Execute domain logic

Send new messages

Save updates to the database

The core of the problem lies with the fact that there are two data-stores, the primary database and a (durable) queue or topic. However, it's only possible to write to one or the other first.

Send then Save

Let's look at what happens when we send messages first, then save to the database.

In this situation, we've sent out messages, but there was an error while writing the updated application state to the database. In this case, no updates are made to the database so as far as our application is concerned nothing happened, but we ended up leaking messages.

Now downstream systems may be working with inconsistent messages. The Service Bus may also redeliver the message, so the application can retry, but if the message is retried a different result might be reached because state and time will continue to change between attempts.

Save then Send

So sending messages first isn't too good, what if we save first instead? Then we wouldn't process invalid messages, and we'll avoid putting bad state into the database. Let's see what that looks like.

However, now we've got a new problem. We successfully save the state but if we failed to send the messages. If we retry now, we'd risk doing the work again, creating duplicates or sending different (invalid) messages. This is a risk because the database could again be in a different state to the first attempt. In fact, the database will always be in a different state because we just wrote to it!

We could also check to make sure we haven't already handled this message and abort processing, but then no messages would be sent at all. The process would stall, and data would fail to propagate.

Sending and Saving Atomically

What we really need is a way to do both at once. We need to store the updated state and any messages we intend to send inside a single transaction. We need this step to be atomic. If we look towards the database, we can achieve this by writing those messages to the database along with the state. In doing so, we won't leak messages before we save, and we won't lose them after either.

Using an Outbox

Saving our state and messages inside the same transaction is the core principle of the Outbox pattern. However, there's a bit more to do before we're finished. We still need to send the messages via the Outbox.

Once the state and the messages have been persisted the next step is to attempt to send the messages in the Outbox. After the messages have been sent to the Service Bus we need to make a second update to the database, marking the messages as sent (dispatched).

However, there are still three separate actions being performed here, and any one of them may fail. We're safe if the first step fails because our system will still be in a consistent state. No messages are lost or leaked. However, if the second step fails, we'll still need to send those messages.

Retry based dispatch

Unlike our earlier attempt of saving first then sending to the service bus, this time we can take advantage of our new Outbox! Now if we encounter an error, we can retry processing the message except we'll only want to send the messages we already have.

Asynchronous dispatch

Messages that aren't redelivered after a failure, require us to also use some method of asynchronous dispatch. Either because the maximum number of tried was reached or because the message isn't configured to retry at all. Asynchronous dispatch allows us to check for unsent messages that fall through the cracks.

At least once delivery

After all that, we should now be in a position where a message will always be delivered to the Service Bus, at least once. At least once delivery is an unavoidable property of distributed messaging. Furthermore, we've been relying on it a bit already. When messages aren't successfully processed, aka there's a failure before the message is acknowledged, we depend on the message being redelivered so we can retry.

So, on the delivery side, there could be a failure to deliver a message:

A failure while processing a message:

Or a failure when sending messages a message acknowledgment, which may be lost even though the message was processed.

This means downstream services may receive messages more than once.

Next up, On the dispatch side, if multiple messages are sent, some may get through, but a failure can make it impossible to know which messages are successfully delivered. Even if the messages are dispatched one at a time, messages may still be sent multiple times due to failures.

For these reasons (and few more I don't have time for right now) it's a good idea to make message processing idempotent. Or in other words, making them safe to process multiple times. As an added optimisation, we can also store the Id's of the messages we have successfully processed and use them to drop any duplicate messages. This can help us to avoid further unnecessary processing. Then we can jump straight to sending any unsent messages.

We can refer to this as the Inbox. We also don't need to keep things in the Inbox forever. I usually opt to keep a certain window, such as Id's no longer than 30 minutes old, or the first few thousand.

Using a relational database

We've talked a lot about a database but not what kind of database because all that matters is that the Inbox, State and the Outbox are all inside the same transaction. In a relational database, I'll usually use a separate table for the Inbox and Outbox. The Outbox will usually have the following columns:

- SequenceNo (Clustered Primary Key)
- MessageId
- AddedAt
- Metadata (json)
- MessageType
- Payload (json)
- Dispatched (bool)
- DeliveredAt (datetime)

You can also remove messages from the Outbox after they're delivered and processed if they're of no further use.

Using a document database

You can also use the Outbox pattern with a Document database. However, it's typical for document databases not to have cross-document transactions. In this case, we can still achieve our requirement of keeping the Inbox and Outbox inside the transaction by storing them inside each document instead.

{
  id: "product-1",
  name: "phone (model x)",
  stockLevel: 9,
  backorderAvalible: false,
  inbox: [ "m-1", "m-2", "m4" ],
  outbox: [
    { id: "m-5", type: "QuantityReserved" ... }
  ]
}

Many Document Databases will let you run queries so you can find unsent messages. If you're using Azure CosmosDB the change feed is a fantastic method of subscribing to changes through the database.

Final Notes

The real world often forces us to think about all the ways our software can fail. The Outbox presents a robust method of reliable message delivery in the face of failure. It's incredibly important to build this reliability into our systems. Otherwise, when things inevitably fail, data inconsistency and data loss will happen.

As an industry, I worry that it's all too common to see companies delay or avoid implementing reliable messaging. Instead, hoping that these failures won't happen (yet). And while some of these failures can be rare, the alternative is often much worse. When message silently go missing or messages are leaked it can quickly become a big issue. One that's often difficult to resolve.

Finally, I'd like to point out this certainly isn't the only way to tackle the problem of reliable messaging. But when it comes to messaging and a ServiceBus it's certainly an invaluable tool to have at your disposal.

Happy Outboxing.

Why you need Reliable Messaging

Wed, 16 Jan 2019 00:00:00 GMT

You might have heard the phrase "there's no such thing as reliable messaging". It's an interesting statement, but aside from being completely untrue, the statement is also entirely unhelpful. What you can't have is "completely reliable messaging", but the real question is how close can you get? How close do you need to get?

The concept of Reliable Messaging is broad and can be somewhat complex at times. I won't get into all the details just yet. Instead, in this post, I'm going to establish what exactly Reliable Messaging is and why it's so important.

What is Reliable Messaging

Reliable messaging is the concept of commutating with certain guarantees about the successful transmission of messages. For example, if the message is delivered, that it is delivered at least once. In contrast, this can be compared best-effort delivery, where there is no guarantee that messages will be delivered at all. I'm going to focus on message delivery and reliability, but reliable messaging may also refer to message ordering.

These guarantees don't need to be baked into the protocol or framework you're using either. An interesting property of reliable messaging is that you can add more reliability at higher levels. You can build reliable protocols on top of unreliable protocols. The best example of this is TCP which is built on top of the IP protocol.

The most important aspect of Reliable Messaging is, at the end of the day, the only way to know if something is done is by receiving a message saying it was done. But before I explore exactly why Reliable Messaging is so important, I'll go on a brief tangent to have a look at why you can't have Completely Reliable Messaging.

Why you can't have Completely Reliable Messaging

Reliable Messaging can't provide complete reliability for a number of reasons, but you only need one negative example to disprove something is not absolute. In a distributed system the receiver could be faulty, offline, or explosively dismantled and spread around all corners of the globe. Even if the receiver works perfectly, the communication medium may be unreliable in many different ways. The sender may also be unreliable, it may crash, or simply contain a bug or other logic error. You can't guarantee anything in a world where anything can fail, but you can plan for failures.

What makes Reliable Messaging important?

It is necessary to understand how a system can fail and how to deal with failures when they inevitability occur.

When building any kind of distributed system it is important to know what guarantees are needed. Otherwise, the default is always best-effort delivery. Which means the system will loose data, without anyone knowing when, why or how. Even if there are no errors, exceptions or obvious signs. If messages fail to be propagated you might not realise anything went wrong for days, or months, or at all.

Issues like this also tend to be expensive to track down and fix. Because they're found so late they may also end up being impossible to track down and fix. The core problem is, it is necessary to understand how a system can fail and how to deal with failures when they inevitability occur. In this sense, Reliable Messaging is a means to an end.

Given that the core problem is dealing with failures, there are situations where you just don't need reliable messaging. In some systems manual monitoring and alerts could be enough; however, this is usually more indirect and coarse. Some actions may not be important at all, in which case best effort would suffice. If an action is user-driven then the user could manually retry instead. Otherwise, if you don't want to be manually adding reliability, because that get's expensive really quickly, then having a good strategy for Reliable Messaging is very important.

Automatic ModelState Validation for ASP.NET Core

Mon, 02 Jul 2018 10:35:58 GMT

Model State Validation for a JSON API isn't too hard because you only need to send back validation errors, especially with the ApiController attribute. However, for all those that are still using server-side rendering and Razor, it's still a bit challenging.

It's challenging because you need to return the correct View after posting a form. And the easiest method of getting the job done is writing this type of code in each Action.

if (ModelState.IsValid)
{
    return RenderTheGetForm();
}

It does the job, but it's usually a lot of extra, tedious, code.

Instead we can keep the Controllers lean by shifting this code into an Action Filter attribute which can be used similarly to the ApiController attribute. So I decided to write one and publish it.

Here's how it works. You install the AutomaticModelStateValidation package and drop the AutoValidateModel attribute on the action that uses a View Model and validation.

[Route("/[controller]")]
public class SubmissionController : Controller
{
    [HttpGet]
    public ActionResult NewSubmission()
    {
        // Load data for the blank form
        return View(new NewSubmissionViewModel(...));
    }

    [HttpPost()]
    [AutoValidateModel(nameof(NewSubmission))]
    public RedirectToActionResult SaveSubmission(SaveSubmissionViewModel model)
    {
        // Save submission to database
        return RedirectToAction(nameof(ViewSubmission), new { Id = 1 });
    }

    [HttpGet]
    public ActionResult ViewSubmission(int id)
    {
        // Load submission from database
        return View(new ViewSubmissionViewModel(...));
    }
}

The AutoValidateModel attribute on the SaveSubmission Action performs the ModelState.IsValid check. If the model is valid everything continues as normal, however, if the model is invalid then the specified fallback Action in is invoked and the ModelState from the previous Action is merged in.

This means you're able to render the invalid Form with validation messages and the appropriate HTTP status code with a single line of code!

If you want to get started, grab the Automatic ModelState Validation NuGet package. And keep reading if you're interested in some more of the details.

The deep dive

One of the highlights of this attribute is to invoke the fallback action programmatically without another round trip to the client. In the past, I've achieved similar functionality by temporarily storing the ModelState and redirecting to the previous action. However, this time around I have made use of the advancements in AspNetCore to bypass that step entirely.

The bulk of the code resides in the AutoValidateModelAttribute which implements the ActionFilterAttribute class. After first checking that the ModelState is invalid the next step is to determine check controller action to invoke.

var controllerName = SansController(controller ?? context.Controller.GetType().Name);

If the controller isn't explicitly specified then the fallback is the Type name of the controller. Here I also remove the Controller suffix.

Next, I have to locate the relevant ActionDescriptor by getting the IActionDescriptorCollectionProvider and finding the matching descriptor.

var controllerActionDescriptor =
    actionDescriptorCollectionProvider
    .ActionDescriptors.Items
    .OfType<ControllerActionDescriptor>()
    .FirstOrDefault(x => x.ControllerName == controllerName && x.ActionName == action);

To invoke the ActionDescriptor the next thing I'll need is an ActionContext. Here is also where I pass along the previous ModelState so the validation errors are carried through to the new Action.

var actionContext = new ActionContext(context.HttpContext, context.RouteData, controllerActionDescriptor, context.ModelState);

The last major piece is getting an IActionInvokerFactory to create a ControllerActionInvoker and then invoking it.

var actionInvokerFactory = GetService<IActionInvokerFactory>();
var invoker = actionInvokerFactory.CreateInvoker(actionContext);
await invoker.InvokeAsync();

After that, there was just one more problem to fix. When no View is explicitly specified when returning a ViewResult, AspNet Mvc will fall back to using the action name from RouteData. Because I'm invoking a second Action inside a single Request the wrong view will be used unless I update the RouteData to match. So, I also set the action name to the name of the fallback action before calling the ControllerActionInvoker.

if (context.RouteData.Values.ContainsKey(ActionNameKey)) {
    context.RouteData.Values[ActionNameKey] = controllerActionDescriptor.ActionName;
}

If you're interested in the full source code you can check it on GitHub under Automatic ModelState Validation.

Running .NET on the Raspberry Pi

Mon, 27 Nov 2017 04:59:57 GMT

I recently got my hands on an older Raspberry Pi, which was a great excuse to see if I could get .NET running on it! If you're interested in playing around with a Pi or just want to see what it's like to get one running, then you're in the right place. I'll take you through setting it up, installing some programs and compiling some C# on the device with a few failures along the way.

I knew this Raspberry Pi was an older model, but I wasn't entirely sure what model it was. Figuring that out seemed like a good first step because that might impact what OS I can run. Plus I wanted to make sure the model I had was able to run .NET. So my first challenge was to figure out what model I had.

At first glance, it looked like it would be a bit more difficult than I was expecting because there's no clear version number printed on the board and quite a few variations which are very similar. I found Wikipedia to be the best way to identify a Raspberry Pi. It lays out each model nicely and shows the mounting holes, components, ports and logo which you can use to determine your exact model.

From there I was able to determine that the model I had was the Model B (Gen 1). Now that I knew what I was dealing with, it was time to see if I could get it to boot. For that, I needed an SD card that would work, a USB Micro cable and a USB adapter. Luckily, I had more than enough phone chargers and a few old SD cards lying around.

My first few attempts at getting the Pi to boot were completely unsuccessful. To this day I don't know if it was because of the SD cards I tried or the Raspberry Ubuntu image I tried. But I may have run into almost every issue you can have (probably not, but it sure felt like it) and the quite extensive guide to troubleshooting boot issues didn't help either. However, somewhere along the way, I got my first image on screen!

Is that the Raspberry Pi rendering a nice rainbow square on screen? No, it's actually just rendering four pixels that have been scaled up to fit the screen. This happens when the GPU firmware successfully loads but then something later in the boot process goes wrong, such as an invalid image on the SD card. Although you'd be forgiven for mistaking it with the small rainbow square in the top right that indicates a low voltage from the power supply.

After trying a few different SD cards, I decided to ditch Ubuntu and try one of the officially supported images. I decided to try NOOBS first, which is an operating system installer that lets you pick from a few different operating systems including Raspbian (the official Raspberry Pi OS). This time I had a bit more luck, not much more, but more nonetheless. The Pi successfully booted! I pick Raspbian and wait for it to install.

And wait. And wait. I think it got to about 20% before refusing to progress any further. It may have just been really really slow, but I was starting to consider that maybe there's something wrong with the device. At this point, I'd been at it for a while, so I decided to leave it for another weekend.

One Month Later

I find myself looking at a Raspberry Pi Zero W, they're cheap and relatively easy to get. But not "already on my desk waiting for me" cheap. It's time to give this thing one last shot.

This time it's straight to Raspbian, in particular, Raspbian Lite (which doesn't contain a GUI). I skipped over the process of getting the OS onto the SD card earlier, but I wanted to point out how nice it is. Once you've downloaded the image file, which is just a zip, in this case, you use Etcher to write the image to the card. It's a three-step process of selecting the image, choosing the drive and clicking start, it's fast and flawless. I plug in the mouse, keyboard and monitor, insert the SD card, and plug in the power. And it Boots!

If you're wondering why that picture is so much clearer than the others, it's because I've switched from using a TV to a computer monitor. I didn't time how long it took, but I'd take a guess the Pi took around a minute or so to boot. After which I was prompted to log in.

Raspbian GNU/Linux 9 raspberrypi tty1
raspberrypi login:

There's already a default user. A quick search for the default credentials reveals Pi's secrets (u: pi p: raspberry). And I'm in!

The first thing I noticed was that the keyboard layout was different. Typing the pipe symbol | instead inserted a hash # which indicated a UK keyboard layout. Most settings for the Raspberry Pi can be updated by entering the command sudo raspi-config. You can also change your password and enable SSH from there.

The steps to change the keyboard layout are fairly straightforward. Well, except for selecting a non-UK keyboard because those are all grouped under Other. Here's the complete set of steps in case you were wondering.

Select Internationalization menu
Select keyboard setup menu.
Select your keyboard (or pick one of the generic models)
Select [Other]
Select [English (US)]

Now I had a fully functioning Pi. So it was time to see if I could install some variant of .NET on it, either .NET Core or mono. What you can run depends on the CPU of the Pi you have. But there are two main versions, arm v6 and arm v7. All the specifications are on the same Wikipedia page I mentioned earlier but if your Pi is running you can also use the following command.

> uname -m
armv61

You might have noticed that the same moniker is printed in the welcome message after you log in. The Raspberry Pi I have contains an armv61 chip the same as the Pi Zero. This is a different model from the new chip in the Raspberry Pi 3 which contains an arm v7 chip. Unfortunately .NET Core (and a few other languages such as node) currently only support the newer arm v7 architecture. This limited my options a bit, but, it turns out that Mono does support arm v6. At this point, I can see the light at the end of the tunnel. I now have a method for running .NET on the Raspberry Pi.

I still wasn't sure if I'd be able to compile .NET on the device or just run applications, so it was time to get empirical. There are two Mono packages you can install via apt-get. The runtime mono-runtime and everything, mono-complete, which includes the compiler. I thought I'd take a stab at installing mono-complete.

The apt-get update command is for updating the local package cache, so I'll get the latest version of the Mono package.

sudo apt-get update
sudo apt-get install mono-complete

My luck's really coming about now. Mono installs successfully, first time!

Time to compile some code. I already had some C# I wanted to try out on the device, and I need my monitor back to get it. So it was time to switch to ssh so I can access the Pi remotely. I used chocolatey to install openssh.

choco install openssh

Plugged the Pi into the network, and realised I had no idea what the IP address was. A friend of mine had pointed me towards arp earlier that day. A great little tool that lists all the IP addresses of devices on your local network. And while I quickly realised it is much nicer on unix-like systems, because it displays hostnames whereas I had to revert to manually looking them up (via nslookup), it was still very useful.

> arp -a

Interface: 10.1.1.227 --- 0x13
  Internet Address      Physical Address      Type
  10.1.1.1              aa-bb-cc-dd-ee-1f     dynamic
  10.1.1.29             aa-bb-cc-dd-ee-2f     dynamic

There weren't too many devices on my local network. I even chanced upon the correct Pi address on my first try.

> nslookup 10.1.1.29

Name:    raspberrypi.lan
Address:  10.1.1.29

Now that I knew the IP address, I could ssh into the Pi and do the rest remotely.

ssh [email protected]

The file I had to copy was fairly small, and I could have copied the file over with scp but echoing out the contents into a file worked for me. Here's how to write hello world to a file.

echo 'using System;

namespace RaspberryPi
{
    public class Program
    {
        public static void Main(string[] args)
        {
            Console.WriteLine("Hello World");
        }
    }
}' > app.cs

Then it was just a case of compiling the program and running it.

> mcs app.cs
> mono app.exe
Hello World

So there we have it! Starting with an unidentified Raspberry Pi, some old SD cards and a phone charger. I managed to (eventually) successfully compile and run a .NET application.

cURL to Postman

Tue, 14 Nov 2017 00:18:32 GMT

Postman is a fantastic tool for testing any HTTP endpoint. But if you're using your browser to look around and you find something of interest it can be a pain to recreate the request in Postman and copy across all the headers. There must be a better way!

Turns out there is! Chrome and Postman both have support for cURL which makes it easy to copy any request from Chromes dev tools and into Postman. You can also export any Postman request as a cURL command which makes sharing much easier as well.

In the Chrome Network tab, you can copy a request via a selection of formats.

Once you have your cURL request you can then use the import command and paste in the cURL command.

It took me a while to find out how to export the request, it's hidden under the code link.

From there you can export the request into a wide variety of formats.

And there we have it, Chrome to cURL to Postman and back again.

Dynamically calling a Function in FSharp

Sat, 11 Feb 2017 06:16:03 GMT

Sometimes you just want to make a dynamic call.

There is no equivalent for Func<>.DynamicInvoke in FSharpFunc so I guess we'll have to roll our own.

In this post I'll go though how to build a function called dynamicInvoke which you use like this:

let sayHello name = printfn "Hello %s!" name
let objectResult = dynamicInvoke sayHello ["Daniel"]

let typedResult = objectResult :> string
// typedResult = "Hello Daniel!"

The example above starts off with a normal FSharp function that takes a typed argument, in this case, a string and returns a string. Then dynamicInvoke is used to call the function with a list of arguments which returns the result as an object.

You could also use dynamicInvoke on its own to build a function with the signature obj seq -> obj. Which I'd say is the functional equivalent of DynamicInvoke, just what we're looking for!

let dynamicFunction = dynamicInvoke sayHello

The actual signature of dynamicInvoke therefore is as follows obj -> obj seq -> obj. In other words, it takes the original function (as an object), an obj seq for the arguments and returns an object, which is the result of the original function.

How it works

At its core, the solution is to use standard .NET reflection to dynamically invoke FSharpFunc<'T,'U>.Invoke : 'T -> 'U. The other tricky bit involves how functions work in FSharp.

In FSharp, functions that take multiple parameters return a function that takes the rest of the parameters. Which itself is a function that takes a single parameter and does the same. All of these functions take a single parameter. So we could also say, there is no such thing as a function that takes multiple parameters!

Let's have a look at a function with multiple parameters:

let sayHello name age =
    printfn "Hello %s, you are %i years old!" name

The type hierarchy of this sayHello function will contain the following type:

FSharpFunc<string, FSharpFunc<int, string>>

In order to dynamically invoke the function, each FSharpFunc that makes up the function can be invoked recursively until a result is reached. And because FSharpFunc is a standard .NET type we can get the MethodInfo for Invoke and dynamically invoke it!

For each argument, the function is partially applied. The code below does exactly this, using reflection to get the MethodInfo for the Invoke method and calling Invoke on it. Such invoke.

let partiallyApply anyFSharpFunc argument
    let funcType = anyFSharpFunc.GetType()
    // Guard: FSharpType.IsFunction funcType 
   
    let invokeMethodInfo = 
        funcType.GetMethods()
        |> Seq.filter (fun x -> x.Name = "Invoke")
        |> Seq.head
    methodInfo.Invoke(anyFSharpFunc, [| argument |])

Now, all it takes to call a function is to recursively partially apply it until we run out of arguments to apply. Add some error handling and we're done! The complete code is below and I'll also publish a NuGet package shortly.

open System
open FSharp.Reflection

type InvokeResult = 
    | Success of obj
    | ObjectWasNotAFunction of Type

let dynamicFunction (fn:obj) (args:obj seq) =
    let rec dynamicFunctionInternal (next:obj) (args:obj list) : InvokeResult =
        match args.IsEmpty with
        | false ->
            let fType = next.GetType()
            if FSharpType.IsFunction fType then
                let (head, tail) = (args.Head, args.Tail)
                let methodInfo = 
                    fType.GetMethods()
                    |> Seq.filter (fun x -> x.Name = "Invoke" && x.GetParameters().Length = 1)
                    |> Seq.head
                let partalResult = methodInfo.Invoke(next, [| head |])
                dynamicFunctionInternal partalResult tail
            else ObjectWasNotAFunction fType
        | true ->
            Success(next)
    dynamicFunctionInternal fn (args |> List.ofSeq )

Now it's possible to dynamically call any FSharp function!

A quick word on error handling. This function can fail if something other than a function is invoked. Sadly there's no IAmAFunction marker interface. So the only option is to use a runtime check and return a ObjectWasNotAFunction InvokeResult.

This is an explicit failure of this function and therefore an explicit result is returned. Exceptions caused by invoking the function, however, are exceptional, and never caught (except at the application boundary), so they pass right through.

If you found this post useful, I'd love to hear about it. So leave a comment if you found this post interesting or if it helped you out!

Comparing MicroBus and MediatR

Sat, 07 Jan 2017 02:09:43 GMT

Mediators are an excellent addition to the developer toolbox. Allowing you to define a pipeline that is decoupled from any particular framework. Having a pipeline makes it easy to deal with cross cutting concerns like security, transaction, validation and others. For these reasons and more using a Mediator has become quite a popular choice. They are also used to improve other aspects of an application in regards to structure and architecture.

Recently there have been a few such Mediator libraries pop-up for .NET including MediatR, GreenPipes and MicroBus. I started MicroBus back in 2015 and have used it in a number of projects since. So I thought now would be great to talk a bit about MicroBus and what it does. In this post, I will be comparing MicroBus and MediatR, so that you can see the similarities and differences.

But before I begin, I'd like to say; I think both MediatR and MicroBus are great solutions to these problems. I've made some different choices along the way, however, both these projects had similar origins (aka doing it numerous times before making a package), have similar goals and tackle similar problems.

Wire Up

I'll start with how to wire-up the Mediators, which involves firstly registering them to a dependency injection container.

With MicroBus I wanted to ensure the experience was the same regardless of which container you prefer. It's syntax, is, therefore, a little more succinct than out of the box MediatR.

// MicroBus
autofacContainerBuilder.RegisterMicroBus(busBuilder);

MediatR takes the approach of zero dependencies and provides the following sample for setting up MediatR with Autofac.

// MediatR
var builder = new ContainerBuilder();
builder.RegisterSource(new ContravariantRegistrationSource());
builder.RegisterAssemblyTypes(typeof(IMediator).GetTypeInfo().Assembly).AsImplementedInterfaces();
builder.RegisterAssemblyTypes(typeof(Ping).GetTypeInfo().Assembly).AsImplementedInterfaces();
builder.RegisterInstance(Console.Out).As<TextWriter>();
builder.Register<SingleInstanceFactory>(ctx => {
	var c = ctx.Resolve<IComponentContext>();
	return t => c.Resolve(t);
});
builder.Register<MultiInstanceFactory>(ctx => {
	var c = ctx.Resolve<IComponentContext>();
	return t => (IEnumerable<object>)c.Resolve(typeof(IEnumerable<>).MakeGenericType(t));
});

However that's not entirely fair, the actual implementation for MicroBus is relatively similar under the covers. MicroBus also currently only provides an Autofac extension. Although you can set-up any other container manually. The Autofac implementation itself consists mainly of this file ContainerExtensions.cs and uses a resolver and scope class similar to ASP.NET MVC instead of factories for resolving dependencies. If you'd like to request a particular container, then feel free to open a GitHub issue.

MicroBus wire-up is slightly more complex in two places, validation, which ensured command and requests only have a single handler and Registrations which support singleton and transient registrations, which is useful for transparently doing complex registrations that can completely change how handlers work. MicroBus uses this to provide Saga support something I'll write more on in a later post.

Pipeline and Handler Registration

Next, we'll take a look at how to register handlers and how to create a pipeline.

Again I wanted to ensure that registration for MicroBus was consistent across containers and that the concept of a pipeline was a first class citizen. There are two main types of handlers in MicroBus, Global/Delegating Handlers which execute for every message and Message handlers which sit at the end of the pipeline.

// MicroBus
var busBuilder = new BusBuilder()

// Global Handlers run in order so these are explicitly registered
.RegisterGlobalHandler<LoggingHandler>()
.RegisterGlobalHandler<SecurityHandler>()
.RegisterGlobalHandler<ValidationHandler>()
.RegisterGlobalHandler<TransactionHandler>()

// Scan an assembly to find all the handlers
.RegisterHandlers(assembly);

Global handlers run in the order they are registered then Message Handlers are executed. The Delegating Handlers are also Task-based which uses async in contrast to before and after methods. Again this is very similar to ASP.NET WebApi and ASP.NET Core, which use a similar approach.

Another one of the goals I had in mind when building MicroBus was Safety. To help prevent you from doing the things you probably don't want to be doing anyway. MicroBus supports three message types, commands, events and queries. Each of these types have different semantics which are enforced through validation. Only a single command handler can be registered for a command. The same goes for queries. Events, however, can have multiple handlers (polymorphic handlers are also supported) but no return types.

Registration via MediatR up until very recently (about two days ago) didn't provide any abstractions, so registrations were made directly via the container with decorators. Leaving the pipeline to the container removed a fair amount of complexity from MediatR and delegates it to the container. The trade-off here though is that the user may have to deal with the complexity instead.

MediatR 3.0 now also provides support for pipelines which makes registration much easier for all containers.

// MediatR
builder.RegisterGeneric(typeof(IPipelineBehavior<,>)).As(typeof(LoggingBehaviour<,>));
builder.RegisterGeneric(typeof(IPipelineBehavior<,>)).As(typeof(SecurityBehaviour<,>));
builder.RegisterGeneric(typeof(IPipelineBehavior<,>)).As(typeof(ValidationBehaviour<,>));
builder.RegisterGeneric(typeof(IPipelineBehavior<,>)).As(typeof(TransactionBehaviour<,>));

The API

The primary interface you use to send commands to MicroBus is the IMicroBus interface.

// MicroBus
public interface IMicroBus
{
    Task SendAsync(ICommand busCommand);
    Task PublishAsync(IEvent busEvent);
    Task<TResult> QueryAsync<TQuery, TResult>(IQuery<TQuery, TResult> query)
        where TQuery : IQuery<TQuery, TResult>;
}

This interface is strongly typed and enforces that a given message type is used with the matching method. MicroBus also provides a secondary generic IMicroMediator interface which can be used with untyped MessageHandlers, we'll look at these a bit later.

// MicroBus
public interface IMicroMediator
{
    Task SendAsync(object message);
    Task PublishAsync(object message);
    Task<TResult> QueryAsync<TResult>(object message);
}

With most choices in MicroBus, I've gone for the opinionated option to get better compile-time safety or better validation. This interface, however, offers much fewer guarantees. In some brownfield projects especially those which already followed a similar pattern but had messages that didn't use the MicroBus marker interfaces, this is extremely useful.

The MediatR interface is very similar to the IMicroBus interface. It has the three primary methods for Commands, Request/Queries and Events but using different names. One nice feature here is the support for CancellationTokens.

// MediatR
public interface IMediator
{
    Task<TResponse> Send<TResponse>(IRequest<TResponse> request, CancellationToken cancellationToken = default(CancellationToken));

    Task Send(IRequest request, CancellationToken cancellationToken = default(CancellationToken));

    Task Publish<TNotification>(TNotification notification, CancellationToken cancellationToken = default(CancellationToken))
        where TNotification : INotification;
}

These interfaces are probably the most similar part of the two libraries. It's also interesting to see that MediatR also took the type-safe route here. One thing I did find slightly confusing was that MediatR uses IRequest with no Response for its commands.

Message Handlers

So far we've covered how to register handlers, and how to send messages to them. Next up is how they are defined. In MicroBus there are three typed handler interfaces and one generic handler interface for the IMicroBus and IMicroMediator interfaces.

The three typed interfaces are for each of the message types, commands, queries and events.

// MicroBus
ICommandHandler<Command>
IQueryHandler<Query, Result>    
IEventHandler<Event>

Here is what a command handler would look like in MicroBus.

// MicroBus
class CommandHandler : ICommandHandler<Command>
{
    public async Task Handle(Command command)
    {
        Console.WriteLine("Command handled!");
    }
}

It's worth mentioning that EventHandlers support polymorphic dispatch so a IEventHandler<AnimalEscaped> and IEventHandler<LionEscaped> would fire if LionEscaped inherits from AnimalEscaped (MediatR also supports this).

The generic handlers are also quite similar. However, in this case, there's only one interface because it's impossible to know what the message will be based on its type. If you're implementing a command or event, you would typically return Unit.

// MicroBus
class Handler : IMessageHandler<Message, Unit>
{
    public async Task<Unit> Handle(Message message)
    {
        return Unit.Unit;
    }
 }

MicroBus is also split up into four main parts.

The core package
Infrastructure contracts, which include IMicroBus and the IHandler interfaces
Message contracts, which includes ICommand, IQuery, IEvent and IMessage
Dependency Injection Extensions

Splitting up MicroBus into multiple packages is a nice feature that lets you selectively include the parts of MicroBus you depend on. Most commonly you see Message Contracts defined in a library that depends only on MicroBus.MessageContracts.

MediatR's Handler are quite similar the main distinction being that MediatR provides both async and synchronous versions of each interface. The example below shows an async command handler.

// MediatR
class CommandHandler : IAsyncRequestHandler<Command>
{
    public async Task Handle(Command message)
    {
        Console.WriteLine("Command handled!");
    }
}

Overall you can see a lot of similarities with a few defining features.

Delegating Handlers

We've already talked a bit about Pipeline and Handler registration and this one of the most useful features of a Mediator. MicroBus has first class support for pipelines and makes writing cross-cutting code effortless.

To start, you implement IDelegatingHandler which has a similarly looking handle method. One of the nice things about IDelegatingHandler is that don't need a constructor, and if you need dependencies, you don't need to worry about where the "inner" handler goes. Instead, the next handler is passed along through Handle keeping the scope of the variable as small as possible. Delegating Handlers are also registered globally which means that you'll only need to register them once and they'll run for every type of message.

The example below shows how you could create a global handler which wraps every message handler in a database transaction.

// MiroBus
public class DatabaseTransactionHandler : IDelegatingHandler
{
    public async Task<IReadOnlyCollection<object>> Handle(INextHandler next, object message)
    {
        using (var transaction = database.NewTransaction()) {
            return await next.Handle(message);
            transaction.Commit();
        }
    }
}

Delegating Handlers also make use of async and await which means you can make using of using and try as opposed to having a before and after method.

MediatR didn't have a global pipeline until recently. Before which made registering cross-cutting code for every handlers somewhat tedious. The latest version 3.0 puts it on par in this respect and the handlers which are called Behaviours are again quite similar. The snippet below shows an example of a behaviour in MediatR.

// MediatR
public class MyBehavior<TRequest, TResponse> : IPipelineBehavior<TRequest, TResponse> {
    public async Task<TResponse> Handle(TRequest request, RequestHandlerDelegate<TResponse> next) {
        //Before
        var response = await next();
        // After
        return response;
    }
}

What's next

Mediators are a great addition to the developer toolbox letting you deal with cross cutting concerns without tying your application to a UI, web or service framework. We've looked at wire-up, declaring and registering handlers, usage, and pipelines while comparing MicroBus and MediatR.

I hope I've been able to show you a bit about MicroBus and what makes it different. So if you're used a mediator before (or if you haven't) I'd recommend trying MicroBus out for yourself, I'm always looking for feedback and suggestions. You can get the get the bits on github here Enexure.MicroBus.

Event Sourcing: What properties should Domain Events have?

Mon, 31 Oct 2016 08:46:13 GMT

What I learned since last time

A while ago I wrote about Generating Read Models with Event Sourcing where I suggest adding derived properties to the persisted domain events also known as Enriched Events. Since writing that post, I have been using this approach. However, over time I found that this was not the solution I was hoping for. In short enriching events also has some problems and recently I have switched over to using only delta style events for persisted events. In this article, I will cover the problems that come from enriching events and suggest a (better) alternative solution to the question "What properties should Events have?".

What problem is Event Enriching meant to solve?

When a Domain Event is raised, it is published to several interested handlers. The first of these is the Apply Method whose job it is to update the state of the Aggregate. Because the Aggregate is the primary source of an event, it has access to all the updated state. The problem stems from needing to pass some of this data to the other interested handlers.

If we look at a bank account, the event would be a transaction resulting in a changed account balance. The other interested handlers could be Read Model Generators, other event handlers or even other services. Some of these are almost certainly interested in the current balance of an account. In this case, Enriching the Transaction Added event would involve adding the Current Balance to the events. Enriching solved the issue of easily providing information to all the interested handlers. More importantly, though, it ensured that there is only one place where derived state is calculated.

Sending derived properties to other handlers and services is an important problem that Enriching Events was able to resolve. So it is equally important to ensure that the new solution solves this problem as well. But first, we'll look into what the problems with Enriching Events are.

As a side note, there are still cases where other handlers for an event keep track of their own running totals or tallies (derived state). However, this is usually because it is data that the Aggregate does not care about and does not already calculate or store. In which case, no logic is being duplicated anyway.

Ok so what’s wrong with Enriching Events

There are a few problems that come from enriching Events which you can avoid by using delta only Events.

Redundant Data

One of the core ideas in event sourcing is storing a list of all the changes that have happened to an Aggregate. By playing back these events you can determine the state of the Aggregate at a point in time. Because the system is deterministic, you will arrive at exactly the same state every time.

For a bank account given all the deposits and withdrawals ever made can determine what the current balance is. If you were to store the current balance along with every transaction, you would have to store much more data for little benefit. In fact, it stops us from using other nice features of Event Sourcing.

You can’t do Replay Testing

Replay testing allows you to use all your current production data as test data for your application. The derived state will be re-calculated using the events as they are replayed. Comparing the results of different versions can then be used to spot any divergences and fix them before they make it to production.

The one downside to this is that we must do slightly more work when replaying events. However, processing events in memory is extremely quick and loading the data is usually the bottleneck anyway. Lots of events can still slow things down, but you can use Snapshotting to win back even more performance.

No Concurrency for Aggregates

Concurrency is probably the biggest issue with Enriched Events. Looking at the Transaction Added event for a bank account. If the balance total is part of the event, then it forces you to process all behaviours in serial. This is what it looks like if we add the current balance to the event.

Event               Amount  Total
TransactionAdded    50      50
TransactionAdded   -20      30
TransactionAdded    10      40
Total    40

But what happens when both the second and third event occurs at the same time.

Event               Amount   Total
TransactionAdded    50      50
TransactionAdded!  -20      30
TransactionAdded!   10      60
Total    60 // Should be 40

If we do not handle concurrency at all, the total from the first event will get overridden by the total from the second event. This would be terrible! Optimistic Concurrency is the next level and a good default, in this scenario, a concurrency exception is thrown instead. This works well for cases where you cannot do work in parallel.

However, if the model does not need the totals to process transactions (at least not in a completely consistent manner), then the order does not matter anymore. You will still end up with 40 regardless of the order you process the behaviours in. So, we can be a bit clever and for transactions that do not conflict, try to insert it again automatically. Speeding up the time it takes to process events.

Event               Amount
TransactionAdded    50
TransactionAdded   -20
TransactionAdded    10
Total    40

Even if behaviours cannot be processed in parallel such if we needed the balance to be completely consistent inside the aggregate, we can derive the totals so we can still avoid persisting them.

Unfocused Events

Enriched Events contain all the derived properties as well as the deltas which can leave you with quite large events. I have found that in most cases individual handlers will only care about one or the other, deltas or totals. For our bank account example if we want to publish the current balance we could have two events instead of one giant event. The first event would still be Transaction Added and the second would be Balance Changed. If you were updating a SQL Read Model you would probably have two tables (transaction and balances) and an event handler for each table, so two events align perfectly.

Multiple events to prevent duplicating logic

Using multiple events is also how we will deal with sending derivable state to other handlers. The second event, which includes the derived state, won't be persisted at all. I'll call this a transient event. The Persisted Domain Events that the behaviour creates are created as normal. However, Transient events need to be handled a little differently because they need the internal state from after the Apply method is called. Adding a Handle method for behaviours that executes after the Apply method gives us a chance to let the world know about derived or calculated data.

Now, when a behaviour is run the persisted event is raised, Apply updates the state, and the Handle method will run. Giving you a place to raise transient events as well. After the behaviour is finished all these events are then published.

Rebuilding the Read Model will also require the transient events to be regenerated. However, we can completely skip the Handle method when Hydrating an Aggregate from an event stream.

Persisted Domain Events

Now we have small delta only events in our event store, the door is open to leveraging the real power of Event Sourcing. It is much easier to keep other handlers informed and avoid duplicate code. Events are more focused, smaller and easier to work with. There are more options for dealing with concurrency. And you can make use of Replay Testing to eliminate a broad range of regressions.

So, what properties should Events have? Persisted events should only store what they need, typically only the change or the delta. The rest can be derived from there.

I am currently building a more Functional Event Sourcing library for C# called Eventual. It's my playground for Event Sourcing ideas and a good place to dig into some real code. It is, however, an early alpha and still a work in progress, so it doesn't cover everything in this post. Nevertheless, I still recommend you take a look if you want a peek under the covers.

Links to Get Started with React and Redux

Wed, 12 Oct 2016 22:38:46 GMT

Getting started to React and want to get right to it? These are the links you're looking for.

What you need!

First up you’ll need Node: https://nodejs.org.au/

You can get the React tooling here: https://github.com/facebookincubator/create-react-app

Once you have React you’ll want Redux: https://github.com/reactjs/react-redux

That’s all you need to get started!

Browser Extensions

React Developer Tools, great for debugging state: https://chrome.google.com/webstore/detail/react-developer-tools

Redux Developer Tools, amazing for debugging state and time travel debugging: https://chrome.google.com/webstore/detail/redux-devtools

Extra Resources

The how and why of React apps: https://css-tricks.com/learning-react-redux/

A random super simple example Redux app: https://github.com/tstringer/create-react-app-with-redux

Extra packages for better redux apps

If you need to make ajax calls this is the best was to do it: https://github.com/yelouafi/redux-saga

For caching updates to speed things up: https://github.com/reactjs/reselect

Remove magic strings in actions and reducers: https://github.com/pauldijou/redux-act

Happy coding! :)

There is always time to write clean code

Sun, 22 May 2016 00:40:02 GMT

The number one most important thing you can do in software development. Always leave the code base cleaner than when you found it. Always aim to improve things.

I recently stumbled on a fantastic podcast by Joshua Wulf featuring Damian Maclennan who brilliantly described why this is so important.

If you see a professional chef vs. a home cook, they enjoy cooking, but they end up with a mess all over the kitchen bench, but if you watch a professional work they'll chop something and then they'll wipe the board down, do something else and clean up a little bit. All along the way they're cleaning up. So they never get this massive build up of crap. Because once you've got that build up of crap you can't move. You end up with this tiny five by five centimeter square of space and you need to chop an onion and all of a sudden onion is going everywhere. That's what software development can be like, that's what code bases get like. You need to clean as you go.

You need to always take that professional mindset. A lot of companies think there's no time to do that. You just need to smash out features, and then you get to the point where people go to their manager and say we've got all this technical debt we need to stop what we're doing and fix our technical debt. And the managers kind of hit the roof over that. We're under all this pressure. I've heard people give these excuses; there's never any time to do it right. But if you go into a hospital they're generally fairly slammed for time as well but you would never see a surgeon say "you know I really should wash my hands but I'm kind of busy, there's no time". They just do it. It's part of their professional ethics to do it. To use hygiene and do things right. And again you'll get fired as a chef if you're not fast enough because you don't clean as you go. But in software it's kind of this top down beat people with a stick approach telling them to go faster and then you wondering why you end up with a mess.

Good software development teams clean as they go and the developers understand their professional responsibilities. You have to keep on eye on the future, on the impacts your decisions today will have. And it doesn't have to be at the expense of the short term. In fact, the future is often closer that it appears. Cleaning up as you go and doing things right can help you as soon as tomorrow. Now don't get me wrong, these aren't rules that apply to all code, we still have throwaways and prototypes. There are still trade-offs to make. But the baseline is set. If you're not cleaning as you go, it's slowing you down.

Announcing MicroBus 3

Wed, 09 Mar 2016 20:47:00 GMT

Awesome! MicroBus 3, what does that mean? Well, I've been using Semver (mostly, sometimes a little Ferver creeps in) for my package versions so it means in this release there's a few breaking changes. Now, first off, this is a much bigger update than 2.0 was and redefines a few of the MicroBus concepts.

There are three main changes, firstly the Registration API has been reworked, global handlers have been smoothed out and there's a new IMicroMediator interface (useful for sending any type as a message). But before I get into the details first here's a little bit of background.

The Story So Far

MicroBus has been a fantastic project to work on. I've been lucky enough to be able to use it quite substantially in my workplace. However, some things didn't turn out quite the way I expected them to. So it was time for a few changes.

Since the beginning, MicroBus has always had the concept of a pipeline which contains two types of handlers. The first was Message Handlers that resided at the end of the pipeline which do the real work for a given message. The second type of handlers were Pipeline Handlers which could be used to intercept a message as it travelled down the pipeline.

In the first version of MicroBus, the Pipeline Handlers are were explicitly added to the message handlers when registering them. However, even though you assigned the Pipeline Handlers individually, they behaved as if they were global. You could only ever have one set of Pipeline handlers for any one message, even if there were multiple different Message Handlers. It had to be the same instance of the Pipeline.

In version 2 support for having a global pipeline was added. However, these handlers were restricted to only running once, so they wouldn't run a second time if you sent messages from inside message handlers. Which was super useful, but slightly confusing with regards so naming.

What's Changed

So it was time to clean up a few things. Version 3 started with wanting a bit more flexibility out of MicroBus. If your message contracts implemented ICommand, IEvent or IQuery then everything was fine. However, there were a few cases, such as handling existing messages that didn't use these interfaces, where handlers needed to support any type of object.

IMicroMediator interface

This new interface has the same three methods (send, publish, query) as IMicroBus except that messages can be any .NET object.

public interface IMicroMediator
{
    Task SendAsync(object message);
    Task PublishAsync(object message);
    Task<TResult> QueryAsync<TResult>(object message);
}

To handle messages that don't implement our interfaces, a new handler IMessageHandler interface has been added. The Handle method of this interface takes a message of any type and returns an object.

class Handler : IMessageHandler<Message, Unit>
{
    public async Task<Unit> Handle(Message message)
    {
        return Unit.Unit;
    }
}

Traditional MicroBus message types can also be sent using the IMicroMediator interface and can be handled by either the type safe or generic message handler interfaces.

Registration API

The next big change was the Registration API, there are two main changes here. Firstly when registering MicroBus to the Container Builder, instead of having to register everything inside a lambda, you can now pass in a BusBuilder like so.

var busBuilder = new BusBuilder();
containerBuilder.RegisterMicroBus(busBuilder);

The second change to the Registration API is to the handler registration methods themselves. Previously the registration code looked like this.

busBuilder
    .RegisterCommand<TestCommand>().To<TestCommandHandler>()
    .RegisterCommand<TestEvent>().To<TestEventHandler>();

The main reason the API was originally designed this way was to make it easier to register events.

busBuilder
    .RegisterEvent<Event>().To(x => {
        x.Handler<EventHandler>();
        x.Handler<EventHandler2>();
    });

In this version of the API you only have to specify the message type once when registering multiple handlers. However, this required quite a lot of code for not much benefit, especially when most of the time you could just use assembly scanning registration methods instead.

.RegisterHandlers(x => x.FullName.Contains("BusMessageTests"), assembly);

The explicit handler registration API is now much more flat, and combines the Register and To methods into a single method in BusBuilder.

busBuilder
    .RegisterQueryHandler<IsItTimeToGetOutOfBedQuery, Result, IsItTimeToGetOutOfBedQueryHandler>()
    .RegisterQueryHandler<BreakfastEvent, MakeBaconHandler>()
    .RegisterQueryHandler<BreakfastEvent, EventHandler>()

Global Handlers

The last big change is Global handlers. I talked about the problems with the way handlers previously worked. Mainly that pipeline Handlers were Global handlers in disguise except individually registered. In v3 these handlers have been pulled out of the individual registrations and replaced with real global handlers, that are globally registered. These handlers will run for every message sent over MicroBus.

busBuilder
    .RegisterGlobalHandler<InnerHandler>()

Global handlers use the new IDelegatingHandler interface which has one main difference to the old IPipelineHandler interface mainly the use of INextHandler instead of a Func to invoke the next handler in the pipeline.

class DoNothingHandler : IDelegatingHandler
{
    public async Task<object> Handle(INextHandler next, object message)
    {
        return await next.Handle(message);
    }
}

Overall global handlers are now much more consistent and much easier to use.

However, this means the global handlers now run all the time, for every message. Needing handlers that know if they're at the Top Level of the call stack is still something I really wanted. So I've brought it back as dependency you can use to tell if it's an outer Handler or not.

Adding IOuterPipelineDetector to the constructor of a handler will provide an instance via Dependency Injection. The IOuterPipelineDetector interface has a single property called IsOuterPipeline which you can use to prevent a handler running a second time if the MicroBus pipeline runs inside an earlier instance of the pipeline.

Conclusion

It's been quite a bit release, with a few new features that will make MicroBus more powerful and easier to use. The new IMicroMediator makes it easier to work with any message types, registration is simplified and global handlers are easier to build.

So there you have it, all the changes in MicroBus 3! Make sure you try it out the new version. Of if you're installing for the first time use the command below. I'm also always on the lookout for feedback so leave a suggestions or issues on the GitHub page. Happy messaging!

PM> Install-Package Enexure.MicroBus.Autofac

Porting MicroBus to DotNetCore

Sat, 30 Jan 2016 13:28:04 GMT

When the new .net core reached RC it looked like the perfect time to start to port some of the Nuget packages I've made starting with MicroBus. I've been watching most of the aspnet standups so I've always been relatively familiar with what was going on, but there were still a few challenges along the way. I usually like to have everything figured out before I write about it, but for this post, I want to give a good picture of the experience as I found it.

I started out using The Road to DNX by Marc Gravell as a guide, which was great for figuring out exactly where to start.

Porting the project

He starts off moving all the old framework specific csproj files so he can build the project the old way as well as with dnx. I ended by just deleting these as I probably won't be using them again even though it's still an RC. So my first step was creating the project.json files, after that you can add them to the solution and Visual Studio will create some xproj files for you.

I almost missed the super important part where the blog explains how to include them.

IMPORTANT: as soon as you add a project.json to a sln you get a second file per project.json for free: {YourProject}.xproj; these should be included in source control.

It breaks the flow a little bit to point it out as an important note but in doing so I guess I missed it on my first read through.

The next part was getting the project building. One of the first things I discovered was that dnx was not just a runtime it's also a build system. And it supports targeting multiple frameworks in the one project file. I wanted to compile my project for both .net 4.5.1 and the new portable dotnet targets. This turned out alright, but there was a lot of digging to figure out exactly which monikers I needed, and the name changes didn't help.

One of the biggest changes I had to make was moving to the new reflection API. I'd used the new API before, but I'm sure I ran into every issue again. The two main things to be aware of here is most of the stuff has moved to TypeInfo and methods like GetInterfaces() are now properties like ImplementedInterfaces. Google didn't help as much here as I expected which probably means it's still a part of .net that is still rarely used.

Getting the Tests working

For the tests, the first thing I tried to do was leave them exactly as they were (as .net 4.5.1 tests) and use a test runner executable just like before. However, it quickly became apparent that this wouldn't work well. When I build each project with dnx I would get the .net451 dlls I needed however the output directories only contain the artifacts for that project. Unlike the previous build system which would also copy all the dependencies. I'm still not sure how it discovers project dependencies, but I'm guessing it's related to the projects being siblings on disk.

I then ported my tests to XUnit so I could use the commands. The IDE lets you then select which commands you want to run. However, you have to have that project selected as the startup project to see it's options. Running the tests this way will still run the tests via the console.

Setting up the Build Server (AppVeyor)

I've been committing my packages to source control recently. The idea of super-deterministic builds is a nice idea. Plus on a build server like AppVeyor restoring packages for each build makes the whole process take much longer than just a git pull.

I tried a few things to get this working first committing the packages folder which doesn't seem to contain all the dependencies. Then I looked into committing the lock files as well. I was having issues just getting the project to build on the server. So after a few attempts, I abandoned the approach in favor of just getting it working first.

I had a few issues getting package restore working. Some project dependencies and package dependencies would just fail with "Unable to resolve dependency". At this point, I was trying to figure out if dnx had the equivalent of the solution file. It does have the concept of global.json but not 100% clear on all the cases it gets used for. I did find that calling dnu restore with the solution path resoved my dependency resolving issue dnu restore $solutionDir. I remember reading somewhere it would search child directories for projects, possibly using the global.json to guide the search.

All this was using the rc1 clr version of the runtime. Initially, it was hard to know what version of the clr I should be using. I actually started out trying to use coreclr for everything, but that can't build .net451. I then switched over to using the clr version and was able to get the project building. However, once I got to the tests, they would then fail. The tests are using dnxcore50, and you can only run them using the new coreclr. So what you have to do is build the project with the clr then use dnvm and switch over to coreclr to run the tests.

The last things to note is that restore and build are incredibly verbose.

Conclusion

Eventually, I got everything working, packages build with dnx pack are pushed to NuGet with the extra support for dotnetcore!

Looking back, I'm not sure if I would have waited until the new dot net cli is ready. There's still a lot of changes happening, and some things could definitely be smoother. It was great to finally get hands on with the new tools, though. Visual studio hid a few things that quickly become apparent when trying to get a project building on a build server. Overall though it feels like a good start although it still feels beta-ish in a few places.

Now that MicroBus is building again it's time to start work on the next release.

Querying AD Groups with PowerShell

Thu, 22 Oct 2015 02:33:15 GMT

Getting the list of the AD groups if nice and easy in PowerShell. All the CmdLets are located in the ActiveDirectory module which you might not have installed on your system.

To install the module you'll need these Windows features

Add-WindowsFeature RSAT-AD-PowerShell, RSAT-AD-AdminCenter

Then you can import the module

Import-Module ActiveDirectory

Then query the user you want to see the groups for. This command can also list the groups for other groups in case of inheritance.

Get-ADPrincipalGroupMembership "username" | select name

Building a MicroBus application

Sat, 19 Sep 2015 05:31:08 GMT

I often like to think of software architecture as the art of structuring software. It's goal being to meet requirements while at the same time being resistant to falling into the state most commonly referred to as a Big Ball of Mud. To fight back against this, applications can use a well-defined architecture along with consistency and readability to ensure they don't become a sprawling, duct-tape-and-baling-wire, spaghetti code jungle. Instead, applications can be structured in such a way as to reduce unnecessary coupling between components. And have a strong foundation that provides consistency, structure and an ideology of the application.

What is MicroBus

MicroBus is a small library designed to help structure applications using the concepts of the Message Bus and Mediator patterns. The mediator pattern focuses on reducing coupling by communicating through a mediator, and the bus depicts the style of communication. These two aspects enable the application to be constructed as a series of pipelines each responsible for its own vertical of functionality.

Sending Messages

MicroBus is centered around sending messages. As an example of what this looks like, here is an example of sending and processing a command.

await bus.Send(new AddItemToCartCommand(cartId, itemId));

The command, in this case, is adding an item to some shopping cart. The message itself is a data only object that implements ICommand. MicroBus also supports events and queries as first class citizens and enforces their semantics though type safety where possible. For instance, events are the only message types that can have multiple handlers registered to them, and queries are the only message types that can return data. The methods to send events and queries are also slightly different to enforce the notion that each has an explicit purpose.

To publish an event to all it's handlers the Publish method is used, the event must implement the IEvent interface.

await bus.Publish(new Event(...));

Queries also have their own method that returns the result for the given query. Queries must implement IQuery<TQuery, TResult> where TQuery is itself, and TResult is the result. The Result must also implement IResult. This enforces the notion that queries must have a result and also enables type inference to work seamlessly for calls to bus.Query().

var result = await bus.Query(new Query(...));

In this case, the type of result would be inferred from the query type.

Handling messages

Once the message has been put on the bus, then the handlers take over. More generally, handlers contain core logic of the application. Above we sent a AddItemToCartCommand command onto the bus so now to handle the command we need a command handler. This is a class that implements the interface ICommandHandler<T>. So our command handler would look something like this.

class AddItemToCartCommandHandler : ICommandHandler<TestCommand>
{
    ctor ⇥ ⇥

    public async Task Handle(AddItemToCartCommand command)
    {
        var cart = repository.Load(command.CartId);

        cart.AddItem(command.ItemId);

        repository.Save(cart);
    }
}

Similarly, events will use IEventHandler<T> and queries will use IQueryHandler<TQuery, TResult>.

After creating the message and handler you then need to register them to the bus. Here is what a registration would look like when using Autofac. In this case handlers will be registered to Autofac with the InstancePerLifetimeScope scope.

container.RegisterMicroBus(busBuilder =>
    busBuilder
        .RegisterCommand<Command>().To<CommandHandler>(...)
        .RegisterEvent<Event>().To<EventHandler>(...)
);

It's also fairly easy to support any other container, and you can even use it without one at all.

var bus = new BusBuilder()
    .RegisterQuery<Query, Result>().To<QueryHandler>(...)
    .BuildBus();

In this case, the built-in BusBuilder is used to create a bus without any IOC container.

The Pipeline

Messages and Handlers are the bread and butter of MicroBus, but the real power comes from the Pipeline. The pipeline consists not only of just the end handler but lets you place any number of intermediary pipeline handlers between receiving the message and handling it.

Pipeline handlers let you intercept messages as they are passed through to the message handlers, and then the responses as they bubble back up through the pipeline. This makes pipeline handlers the ideal place to handle all the cross-cutting concerns in an application. Such as logging, security, unit-of-work/transactions to name a few.

Here is an example of a pipeline handler responsible for starting and committing a Unit of Work.

class TransactionHandler : IPipelineHandler
{
    private readonly UnitOfWork unitOfWork;

    public PipelineHandler(UnitOfWork unitOfWork)
    {
        this.unitOfWork = unitOfWork
    }

    public async Task<object> Handle(Func<IMessage, Task<object>> next, IMessage message)
    {
        try {
            var response = await next(message);
            unitOfWork.Commit();
        } catch {
            unitOfWork.Rollback();
        }
    }
}

The same IPipelineHandler interface is used for all messages. This enables the use of the same pipeline can be used across all types of messages. This interface is fairly typical, but there's one thing worth point out here which is the first parameter of the Handle method. This method takes a Func<IMessage, Task<object>> which instantiates or resolved the next handler in the pipeline and calls its handle method. This is a kind of Russian doll model in that each handler contains the next handler.

Creating a pipeline just consists of creating a new Pipeline object and adding a series of pipeline handlers to it.

var pipeline = new Pipeline()
    .AddHandler<PipelineHandler>();

Once the pipeline has been created you can use it for as many different messages as you want. For example here the same pipeline is used to register a command and a query.

container.RegisterMicroBus(busBuilder => busBuilder
        .RegisterCommand<Command>().To<CommandHandler>(pipeline)
        .RegisterQuery<Query>().To<QueryHandler>(pipeline)
);

Compared to MVC/WebApi

So far we've seen MicroBus handling messages and handling cross-cutting concerns using the pipeline. While most frameworks, such as ASP.NET Web API, ASP.NET MVC, and Nancy will let you do something similar with handlers and action filters. One of the advantages behind the MicroBus is that it allows the application code to define the pipeline itself as opposed to embedding it in the frameworks themselves. These frameworks many also have other concerns going on besides the very simple “message in, message out” pattern.

Decoupling the application from these frameworks also has the added benefit of being able to use the same code for multiple different entry points. For example, you have a web app that can also consume messages from a service bus. Because all the cross cutting concerns are handling by MicroBus, it becomes trivial to support.

Entry points can also include Tests making Integration and Unit testing much easier. Part of the reason for this is each feature already maps to one command and handler pair. Integration tests can simply use the bus as is with the existing pipeline, and Unit Tests can focus on handlers as a unit.

MicroBus Lifecycle

Lastly, I wanted to touch a little bit on Object Lifecycles in MicroBus. For a Bus setup using Autofac, most components will be registered as instance per lifetime scope except the bus itself. The bus is registered as transient and will create a new lifetime scope for each message. So even without using Autofac in WebAPI, each request would get a new instance of the bus and the pipeline.

Things get a little more complicated when handlers themselves send messages. In that case, no new scope is created so lifetime scope registrations can be shared across nested pipelines as well. It's also possible to override when MicroBus creates new scopes by implementing the IDependencyScope and IDependencyResolver interfaces.

Getting Started

So there you have it. MicroBus, a tiny in memory message bus that can help you structure your app, reuse your cross-cutting concerns and decouple it from communication concerns.

I hope you enjoyed learning a little bit about what MicroBus is and how it can help you. If you're, keep to see more make sure you check out the MicroBus GitHub page. Or just go right ahead and install the NuGet package and have a play yourself.

PM> Install-Package Enexure.MicroBus

For more examples, you can also check out the Enexure.MicroBus.Tests project.

Generating Read Models with Event Sourcing

Tue, 21 Jul 2015 22:28:00 GMT

Event sourcing is a very powerful tool that allows you to capture information you didn't even know you wanted. One of the things you usually find yourself doing when event sourcing is applying the principle of CQRS. CQRS effectively splits your application into two parts, a read side for queries and a write side for commands. These two sides are kept in sync by events that the read side listens to, which represent the results of the commands. One of the issues I've tackled recently is how to go about generating the read model from these events.

Update! I've learnt a lot since writing this article and no longer recommend the conclusion I came to here. In short enriching events had unforeseen problems and I would instead recommend using only delta style events in a persisted event streams. To learn more read Event Sourcing: What Properties should Domain Event have?

So to set the scene, imagine a bank account, it contains a series of transactions and the total for the account. The two main events that we have to handle here are credits and debits. I'll skip over why they're not just one transaction event for this post. The part I want to focus on is what properties these events need. My first thought was that these events should just represent the deltas of the system. So my credit event would only need to specify the amount. The event would then be saved to the event store and published so any other potential listeners could handle it as well.

Handlers keeping track themselves

Ok, now it's time to update the read model! My read model is a typical set of database tables, accounts and transactions. I'll start with transactions because it's easy, all I need to do here is write in my event. Now for the tricky bit. My accounts table has a balance total column that I need to update in response to the event as well, but I don't know what the new total is. Could I use the existing value in the read model and just add my amount to it? No, that doesn't feel right at all, worse my values could drift between models. There's also another problem. If another service wanted to listen for these events and keep track of the total. They'd have to deal with the same problem, doing the calculation again themselves. They might not even have persisted memory to keep track of the total.

The Credit Added event containing just the amount to add

/* CreditAddedEvent */ {
    'amount': 10.0
}

With deltas each listener has to duplicate the work done

Who needs the total:
- Domain model
- Read model to display total
- Reporting

Using the domain model's current state

It would be fantastic if I somehow had access to the information when the credit event is raised. The model in the write side keeps track of the balance total already. Can I get the information from the model when processing the event? This could work, but it would require exposing the total via a property so the handler could access it.

I could get the information from the model when processing the event

// Maybe something like this
public void Handle(Metadata<CreditAddedEvent> metadataEvent) {
   
    metadataEvent.DomainEvent.Amount
    metadataEvent.DomainModel.Total
}

This didn't quite sit well with me either, one of the great things about the models in CRQS+ES is they can shed all their properties and properly encapsulate their state. If I expose properties here command handlers would then have access when they really shouldn't. There's also one other concern, tests. I also need to check the total in my tests and exposing this makes testing harder because my model is exposing more of its implementation. So either I can't change my model, or my tests become more prone to change.

Enriching the event

My last option to get the total is to put the total on the event itself (enriching the event). So that the event would then contain the amount of credit and the new total for the balance. I'd been avoiding this because the total could be derived from the events that just contained the credits and debits. Adding the derived totals would then mean they had to be persisted in the event store. However looking back this gives us exactly what we want and solves some other problems along the way.

Enriching the event means that it now represents the state change

/* CreditAddedEvent */ {
    'amount': 10.0,
    'newTotal': 70.0
}

Enriching the event means that it now represents the state change of the aggregate. Therefore, the state change becomes completely explicit and simplifies usage of the event stream immensely. Consumers don't have to keep track of any state themselves, in my case keeping their own tally. Enriching the events also helps keep my code dry, all my calculations take place in the behaviours of my models. This also makes replaying my events blisteringly fast because only have to set fields; no other calculations are needed. This would be even more beneficial if derived values were computationally expensive. As for the idea of exposing properties, enriching the event means I don't need to expose any properties for tests or to generate the real model. Everything they need is on the events, and the aggregates completely encapsulate all their internal state. The domain models should only need to process commands and emit events as side effects. This also helps unit tests become more resilient to change while giving me more freedom to change the internals of the models. You'll also find that some state doesn't even need to be kept in the memory of the models but may only be used in the events and the read model.

Enriching the event but not storing it

However, there's one more possible path I could have taken. Which is to have two different events. The delta that is stored in the event store and the enriched event that is only in memory. There are also problems with this approach, but I want to include it for completeness. While this gives me most of the benefits from just having one event, replaying the events becomes a much more expensive operation. I'd also have to write two events instead of just one which adds a whole bunch of other complexities and raised a whole bunch of questions. Would the events have the same identity? Would I need to store things that are not deltas anyway, such as answers from external services? What would I call the new event? They are the same event but have different models, it feels awkward giving two events the same name. I'd much rather just store the extra data and have just one event.

In the end

Enriching my domain events and saving them in the event store lets me generate the read model I need easily and avoids unnecessary complexity. Thus keeping the application in a maintainable state. I get the benefits of fast replaying of events, having all state change explicitly represented in the events and fully encapsulated state in my domain models. I could also data mine the raw event stream much more easily without using my application to process the events or duplicating the logic in it.

The Perfect Pot

Sat, 24 Jan 2015 06:25:49 GMT

Whether it's building software, a business, or some other project, chances are you'll almost never get it right the first time. Years ago, I heard a story of a pottery teacher that tried an experiment on the class to find out the best way to teach pottery to the students.

A high school pottery teacher was addressing the class, regarding their new assignment. The teacher told half of the class that they had the semester to create their best pot, which they would submit to be graded. To the other half of the class, the teacher said they would have to make 50 pots and that they would be graded on their best pot.

Who do you think produced the best pot? If you said the second half of the class, you'd be correct. I think the story applies well to the art of software development and I don't mean just building 50 of something. So lets explore this a little bit, why did they do better?

Practice

This is the classical answer, and it has a lot to do with the way the mind works. Practice strengthens the neural pathway in the brain. When you first learn something, it requires a very conscious effort. But as you practice, you become more efficient until you can perform the task almost without thinking. Regardless of whether you're doing pottery or writing software, this not only makes you faster, but when you can do these things "without thinking," you can spend more time thinking about other things. In other words, it will free up your mind so you can focus on the finer details.

Blinded by Perfection

The first group were too busy trying to decide what makes the perfect pot, and they are never able to make it because they don't have the necessary skill to achieve their goal. However, the second half of the class was forced to make pot after pot and refine their process.

In software, this is commonly referred to as analysis paralysis. Where you spend so long trying to figure out everything, you never get around to actually starting or finishing. However, when you're forced to spend more time just building things and finishing things, you get to experience a wider range of problems and solutions.

Destination Unknown

To me, this was the most important point. But what do I mean by destination unknown? Well, the first group did not know what the perfect pot would look like or how to make it. They did not know what would make the pot collapse and what would make it beautiful. They had a vague idea of what they wanted to make it but lacked the skills to get there. So, by focusing on building the perfect pot, they did not get the opportunity to learn as much. The second group, on the other hand, started out in the same place but through trial and error they built on their knowledge. They were free to experiment and refine their process as they went.

The same applies to software development as well. Jumping in and building things lets you discover how a solution should look and how to build it. As you progress and encounter real issues, you can find the solutions as you need them. This is a much better way for us to learn, as we can see the reasons behind them much more clearly. You also don't need to solve everything upfront, which may be too much to hold in your head at one time. It's impossible to exactly know what to do upfront, you have to discover it along the way.

The first half of the pottery class spent hours on a single pot, throwing it and rethrowing it. But by the end, none of them had made a perfect pot. Students that had to make 50 pots, on the other hand, made so many pots that they figured out what worked and what didn't. By the end of the semester, they could all throw a perfect pot.

When writing software, the only time I knew what I should have done is after I've already tried something. You can't think of everything upfront. The best way to learn something is to try and do it.

Calling a Rest (Json) API with PowerShell

Tue, 11 Nov 2014 04:39:46 GMT

PowerShell makes working with rest API's easy. In PowerShell version 3, the cmdlets Invoke-RestMethod and Invoke-WebRequest where introduced. These cmdlets are a huge improvement coming from the .NET model you had to work with previously turning a request into a concise one liner similar to curl (Which is also an alias for Invoke-WebRequest in PowerShell).

The difference between the two is quite small, Invoke-RestMethod simply being a slightly more convenient wrapper around Invoke-WebRequest as it only returns the content, omitting the headers.

The most common case I tend to use this method for is querying or posting to a json rest API's. I usually end up just using Invoke-RestMethod so I'll focus on it.

The simplest call you can make is to just provide the URL. This will default to a GET request, and any unsupplied optional parameters are omitted from the request. For example, this GET request won't have a content type. However, if this parameter is omitted and the request method is POST, Invoke-RestMethod sets the content type to "application/x-www-form-urlencoded". Invoke-RestMethod -Uri $uri

The value returned will be automatically parsed depending on the content type of the response. For a json endpoint, I'll automatically get a PowerShell object (hashtable) that represents the json response.

So getting data from an endpoint is pretty easy but most rest APIs require an authentication token in order to verify your request. The most common way of supplying the token is via a HTTP header, which looks like this.

$apiKey = "SomeKey"
$resource = "http://localhost/api/books"

Invoke-RestMethod -Method Get -Uri $resource -Header @{ "X-ApiKey" = $apiKey }

Now we can query a json endpoint, but what about sending json. PowerShell objects can be represented using hashtables in a similar way to Javascript objects. PowerShell 2.0 also provides two functions (ConvertTo-Json and ConvertFrom-Json) for easily converting back and forth between JSON and PowerShell objects. The final step is to just construct a PS object and convert it.

$body = @{
    Name = "So long and thanks for all the fish"
}

Invoke-RestMethod -Method Post -Uri "$resource\new" -Body (ConvertTo-Json $body) -Header @{"X-ApiKey"=$apiKey}

So in just four lines of code you an submit a simple POST to any HTTP endpoint. If you want some further reading, you can also find the documentation for Invoke-RestMethod at Microsoft TechNet.

MVC Architecture

Thu, 25 Sep 2014 09:40:14 GMT

The typical scaffolding tool and sample applications for MVC web apps are usually based on simple CRUD like systems. They'll, usually, have Views written in a templating language, with some kind of ORM and Models for persistence and Controllers to tie the two together.

But there's a problem with this layout, and it's causing the Controllers to grow out of control. Because this scenario is based on CRUD the design here has found it's way into all kinds of applications, such as applications that need more than trivial amounts of business logic. Hopefully, now we can start to see the issue. All this logic ends up being placed in the Controllers which as a result become bloated. This also negatively impacts the Model, which effectively becomes just a data access layer.

So what's the alternative?

In MVC, the Model is often confused with various other similar concepts such as DTOs or a data layer. However, the Model is a more general concept that refers to the entire set of concepts your application is modelling. Persistence isn't a concern of the MVC pattern. So you can think of the Model as the entire domain, which includes the behaviours (business logic) and state. It's also useful to remember that the MVW (Model View Whatever) patterns are user interface patterns and do not attempt to describe or constrain any of these parts other than to split the View from the Model.

Moving the majority of application logic into the Model frees the Controller and View from dealing with those concerns and leaves them to focus on their primary role; which is to act as an interface to the outside world. This allows the Model to focus on the domain. Letting it properly encapsulate internal concepts, and exposing it's own facades where needed. It can even be self-contained with a well defined API of its own. With that in mind, the purpose of the controller becomes a bit clearer. Its job is simply to translate a request for the Model and any results back for the views.

In new applications, it's tempting to use the Models to generate the views and bind changes directly back onto the Models. However, this tends to lead back to large controllers that do too much, and has other problems as well. To avoid these issues instead of binding directly onto the Model (which may not even be possible such as when doing Domain Driven Design) data is bound onto a ViewModel. While View Models can sometimes look similar to objects in the Model they are distinct and ideally are never referenced from the Model. Separating them have a number of benefits including security, separation of concerns (think web validation) and most importantly decouple the Model from the View.

Decoupling the Model is extremely important as not doing so will severely limit how easy it is to make changes to the system. As the View will diverge from the Model in most systems. A simple example would be a markdown editor where you stored both the original Markdown and generated HTML in the Model as an optimisation. Sharing the Model with the View would have effectively limited the application to basic CRUD as the View must always mirror the Model.

However, there's still a problem even after introducing View Models. Applications not only diverge between the Model and the View Models. They also Diverge depending on the action you're performing. For example, when creating a record you may wish to include several fields that are read-only after that point. So updates need to use a different View Model from creates. They may even diverge depending on if the View Model is used for Input or Output. Using the same example, we may want to then show these fields as read-only when updating. Which would require them to exist in the Output Update View Model but not the Input Update View Model. In practice, I've found these View Models can start out looking very similar to your Models. But as time goes on and they diverge it becomes apparent the input (parameter) and output (return value) of each Action in the controller can be deserving of its own View Model.

Because of the similarities that View Models can have with the Model it can be possible to automate this process of converting between the two. In the past, I've used AutoMapper to do exactly this. However, as the View Models inevitably diverge you end up with so many mapping rules that it becomes easier to do the mappings explicitly. To read more about why using a mapper should be avoided check out the post Friends don't let friends use AutoMapper.

So to recap, the aim of MVC is to separate the View from the Model of the application. The Model should be where the bulk of your application lies and is effectively your application without the user interface. The View and the View Models represent the user's representation of the system. The controller then sits between the two and is the glue that brings these two parts together. This results in increased flexibility due to the decoupling between the two sides. Which means changes to the application are both easier and faster as you don't have to fight the Model to fit the design.

Avoid null with Containers

Wed, 27 Aug 2014 22:42:52 GMT

In C# It's often recommended that you should avoidi using nulls wherever possible. Avoiding nulls is a great concept that can simplify your code. Even so, nulls (or some such equivalent concept) will always have their place, such as to convey data is not available.

However, one case where I always pay close attention to using nulls is when working with what object-oriented developers call Containers. A container simply wraps around an instance or multiple insances of some type. This includes types such as Enumerable, List, Lazy, Observable, Task, Nullable (which is partially a special case because it never throws a NullReferenceException) arrays and other similar non-generic types. Typically in these cases returning an empty list or an empty Task over returning null makes the code much more clean and clear. By not returning null, you don't have to check for nulls and hence your code does not need to follow a different path based on the return value.

Comparing between the two even in a simple case greatly increases the complexity of the code. The first example is a standard foreach over an enumerable.

foreach (var item in enumerable) {
    item.Dump()
}

Which when you need to worry about nulls would become the following

if (enumerable != null) {
    foreach (var item in enumerable) {
        item.Dump()
    }
}

And for Tasks, here is the what it looks like without nulls

var result = await DoAsync()

And when the function can return nulls you loose type inference or end up with deeply nested code.

object result;
var resultAwaitable = await DoAsync();
if (resultAwaitable != null) {
    result = await resultAwaitable;
}

There are two main cases where you still might want to return a null value. Such as to be explicit about the difference between unavailable data and no data. The other is to avoid extra memory allocations in high-performance scenarios. However, I find both of these cases to be relatively rare.

In the end, the most important thing is usually maintainability and standardizing on empty containers offers more reliable and concise code.

Example Web.Config files (3.5 and 4.5)

Mon, 04 Aug 2014 07:00:35 GMT

These are the default generated Web.Config files from Visual Studio 2013 Update 3.

Web.Config for .NET Framework 4.5.1
Web.Config for .NET Framework 3.5

Other variations are all quite similar but if you think I'm missing one that's useful leave a comment.

Web.Config for .NET Framework 4.5.1

<?xml version="1.0" encoding="utf-8"?>
<!--
  For more information on how to configure your ASP.NET application, please visit
  https://go.microsoft.com/fwlink/?LinkId=301880
  -->
<configuration>
  <configSections>
    <!-- For more information on Entity Framework configuration, visit https://go.microsoft.com/fwlink/?LinkID=237468 -->
    <section name="entityFramework" type="System.Data.Entity.Internal.ConfigFile.EntityFrameworkSection, EntityFramework, Version=6.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" requirePermission="false" />
  </configSections>
  <connectionStrings>
    <add name="DefaultConnection" connectionString="Data Source=(LocalDb)\v11.0;AttachDbFilename=|DataDirectory|\aspnet-WebApplication45-20140804053515.mdf;Initial Catalog=aspnet-WebApplication45-20140804053515;Integrated Security=True"
      providerName="System.Data.SqlClient" />
  </connectionStrings>
  <appSettings>
    <add key="webpages:Version" value="3.0.0.0" />
    <add key="webpages:Enabled" value="false" />
    <add key="ClientValidationEnabled" value="true" />
    <add key="UnobtrusiveJavaScriptEnabled" value="true" />
  </appSettings>
  <system.web>
    <authentication mode="None" />
    <compilation debug="true" targetFramework="4.5.1" />
    <httpRuntime targetFramework="4.5.1" />
  </system.web>
  <system.webServer>
    <modules>
      <remove name="FormsAuthenticationModule" />
    </modules>
  </system.webServer>
  <runtime>
    <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
      <dependentAssembly>
        <assemblyIdentity name="System.Web.Helpers" publicKeyToken="31bf3856ad364e35" />
        <bindingRedirect oldVersion="1.0.0.0-3.0.0.0" newVersion="3.0.0.0" />
      </dependentAssembly>
      <dependentAssembly>
        <assemblyIdentity name="System.Web.Mvc" publicKeyToken="31bf3856ad364e35" />
        <bindingRedirect oldVersion="1.0.0.0-5.1.0.0" newVersion="5.1.0.0" />
      </dependentAssembly>
      <dependentAssembly>
        <assemblyIdentity name="System.Web.Optimization" publicKeyToken="31bf3856ad364e35" />
        <bindingRedirect oldVersion="1.0.0.0-1.1.0.0" newVersion="1.1.0.0" />
      </dependentAssembly>
      <dependentAssembly>
        <assemblyIdentity name="System.Web.WebPages" publicKeyToken="31bf3856ad364e35" />
        <bindingRedirect oldVersion="1.0.0.0-3.0.0.0" newVersion="3.0.0.0" />
      </dependentAssembly>
      <dependentAssembly>
        <assemblyIdentity name="WebGrease" publicKeyToken="31bf3856ad364e35" />
        <bindingRedirect oldVersion="1.0.0.0-1.5.2.14234" newVersion="1.5.2.14234" />
      </dependentAssembly>
    </assemblyBinding>
  </runtime>
  <entityFramework>
    <defaultConnectionFactory type="System.Data.Entity.Infrastructure.LocalDbConnectionFactory, EntityFramework">
      <parameters>
        <parameter value="v11.0" />
      </parameters>
    </defaultConnectionFactory>
    <providers>
      <provider invariantName="System.Data.SqlClient" type="System.Data.Entity.SqlServer.SqlProviderServices, EntityFramework.SqlServer" />
    </providers>
  </entityFramework>
</configuration>

Web.Config for .NET Framework 3.5

<?xml version="1.0"?>
<configuration>

    <configSections>
      <sectionGroup name="system.web.extensions" type="System.Web.Configuration.SystemWebExtensionsSectionGroup, System.Web.Extensions, Version=3.5.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35">
        <sectionGroup name="scripting" type="System.Web.Configuration.ScriptingSectionGroup, System.Web.Extensions, Version=3.5.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35">
          <section name="scriptResourceHandler" type="System.Web.Configuration.ScriptingScriptResourceHandlerSection, System.Web.Extensions, Version=3.5.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35" requirePermission="false" allowDefinition="MachineToApplication"/>
          <sectionGroup name="webServices" type="System.Web.Configuration.ScriptingWebServicesSectionGroup, System.Web.Extensions, Version=3.5.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35">
            <section name="jsonSerialization" type="System.Web.Configuration.ScriptingJsonSerializationSection, System.Web.Extensions, Version=3.5.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35" requirePermission="false" allowDefinition="Everywhere" />
            <section name="profileService" type="System.Web.Configuration.ScriptingProfileServiceSection, System.Web.Extensions, Version=3.5.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35" requirePermission="false" allowDefinition="MachineToApplication" />
            <section name="authenticationService" type="System.Web.Configuration.ScriptingAuthenticationServiceSection, System.Web.Extensions, Version=3.5.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35" requirePermission="false" allowDefinition="MachineToApplication" />
            <section name="roleService" type="System.Web.Configuration.ScriptingRoleServiceSection, System.Web.Extensions, Version=3.5.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35" requirePermission="false" allowDefinition="MachineToApplication" />
          </sectionGroup>
        </sectionGroup>
      </sectionGroup>
    </configSections>

    <appSettings />
    <connectionStrings />
    <system.web>
        <compilation debug="true">

          <assemblies>
            <add assembly="System.Core, Version=3.5.0.0, Culture=neutral, PublicKeyToken=B77A5C561934E089"/>
            <add assembly="System.Data.DataSetExtensions, Version=3.5.0.0, Culture=neutral, PublicKeyToken=B77A5C561934E089"/>
            <add assembly="System.Web.Extensions, Version=3.5.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35"/>
            <add assembly="System.Xml.Linq, Version=3.5.0.0, Culture=neutral, PublicKeyToken=B77A5C561934E089"/>
          </assemblies>

        </compilation>
        <!--
            The <authentication> section enables configuration
            of the security authentication mode used by
            ASP.NET to identify an incoming user.
        -->
        <authentication mode="Windows" />
        <!--
            The <customErrors> section enables configuration
            of what to do if/when an unhandled error occurs
            during the execution of a request. Specifically,
            it enables developers to configure html error pages
            to be displayed in place of a error stack trace.

        <customErrors mode="RemoteOnly" defaultRedirect="GenericErrorPage.htm">
            <error statusCode="403" redirect="NoAccess.htm" />
            <error statusCode="404" redirect="FileNotFound.htm" />
        </customErrors>
        -->

      <pages>
        <controls>
          <add tagPrefix="asp" namespace="System.Web.UI" assembly="System.Web.Extensions, Version=3.5.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35"/>
          <add tagPrefix="asp" namespace="System.Web.UI.WebControls" assembly="System.Web.Extensions, Version=3.5.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35"/>
        </controls>
      </pages>

      <httpHandlers>
        <remove verb="*" path="*.asmx"/>
        <add verb="*" path="*.asmx" validate="false" type="System.Web.Script.Services.ScriptHandlerFactory, System.Web.Extensions, Version=3.5.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35"/>
        <add verb="*" path="*_AppService.axd" validate="false" type="System.Web.Script.Services.ScriptHandlerFactory, System.Web.Extensions, Version=3.5.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35"/>
        <add verb="GET,HEAD" path="ScriptResource.axd" type="System.Web.Handlers.ScriptResourceHandler, System.Web.Extensions, Version=3.5.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35" validate="false"/>
      </httpHandlers>
      <httpModules>
        <add name="ScriptModule" type="System.Web.Handlers.ScriptModule, System.Web.Extensions, Version=3.5.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35"/>
      </httpModules>

    </system.web>

    <system.codedom>
      <compilers>
        <compiler language="c#;cs;csharp" extension=".cs" warningLevel="4"
                  type="Microsoft.CSharp.CSharpCodeProvider, System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089">
          <providerOption name="CompilerVersion" value="v3.5"/>
          <providerOption name="WarnAsError" value="false"/>
        </compiler>
     </compilers>
    </system.codedom>

    <!--
        The system.webServer section is required for running ASP.NET AJAX under Internet
        Information Services 7.0.  It is not necessary for previous version of IIS.
    -->
    <system.webServer>
      <validation validateIntegratedModeConfiguration="false"/>
      <modules>
        <remove name="ScriptModule" />
        <add name="ScriptModule" preCondition="managedHandler" type="System.Web.Handlers.ScriptModule, System.Web.Extensions, Version=3.5.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35"/>
      </modules>
      <handlers>
        <remove name="WebServiceHandlerFactory-Integrated"/>
        <remove name="ScriptHandlerFactory" />
        <remove name="ScriptHandlerFactoryAppServices" />
        <remove name="ScriptResource" />
        <add name="ScriptHandlerFactory" verb="*" path="*.asmx" preCondition="integratedMode"
             type="System.Web.Script.Services.ScriptHandlerFactory, System.Web.Extensions, Version=3.5.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35"/>
        <add name="ScriptHandlerFactoryAppServices" verb="*" path="*_AppService.axd" preCondition="integratedMode"
             type="System.Web.Script.Services.ScriptHandlerFactory, System.Web.Extensions, Version=3.5.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35"/>
        <add name="ScriptResource" preCondition="integratedMode" verb="GET,HEAD" path="ScriptResource.axd" type="System.Web.Handlers.ScriptResourceHandler, System.Web.Extensions, Version=3.5.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35" />
      </handlers>
    </system.webServer>
    <runtime>
      <assemblyBinding appliesTo="v2.0.50727" xmlns="urn:schemas-microsoft-com:asm.v1">
        <dependentAssembly>
          <assemblyIdentity name="System.Web.Extensions" publicKeyToken="31bf3856ad364e35"/>
          <bindingRedirect oldVersion="1.0.0.0-1.1.0.0" newVersion="3.5.0.0"/>
        </dependentAssembly>
        <dependentAssembly>
          <assemblyIdentity name="System.Web.Extensions.Design" publicKeyToken="31bf3856ad364e35"/>
          <bindingRedirect oldVersion="1.0.0.0-1.1.0.0" newVersion="3.5.0.0"/>
        </dependentAssembly>
      </assemblyBinding>
    </runtime>

</configuration>

Unlocking locked files in Windows

Fri, 01 Aug 2014 09:21:35 GMT

Every developer has run into this issue at least once. There are a few different tools you can use to find out what process is locking a file. In the past I've used a few different tools.

My favorite tool today is Lock Hunter which is powerful and easy to use. It's packed with a bunch of useful features, integrates nicely into explorer and installs without any "bonus" software. I can't recommend it enough.

Before Lock Hunter I used Unlocker by Empty Loop as my go to solution but today it's bundled with a toolbar and isn't early as pleasant as Lock Hunter.

If you're already using Process Explorer it can be a great solution. You can use it's Find Handle or DLL to locate the culprit. On the downside (or maybe the upside depending on your point of view) there are is no explorer integration.

Handle is another tool by Systernals but aimed solely at the command like. This tool focus on the basics and leaves the action up to the user.

Missed a tool you love to use? Leave a comment!

Debugging Rx with Seq

Mon, 21 Jul 2014 08:34:02 GMT

Debugging Reactive Extensions can sometimes be a bit tricky. When you have a few stream to watch, pinpointing failures is time consuming, and a can easily become a burden.

Catching first-chance exceptions can be extremely helpful when debugging Rx. However, it's much more useful to see how events progress over time. After coming across a thread about tracing in Reactive Extensions I thought I could make a few improvements to their tracing methods by making use of Seq and Serilog.

Seq is fantastic tool that lets you visualise and explore log data quickly and easily. You can get Seq up and running within minutes. Just download the NuGet for Serilog and install the latest version of Seq.

Some events in Seq.

To hook this up in your application grab this updated Trace extension method using Serilog.

public static IObservable<TSource> Trace<TSource>(this IObservable<TSource> source, string name)
{
	var id = 0;
	return Observable.Create<TSource>(observer => {

		var itemId = ++id;
		Action<string, object> trace = (m, v) => Log.Information("{name}{id}: {method}({value})", name, itemId, m, v);

		trace("Subscribe", null);
		IDisposable disposable = source.Subscribe(
			v => { trace("OnNext", v); observer.OnNext(v); },
			e => { trace("OnError", e); observer.OnError(e); },
			() => { trace("OnCompleted", null); observer.OnCompleted(); });

		return () => { trace("Dispose", null); disposable.Dispose(); };
	});
}

Next wire up the trace.

Observable
	.Interval(100).Trace("Interval")
    .Take(2).Trace("Take")
    .Count();

Then just register Seq with Serilog and log away.

Log.Logger = new LoggerConfiguration()
	.WriteTo.Seq("http://localhost:5341")
	.CreateLogger();

Making great Octopus PowerShell step templates

Mon, 14 Jul 2014 07:52:44 GMT

Step templates, introduced in Octopus Deploy 2.4, are a great way to share and reuse useful PowerShell scripts.

Anyone can make a step template and submit it via a pull request over at Github. I've come up with a few tricks to make writing step templates easier.

I wanted to make sure I could do a few things easily like testing the script outside of Octopus, handle errors as well as defaults and provide some structure around the layout of the script.

The first thing I do is declare the parameters I want the script to have. Note the inclusion of the whatIf switch parameter which makes it easy to test potentially dangerous scripts with a dry run.

param(
    [string]$parameter1,
    [int]$parameter2,
    [switch]$whatIf
)

By default PowerShell will attempt to continue to run a script after any errors so next I like to change that so an error will stop the script. Note this is just for running outside Octopus, as Octopus also sets this for you.

$ErrorActionPreference = "Stop"

Next I declare a helper function called Get-Param. This function makes it much easier to run scripts outside Octopus by checking for an Octopus parameter and automatically falling back to use a variable. Additionally it also handles errors for required parameters and default values for optional ones.

function Get-Param($Name, [switch]$Required, $Default) {
    $result = $null

    if ($OctopusParameters -ne $null) {
        $result = $OctopusParameters[$Name]
    }

    if ($result -eq $null) {
        $variable = Get-Variable $Name -EA SilentlyContinue
        if ($variable -ne $null) {
            $result = $variable.Value
        }
    }

    if ($result -eq $null) {
        if ($Required) {
            throw "Missing parameter value $Name"
        } else {
            $result = $Default
        }
    }

    return $result
}

Following that you can then declare any other functions that you need.

Finally, and most importantly is the main body of the script. It's executed last because of PowerShell requires you declare a function before you can use it. The body is a self executing anonymous function similar to how you would do so in JavaScript. This lets you provide a child scope for the parameters and variables your script uses. Declaring the parameters here also ensures that the parameters have the correct types for both Octopus and script parameters.

& {
    param(
        [string]$parameter1,
        [int]$parameter2
    )

    Write-Host "Script Title"
    Write-Host "PrameterName1: $parameter1"
    Write-Host "PrameterName2: $parameter2"

    # Main body of the script goes here!

 } `
 (Get-Param 'parameter1' -Required) `
 (Get-Param 'parameter2' -Default 10)

If you're wondering about the backticks, they allow the command to span multiple lines.

Here's the whole script.

# Running outside octopus
param(
	[string]$parameter,
	[switch]$whatIf
)

$ErrorActionPreference = "Stop"

function Get-Param($Name, [switch]$Required, $Default) {
	$result = $null

	if ($OctopusParameters -ne $null) {
		$result = $OctopusParameters[$Name]
	}

	if ($result -eq $null) {
		$variable = Get-Variable $Name -EA SilentlyContinue
		if ($variable -ne $null) {
			$result = $variable.Value
		}
	}

	if ($result -eq $null) {
		if ($Required) {
			throw "Missing parameter value $Name"
		} else {
			$result = $Default
		}
	}

	return $result
}

# More custom functions would go here

& {
	param(
		[string]$parameter
	)

	Write-Host "Script Title"
	Write-Host "PrameterName: $parameter"

    # Main body of the script goes here!

 } `
 (Get-Param 'parameter' -Required)

Powershell typeof

Mon, 14 Jul 2014 07:00:16 GMT

Coming from C#, it provides the built in function typeof() that you can use to get the Type of a class. Powershell also makes it easy to get Type information.

Wrapping the type in square brackets [TypeName] evaluates to an instance of System.Reflection.TypeInfo. Which you can then use to compare it as per normal.

PS>"".GetType() -Eq [string]
True
PS>(1).GetType() -Eq [int]
True
PS>(1.0).GetType() -Eq [int]
False

You can also assign the value to a variable if needed.

PS>$x = [bool]
PS>$x

IsPublic IsSerial Name                                     BaseType
-------- -------- ----                                     --------
True     True     Boolean                                  System.ValueType

I hope this will make it easier to find next time!

Octopus Deploy

Tue, 27 May 2014 21:43:04 GMT

I'm happy to announce that I've joined forces with the amazing developers at Octopus Deploy. It's a fantastic product and if you havn't heard of them yet make sure you take a look.

Deployments are such an important part of every project, yet they're often overlooked until the last minute. Even the best of us make mistakes and when we're under the pressure of a new deployment things can easily be rushed or forgotten. Often resulting in more deployments. Which is just one reason why automated deployments and Octopus Deploy is such an important part of my toolset.

I think that all deployments should be automated. Once you start automating deployments, you can guarantee they're consistently fast and reliable. Automating deployments isn't just about making them faster either. It opens up possibilities that simply weren't there before.

I'm glad I'll get to contribute some of my ideas in this space, and help make automated deployments a part of everyone's toolset.

An update on background tasks in ASP.NET

Thu, 15 May 2014 10:16:12 GMT

There are two types of background tasks that you'd want to use in .NET. Long running tasks which include things like background services that run every 5, 10 minutes or another interval. Long running tasks could be continuously running, possibly spanning several days. Then there are Short lived tasks which are great for running smaller asynchronous tasks. Long running tasks are not supported in ASP.NET however short lived tasks have always been possible, and just got a little easier.

But first, To recap on why long-running tasks are problematic. The main issue is that IIS is a request driven web server. Which means, if there's no one hitting your website nothing will happen. You could work around the problem as Phil Haack describes in his post The Dangers of Implementing Recurring Background Tasks In ASP.NET although you still need some kind of request to trigger the website. In the end, it's much better to just pull the background "Service" out of your website and run it as a Windows Service instead.

On the other hand, ASP.NET provides much better support for reliable short lived tasks through HostingEnvironment.RegisterObject. Using RegisterObject lets you register objects in IIS so it can provide a warning when the App Domain is being shutdown. Letting you finish up your work and terminate these background threads gracefully. This is extremely useful when you need to run asynchronous tasks like sending a batch of emails on a separate thread. However, implementing this interface is tedious and overly complicated.

In ASP.NET 4.5.2 this has become much easier with the introduction of HostingEnvironment.QueueBackgroundWorkItem which provides a much simpler alternative. By making use of the Task library for .NET you can invoke a lambda as a short running task. Two overloads are provided for the function.

  1. Action<CancellationToken>
  2. Func<CancellationToken, Task>

These methods are both functionally equivalent, fire and forget async. The first simply takes an Action and executes it in the background. The second is exactly the same but returns a Task which is required in order to use async lambda's, like so.

  HostingEnvironment.QueueBackgroundWorkItem(async cancellationToken => {    
      	// Perform task
  });

Under the covers the QueueBackgroundWorkItem method still uses HostingEnvironment.RegisterObject along with ThreadPool.QueueUserWorkItem to execute the tasks. Meaning you'll still only get 30 seconds to finish what you were doing if cancellation is requested. Even so, this is still an addition that will be greatly appreciated.

Implementing Domain Driven Design

Mon, 28 Apr 2014 01:55:35 GMT

I've talked about what Domain Driven Design is and what kind of benefits you can expect from it.

The next concept to explore is what the implementation looks like. I'll be focusing on the architecture of such a system more than how to model a domain, which Eric Evans covers in his book.

The domain in DDD contains all the business logic in the application. Its equivalent in a layered system would be the domain/business layer. A typical domain implementation can be said to look like this:

In Domain Driven Design there are three main types of objects the Aggregate Root, Entities and Value Objects. These are objects in the typical object orientated sense and represent a concept from the domain/business. The other two concepts show are Aggregates and Bounded Context, which relate to the logical grouping of these domain objects.

Entities

Entities are a model of an object or concept in the domain and always have a distinct Id, usually a Guid. An Entity could be anything from such as a user, a product, or any other object. The Aggregate Root shown is actually also an entity but it sits at the root of the hierarchy and has no direct references to it.

Entities are a model of the business, but what does an Entity actually look like? They are objects like, Product, Customer and Order which contain the behaviours (methods) of domain objects they are modeled after. These behaviours are owned by the Entities, not stowed away in a manager or service. This also means that Entities don't need to expose all those properties that services or managers would otherwise need. Instead, the state is encapsulated inside the object. Entities are not Data Transfer Objects and they should have very few, if any, properties. It also makes the behaviours discoverable, more expressive and more consistent as they are all local members of objects.

Take the following two lines for example:

company.Hire(newEmployee);

companyService.Hire(company, newEmployee);

When Hire is simply another behaviour of the company it becomes easy to find and clear to read.

So if Entities don't have any Properties how do you perform queries? The answer is; by using a read model not the domain model. The domain is primarily concerned with the business logic. It is only used to change the state of the domain. All queries are separated from the domain and instead a thin read layer is used. This is also known as Command and Query Responsibility Segregation or CQRS, which I'll talk about in a future post.

Once entities have behaviours they sometimes need to use services or take other dependencies. However, an entity should never have any dependencies injected into it. Instead, it's better to make things explicit. It's actually the behaviours that have dependencies, not the Entities. Therefore, when a service is needed, it can be provided as an argument in the method where it is used.

user.ChangePassword(hashingService, newPassword);

In regards to validation in the domain, we can view it as just another aspect of the behaviour. Once the entities have behaviours it's quite straightforward to check the arguments are valid and invariants and are maintained. Then the entity itself can ensure it is never put into an invalid state. This also applies for any constructors, which really are just another behaviour.

The final concern to address is persistence of the domain. The domain model should be persistence ignorant as it’s only concerned with the domain. Object relational mappers tend to make this more difficult although it is possible. However, I tend to use Domain Events to handle persistence and avoid ORMs altogether.

Values

Values on the other hand are just data and never have an Id. Their equality is determined as the sum of its parts. For example the value Position could contain Latitude and Longitude components. Two values where all the components are equal are also equal. For this reason Values are usually also immutable.

Now that I've explained Entities and Values I'll move up to Aggregates and the Bounded Context.

Aggregates

An Aggregate is a boundary around one or more Entities. This boundary is also a transactional boundary. Such that all Entities in one Aggregate cannot directly reference Entities in another. An Aggregate would represent a single unit, for example a Tree. The aggregate is composed of the tree or trunk (the Aggregate Root) and leaves (the other Entities). Aggregates are also the basic element of data storage, much like in a document database. You only save or load whole Aggregates.

I mentioned earlier that you cannot directly reference Entities in another Aggregate. Any references from outside the aggregate should only go to the aggregate root, as you cannot cross the Aggregate boundary. Unlike references inside an Aggregate that link directly to an instance. References to another Aggregate Root will only use its identifier. This would also apply to circular references to the Aggregates own Aggregate Root.

Contexts

Context is extremely important in Domain Driven Design. An entity could logically exist in multiple Bounded Contexts or even Aggregates with different representations. As an Example, Human Resources sees a person as a resource with a different set of information and behaviours than Accountant would. In an online store items in a user’s cart could be different to items in their order, even though conceptually they're the same thing.

Modeling one concept as multiple Entities is more common in larger domain where it becomes much more difficult to find a single unified model. Especially when different groups of people need different things from the model.

Other Concepts

There are three other concepts I didn't fully cover here; Repositories, Domain Events and Services.

Repositories are a very simple concept and simply load or save Aggregates.

I did briefly mention Domain Events; the basic premise here is that instead of an ORM implicitly tracking changes Events are used to describe what changes happened.

The concept of services which could be application, domain or infrastructure services.

In Conculsion

These are the core concepts of a Domain Driven Design implementation. The design of the domain model is built around the concepts of Aggregates and Entities. Modelling a domain in this way is results in code that is much more expressive, clear and cohesive.

Internet Explorer Won't Save Cookies On Domains With Underscores In Them.

Tue, 15 Apr 2014 23:41:23 GMT

All versions of Internet Explorer (version 5.5 all the way to version 11) will discard all cookies if the domain has an underscore in it. This can lead to the following issues, which can be quite difficult to debug:

Cookies are not set on the client system.
ASP.NET Session variables are lost.

When debugging web applications I can't recommend Fiddler enough. It will be able to pick up these kinds of issues and in this case provides the following warning.

!! WARNING !!: Server hostname contains an underscore and this response sets a cookie. Internet Explorer does not permit cookies to be set on hostnames containing underscores. See https://support.microsoft.com/kb/316112.

It's perfectly legal for a domain to have an underscore in it, so this issue goes against the standard (See RFC 2181, section 11, "Name syntax"). Part of the reason it is hard to debug is that the domain will work fine in all other browsers.

The support article has the following to say about the issue:

PRB: Session Variables Do Not Persist Between Requests After You Install Internet Explorer Security Patch MS01-055

A potential security vulnerability exists in Internet Explorer versions 5.5 and 6.0 in which a malicious user could create a URL that allows a Web site to gain unauthorized access to cookies that are stored on a client computer and then (potentially) modify the values that are contained in these cookies. Because some Web sites use cookies that are stored on client computers to store sensitive information, this security vulnerability could expose personal information.

So this issue is caused to prevent a security issue on a deprecated version of Internet Explorer. I think it's about time this restriction was removed.

You can vote to fix this issue at Microsoft Connect.

Self executing anonymous function in Powershell

Fri, 11 Apr 2014 02:40:27 GMT

One of the great features of JavaScript is the self-executing anonymous function. It's extremely useful because you can avoid polluting the global scope and express your dependencies in an explicit manner.

PowerShell lets you do something similar by taking advantage of blocks, the param keyword and the call operator &.

& { param($msg) Write-Host $msg } "Hello World"

This function will be automatically invoked with the string "Hello World".

Ninject Modules

Fri, 04 Apr 2014 21:08:50 GMT

Ninject is a fantastic Dependency Injection framework for the .NET framework. You can bind to the kernel whenever you want however you usually want to setup all your binding in once place right, at the start of your application. Ninject provides the perfect place to declare all you bindings in a single place in the form of a NinjectModule.

To create a Kernel using a Module it is passed as a constructor parameter.

var kernel = new StandardKernel(new ApplicationBindingsModule());

Then you can simply declare the Module which requires you to implement the abstract Load method.

public class ApplicationBindingsModule : NinjectModule
{
	public override void Load()
	{
		Bind<IClock>().To<Clock>().InSingletonScope();

		// If you use Ninject.Extensions.Convention you have to use `this` to use the extension method syntax.
		//this.Bind(x => x.FromThisAssembly()...
	}
}

Using Modules allows you to easily swap out bindings when creating a Kernel and provides a standard location for all your bindings.

What is Domain Driven Design?

Mon, 31 Mar 2014 08:57:51 GMT

In 2004, Erick Evans coined the term Domain Driven Design in his book also called Domain Driven Design.

Since then there has been lots of talk about using Domain Driven Design to tackle high complexity business problems, but what about everyday development. Just like all software there are tradeoffs. The difference between projects not complex enough for Domain Driven Design and projects that are is not discrete and is entirely superficial; the domain itself is much more important. I'm hoping I can look at some key ideas presented in the Domain Driven Design space and see how they apply to everyday software.

Traditionally when we set out to design a system we'd start by looking at the database. Relational databases have been so ingrained into us that it's become the only way that we have thought about our applications. We'd model our systems after the tables and their columns. As a result, the business logic was spread between all kinds of manager objects and in the heads of the users.

The focus of Domain Driven Design is to develop a consistent language that describes both entities and their behaviours in the domain. Identifying and modelling the behaviours, instead of properties and data, makes the system self describing allowing us to clearly see the business requirements. This is fantastic for the developers as it becomes much easer to see intent. The second thing it provides developers is the ability to use encapsulation instead of exposing all their data via properties. I'll explain why this leads to better software in a later post. On the business side it enables developers and the domain experts to share the same understanding of the domain and communicate effectively. This is very valuable because changing business requirements map very easily back into the application.

Domain Driven Design has the potential to provide many benefits to business and to developers. If your software is the driving force in your business, Domain Driven Design is definitely worth looking into.

Why use NuGet

Sat, 22 Mar 2014 03:04:31 GMT

I've had a few people ask me why they should use NuGet and the occasional person that's never heard of it. If you are currently developing software without a package manager, here is why you should be interested.

What is NuGet?

NuGet is the official package manager for .NET managed by the Outercurve Foundation.

But firstly what is a package manager?

In software, a package management system, also called package manager, is a collection of software tools to automate the process of installing, upgrading, configuring, and removing software packages.

A typical NuGet Package consists simply of a metadata file and some dll files for the supported .NET versions.

![Package structure](https://pfcjjq.dm2302.livefilestore.com/y2pCBjkRcauQ0pM0ZHiJea7P_cFZgdGtfN6PqYQNV5ldF7nphagBs5D_TNrlGoctoObNNSvNZ_qj82nD3EhJ-LAW4TebxfVtb5GC0_eEYGf7fM/NuGet.png)

These packages don't just have to contain dlls, they can also contain content such as JavaScript libraries and other text files.

I've never needed NuGet before, what's wrong with just referencing them directly like it's always been?

Let’s say you just started working on a new project. You want to use a library to help you out. It could be an SQL Generation library to help write a dynamic report viewer.

So you search around for a library and find a one you're happy with. Next you locate and download the binaries for the library and copy them to your project. Now you can reference them from Visual Studio and start work.

The next thing you'll need to worry about is updating that library. If you run into a bug and or want to update the project to get the latest updates or features you have to go through most of the download process again. You'll also never be able to tell if there's an update ready without going to their website.

The worst part is you have to do this for every library you want to use. This is a lot of work.

It gets even worse if that library happened to depend on another project, such as logger, which you or any other projects might also want to depend on.

How does NuGet make it easier for me?

NuGet can provide consistency, every library you need can be added with just one click. All configuration you need is automatic and any dependencies are automatic added as well.

Every library you have can be updated with one command. With each package automatically updated to the correct version taking into account each package that depends on it.

Is NuGet perfect?

Nothing is without flaws and NuGet is no exception. NuGet is community driven and while this is great it also means packages, just like libraries, are only as good as their creators.

It's also leaky abstraction, which just means you still need to know how assemblies work and get referenced.

Who is actually using it?

The .NET framework team itself has been adopting NuGet as the primary method of delivery over the last few years. The ASP.NET and EntityFramework teams have been leading that trend. The core .NET team started using NuGet in 2012 and have now been releasing parts of the Base Class Library with NuGet.

Along with a community including thousands of projects NuGet has become the standard method of delivering and using .NET libraries.

How do I get started?

Once you open Visual Studio and create a new project you're already using NuGet.

To check out the packages you're using and to add new ones select Manage NuGet Packages... from the References menu.

From here you can browse through and install the great range of available packages.

NuGet allows people to share and take advantage of numerous fantastic projects. Allowing the .NET community to innovate and deliver at an accelerated rate. So next time you're looking for a great library, check NuGet first.

What's in a Buzzword

Fri, 14 Mar 2014 11:08:12 GMT

It's sometimes tempting to dismiss things as the next Fad or Buzzword. There's never enough time to learn everything and some things are genuinely bad but if you get into the habit of dismissing everything you're also going to miss a lot of good things.

Next time you see a new name or word that's buzzing, take the time to investigate what it's about and make your own choices. Names help us to convey our ideas. Terms like Domain Driven Design and Microservices hold with them a host of implicit knowledge. Wrapping up a bunch of existing concepts and providing it to us a concise whole.

Don't feel bad because you don't know a word, everyone has to say i don't know every once in a while.

So next time you see a new library, pattern or architecture take a deeper look. Who knows, you might just learn something.

OWIN and System.Web.Optimizations

Sat, 01 Feb 2014 05:37:07 GMT

I struck out to see if I could make use of the Microsoft ASP.NET Web Optimization Framework (version 1.1.2) in my self hosted OWIN Nancy application.

The only real resource I could find on the topic was the page How to use System.Web.Optimization Bundling with Nancy in the Nancy wiki on GitHub. Which looked good at first sight untill I looked a little deeper. This page is not only wildly out of date but didn't even get me close to a working solution.

The System.Web.Optimization project unfortunately has a large dependency on System.Web. Which for all practicle purposes makes it unusable when using Microsoft.Owin.Hosting.

While might still be possible to get it working without IIS by replacing the BundleHandler, providing a HttpContext and emulating a virtual directory for the app domain. Which would look something like the this...

var domain = Thread.GetDomain();
domain.SetData(".appDomain", "");
domain.SetData(".appVPath", "/");
domain.SetData(".appPath", "/");
domain.SetData(".appId", "");

I think I'll just wait for some official OWIN support and see how Cassette goes in the meantime.

Git - When to Merge

Thu, 12 Sep 2013 01:00:00 GMT

Many people use git pull and merge frequently in an effort to keep a branch up to date while working a feature. However it's actually recommended not to merge into a feature branch until it's complete and ready to be integrated back into the development branch.

The reason for this is that merge is a is a semantic action and actually has additional meaning than just update this branch. For example a feature branch should only be concerned with adding a single feature to a point in time. This makes development easier, you don't want the system changing around you as you develop a feature. Merge does a great job at integrating branches when they're ready (As long as you're project isn't completely different [if it is you have bigger problems]). It also lets you become more flexible, branches can merge with other branches or features without getting changes they're not interested in.

It's also good to remember if you just want to sync with a remote such as the origin you can use fetch which avoids the implicit merge of pull.

TechEd Australia 2013

Wed, 04 Sep 2013 01:00:00 GMT

The first day at TechEd offered two main sessions the developer kick off and the Keynote. The developer kick off touched on a new Visual Studio 2013 feature called Browser Link for ASP.NET and Team Foundation Service.

The kick off started by introducing One ASP.NET which is the unification of all web related .NET technologies, frameworks and libraries like MVC, SignalR and Web Forms can be mixed and matched however you like and all fall under the one ASP.NET umbrella. There has been a range of improvement to intellisense across the board particularly in html and CSS editors. The main highlight was the introduction to Browser Link by the creator of Web Essentials for Visual Studio.

Brower link enables a persistent link from Visual Studio to whatever web browser you use. It enables a few interesting scenarios the first of which being changes made to CSS are instantly reflected in all the browsers connected by Browser Link. Edits to razor files require a little more than a save (one key combo) because behind the scenes you're telling each connected browser to refresh the page but you neve have to leave Visual Studio. Imagine you're trying to iron out a few CSS quirks across two or more browsers. You can have two browsers open at the same time and CSS changes appear instantly on both browsers as soon as you save the file making comparisons efficient and easy.

Browser link also lets you do the inverse, you can use the web browser to jump to the exact line in your razor template that an element corresponds to via the inspector (Ctrl-Alt-I). You can also go one step further, ctrl-alt-d enables edit mode which can let you do thinks link edit html or Action Links and other Html helpers which are updated in real time and this all works in any browser.

The next session introduced Team Foundation Service which has a set of features coming to the next version of Team Foundation Server. There are a few interesting features including git support, a free tier for up to 5 users, a scrum style backlog and Trello-esk cards. Also notably was the addition of an integrated persistent chat room through which you can share news and chat with other members but also see general activity in the project such as build and code review events.

The primary keynote had a few speakers, the main talk was from the CEO of Yammer. This was the one of the best talks I've seen and it talked about organisations and change. The theme was Blink and you'll miss it. The rate at which technology is changing is happening so fast that it's becoming even more important for people and organisations to be able to learn and react faster. If you wait too long it's already irrelevant. This means instead of long release cycles where failure means 2-3 years of work goes down the drain, moving to pushing updates every month. This means the cost of failure is much smaller as well. What's worse changing one month of work or one year's worth. This is so important for organisations and this quote by Jack Welch describes it perfectly.

"If the rate of change on the outside exceeds the rate of change on the inside, the end is near."

There have been an increasing number of disruptive changes happening in the last 5 or so years. Communication has enabled modern society to move ideas and information much faster than ever before. The problem we see in companies these days is that people inside organisations don't communicate very well, especially across departments. The outside world is moving so fast because it's a network of self motivated people and we need to bring that into our companies and organisations. These types of organisations are called Responsive Organisations and you can read more about what it means to be one at https://www.theresponsiveorg.com/manifesto.

John Carmack Keynote

Sat, 03 Aug 2013 01:00:00 GMT

John Carmack's keynote this year was a fantastic talk about software engineering. Parts three and four of the keynote touched on all the key hot topics including Code Quality, Functional Programming, Static Typing vs Dynamic Typing, Static Analysis and Garbage Collection. Defiantly worth watching if you're into professional development, and if not it's still worth watching.

My PowerShell Profile

Wed, 03 Jul 2013 01:00:00 GMT

This is my PowerShell profile, it does a few nice things such as add open and edit commands to the shell. Most importantly though it customizes my prompt to shorten it while still showing how deep I am in the directory hierarchy. I've also set up poshgit as well as a few other helpers and functions.

function Open($path) {
	explorer $path
}

function Edit {
 
	[CmdletBinding()]
	Param(
	
		[Parameter(Mandatory = $False, ValueFromPipeline = $True, ValueFromRemainingArguments = $True, Position = 0)] 
		$File
	)
	 
	Process {
		$app = "C:\Program Files (x86)\Notepad++\notepad++.exe"
	
		if ($File -ne $null) {
			$parameters = '"' + $File + '"'
			
			$options = New-Object "System.Diagnostics.ProcessStartInfo"
			$options.FileName = $app
			$options.Arguments = $parameters
			$options.WorkingDirectory = $pwd
			
			$temp = [Diagnostics.Process]::Start($options).WaitForInputIdle(500)
		}
		Invoke-Item $app
	}
}

# This was already here
function elevate-process
{
	$file, [string]$arguments = $args;
	$psi = new-object System.Diagnostics.ProcessStartInfo $file;
	$psi.Arguments = $arguments;
	$psi.Verb = "runas";

	$psi.WorkingDirectory = get-location;
	[System.Diagnostics.Process]::Start($psi);
}
set-alias sudo elevate-process;


# WebGet
function get-html([string]$url)
{
	$webClient = (New-Object System.Net.WebClient);
	$webClient.DownloadString($url);
}

function shorten-path([string] $path) { 
   $loc = $path.Replace($HOME, '~') 
   # remove prefix for UNC paths
   $loc = $loc -replace '^[^:]+::', ''
   # make path shorter like tabs in Vim,
   # handle paths starting with \\ and . correctly
   return ($loc -replace '\\(\.?)([^\\])[^\\]*(?=\\)','\$1$2') 
}

function prompt { 
	# our theme
	$cdelim = [ConsoleColor]::DarkCyan 
	$chost = [ConsoleColor]::Green 
	$cloc = [ConsoleColor]::Cyan 
	$default = [ConsoleColor]::White 
	
	# Reset color, which can be messed up by Enable-GitColors
	$Host.UI.RawUI.ForegroundColor = $default

	write-host (shorten-path (pwd).Path) "" -n -f $cloc
	Write-VcsStatus
	return '> '
}

# gci . *.cs -Recurse | select-string . | Group Filename | Measure-Object Count -Min -Max -Average
function CountLines($directory)
{
	$pattern = "*.cs"
	$directories = [System.IO.Directory]::GetDirectories($directory)
	$files = [System.IO.Directory]::GetFiles($directory, $pattern)

	$lineCount = 0

	foreach($file in $files) {
		$lineCount += [System.IO.File]::ReadAllText($file).Split("`n").Count
	}

	foreach($subdirectory in $directories) {
		$lineCount += CountLines $subdirectory
	}

	$lineCount
}

$msbuildPath = "C:\Windows\Microsoft.NET\Framework\v4.0.30319\"
$gitPath = (Get-Item "Env:LocalAppData").Value + "\GitHub\PortableGit_93e8418133eb85e81a81e5e19c272776524496c6\cmd\"

$env:path += ";$gitPath;$msbuildPath;"


function Enable-Git {
	. 'C:\Users\me\Documents\WindowsPowerShell\Modules\posh-git\profile.example.ps1'
}
	
#Write-Host "For git use 'Enable-Git'"

function Load-Profile
{
	. $Profile
}

function Edit-Profile
{
	edit "C:\Users\me\Documents\WindowsPowerShell\Profile.ps1"
}

function Git-Log([switch]$OneLine, $Length)
{
	$Length = 1000;

	if ($OneLine) {
		git log --pretty=oneline -n $Length
	} else {
		git log -n $Length
	}	
}

Testing emails in .NET

Mon, 29 Apr 2013 01:00:00 GMT

Testing emails can sometimes be a bit challenging. smtp4dev is a small program that intercepts all received emails so you can quickly send and view emails from any application that uses it as the SMTP server.

For a .NET application just add this to the config file.

<mailSettings>
	<smtp>
		<network host="localhost"/>
	</smtp>
</mailSettings>

Now all emails that the application sends will appear for your viewing pleasure in the email list (pictured below).

Loading Settings From web.config

Sat, 20 Apr 2013 01:00:00 GMT

It's always a good idea to be as consistent as possible when you're writing code. So when you're using settings from a configuration file it's a good idea to avoid sprinkling ConfigurationManager.AppSettings["MySetting"] around the codebase, especially if you've making multiple calls to it.

A great way to provide consistency and to remove this duplication is to create a settings class either one static global one or multiple instance classes which I'll then manage with dependency injection. Then I load all the settings from configuration into that class on start up.

I've also written a little library that makes use of reflection to make this even easier.

Once my settings are in my config file

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
	<appSettings>
		<add key="Domain" value="example.com" />
		<add key="PagingSize" value="30" />
		<add key="Invalid.C#.Identifier" value="test" />
	</appSettings>
</configuration>

I make a static or instance class depending on my needs. For simple applications with only a few settings one static class is fine.

private static class Settings
{
	public string Domain { get; set; }

	public int PagingSize { get; set; }

	[Named("Invalid.C#.Identifier")]
	public string ICID { get; set; }

}

Then using my library call either Inflate.Static or Inflate.Instance and the cool thing is I can use any key value source.

using Fire.Configuration;

Inflate.Static( typeof(Settings), x => ConfigurationManager.AppSettings[x] );


var settings = new SpecificSettings();

Inflate.Instance( settings, x => ConfigurationManager.AppSettings[x] );

All the code for this is in bitbucket at https://bitbucket.org/Lavinski/fire

There is even a nuget package https://nuget.org/packages/Fire/

Install-Package Fire

Wildcards hostname in your hosts file

Wed, 10 Apr 2013 01:00:00 GMT

If you've ever wanted to setup a wildcard entry in your hosts file so you don't have to add an entry for every IIS binding you'll be happy to know there is a way.

The trick however is not to use the hosts file but a local DNS proxy instead. Acrylic DNS is a free and open source which you can download here. You use it in much the same way you would use a hosts file.

Stackoverflow has a great answer detailing how to set it up:

Configuring Acrylic DNS Proxy

To configure Acrylic DNS Proxy, install it from the above link then go to:

Start
Programs
Acrilic DNS Proxy
Config
Edit Custom Hosts File

Add the folowing lines on the end of the file:

127.0.0.1	*.localhost
127.0.0.1	*.local

Restart the Acrilic DNS Proxy service:

Start
Programs
Acrilic DNS Proxy
Config
Restart Acrilic Service

You will also need to adjust your DNS setting in you network interface settings:

Start
Control Panel
Network and Internet
Network Connections
Local Area Connection Properties
TCP/IPv4

Set "Use the following DNS server address":

Preferred DNS Server: 127.0.0.1

Multiple https bindings in IIS7 for developers

Wed, 20 Mar 2013 02:00:00 GMT

When working with websites that require https (SSL) developers always ending up facing the problem: Why can't you use hostnames with https. This is because hostname is encrypted so IIS needs to establish the SSL connection before it can even read the hostname.

However we can get around this problem by making use of a wildcard certificate. Then it won't matter what the hostname is and we can happily use SSL on all our internal dev sites.

You can't actually use IIS to setup a wildcard certificate so I'll be using a bit of Powershell to move things along.

The first step is to create the self signed certificate. Any clients will also have to trust this certificate.

$cert = New-SelfSignedCertificate -DnsName "*.company.internal" -CertStoreLocation cert:\LocalMachine\My
Export-Certificate -Cert $cert -FilePath "company.internal.cer"

$password = ConvertTo-SecureString -String "SecretHeHe" -Force –AsPlainText
Export-PfxCertificate -Cert $cert -FilePath "company.internal.pfx" -Password $password

Next add the bindings to IIS, this script will add a http and https binding for each combination of site and environment.

$sites = @(
	"website"
)

$envs = @(
	"test",
	"stage"
)

Import-Module WebAdministration
foreach ($name in $sites) {
	
	foreach ($env in $envs) {
		$name = "$name ($env)"
	
		$siteName = "$name ($env)"
		New-WebBinding -Name $siteName -Port 80 -Protocol http -HostHeader "$env.$name.company.internal"
		New-WebBinding -Name $siteName -Port 443 -Protocol https -HostHeader "$env.$name.company.internal"
	}
}

Just one thing left, setup IIS to use the certificate. To do that open IIS and import the certificate. Then select the open one of the https bindings and select the certificate (it won't matter which one, any site on that port will use it).

Properties Visual Studio passes to MSBuild

Fri, 08 Mar 2013 02:00:00 GMT

It always takes me ages to kind what properties Visual Studio makes available to the MSBuild project files so here's the full set.

$(RemoteMachine)

Set to the value of the Remote Machine property on the Debug property page. See Changing Project Settings for a C/C++ Debug Configuration for more information.


$(Configuration)

The name of the current project configuration (for example, "Debug").


$(Platform)

The name of current project platform (for example, "Win32").


$(ParentName)

(Deprecated.) Name of the item containing this project item. This will be the parent folder name, or project name.


$(RootNameSpace)

The namespace, if any, containing the application.


$(IntDir)

Path to the directory specified for intermediate files relative to the project directory. This path should have a trailing slash. This resolves to the value for the Intermediate Directory property.


$(OutDir)

Path to the output file directory, relative to the project directory. This path should have a trailing slash. This resolves to the value for the Output Directory property.


$(DevEnvDir)

The installation directory of Visual Studio 2010 (defined as drive + path); includes the trailing backslash '\'.


$(InputDir)

(Deprecated; migrated.) The directory of the input file (defined as drive + path); includes the trailing backslash '\'. If the project is the input, then this macro is equivalent to $(ProjectDir).


$(InputPath)

(Deprecated; migrated.) The absolute path name of the input file (defined as drive + path + base name + file extension). If the project is the input, then this macro is equivalent to $(ProjectPath).


$(InputName)

(Deprecated; migrated.) The base name of the input file. If the project is the input, then this macro is equivalent to $(ProjectName).


$(InputFileName)

(Deprecated; migrated.) The file name of the input file (defined as base name + file extension). If the project is the input, then this macro is equivalent to $(ProjectFileName).


$(InputExt)

(Deprecated; migrated.) The file extension of the input file. It includes the '.' before the file extension. If the project is the input, then this macro is equivalent to $(ProjectExt).


$(ProjectDir)

The directory of the project (defined as drive + path); includes the trailing backslash '\'.


$(ProjectPath)

The absolute path name of the project (defined as drive + path + base name + file extension).


$(ProjectName)

The base name of the project.


$(ProjectFileName)

The file name of the project (defined as base name + file extension).


$(ProjectExt)

The file extension of the project. It includes the '.' before the file extension.


$(SolutionDir)

The directory of the solution (defined as drive + path); includes the trailing backslash '\'.


$(SolutionPath)

The absolute path name of the solution (defined as drive + path + base name + file extension).


$(SolutionName)

The base name of the solution.


$(SolutionFileName)

The file name of the solution (defined as base name + file extension).


$(SolutionExt)

The file extension of the solution. It includes the '.' before the file extension.


$(TargetDir)

The directory of the primary output file for the build (defined as drive + path); includes the trailing backslash '\'.


$(TargetPath)

The absolute path name of the primary output file for the build (defined as drive + path + base name + file extension).


$(TargetName)

The base name of the primary output file for the build.


$(TargetFileName)

The file name of the primary output file for the build (defined as base name + file extension).


$(TargetExt)

The file extension of the primary output file for the build. It includes the '.' before the file extension.


$(VSInstallDir)

The directory into which you installed Visual Studio 2010.

This property contains the version of the targeted Visual Studio, which might be different that the host Visual Studio. For example, when building with $(PlatformToolset) = v90, $(VSInstallDir) contains the path to the Visual Studio 2008 installation.


$(VCInstallDir)

The directory into which you installed Visual C++ 2010.

This property contains the version of the targeted Visual C++, which might be different that the host Visual Studio. For example, when building with $(PlatformToolset) = v90, $(VCInstallDir) contains the path to the Visual C++ 2008 installation.


$(FrameworkDir)

The directory into which the .NET Framework was installed.


$(FrameworkVersion)

The version of the .NET Framework used by Visual Studio. Combined with $(FrameworkDir), the full path to the version of the .NET Framework use by Visual Studio.


$(FrameworkSDKDir)

The directory into which you installed the .NET Framework. The .NET Framework could have been installed as part of Visual Studio 2010 or separately.


$(WebDeployPath)

The relative path from the web deployment root to where the project outputs belong. Returns the same value as RelativePath.


$(WebDeployRoot)

The absolute path to the location of . For example, c:\inetpub\wwwroot.


$(SafeParentName)

(Deprecated.) The name of the immediate parent in valid name format. For example, a form is the parent of a .resx file.


$(SafeInputName)

(Deprecated.) The name of the file as a valid class name, minus file extension.


$(SafeRootNamespace)

(Deprecated.) The namespace name in which the project wizards will add code. This namespace name will only contain characters that would be permitted in a valid C++ identifier.


$(FxCopDir)

The path to the fxcop.cmd file. The fxcop.cmd file is not installed with all Visual C++ editions.

Source: https://msdn.microsoft.com/en-us/library/c02as0cs.aspx

Switch from 'Entity Framework Linq' to 'Linq to Objects' with AsEnumerable

Thu, 14 Feb 2013 02:00:00 GMT

Most people know what when writing a Linq query with Entity Framework that until that query is enumerated it won't call out to the database. Sometimes it's necessary to do some more work with the results such as use a method as part of the projection.

var result =
    from people in Context.People
    where people.DateOfBirth < twentyYearsAgo
    select people;

function GetAge(DateTime dateOfBirth) { ...

This is where AsEnumerable comes in. This generic function allows you to switch from the IQueryable source to IEnumerable which will use Linq to Objects. Using AsEnumerable instead of ToList allows us to still delay the execution of the query until it's needed.

var query =
    from people in Context.People
    where people.DateOfBirth < twentyYearsAgo
    select people;

var result =
    from people in query.AsEnumerable()
    select GetAge(people.DateOfBirth);

You could do the same thing by casting to the IEnumerable type but for anonymous objects you'll find you need to use AsEnumerable anyway to infer the type from the query.

Using ProviderName to get the database connection

Fri, 01 Feb 2013 02:00:00 GMT

When setting your connection string in a .NET config file you can make use of the providerName attribute by using DbProviderFactories.GetFactory and passing it the provider name.

var connectionStringSettings = ConfigurationManager.ConnectionStrings["DefaultConnection"];

connection = DbProviderFactories.GetFactory(connectionStringSettings.ProviderName).CreateConnection();
connection.ConnectionString = connectionStringSettings.ConnectionString;
connection.Open();

Entity Framework and Foreign key constraint may cause cycles or multiple cascade paths

Mon, 21 Jan 2013 02:00:00 GMT

Today I ran into the Foreign key constraint may cause cycles or multiple cascade paths issue for the first time. The first thing to note is the error isn't actually from Entity Framework it's from SQL Server.

Introducing FOREIGN KEY constraint 'ConstraintName' on table 'TableName' may cause cycles or multiple cascade paths. Specify ON DELETE NO ACTION or ON UPDATE NO ACTION, or modify other FOREIGN KEY constraints.

I was however using EF5 Code First so I fist needed to figure out why my model was giving me back this error. The error message itself pointed me to the Schema table referring to the Project table. Note my schema looks a bit like this:

Project { Id, Name }
Schema { Id, ProjectId, ... }
Node { Id, ProjectId, ... }

As you can see there are two relationships to the Projects table, which is usually fine. The issue comes from the fact that EF using the OneToManyCascadeDeleteConvention convention my default. Which was not what I wanted for these two relationships.

My EF model takes the convention and attribute approach, I try to avoid the fluent API to keep everything in the one place. However there is currently no attribute to turn off cascade delete.

Here is how to do it using the fluent API:

internal sealed class SchemaConfiguration : EntityTypeConfiguration<Schema>
{

	public SchemaConfiguration()
	{
		this
			.HasRequired(x => x.Project)
			.WithMany()
			.WillCascadeOnDelete(false);
	}
}

/// <summary>
/// Model Creating Event Handler.
/// </summary>
protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
	modelBuilder.Configurations.Add(new NodeConfiguration());
}

Need more help

Ninject and Singletons in the WebAPI with Scopes

Fri, 11 Jan 2013 02:00:00 GMT

The new Activation Blocks in Ninject are great for automatically resolving dependencies in the request scope. However they suffer from one flaw which is they don't respect Constant or Singletons in fact they ignore the scope completely!

In my application I needed a reference my dependency injection container in one of my objects. This would end up throwing Error loading Ninject component ICache at the end of the request because now I had two kernels. Not good at all however I still had hope.

Luckily there is one type of binding the ActivationBlock can't ignore. The ToMethod binding. So now I can make a small ScopedResolver class which I can then inject into my class instead.

Binding

kernel.Bind<ScopedResolver>().ToMethod(x => {
    return new ScopedResolver((IActivationBlock)x.GetScope());
});

ScopedResolver

public class ScopedResolver
{
	IActivationBlock resolver;

	public ScopedResolver(IActivationBlock resolver)
	{
		this.resolver = resolver;
	}

	public T TryGet<T>()
	{
		return resolver.TryGet<T>();
	}

}

Silverlight Page Transition Complete Event

Sun, 07 Oct 2012 01:00:00 GMT

While creating a mixed silverlight and xna WP7 application I started to add in page transitions with the silverlight toolkit. For the silverlight part of the application they work perfectly however when transitioning to the game screen the animation was not present.

This is because in OnNavigatedTo i'm obtaining the graphics device which overrides the rendering and this occurs before the animations starts.

At this point you have two choices, implements a transition like effect manually in xna or delay obtaining the graphics device until the animation completes and display some content to transition with.

I'm going to give a sample of how to do the latter.

public GamePage()
{
    tansitionEnd = new RoutedEventHandler(GamePage_EndTransition);
}

protected override void OnNavigatedTo(NavigationEventArgs e)
{
    TransitionService.GetNavigationInTransition(this).EndTransition += tansitionEnd;
    base.OnNavigatedTo(e);
}

protected void GamePage_EndTransition(object sender, RoutedEventArgs e)
{
    TransitionService.GetNavigationInTransition(this).EndTransition -= tansitionEnd;
}

The reason I picked the latter is because I will be using silverlight to render some parts of the UI. It's also important to note that as soon as you share the graphics device page level transitions will not render as xna takes over. However as stated above it's still possible to create a transition manually inside xna.

SharePoint WebConfig Modifications

Wed, 25 Jul 2012 01:00:00 GMT

Example for adding or removing a webconfig modification in SharePoint 2010.

private void UpdateWebConfig(string name, string path, string owner, SPWebConfigModification.SPWebConfigModificationType type, string value, SPWebApplication app, bool removeModification)
{

	if (removeModification) {
		var removals = app.WebConfigModifications.Where(item => item.Name == name).ToList();

		foreach (var item in removals) {
			app.WebConfigModifications.Remove(item);
		}

	} else {

		SPWebConfigModification modification = new SPWebConfigModification();
		modification.Name = name;
		modification.Path = path;
		modification.Owner = owner;
		modification.Sequence = 0;
		modification.Type = type;
		modification.Value = value;

		app.WebConfigModifications.Add(modification);
	}

	app.Update();
	app.Farm.Services.GetValue<SPWebService>().ApplyWebConfigModifications();
}

Usage:

UpdateWebConfig(
	"add[@name='handlerName']",
	"configuration/system.webServer/handlers",
	"FeatureName",
	SPWebConfigModification.SPWebConfigModificationType.EnsureChildNode,
	"<add name='handlerName' verb='*' path='service.axd' type='Namespace, Assembily, Version=1.0.0.0, Culture=neutral, PublicKeyToken=0x0xx00000x00xx0' />",
	webApplication,
	false	
);

Inheriting SharePoint masterpages with Powershell

Wed, 06 Jun 2012 01:00:00 GMT

If you want your subsites to inherit their masterpages from their parents the first thing you need to know is the difference between normal sites and publishing sites: Only publishing sites actually support inheriting masterpages. If you're site is not a publishing site you have to iterate over all the Webs and set the property manually.

If you have a publishing site your in luck SharePoint makes it easy for you but it's not completely obvious what to do. My first attempt at setting these properties to Inherit was by setting the site property. And while this does make the UI say the right thing, it doesn't actually change the property CustomMasterUrl in any way and you'll just get the old masterpage.

$web.AllProperties["__InheritsCustomMasterUrl"] = "True" // This is wrong

What you have to do is update the CustomMasterUrl as well, but then you're getting into unsupported functionality and it's also unnecessary work. Conveniently SharePoint already provides you with a PublishingWeb class that supports inheritance for these properties.

Here is an example:

$isPubWeb = [Microsoft.SharePoint.Publishing.PublishingWeb]::IsPublishingWeb($web)
if ($isPubWeb) {
	$pubWeb = [Microsoft.SharePoint.Publishing.PublishingWeb]::GetPublishingWeb($web)
}

if ($xmlNode.CustomMaster -ne $null) {
	if ($xmlNode.CustomMaster -eq "Inherit") {
		if ($isPubWeb) {
			# https://msdn.microsoft.com/en-us/library/ms562472.aspx
			$pubWeb.CustomMasterUrl.SetInherit($true, $false)
		} else {
			Write-Host -Fore Red "You cannot inherit without a publishing web!"
		}
	} else {
		$url = $($site.ServerRelativeUrl + "/_catalogs/masterpage/" + $xmlNode.CustomMaster)
		if ($isPubWeb) {
			$pubWeb.CustomMasterUrl.SetValue($url, $false)
		} else {
			$web.CustomMasterUrl = $url
		}
	}
}

If you are going to do something, do it right

Sat, 26 May 2012 01:00:00 GMT

I love this quote and I think it really fits in with what I'm doing with this blog, and in my everyday life. Taking pride in your work is the only way to ensure that it's as good as it can be. Every day I like to feel that things are better than when I started. It's oftern the little things that count the most and here I have the opportunity to explain why.

Physics for games

Tue, 22 Nov 2011 02:00:00 GMT

I'm revisiting a lot of the physics and I predict trigonometry that was once a daily part of my life before graduating university.

I'm surprised by how much I'd forgotten and writing it down always helps you remember it. Especially when it's on a blog and you can search for it later ;).

The system I'll be focusing on is Rigid Body Dynamics so lets start with the basics.

Any input into the system should be in the form of an impulse, which is basically a force against a point on a body. If all the impulses act against the center off mass we can deal with it just like a simple force.

The formula for a force:

force = mass x acceleration

Which can be rewritten as:

acceleration = force / mass

Which is great because our input is a force and mass can be whatever fits for our simple system which gives us acceleration.

The next thing we need is the velocity formula:

velocity = acceleration x Δtime

Now for those of you that don't know Δ or in in English Delta means change or in this case change in time.

You should keep track of your objects velocity and add to it whenever the body experiences acceleration.

So now we have velocity and here comes position which we also keep track off and add to.

Position formula:

position = velocity x Δtime

Now we can do all the way from the initial push to there the body ends up after a known change in time. The same also applies to angular forces and motion but that's something I'll get to later. To be honest I'm still trying to wrap my head around inertia tensors which you need to

Helpful links Dynamical simulation

WP7 Mango Unit Testing Framework "Cannot find a Resource with the Name/Key typeNameConverter"

Tue, 18 Oct 2011 01:00:00 GMT

When trying to setup unit testing I was slightly dissapoited you couldn't test a silverlight library using a normal test project. However you can use the testing framework by using the Silverlight 3 libraries here.

However you have to make sure you get the right version. I didn't and instead I found a link to an older tutorial and got the nice error Cannot find a Resource with the Name/Key typeNameConverter and the only real match for the error had a broken link. The solution was simple though, find the source and update to the Mango version.

C# and casting

Tue, 27 Sep 2011 01:00:00 GMT

I see alot of people using the as keywork where they really just want to do a cast. Most of the time a cast will be what you want because it's faster and will fail early when somthing goes wrong.

When should you cast? You should cast when you already know what type the object is. A cast also will **not ** fail your value is null.

var value = (Apple)appleDictionary[key];

When should you use as? When you don't know the type of object.

bool makeA = true;
BaseClass baseClass = null;
baseClass = makeA ? new SuperClassA() : new SuperClassB();

SuperClassB val = baseClass as SuperClassB;
if (val != null) { //...

You will almost always be compairing the value to null after using as.

It's important to know as does not replace cast in any way, always pick the one you need for your situation.

IIS Background Thread Abort Exception

Sat, 27 Aug 2011 01:00:00 GMT

One of my applications creates a background thread on startup which executes a series of tasks at a regular interval. However I get a ThreadAbortException after my application pool is recycled.

The problem is simply IIS is aborting my thread and my application is not handling it well. I found the solution in this Stack Overflow question. You have to tell your thread to stop via the Application_End method in Global.asax.cs which gets called when the application is recycled.

C# and AOP (Aspect-oriented programming)

Fri, 26 Aug 2011 01:00:00 GMT

I just had a very interesting experience with AOP in C#. I have a function with a return type List which is being intercepted and that's all well and good. However the interceptor function is a validator style function and can prevent the real function by being called and returning the boolean false.

So the code looks a little bit like this:

List<Update> updates = Manager.ValidateAndCreate();

// protected void Save(List<Update> updates) { ....
Save(updates);

The Method Interceptor looks like the following

public class ExceptionAdvice : AopAlliance.Intercept.IMethodInterceptor {

    public object Invoke(AopAlliance.Intercept.IMethodInvocation invocation) {

        if (isValid(invocation)) {
            return invocation.Proceed();
        } else {
            return false;
        }
    }

    private bool isValid( ...
 }

Now after validation fails the value of updates is actually a boolean not a List, I thought there would be some kind of runtime error here but there was not, so:

updates.GetType().Name == "Boolean"

But:

updates is bool == false

So save will still accept its mutated list of updates and will complain later on when you try to use it.

So how is this possible in a type safe language like C#? btw it's spring-aop.

Link to the StackOverflow question

Object-Relational Mapper Exceptions

Fri, 19 Aug 2011 01:00:00 GMT

When working with Object-Relational Mappers like nhibernate and Entity Framework 4 it can seem like a good idea to setup a Session or Context at the start of a request and finalise it at the end of a request.

This is exactly what I was doing until I figured out how wrong it was. I'll be using Entity framework for my examples so I'm not repeating myself. So the problem is basically that a Context is not your database access object but a Unit of Work that will be committed to the database. This is an extremely important point and this is because of exception handling.

Now for a nice example, say you want to save a record to the users table but the username has a uniqueness constraint. If we're using one context per request and try to add a new user with an unavailable username we get a nice UpdateException. Now once this happens you can't use the context anymore. As a work around you can test for error states in a transaction before they happen, but that's allot of work and doesn't sound too nice.

The solution therefore is to perform actions as a Unit of Work. That way if one doesn't work out you can handle it properly and recover.

Take a look at this article to point you in the right direction Using Repository and Unit of Work patterns with Entity Framework 4.0.

Sharepoint People Picker Displays Selection as Html

Tue, 02 Aug 2011 01:00:00 GMT

When adding a people picker to a custom Html page I encountered an interesting error. When submitting the page the content would return hidden html from the control along with the message “You are only allowed to enter one item”.

The html looks like this.

< span id=’span xxxxx’ tabindex=’-1’ contentEditable=’false’ class=’ms-entity-resolved’ title=’xxxxx’ />

The issue seems to be I’m using IE 9 Standards mode, as SharePoint usualy runs in quirks mode.

Sharepoint Permissions

Tue, 05 Jul 2011 01:00:00 GMT

The SharePoint permission model uses user impersonation, where a typical asp.net application will run under the permissions of the application pool SharePoint runs under the users credentials.

The scenario we’ll be looking at is updating a sharepoint list programmatically.

SharePoint has four default user groups, Read, Contributor, Designer and Full Control. Users would usually be in any of these groups or anything in between. This means that unless the users are also given explicit access to the list directly or via a group they may not have access to update a list even if allow unsafe updates is on.

To resolve this issue there are two solutions.

*The first is no revert back to the app pool, using this method means you have to identify the current user manually and by default all updates will appear as created or modified by the system account. *The second option is to assign users access to the list explicitly or create a group with access and add the group to the users (either automatically or manually via SharePoint).

In my case I needed a solution where users could not access the list normally but using my custom solution allowed them to make updates. Therefore option one was the way to go for me.

Audit Data of Sharepoint List Items

Fri, 01 Jul 2011 01:00:00 GMT

Here is how to read list item data in sharepoint 2007.

var listItem = getMyItem();

string creator = listItem["Created By"];
DateTime created = listItem["Created"];

AppHarbor - First Impressions

Tue, 28 Jun 2011 01:00:00 GMT

App Harbor is a great new cloud hosting service, which I’ve been trying out over the last few weeks. It deploys straight from source code using msbuild to transform your webconfigs, run your tests and deploy to the servers.

However there’s a few things that I didn’t realise untill I’d started using it for a while. Firstly GIT support on windows if not the greatest and while it has been working fine Mercurial has been getting some popularity as an alternative and appharbor even has some support for working with it with bitbucket.

Secondly its cloud based and when you try to get a CMS running you quickly realise they don’t work too well across multiple servers. As you need a central storage location for uploaded files, and CMS managed files. You can get it working and it’s something App Harbor is also looking into but it’s no simple deploy at the moment.

About Daniel Little

Fri, 23 Apr 2010 00:00:00 GMT

I'm a Software Engineer based in Brisbane Australia, with a focus on developing web based systems. I enjoy functional programming, domain driven design, distributed systems and event sourcing. But most of all I enjoy solving problems. As a developer I have the ability to create something from nothing. To make peoples jobs and lives, better and more enjoyable. It's an amazing feeling to make a real impact to companies and people everyday.

In software development, you never get far alone. The best software is backed by amazing teams built on trust, empathy and humility. These teams know that working software is the best measure of progress; you must deliver working software frequently. They know technical excellence always aligns with the business objectives; you can't go faster by cutting corners. And they know communication is the most crucial element in software development.

I value good work culture because people and technology are intrinsically intertwined. I know having context is key for building great software and I try to always optimise for people. I believe technical excellence always aligns with the business objectives and I value and strive to be more authentic to share and discover real stories.

You can find me on twitter at https://twitter.com/daniellittledev

And at other places on the web such as: