Getting started with puppet

I’ve recently started using Puppet.  One of the things I first struggled with was creating a hello world application.

If you look at the getting started documentation you are pointed to a Puppet hello world.  This  was promising but turned out to be more complicated than I had hoped as it was configuring a client and a server.

Puppet Standalone

The installation guide mentioned Puppet Standalone. This sounded like what I wanted but I couldn’t find any other information about installing in this mode.

This isn’t surprising as the documentation is aimed at enterprise use and this wouldn’t be valuable in that context.

Install Standalone puppet

After some trial and error I realised that to install standalone puppet you simply download the puppet agent MSI.

Once installed, we just need to write a script to do something.  The simplest thing I could think of is to create a folder and write a file into the new folder.

Hello World Script

Puppet declares the steps to set up a machine in puppet classes.  Once I found an example the file type on Puppet Cookbooks it was pretty simple to create this script;

class MyFirstScript{

file{'c:\HelloWorld\hellowWorld.txt':       

ensure =>file,        

content => 'Hello world!',       

}

file{"c:\HelloWorld":
ensure =>"directory",

}

}

include MyFirstScript

Save this as MyFirstScript.pp.  pp is the extension used for puppet scripts.

The script creates a file called hellowWorld.txt in the c:\HelloWorld folder and adds some content to the file.

Running the script

Now we’re going to check the script is valid and execute the script.

Open a command window and use the puppet command to check the syntax of the script;

puppet parser validate MyFirstScript.pp

Now to run this locally at the command line.

puppet apply MyfirstScript.pp

You should now be able to  navigate to the file to check it’s been created.

Next steps

This isn’t particularly useful but it shows how to install and run your first script in puppet.

Puppet labs have some great resources to learn about puppet and there are plenty of example modules and cookbooks.

Code tag cloud

I thought it would be a bit of fun to pass my code through a tag cloud creator. I thought it might give some insights into the domain language.

I used Boomerang for this experiment.

I merged the projects class files into one on the command line;


    for /r ".\boomerang.host" %f in (*.cs) do type %f >>merged_code.txt

Then when over to TagCrowd.com and created the image below.

Tag cloud image

I didn’t learn much about my project but it was a bit of fun and maybe I’ll do some renaming the next time I’m in the code.

What’s the problem with TDD?

Have you ever worked on a system where the tests slow you down?

Do the tests break even with the smallest change?

Are the tests brittle?

Is it really hard to write a test? Lots of setup, mocks and expectations?

I’ve worked on these and I’m pretty sure I’m not alone.

So what’s the problem with TDD?

Feedback loops

To understand the problem we have to understand feedback loops.

The use of feedback loops is crucial in engineering; the components in a system provide feedback that reinforces or reduces change.

There are two types of feedback loops; positive and negative.

Negative feedback

Negative feedback loops move systems towards stability.

Negative feedback loops act as a damper to, or prohibits, change.

An example of a negative feedback loop in engineering is the centrifugal governor. This limits the speed of the engine by limiting fuel supply.

Positive feedback

Positive feedback loops moves systems towards instability.

Positive feedback loops reinforce change. A change reinforces itself and creates more of the same change.

An oscillator is an example of a positive feedback loop.

Tests as a feedback loop

In much the same way, unit tests are an example of a negative feedback loop.  When one breaks we stop and fix it. It is enforcing stability.

However, tests are also providing feedback when you are writing them. If it’s hard to write a test, your design is likely sub-optimal for the change you are trying to make.

For example, if there are lots of dependencies to mock and expectations to set up, we may have missed a needed abstraction that combines them.

The longer we put up with tests that are getting harder to write, the more our software degrades.

Designing feedback loops

We have to be careful how we design our feedback loops. If our tests are too fine grained our feedback loop will be too tight, limiting the changes we can make to our system. This will reduce our ability to change our software.

We need the feedback to ensure we haven’t broken anything while at the same time we want to be able to change the implementation details of our software.

Conclusion

Our tests are feedback loops, telling us, not only when changes are wrong, but also when our changes cannot easily be supported in the design.

If we ignore our tests it will become harder to work within our system over time.

We need to be careful how fine grained we make our tests; too fine and we won’t be able to change our software. Not fine enough and we will inadvertently break some behaviour of our software.

Testing emails

We’ve recently added email notifications to our application.  I really wanted to avoid configuring email servers or setting up email accounts to test our software. Luckily there are a number of tools to help with this for development, and testing.

Development

We’re using .Net so we can configure our application to send all SMTP to a file directory. This can be done in the application configuration file with the below snippet;

<system.net>
    <mailSettings>
        <smtp deliveryMethod="SpecifiedPickupDirectory">
            <specifiedPickupDirectory pickupDirectoryLocation="c:\temp\mailbox\"/>
        </smtp>
   </mailSettings>
</system.net>

Our acceptance tests relied on this mechanism to redirect SMTP traffic to the file system. We then had an interface in our automation framework to parse these files so we could easily check the contents.

Component Testing

For manual testing we wanted to avoid having to change the application configuration so looked for a simple way of setting up SMTP.

We decided to use an SMTP fake. There are a few open source project for this.  Two examples are SMTP4Dev and PaperCut.

I use SMTP4Dev simply because I’ve used it before but will try papercut when I get some time.

System Testing

This is the only time we needed to have a real SMTP server configured.  For this we have a Dogfood environment that our operations department takes care of. Taking this approach completely removed the need to become experts in configuring email servers to test sending emails.

Mocking dangers

Mocking dependencies allow us to create tests that are fast by removing our dependencies on external systems.

However, it’s easy to overuse and create brittle tests that become a maintenance burden.

For example, imagine we are creating a To-do list application. We might come up with the below simple design;

todo_design

Given this design we might end up with tests similar to the ones below;

public class TodoSpecs
{
    public class TodoSpecs
    [Test]
    public void Service_invokes_repository()
    {
        var repository = Substitute.For<IRepository>();
        var service = new ToDoService(repository, Substitute.For<IMapper<ToDoTask, TaskResource>>());
        service.GetAll();
        repository.Received(1).GetAll();
     }

     public void Service_should_return_resources()
     {
         var repository = Substitute.For<IRepository>();
         var toDoTask = new ToDoTask() { Note = "new todo task" };
         repository.GetAll().Returns(new[] { toDoTask });
         var mapper = Substitute.For<IMapper<ToDoTask, TaskResource>>();
         mapper.Map(Arg.Any()).Returns(new TaskResource() { Note = "new todo task" });
      
         var service = new ToDoService(repository, mapper);
         var todos = service.GetAll();
      
         <mapper.Received(1).Map(Arg.Is(toDoTask));
         todos.Count().ShouldBe(1);
      }

      public void mapper_should_map_task_to_resource()
      {
          var mapper = new TaskResourceMapper();
          var task = new ToDoTask() { Note = "new todo task" };
          var resource = mapper.Map(task);
          resource.Note.ShouldBe(task.Note);
      }
}

These tests are trying to prove the service gets all tasks from the repository, converts them to a resource and returns them. At first glance this seems fine. The first test mocks the repository dependency and tests we call the getall() method. This is reasonable as the real repository will be going to a database of some kind.  If we didn’t mock this we would have a slow integration test.

Brittle Tests

The problem is in the second test. The first smell that something is wrong it that the test is hard to understand. We have to mock two things – the repository and the mapper.

This means that any change to the mapper requires us to change our tests in two places; the tests that cover the mapper and the tests that create mock mappers.

Another problem is that to prove we return the correct result to the client, we are testing the collaboration between the mapper and the service. If we remove the mapper we have to change all our tests.

Testing from the outside

Now, imagine if we just tested this code from the service and treated the mapper as an internal object. We’d probably have something like this;


[TestFixture]
public class TodoSpecs2
{
    [TestFixture]
    [Test]
    public void Service_invokes_repository()
    {
        var repository = Substitute.For<IRepository<ToDoTask>>();
    
        var service = new ToDoService(repository);
        service.GetAll();
    
        repository.Received(1).GetAll();
    }

    public void Service_should_return_resources()
    {
        var repository = Substitute.For<IRepository<ToDoTask>>();
        var toDoTasks = Builder<ToDoTask>.CreateListOfSize(1).Build();    
        repository.GetAll().Returns(toDoTasks);

        var service = new ToDoService(repository);
        var todos = service.GetAll().ToList();

        todos.Count().ShouldBe(1);
        todos[0].Note.ShouldBe(toDoTasks[0].Note);
    }
}

We have removed the mapper from the services constructor and are not testing the mapper anymore. We are still using the mapper in the service but it is being created in the service.

As a result, the test is much more readable and nothing is lost – the mapper is still covered by this test. However, rather than testing it directly, requiring us to create unneeded dependencies and brittle tests, it is being tested as a side-effect of this test.

That’s completely valid – the mapper is cheap to create and consume and it only collaborates privately with the service. We could refactor the code to remove the mapper and our test would still be be valid.

Conclusion

Overusing mocking creates brittle tests that will inhibit flexibility and decrease productivity. We should only mock objects that are expensive to create or use.

In the example, the mapper was cheap to create and use and did not need to be an external dependency. Treating it as private to the service resulted in more flexible and easier to understand tests.

HttpClient performance

I was able to dramatically improve the performance of HttpClient by removing one line of code. Take the below snippet;

 var httpClientHandler = new HttpClientHandler()
 {
      UseDefaultCredentials = true,
      PreAuthenticate = false,
      ClientCertificateOptions = ClientCertificateOption.Automatic
 };
 var webclient = new HttpClient(httpClientHandler);

The offending line of code is highlighted.

This code averaged 20 seconds per 100 requests. Using the default ClientCertificateOption.Manual this improved to 4 seconds.

I couldn’t understand why this would be, so I cracked open HttpClient with Jetbrains dotpeek to take a closer look.

HttpClient eventually calls it’s base class SendAsync method.  This calls SendAsync on the handler and it is here the importance of this property becomes clear.

The handler calls the private method CreateAndPrepareWebRequest().  This in turn calls the innocent sounding SetDefaultOptions(…) method where the below code lurks;

 X509CertificateCollection clientCertificates = UnsafeNclNativeMethods.NativePKI.FindClientCertificates();
 if (clientCertificates.Count <= 0)
    return;
 webRequest.ClientCertificates = clientCertificates;

It turns out the FindClientCertificates() method enumerates the users certificate store on every request.

I found this amazing and it seems extremely wasteful.

I am still trying to figure out if there is a good reason why the certificate store couldn’t be enumerated once and the certificate list stored for future requests.

But that’s for another day!

Faking third party web services

I’ve worked on a number of projects that relied on third party web services or API’s.  Most of these had a test service but didn’t provide a way to return a predetermined response.

This left two choices when testing our software; call the test service or create a fake service of our own

Third party test services

Usually the purpose of a test service is to ensure that your production systems will work with their production systems.  So it is used for testing the integration of two systems.

This usually means the only difference is the data is not real or is a sub-set of the live data.  Using it still requires calls across service boundaries (HTTP), expensive computation and storage (like databases).

This creates a number of issues when executing an automated test suite for your software;

  • Calls across services boundaries are expensive, resulting in slow tests and slow feedback
  • Tests will be brittle because the data can change, leading to different results over time
  • You may not have access even to the test service in the environment you are running your tests

Faking services

The solution is invariably to create a fake service that can be configured to return a specific response.

I’ve done this a few times and without exception the solution has become a maintenance burden. Some  of the problems I’ve experienced are below;

  • Fake is not re-usable across projects
  • Duplication of fakes across teams
  • Different fakes for different third party services

This leads to a lot of duplication of effort and lost time.

Proxy servers

After realising this could be solved with a proxy server I created Boomerang to make this easier.  This enables developers to set predetermined responses for HTTP requests.

For example, I can specify a json object should be returned from a GET for a specific relative address;

var dictionary = new Dictionary<string, string>() { { "content-type", "application/json" } };
Boomerang.Server(5100).Get("/api/products").Returns(products.SerialiseToJsonString(), 200, dictionary);

This creates a proxy server on port 5100 and specifies the response to a HTTP GET request to the relative uri /api/products should be a list of Products formatted as json.

I created an extension method to help with the json serialisation;

public static string SerialiseToJsonString(this object target)
        {
            string str;

            var dataContractJsonSerializer = new DataContractJsonSerializer(target.GetType());

            using (var mem = new MemoryStream())
            {
                dataContractJsonSerializer.WriteObject(mem, target);
                mem.Flush();
                mem.Position = 0;
                str = Encoding.Default.GetString(mem.ToArray());
                Console.WriteLine(str);
            }

            return str;
        }

Conclusion

GET, POST, PUT and delete are supported and the base address is ignored.  If Boomerang receives a request it hasn’t got a response for it will return an error response.

I hope this makes testing with third party services a bit easier!