{"id":431,"date":"2016-03-17T15:32:46","date_gmt":"2016-03-17T15:32:46","guid":{"rendered":"http:\/\/www.crazygaze.com\/blog\/?p=431"},"modified":"2016-08-24T15:49:04","modified_gmt":"2016-08-24T15:49:04","slug":"how-strands-work-and-why-you-should-use-them","status":"publish","type":"post","link":"https:\/\/www.crazygaze.com\/blog\/2016\/03\/17\/how-strands-work-and-why-you-should-use-them\/","title":{"rendered":"How strands work and why you should use them"},"content":{"rendered":"

If you ever used Boost Asio, certainly you used or at least looked at strands<\/a> .<\/p>\n

The main benefit of using strands is to simplify your code, since handlers that go through a strand don’t need explicit synchronization. A strand guarantees that no two handlers execute concurrently.<\/p>\n

If you use just one IO thread (or in Boost terms, just one thread calling io_service::run<\/em>), then you don’t need synchronization anyway. That is already an implicit strand.
\nBut the moment you want to ramp up and have more IO threads, you need to either deal with explicit synchronization for your handlers, or use strands.<\/p>\n

Explicitly synchronizing your handlers is certainly possible, but you will be unnecessarily introducing complexity to your code which will certainly lead to bugs. One other effect of explicit handler synchronization is that unless you think really hard, you’ll very likely introduce unnecessary blocking.<\/p>\n

Strands work by introducing another layer between your application code and the handler execution. Instead of having the worker threads directly execute your handlers, those handlers are queued in a strand. The strand then has control over when executing the handlers so that all guarantees can be met.<\/p>\n

One way you can think about is like this:<\/p>\n

<\/p>\n

Possible scenario<\/h4>\n

To visually demonstrate what happens with the IO threads and handlers, I’ve used Remotery<\/a> .<\/p>\n

The code used emulates 4 worker threads, and 8 connections. Handlers (aka work items<\/em>) with a random workload of [5ms,15ms]<\/code> for a random connection are placed in the worker queue. In practice, you would not have this threads\/connections ratio or handler workload, but it makes it easier to demonstrate the problem. Also, I’m not using Boost Asio at all. It’s a custom strand implementation to explore the topic.<\/p>\n

So, a view into the worker threads:<\/p>\n

<\/p>\n

Conn N<\/em> are our connection objects (or any object doing work in our worker threads for that matter). Each has a distinct colour. All good on the surface as you can see. Now lets look at what each Conn<\/em> object is actually doing with its time slice.<\/p>\n

<\/p>\n

What is happening is the worker queue and worker threads are oblivious to what its handlers do (as expected), so the worker threads will happily dequeue work as it comes. One thread tries to execute work for a given Conn<\/em> object which is already being used in another worker thread, so it has to block.
\nIn this scenario, ~19%<\/em> of total time is wasted with blocking or other overhead. In other words, only ~81%<\/em> of the worker thread’s time is spent doing actual work:<\/p>\n

<\/p>\n

NOTE: The overhead was measured by subtracting actual work time from the total worker thread’s time. So it accounts for explicit synchronization in the handlers and any work\/synchronization done internally by the worker queue\/threads.<\/p>\n

Lets see how it looks like if we use strands to serialize work for our Conn<\/em> objects:<\/p>\n

<\/p>\n

<\/p>\n

Very little time is wasted with internal work or synchronization.<\/p>\n

Cache locality<\/h4>\n

Another possible<\/strong> small benefit with this scenario is better CPU cache utilization. Worker threads will tend to execute a few handlers for a given Conn<\/em> object before grabbing another Conn<\/em> object.<\/p>\n

Zoomed out, without strands<\/strong><\/p>\n

<\/p>\n

Zoomed out, with strands<\/strong><\/p>\n

<\/p>\n

I suspect in practice you will not end up with handlers biased like that, but since it’s a freebie that required no extra work, it’s welcome.<\/p>\n

Strand implementation<\/h4>\n

As a exercise, I coded my own strand implementation. Probably not production ready, but I’d say it’s good enough to experiment with.<\/p>\n

First, lets think about what a strand should and should not guarantee, and what that means for the implementation:<\/p>\n

    \n
  1. No handlers can execute concurrently.\n