Micro-service architecture and the human body

Let’s assume I am being chased by a lion. If I he scratches my leg, will that result in a sudden death, i.e. a collapse of my whole body?

No it does not. It might hurt big-time, and I might need to get some help on the long term, but the first few minutes, seconds or maybe even hours, my brain will probably ignore the signals sent by the nerve receptors, and I should probably try to reduce the amount of blood loss I have.

My adrenaline levels will rise, I will ignore the pain signals, and I might be able to run faster and longer then I ever imagined.

The information highway

Think about the way our body communicates with our brain: the brain has two main paths for communicating with the body: the nerves and the blood vessels. They both have a distinct function:

• Nerves control muscles and report back status using nerve receptors.
• Blood vessels report back information to your brain using hormones generated by your organs; for example the leptin hormone reports satiety to the brain. They are also used to transport essential nutrients to your cells.

This is a gross simplification, but is in essence how your body works.

Some remarks

• The brain responds to both hormones and nerve receptors with a distinction; nerves are about state of an organ or muscle(i.e. lift that arm), while hormones are more about meta-information: how can I survive/multiply/…
• An organ might fail, but that does not imply that you die. The body can adapt itself in numerous ways by regulating hormones first, and finding new ways after that.
• The brain is a mystery; we know some things about it, but we don’t know everything. If we would have unlimited bodies, computing power and engineering skills, we could probably attach all nerves and hormone detectors to a machine, put them all in a room, and use survival of the fittest to figure out the best path to enlightenment. The problem is in defining what “survival” actually means, i.e. live the longest in the room, grow the oldest, have the fullest life, …
• Some organs are essential (f.e. your heart).
• The body is adaptive, f.e. nerve receptors in the brain have been retrained to allow blind people to “see” (= a perception of awareness).

This brings us to the MSA

Does your brain know how your nerves and blood vessels are organised? I don’t think so; what matters to the brain is cause and consequence. MSA’s work just like that:

• Your nerves report the well-being of the system: do all my organs function, is one missing, it’s the state of your system:
  • does my service still run
  • when was the last time it emitted an event (i.e. is it actually used).
  • how much events were emitted in a time-frame
  • does the system reply to a heart-beat.
• While your nerves are important, the value is in the well-being (i.e. the hormones flowing through your blood stream)
  • do we have visitors on my web site
  • are they ordering stuff
  • do we get payments
  • but also very short-term other goals
    • do people buy our new product X
    • does offering the 2nd item for free help

The gist of it

Here’s the difference between SOA and MSA: as long as the “hormones”(=used to generate business metrics) do not signal a problem, you don’t care about the organs (i.e. individual services), f.e. if you don’t know what a service is doing, just shut it down. If the business metrics don’t show it, it was useless. If people start making a fuss about it, add the proper business metric, and reboot the service.

Have an extra feature or a customer-specific thing? Just let both services run, and let the one processing those responses decide which answer to take.

Does a service fail? As long as your business metrics don’t show it, you don’t have a problem… In fact, because your business rules might change every once in a while, services will get redundant.

Embrace failure and shut-down services at random during office hours. If your business metrics show failure, make sure that there are always at least 2 of them running, so the next time one of them fails, you will no longer have that issue… Or just write another service - after all, it’s just a couple of 100 lines - that does what it is supposed to do, and maybe even way better then before.

If a system fails, you go back to the nerves to check what might be the cause, or just decide to fix it by launching another service. Experience will tell you.

Where all of this is coming from

As Nassim Taleb explains in Antifragile: Things That Gain from Disorder , the risk of a lot of small things all failing simultaneously is way smaller then the risk of one big thing failing at once. To get good at handling failure, it is better to fail all the time. This will allow you to have confidence in the system as a whole, while individual components might fail.

The essence of MSA is monitoring the “what”, while EDA and SOA typically monitor the “how”. You don’t care about how the system works, just that it works… If it fails, you fix it by adding another service or adjusting the existing service. By making failure a non-event, you become essentially resistant to it.

High Availability for Amazon VPC NAT Instances: An Example

nano /etc/haproxy/haproxy.cfg

frontend  TEST-http-in
    bind *:80
    acl root_url url /
    redirect code 301 location https://correct.domain.com/context/ drop-query append-slash if root_url

frontend https-c-in
   bind *:443
   mode tcp
   default_backend app_servers
   acl root_url url /
   redirect code 301 location https://correct.domain.com/context/ drop-query append-slash if root_url


backend app_servers
   balance source
   mode tcp
   option ssl-hello-chk
   server  app 10.10.10.10:8443 check inter 2000 rise 2 fall 5

nano /etc/rc.conf

# rc.d(8) packages scripts
# started in the specified order and stopped in reverse order
pkg_scripts="haproxy"

Watch this animation and learn how the Nutanix Virtual Computing Platform Works.

The Nutanix Solution