After some reflection on my recent series of posts about Paremus ServiceFabric on EC2 I realise that I never provided a high level commentary on what each of the moving parts does, and why they’re important.

• Paremus ServiceFabric – this is a distributed OSGi runtime framework. The point is that you can package an application as a set of OSGi bundles, and deploy them onto the fabric without having to care too much about underlying resources – that’s what the fabric management is there to take care of. What’s especially neat about the Paremus implementation of this is that the fabric management itself runs on the fabric, so it gets to benefit from the inherent scalability and robustness that’s there (and avoids some of the nasty single points of failure that exist in many other architectures).
  • OSGi is a good thing, because it provides a far more dynamic deployment mechanism for applications (making it easier to design for maintenance).
  • ServiceFabric also makes use of Service Component Architecture (SCA), which allows better abstraction of components from underlying implementation details. This allows parts of the overall architecture to be swapped out without having to reach in an change everything. Jean-Jacques Dubray from SAP provides an excellent explanation of how this improves upon older approaches on his blog.
• CohesiveFT Elastic Server on Demand – this is a factory for virtual appliances. I used it to build the Amazon Machine Images (AMIs) that I needed. A bit like OSGi it uses a concept of bundles, and for some of the software that wasn’t already there in the factory (e.g. the Paremus stuff) I had to create my own. Once I had the bundles that I needed I was then able to choose an OS, and build a server to my recipe (aka a ‘bill of materials’). The factory would send me an email once a server was ready (and optionally deploy and start it for me straight away).
• CohesiveFT VPNcubed – this was the overlay network that ensured that I had consistent network services (that supported multicast) covering the private and public pieces of the project. It basically consists of two parts:
  • A manager – which can exist in the private network or the cloud (or both). For simplicity I went with a pre packed AMI hosted on EC2
  • A set of clients. These are basic OpenVPN clients. For my AMIs I used a pre packed bundle. For the machines on my home network I just downloaded the latest version of OpenVPN. The manager provides ‘client packs’ containing certificates and configuration files, which need a little customisation to specify the manager location.
• CohesiveFT ContextCubed – this provides the ability to start and customise a bunch of virtual appliances (AMIs) automatically. With the help of their CTO, Pat Kerpan, I was working with a pre release of this service (hence no link). ContextCubed (which I accidentally called ConfigCubed in my post about it) provides an init.d style mechanism that sits outside of the virtual machine itself. I used it to download and install VPNcubed client packs, start the VPN, stop some services I didn’t want, reconfigure the firewall to allow multicast, and add binding config to the Paremus Atlas service (before starting it up). I could have also used it to create hosts files to work around some of the naming issues I encountered, but I think I’ll wait for Pat to fix things up with DNScubed or whatever he ends up calling it. Hopefully in due course the *cubed services will all find their way onto the same virtual appliance, so there can be a one stop shop for stuff that makes an application work in a hybrid cloud (or whatever suits your taste from private to public).

One thing that would have been fun to try (but that I didn’t attempt) is closing the loop between the PaaS management layer in ServiceFabric, and the IaaS management layer in ContextCubed. This would allow (for instance) extra machines to be deployed dynamically to satisfy peaky workloads (or deal with failure) running on ServiceFabric. I’ll leave that for another day.