The answer is “Hybrid”

“Hybrid,” is the answer.

I’m talking about “on-prem vs cloud,” the bugaboo trick question that has plagued us for nearly a decade.

What I mean by reframing the question like this is that – absent other details – the location and ownership of servers is much less important than the architecture of the solutions.

So let’s get on with it. This is more than just philosophy. We are engineers – we thrive on specifics:

A colleague of mine likes to say, “ssh is cheating.” What he means is that it is unacceptable, in 2017, to leave your system so incomplete that it requires a human to log in and perform manual configuration in the event of an update or (heavens forfend) a reboot. Ssh as a protocol is fine (most of the tools below run over that protocol), it’s the manual intervention that constitutes cheating.

System configurations should be defined in software. It doesn’t really matter which domain specific language we pick, any of Puppet, Chef, or Ansible will work fine. It is important to pick one and get good at it, rather than trying to maintain a polyglot mashup.

Configuration scripts need to be under version control, just like any other software. Github is the default code repository these days, though there are reasons to go with something slightly more pre-integrated like Bitbucket.

Configuration changes should follow a structured process like Gitflow. It is worth noting that this tool is only helpful if coupled with human processes of communication and trust. Human beings need to check and validate each other’s work, to avoid overwriting or colliding with each other’s changes.

Once a change is checked in and reviewed, continuous integration, test, and deployment is the order of the day. Tools like Jenkins remove all of the manual interventions between approving a change and seeing the code built, tested, and pushed to production on whatever schedule the team has picked. Note that this is not an argument for the wild west. For most teams, most of the time, I’m an advocate of “read only Fridays,” since changes on a Friday frequently lead to long weekends at the keyboard.

All of this is just as true in the on-premises data center as it is in Amazon’s east-coast-2 availability zone. You don’t get to ignore modern systems engineering practices just because finance negotiated a really killer deal with Dell.

So when someone asks me, “on-prem vs cloud,” without further elaboration, I say “hybrid.” It’s an answer that allows me to get on with building robust, scalable, agile systems no matter who happens win out as the infrastructure provider.