Our Kubernetes Workshop. Or Why did we start using Kubernetes in Production?

It all started with one goal: We needed a platform to ship our new battalion of microservices.

We wanted a place where our applications could benefit from the latest cloud practices in the microservice ecosystem.

Guestline is a Cloud-based company. We have been in this journey for a while, and as many in the industry we started by lifting and shifting our main applications to VMs, after that we saw ourselves slicing the monolith either with MicroFrontEnds or MicroServices, and soon we saw that we could benefit from more innovative technologies.

We did a few POCs on Kubernetes and these are the main reasons why we chose it:

On top of that, we chose AKS over running our own Kubernetes cluster because we wanted a managed system that could help us testing the waters. We don’t have — yet — the people to run our own Kubernetes cluster, so we took a leap of faith as soon as it went GA. Our very last reasons:

4. Being Cost effective. We didn’t want to run all the maintenance and the hassle to keep everything running smoothly. With AKS many of those things such as the master nodes come for free.

5. We needed to integrate with a private Virtual Network with an existing Express Route. So AKS helps you do that if you choose to use Advance Networking.

To start off on the right foot we needed a culture shift, so we rebranded a central team dedicated to many things from provision infrastructure, to maintain SQL servers or the legacy applications, manage secrets, create pipelines…to solely being responsible to provide a platform where teams could ship our new set of applications.

As you can imagine, a culture shift doesn’t happen overnight, so it was challenging to balance your legacy while you try to build your new shiny flagship.

The spirit was to create a team with the whole idea of making themselves redundant from their current job by automating every task that they will face along the way. So soon they could spread themselves to other teams to help them embrace the learning curve. At the end of the day, everything is about reducing your lead time to production. Everything is about bringing maturity and speed.

We are heavily invested in AzureDevOps (the old VSTS Online), so we took some time automating Task, Variable Groups, and Service Connections. We created a pipeline to provision AKS, and there was plenty of research around deployments and security. At the end of the day:

While provisioning the clusters we encountered many issues that we felt defeated so many times, but every small win counted towards the goal we set ourselves: To be able to run a workshop by December.

We mapped how many things were definitely mandatory, and how many things we could live without in our first release so we could get back to them to improve later.

We started off with a theoretical talk about why Kubernetes and which are the benefits we can see there.

We built a simple hello world application that teams will need to ship to our Continuous Integration environment. We provided documentation on how to troubleshoot most common scenarios when deploying, either using AzureDevops or running some queries in OMS.

Our default pipeline looked like that:

And when the day of the workshop finally arrived, we faced two main issues:

Consequently, the outcome was that not many were able to ship a simple hello world to CI. And if that is proven to be so complicated, how can we face real production workloads? How can we ship business critical applications?

So we ran a retrospective.

Yaml is about experience, it’s true, but there are many things we can do out there to help teams help themselves. So we built our own helm template and started to teach teams how to use it.

We discarded the idea of one tiller per namespace. We still want to use Hashicorp Vault to manage certificates, but isolating tiller per namespaces was a bit too much. From here, we are evaluating how to be secure without being a bottleneck, while still providing easy ways of deploying.

How one can achieve balance among all those things?

Those are indeed the challenges that our team will need to face in the following quarters.

We managed to ship three applications to Production following the new template, and the teams feel happier using that.

Our next challenges are: Improve monitoring and alerting (we are already using WeaveScope, and we love it), spread the knowledge about this new paradigm so teams start shipping containers to K8S, continue improving security and start working toward chaos monkey scenarios. Quite a lot to keep us busy for a while!

Keep tuned if you would like to know more about our journey!

Related posts:

Nowadays everyone is excited about doing projects using machine learning or deep learning. Through this blog, I will give you a chance to be ‘ Picasso ‘ of deep learning as we are going to explore…

Technology is in a state of continuous flux. While some businesses are adapting fast enough to brace this rapid evolution of technology, most businesses are reactive and unable to leverage…

The central flaw in the bank system is one of the basic activities carried out by banks, namely “borrow short and lend long” (BSLL), which has long been recognised as being risky. The alleged benefit…