Running TurboTax On Kubernetes: Lessons Realized From Our Journey

It began with a problem. At KubeCon 2018, one of many Kubernetes Founding Engineers at Google requested us – with a wink – to let him know after we deliberate emigrate TurboTax to Kubernetes.

At the time, this was a daring problem – the duty appeared nearly insurmountable. Very few organizations had completed greater than tiptoe to run mission-critical purposes on Kubernetes.

Intuit was already working microservices in manufacturing, however as a posh software with a whole lot of microservices serving hundreds of thousands of consumers, TurboTax was a unique matter. Would the know-how be capable to deal with Intuit’s scale and meet all safety and compliance necessities for this international monetary providers platform?

Fast-forward to 2020. In lower than 12 months – from May 2019 by way of April 15, 2020 – we rose to the problem. We now run most of TurboTax’s vital providers on Kubernetes in our manufacturing surroundings, together with core providers such because the Identity platform and Financial Data platform, which additionally serve Mint and QuickBooks prospects. We’ve achieved efficiency at scale to satisfy the calls for of even the busiest tax submitting seasons, proving the viability of Kubernetes for our most mission-critical purposes. Along the best way, our efforts had been acknowledged by the Cloud Native Computing Foundation (CNCF) with its coveted prize for the most effective finish consumer.

The venture has delivered important advantages to Intuit, together with:

• Rapid, iterative product growth and rollout
• Consolidation beneath one platform for all growth groups
• Efficient use of sources at scale for value discount
• Strong ecosystem assist
• Unified distribution mechanism for service artifacts

And we have gained the experience and confidence to maneuver all remaining assist providers from TurboTax to Kubernetes.

Of course, migrating from TurboTax to Kubernetes wasn’t simple. We’ve needed to develop new abilities and overcome some technical obstacles alongside the best way. If you are contemplating transferring to Kubernetes your self, you’ll be able to dive deeper into the technical particulars of the Intuit journey right here:

Top 5 Lessons We Learned During Our Trip

1. Gather necessities prematurely and plan infrastructure accordingly
   The key necessities to determine embrace:

• • The variety of microservices you run on Kubernetes.
  • Compute sources, together with the variety of clusters and Kubernetes nodes you want, the AWS Availability Zones and areas the place they are going to be deployed, and non-Kubernetes configuration prepared for catastrophe restoration.
  • The scale you want to assist, together with most TPS load and variety of concurrent customers.

2. Invest in coaching your technicians to satisfy migration necessities
   There isn’t any shortcut to human experience. We’ve taken the time to conduct a number of weeks of coaching packages for a whole lot of engineers throughout the corporate inside our TurboTax, QuickBooks, and Developer Experience groups. The coaching coated foundational applied sciences equivalent to Docker and Kubernetes, together with superior ideas associated to multi-tenancy, cluster add-ons, load balancers, and auto-scalers. To construct familiarity and proficiency, we have offered our engineers with playground clusters of totally functioning Kubernetes namespaces to tinker with.

3. Keep compute and information separate and create a strong entry path between the 2
   A vital a part of Kubernetes migration is making certain entry to current information. In our case, a lot of our current TurboTax information layer was already accessible by way of APIs (software programming interfaces) by way of NAT Gateway [network address translation gateway in the AWS GovCloud (US) region]. For instances the place information wanted to be accessed straight utilizing AWS managed providers, entry to completely different AWS accounts was arrange between the Kubernetes account and the information providers account. This method, our software groups can proceed to handle entry to their Data Service AWS account and use their favourite instruments to handle the information. This structure additionally permits service groups to retain their non-Kubernetes-based configuration for catastrophe restoration functions.

4. Test for reliability and efficiency at scale
   It goes with out saying that manufacturing Kubernetes should meet manufacturing necessities. For us, that meant performing reliably on the dimensions of peak tax seasons. To see if the infrastructure may maintain up, we ran weekly checks with thrice the anticipated load, in addition to failure checks with visitors spikes, regional outages, and availability zone outages. Of course, we found a number of technical points. You will discover the main points within the second a part of our Medium weblog, however the points and their options gave us a a lot deeper understanding of working Kubernetes on AWS. In the top, we gained confidence that the entire setup would be capable to carry the load by way of tax season.

5. Automate, automate, automate
   The extra you automate, the extra dependable and strong your infrastructure turns into. We’ve automated every part from onboarding new providers on Kubernetes to monitoring and resolving identified points. The governor, lifecycle supervisor, iam supervisor, occasion supervisor, improve supervisor, and lively monitor parts are open supply in Intuit’s Keiko initiatives are a direct results of such automation. And extra automation is on the best way.

What did success appear like?

The infrastructure held up. During tax season, all providers ran easily and met availability, scale, and efficiency ensures. And that is saying rather a lot, given the character of our seasonal visitors and scaling necessities for core providers that assist greater than 80 p.c of the visitors that drives TurboTax.

What’s subsequent for Intuit on our Kubernetes journey

This monumental achievement is a tribute to the exhausting work and dedication of extremely gifted people from throughout the corporate. Throughout the journey, Intuit TurboTax and Developer Platform engineers utilized steady testing, data-driven choices, and focused automation to efficiently handle this daring problem.

And we’re happy with it.

The first technology of Intuit’s Kubernetes platform is constructed on Kubernetes primitives and a customized management aircraft, which has served us properly for the dimensions and efficiency we wished. However, there’s at all times extra to do, particularly for even greater scalability, monitoring, observability and manageability. Going ahead, we are going to proceed emigrate the remaining providers in TurboTax, in addition to different Intuit choices. And we’re constructing the following technology of a supercharged Kubernetes platform utilizing broadly accepted Intuit open supply initiatives equivalent to Keiko, Argo And Admiral.

Visit us at for extra data KubeCon + CloudNativeCon 2020 North America from November 17–20, 2020 at our digital sales space within the Silver D convention room. You may discover Intuit on the convention schedulethe place we take part in 5 session talks on November 18, 19 and 20.

Shrinand Javadekar is a lead engineer for the Intuit Developer Platform group and is liable for constructing a contemporary SaaS platform to develop, deploy and run all apps at Intuit. The open supply venture Keiko was born from this work.