Jun 19, 2017

Scaling from localhost app to a SaaS product

Almost every SaaS product was once a little application running on localhost of someone's computer. The journey from here to being a SaaS product that works at scale is unique for every product but also similar in many ways. Having worked on both sides of the spectrum and in the transition here are few things I have learned along the way. These are nowhere close to being comprehensive and omit many aspects of development. I have split these into 3 phases viz. Getting Started, Taking off and On to greatness, as the needs in each phase are different.

Getting Started

You will build a monolith and its ok
The best way to build software is to quickly reach a point where you can iterate. That is why your "hello world" commit is probably the most important. Thereafter, at least for application development, it is best to use the time-tested tools, libraries, and frameworks available to build upon that. What you will end up building is a Monolith and it is ok. Ever successful product starts from there.

Don't optimize till you really need to
Optimization is easy and fun. Knowing what to optimize is very difficult. It requires a deep understanding of the problem and the context which only comes with experience and time. So do it in the best possible way within the time bound and leave it at that. The time will come to do it right and it will be the better right then.

Don't fix a black box that is not broken
We all like to get rid of the technical debt. But this is not the time. Create lots of black boxes and don't worry much about what is inside. The reasoning here is as the product evolves and needs to scale, some of those may be irrelevant and others you may want to do in a different way

Do it manually before you can automate
Doing things that don't scale help you scale it in the right way. Development is expensive in cost but more in time. Before you automate anything, make sure you manually do it so you understand the problem thoroughly and the effort in automating is worth the future time spent doing it manually.

Taking off
Interface the crappy code with the good code
Once you are off to a good start, there comes a time when you need to start looking back and improving things. The first thing is to have two types of code, the good code, and the crappy code. The good code is the abstraction layer upon which the crappy code is written. Quickly create the abstraction layer and improve the good code. Also, shield the crappy code from the good one.

Optimize when it starts to annoy
Some of the shortcuts we took before now need to be rectified. By this time you know what is important and top of the priority list. You also understand the problem much better. So start improving/optimizing those pieces.

Start breaking it one thing at a time
Testing a monolith is no fun. When quality control starts becoming a bit of an issue, start breaking things out. There are few low hanging fruits here.
  • Sort your database table by their size. The top one is a good candidate for separation.
  • Build API documentation. It will force you to write better APIs.
  • Stop serving files directly that a CDN can serve.
  • Look at what function consumes maximum CPU on your server and separate it out
  • Implement queues and process few things asynchronously
Protect your critical path. Make it simpler and dumber
The front facing servers that take the requests should be doing little. Robustness comes with simplification. Simplification comes with a better understanding of the problem. Monitor everything and simplify on every alert.

On to greatness
Move towards micro services
Complexity and reliability don't go well together. You need small simple disparate components to make them reliable. This happens when the pieces are so small, they cannot be smaller (Don't overdo). They work with a small set of data and are interdependent upon other services.

Create focused teams
At this stage of the journey, we realize about the myth of a full stack developer. They do exist but full stack rockstar developers don't and you need rockstar developers for every piece of your tech stack. So start creating focused teams. Let them own their part. The architecture now reflects the organization. Careful here, the reverse becomes true as well over time.

Embrace PaaS and FaaS
I think they are inevitable to scale at this stage of technology. Do away with as many pieces of the infrastructure as you can. IaaS is great, but you still have to manage those instances. With PaaS and FaaS, you can focus on the application and the micro service and not have to deal with operational issues.

Automate infrastructure
Whatever infrastructure remains needs to be in code. Ideally, you should be able to spawn up an entire production instance of your SaaS stack with one command and put it to use for production. With the tools and APIs available today, it is easily doable. Once that has been built up, use it daily. Infrastructure must be managed in the same way as the code is.