Jun 19, 2017

Scaling from localhost app to a SaaS product

Almost every SaaS product was once a little application running on localhost of someone's computer. The journey from here to being a SaaS product that works at scale is unique for every product but also similar in many ways. Having worked on both sides of the spectrum and in the transition here are few things I have learned along the way. These are nowhere close to being comprehensive and omit many aspects of development. I have split these into 3 phases viz. Getting Started, Taking off and On to greatness, as the needs in each phase are different.

Getting Started

You will build a monolith and its ok
The best way to build software is to quickly reach a point where you can iterate. That is why your "hello world" commit is probably the most important. Thereafter, at least for application development, it is best to use the time-tested tools, libraries, and frameworks available to build upon that. What you will end up building is a Monolith and it is ok. Ever successful product starts from there.

Don't optimize till you really need to
Optimization is easy and fun. Knowing what to optimize is very difficult. It requires a deep understanding of the problem and the context which only comes with experience and time. So do it in the best possible way within the time bound and leave it at that. The time will come to do it right and it will be the better right then.

Don't fix a black box that is not broken
We all like to get rid of the technical debt. But this is not the time. Create lots of black boxes and don't worry much about what is inside. The reasoning here is as the product evolves and needs to scale, some of those may be irrelevant and others you may want to do in a different way

Do it manually before you can automate
Doing things that don't scale help you scale it in the right way. Development is expensive in cost but more in time. Before you automate anything, make sure you manually do it so you understand the problem thoroughly and the effort in automating is worth the future time spent doing it manually.

Taking off
Interface the crappy code with the good code
Once you are off to a good start, there comes a time when you need to start looking back and improving things. The first thing is to have two types of code, the good code, and the crappy code. The good code is the abstraction layer upon which the crappy code is written. Quickly create the abstraction layer and improve the good code. Also, shield the crappy code from the good one.

Optimize when it starts to annoy
Some of the shortcuts we took before now need to be rectified. By this time you know what is important and top of the priority list. You also understand the problem much better. So start improving/optimizing those pieces.

Start breaking it one thing at a time
Testing a monolith is no fun. When quality control starts becoming a bit of an issue, start breaking things out. There are few low hanging fruits here.
  • Sort your database table by their size. The top one is a good candidate for separation.
  • Build API documentation. It will force you to write better APIs.
  • Stop serving files directly that a CDN can serve.
  • Look at what function consumes maximum CPU on your server and separate it out
  • Implement queues and process few things asynchronously
Protect your critical path. Make it simpler and dumber
The front facing servers that take the requests should be doing little. Robustness comes with simplification. Simplification comes with a better understanding of the problem. Monitor everything and simplify on every alert.

On to greatness
Move towards micro services
Complexity and reliability don't go well together. You need small simple disparate components to make them reliable. This happens when the pieces are so small, they cannot be smaller (Don't overdo). They work with a small set of data and are interdependent upon other services.

Create focused teams
At this stage of the journey, we realize about the myth of a full stack developer. They do exist but full stack rockstar developers don't and you need rockstar developers for every piece of your tech stack. So start creating focused teams. Let them own their part. The architecture now reflects the organization. Careful here, the reverse becomes true as well over time.

Embrace PaaS and FaaS
I think they are inevitable to scale at this stage of technology. Do away with as many pieces of the infrastructure as you can. IaaS is great, but you still have to manage those instances. With PaaS and FaaS, you can focus on the application and the micro service and not have to deal with operational issues.

Automate infrastructure
Whatever infrastructure remains needs to be in code. Ideally, you should be able to spawn up an entire production instance of your SaaS stack with one command and put it to use for production. With the tools and APIs available today, it is easily doable. Once that has been built up, use it daily. Infrastructure must be managed in the same way as the code is.

Sep 8, 2016

The 10x Programmer

Who is the 10x programmer? How do you know one from the other? How do you hire a 10x programmer? There is this perception I have seen about the 10x programmer who is this insane work machine, who can spit out code faster than most can type. I feel the term is highly misunderstood.

Lot of people think developing software is a linear task. That productivity is directly proportional to number of commits, lines of code and number of hours spent. I even came across a startup that provides tools to measure team productivity and performance using those stats. Those are not fun places to work at for a programmer.

To understand the 10x programmer is to understand what writing software is. Writing software I think is a form of art. If I compared paintings in terms of how many shades of colors were used or how many gallons of paint went into its making, would that be a fair evaluation? Or how about measuring the quality of a book by the number of pages / words written?

A piece of code solves a problem. A good piece of code solves a pattern of problem. An excellent piece of code solves the right pattern of problem. And that is what the 10x programmer does. They solve the right pattern of problem.

To know the right pattern is no trivial task. Not many can see those patterns. For someone who does not understand art, all paintings are more or less the same. Leadership that don't understand this, will not appreciate the 10x jobs hence will eventually loose the programmer or the gains.

To hire the 10x programmer thus is also difficult. One needs to understand not only how they solved certain problems but also the context and the alternatives. The spark of genius does not necessarily light during the hour of the interview, hence it needs to be derived going into the details of the past. It would help to see how they understand a problem v/s solving it. An alternative approach to hiring 10x programmers is to spend time identifying those from existing team. Then nurturing them to grow and solve larger problems.


Sep 14, 2015

How technical debt creeps in

P1: I want to see 1, 2, 4 and 5.
D1: print 1, print 2, print 4, print 5
P2: I want to see 3
D2: print 3
P2: 3 should be after 2 and before 4
D2: output.replace /(2) (4)(.+)3/\1{3}\2\3/
P3: I want to see 6