Peter Marklund's Home

How often have you as a software system come across a system that is really well designed and well tested and that isn't also over engineered in some ways?

How often is there a cost/benefit consideration when we introduce a new abstraction or levels of indirection in our software? At the time of writing the abstractions it feels like all benefits and i all makes perfect sense. There is no cost other than the time it takes to write the code and as developers we can write code really fast. New developers who discover the abstractions down the road may have very different feelings about them though.

Let's say you are building a REST API on AWS Lambda with a database like PosgreSQL or MongoDB. How many layers of abstraction do you need for a system like that? Should you abstract away the cloud provider, the database, SQL, and HTTP? Should you build those abstractions regardless of the likelihood of change? Sometimes we seem to be suffering from the "not invented here" syndrome and we seek to shield and cushion our delicate code from the capricious outside world. One maybe overlooked advantage of directly using a standard interface (like HTTP, or SQL, or even AWS Lambda) is that it tends to be well specified, documented, battle tested, and understood. That is typically not the case with custom/proprietary/internal interfaces and abstractions (interfaces that we create to wrap standard ones). Suppose you abstract away your cloud provider or database and that the likelihood of that dependency changing is 1%. You will then be paying the cost of the abstraction in increased complexity and code size in 100% of future scenarios. Only in the 1% scenario will you possibly reap the benefit of the abstraction making the technology change easier. But even in that case you will probably end up needing to do some work to accommodate the change and you haven't necessarily benefited from the investment.

Let's remind ourselves of the "You aren't gonna need it" (YAGNI) and "do the simplest thing that could possibly work" principles. Maybe instead of investing in speculative generalizations up-front we should build the abstractions and adapters once we actually need them? Maybe in the future once we have the concrete needs and use cases we will better know what to build.