Don't Improve Your Shit

You have a huge hack in your system, something that blushes the cheeks out of the most senior engineers and is an example to be avoided by the more junior ones. A kind soul proposes: how about we improve it a little bit? It’s a no-brainer, right? With a small effort the hack will be almost okay, rather than the hot steaming pile of shit it’s now. Right? WRONG! You should not improve your shit. 

Perceived Shitness is Potential Energy for Change

We all want great systems, and the most successful products and companies are full of them, but how do they come about? I argue that those companies utilize a scarce resource well: perceived shitness.

Engineers with more experience and taste have a honed sense of smell to the shitness of systems, and will naturally build judgements of which ones are more shitty than others. This perceived shitness is what motivates them to propose improvements. Thus, greatness can only be achieved by channeling this perception into building great systems.

On the other hand, when you ameliorate a huge shit, you spend a precious amount of perceived shitness and in exchange you get a mediocre system. That’s what ordinarily competent people do, but you can do better. You can be excellent.

Suppose for instance that your backend sometimes crashes in production. It has no monitoring, so customers often complain something is failing.

An eager competent engineer might suggest integrating error monitoring, like Sentry, so that when there’s an exception the team will get a ping. It’s a quick and simple project, with good cost benefit. The problem is, once the pain of constant customer complaints is alleviated, there will be no political will to add proper monitoring. All the perceived shitness will have been spent. With more complete monitoring you could measure failure rate or uptime, but alas no - you will get shitty spammy alerts from Sentry instead.

Let the Shit Simmer

What should you do instead? Endure the shit, let its foul smell intoxicate the nostrils of everyone in the company. Not only you must think it’s shit, but it should become consensus. When that happens, and it’s a subtle art to not let the shit out too long, the potential energy will be built up enough that the ideal solution becomes viable.

Know Your Shit

But what’s the ideal? It’s just the maximally non-shitty version of something.

To find it you should hone your sense of smell by always considering: how could this system be the least shitty? In other words, how could it be ideal?

The ideal should be recurrent topic of conversation in your team. Mind you, you should not propose concrete plans for now, you should just talk about what could be. The ideal design does not sprout into being fully formed. It’s borne our of the struggle - the constant debate of competent knowledgeable people. This debate is also vital for the ideal to be a foregone conclusion once it’s proposed. “Of course we should do that! how could it be any different?”

Own the shit

You’d be surprised at how much of one’s sense of perceived shitness is just “it’s not mine”. You must actively work against this bias. You should be one of the people most knowledgeable about a system before you highlight its shitness. You should become an actual owner, so that people understand what you say as “my system has such and such problems”, never as “your system does”. Without that, diagnosing shitness devolves into zero-sum turf-war, and that helps no one.

Say No to Shit Improvers

Other people will also perceive what is shit, but since they have not read this essay, they will try to improve it. They are doing harm, and they must be stopped. Good intentions be dammed, what matters is impact, and impact of their shit improving ways is mediocre systems.

You have to show them how perceived shitness of the current system must be allowed to fester. They should agree with you on what the ideal is, and that improving the shit actively prevents it.

In other words, you should convert them from shit improver to a fellow shit killer.

Ideally that should be done by reasoned argument (sending them this essay might help), but, if need be, pull rank. When my boss’s boss first stopped my shit improving by blocking my proposal I did not immediately agree, but it became the seed of my long-held no-shit-improving philosophy. This essay only exists because he ordered me to not improve the shit.

Fix the Shit

Now it’s the time to cash out, to trade all that built-up perceived shitness for the opportunity to build the right solution. Take care to not go overboard though. The companion of knowing the ideal is less is more. You should solve few problems, choose them wisely, and solve them well.

The normal evolution of all systems is towards shitness (a specific instance of the more general principle that the entropy of a system always grows), but your strenuous effort can prevent that. With care and taste you can build something non-shitty, something you can be proud about.

Keep discussing it though. The same way perceived shitness has to be socially discovered, so has excellence. Bring others into the never ending discussion of how to make it not shitty. Not only will there be better ideas but there will be less politics and conflicts. You will be all in it together, just trying to prevent shit.

Now that you are done, congratulations! Nothing beats the feeling of turning a shit system into a great one.

Not Everything is Shit

This essay might be misconstrued as a call to never improve anything, and instead always replace solutions in full. Of course not, that would be foolish. If everything smells like shit to you, maybe it's time to check your sniffer. You are just the flip-side of those who swim in shit and think it’s unicorns and rainbows, and just as harmful.

At normal places only a minority of things are shit, and for your sense of smell to be respected it must accurately identify them. 

Also it’s not usually whole systems that are shit, but aspects of them, and you need to be granular. For example, a system might be great overall, with only its ORM or its monitoring being shit.

By judiciously separating the shit from the gold, and within the gold the remnants of shit, you will become a sensible respected engineer as opposed to a loudmouth.

In Conclusion

Pay attention to the quality of systems you work on. Argue with competent coworkers about their strengths and weaknesses. Hone your sense of smell and your taste. Keenly split ordinary stuff - to be improved - and shit - to be replaced. Use every ounce of the perceived shitness effectively as it’s a scarce resource. Build something you can be proud about, but most important of all: don’t improve your shit.

Acknowledgements:

• A more rudimentary version of the idea in this essay was taught to me by my former skip-level manager and almost homonym Davi Reis. While I developed the idea further the seed of it is his.