Artfully Clean

Imagine buying a house and then never doing the laundry, vacuuming, taking out the trash, or washing the dishes. You just let stuff pile up until you run out of space and then you throw your up hands and go “oh, it must be time for a new house.” Sounds ludicrous eh?

In a highly competitive development environment though you build tables and files and directory folders, develop software, test a bit, and away you go. You patch up the bugs, optimize the queries, add a handful of indices, and you’re flying.

After a couple years though people are wondering why things take so long. Unsurprisingly the typical reason is a failure to archive and then remove the old data. Nobody is taking out the trash! Data retention and archival policies never get baked into the original specifications as the sponsors seldom see any immediate competitive value from them. Eventually however a clean system becomes less of a luxury and more a necessity. Folks start to notice the pile of dirty dishes.

The effort required to retrofit for cleanliness is about the same as the effort for planning for this up front. Strictly operationally however it is better to delay this expenditure until a system becomes a mess as you will then have a better understanding of the corners with the most fuzzballs.