An Artful Chunk

A fairly substantial design issue when planning the logical design of a database is how to "chunk" it. I don't mean vertical or horizontal partitions (for performance optimization) nor segmentation for security constraints. Rather, I use the term chunk here to signify how you associate large swaths of data with business processes such that you can restore a subset of the database.

Here's an operational example. You have several customers for whom you retrieve monthly or weekly datasets of new input, at random intervals for each customer. After you ingest the data you run three large processes to end up with a pick list of what to ship to each one. Say the whole process takes a couple weeks of elapsed time.

One day, after the second big processing step, you are informed that one of your customers sent you the data for the wrong store. What do you do now?

Planning for this type of occurrence takes careful attention to detail. Any manual data updates that occur outside of automated production need an audit trail so you may re-apply them after your data is restored. Large aggregation processes need checkpoints or staging tables so you may recalculate after a backout.

Making contingencies for large contiguous corrections, from the start of logical database design, is the artful way to safely chunk your database.

Artistic Teaching

Any company that has been around for more than six or seven years has had to deal with changes in the technology stack. Naturally this has serious implications to the clerical staff: those that actually deal directly with the customers. Of course most successful companies hire employees who are quick to learn anyhow. Yet when you have major system wide changes, it is of tremendous help to have a couple folks on board who are both astute at learning on their own as well as at communicating to and teaching the new technology to others.

It takes a certain type of personality to be comfortable with teaching coworkers: this is not like an elementary school teacher nor even a college professor. To be effective the instructor needs to be comfortable enough in their own learning abilities to stay "ahead of the curve" and hence secure in their position.

They need to possess patience for those who are slower to grasp new concepts, and yet appreciate the effort their fellow employees are making, and have a means to be continually encouraging. Companies that have found the keys to long term survival have also found how to keep their internal "teachers" engaged and on board.

Artful Champions

There's an old saying, "every social war is a battle between the very few on both sides who care and who fire their shots across a crowd of spectators." Analysis is very much like that. You have perhaps two or three managers who have different opinions about what your objectives should be, and a couple large groups of customers who care in the collective sense, but who individually just view your project with the valid skepticism of the small part it plays in their lives.

To really nail down the ethos, and for a large project to succeed, you need to have (or manufacture) a champion for each of the user, customer, and business interest groups. These champions needs to sincerely stand up for the benefits and beliefs of their group, and be gregarious enough to express them.

As you cycle through the balance points you then build up goodwill amongst the champions, and this enables you to negotiate concordance on the more difficult decisions.

This can be an especially hard sell: first, when a champion is less than sincere in standing up for their group. Secondly, when a champion changes employment, through their own choice or otherwise. And thirdly, when you can't get sufficient permanent heavy artillery support from a senior manager to resolve those instances where negotiations fail.

A solid 25% of full-metal Analysis involves the psychic steps to avoid these three pitfalls.