Artful Tool Use

A work acquaintance once posited “if we have the stupid people develop it, then it will be easier to maintain. If the smart people develop it then only the smart people will be able to maintain it.”

One of the classic management dilemmas remains how to select the creative workforce for a long-term project (or in software a neverending project) such that the system will not only stay maintainable but that the developers can accomplish the project in a subtle and efficient manner. It’s rather a deep question.

Your philosophy towards tool use and the Development Lifecycle may precipitate how you answer this question correctly in your surroundings. Imagine a chart with company strategy on one axis and expected system longevity on the other axis. Then we should ideally expect the mix of tool use and the level of employee expertise to look something like this:


Legend: HT = high tool, ME = medium expertise

At first glance this chart makes little sense: the ranges of expertise and tool use jump about and seem to lack any expected sort of gradient. This is attributable to the two dominions for when tools make a good bet: first when getting a system out fast has high strategic value, and secondly when time or money are likely to constrain your resources.

Similarly two situations require a higher level of expertise: when a system has a very specific focused utility and when the company needs the software to survive an expected future path of high business volatility.

Choose your tools and your creative workforce to match your company’s strategic plan. This is a key goal of I.T. strategy alignment and helps to avoid the dreaded swamp monster of impedance mismatch.

Artful Commitment

People arrive at the ocean of project management from various rivulets of experience. Some have navigated programming or quality assurance while others set out with a business management education. Regardless of how you get there, once you have responsibility for overseeing the movement of creative team processes you find yourself in the delicate position of asking people to do things.

To a certain extent folks enjoy being challenged and pushed. They dislike, however, being stressed and overworked. The trick is to find the sweet spot where you commit them to the fair amount of pressure. At the same time folks have their own personal quirks and either tend to over-commit themselves or alternatively play up the amount of time that they are being productive when they are actually socializing instead.

It’s sort of management science: if you don’t assign the work then who is going to do it? Obviously you want to show some efficiency in completing the workload, but you also want to maintain good morale. This engages several competing foci and proves to be a riveting point in project management.

And I mean riveting in the sense of steel beam construction: the rivets only go through the steel in strategic locations, and for good reason. A rivet in the wrong place — too close to other rivets or adjacent to a concentration of shear — will weaken rather than strengthen a structure.

It’s your job when you are managing a project to rivet the team together to make sure that all of the pieces, players, and components are carrying their load under equal stress.

Artistic Teaching

Any company that has been around for more than six or seven years has had to deal with changes in the technology stack. Naturally this has serious implications to the clerical staff: those that actually deal directly with the customers. Of course most successful companies hire employees who are quick to learn anyhow. Yet when you have major system wide changes, it is of tremendous help to have a couple folks on board who are both astute at learning on their own as well as at communicating to and teaching the new technology to others.

It takes a certain type of personality to be comfortable with teaching coworkers: this is not like an elementary school teacher nor even a college professor. To be effective the instructor needs to be comfortable enough in their own learning abilities to stay "ahead of the curve" and hence secure in their position.

They need to possess patience for those who are slower to grasp new concepts, and yet appreciate the effort their fellow employees are making, and have a means to be continually encouraging. Companies that have found the keys to long term survival have also found how to keep their internal "teachers" engaged and on board.

Artful Ownership

In a relationship-driven business world, job security and software maintainability are directly and inversely related. Gothically this happens both through a carrot and stick approach from the developer toward the management team and also from a developer to his own future self.

Promoting ownership stimulates the desire to create maintainable code. Much like the difference between renting an apartment and owning a house, the developer who recognizes that he owns (and is responsible for) the code has a vested interest in being able to go back years from now to understand how it works.

Management teams that swap out developers like machine cogs create a separation between a developer and his code: in essence the developer becomes a "renter" and he will deliberately obfuscate his work. One because he has no incentive to spend the extra effort to make it maintainable, and two because he will strive for job security by obscurity: he punishes the management for his insecurity by complicating their desire for robotic interchangeable developers.

In some cases though software may be a one-shot throwaway: if management expects the march of technology to obsolete a software platform rapidly then it might not particularly care whether it is maintainable. Hence if a developer would rather own than rent, he better darn well be sure that he works on creating a substantial and mission-critical system that his employer will still be relying on, with the same platform, six or seven years from now.

Yet... ownership is a chimera. It is an intentional feeling rather than an actual reality. Legally the developer does not own the fruits of his work. How can management promote the feeling of ownership even though the legal reality is otherwise? If it fails to do so it can never expect developers to create lasting platforms.

An Artful Environs

In an earlier post I discussed the vagaries of herding the cats of developers. Here I want to chat a bit about motivation of creative developers in a more diffuse sense -- about how to create the environment that allows for quality and valuable software to "materialize."

This is nearly a diametrically opposite approach to the conservative management of a formally enforced SDLC. And it only tangentially has to do with software. First, consider in general what motivates an artist: a desire to be creative, a desire for recognition, a desire to be challenged, and a desire to stand out amongst their peers. Maybe they seek glamour, or grunge, or just to make an aesthetic statement. Maybe they want to promote a socially redeeming view.

Considering the amount of demanding self-discipline, focus, sensitivity, and work that this requires, how do you create a comfortable supportive space in a world of abrupt changes, insecurities, hidden stakeholders, control freaks, and conflicting priorities?

Mostly it helps to be honest and open in your approach. You don't have to necessarily build a consensus, but it does help to lay all of your cards on the table: acknowledge both the staff needs and the strategic challenges as well as how you plan to enable the former through the onslaught of the latter.

Yeah that by itself seems like a considerable undertaking, but the mere process of discussing it with your staff and your superiors will get you on the same page. At least it will settle the expectations toward how much of the developers art you can bear under the freedom of the muses versus how much of their work must yield to the dogma of the Methodology.

Artistic Reality

Sometimes when you are dealing with the high politics of a system that might have major impact on an industry, you may run into a wall of confusing interests. How did the status quo come about, what are all the vested interests, if you change a practice what else might be affected? I find that when I have allowed my hips to be buried in the muck, first I doodle for about fifteen minutes, and then I set myself down and draw a Reality diagram. This is a concise way to describe, on a single sheet of paper, all the psychics impinging on a project. It is comprised of five quick sketches, each a slightly different flavor, describing various elements and relationships surrounding the projects "reality."


The first diagram of Physical space, shows a hypothetical activity that encapsulates a typical event ranging across a variety of actors. Man A stabs man B, who seeks help from doctor C. Okay relax it was an accident. B works at factory D.


The second diagram shows Ethical Space, basically who has spiritual "claims" on others for their actions (or omissions).


The third diagram shows Contractual Space: the understandings, paper, and legal relationships between the parties.


This fourth diagram is more of a list, a high level classification of the types of information each party keeps that are relevant to your system.


The last chart shows Financial space: simply how the money flows between the parties. So now you've got a piece of paper in front of you that succinctly summarizes the entire Reality surrounding your project. Does this solve anything? No. Does this tell you what to do next or make your life any easier? Probably not.

Yet I have found that this diagramming is an invaluable tool, because it shows you why things are the way that they are, the nature of their balance, and the interplay of the forces between them. It frees you from the muck because it shows you the difference between the hard and the soft limits. So unstick yourself: get real!

The Art of Survivability

In the rough and tumble world of the development of corporate software most companies establish "checkpoints" to assure that their engineers don't get too far down the river before they realize that they needed a paddle (or an outboard motor). How they implement these inspections (with paperwork or process) varies greatly, but the overarching principles are standard. What a company needs from its software operationally in order to prevent a black-swan failure is:

1.       Scalability – developers need to demonstrate right from the start that each successive implementation will meet the *eventual* maximum usage envisioned (in terms of concurrent users and fully loaded database tables).

2.       Trackability – every release should document what changed or got added -- in a public location -- for everyone to see.

3.       Knowledge base – all modern software is so immensely complicated and rapidly changing that it is impossible to keep the technical documentation up-to-date.  Therefore to support maintainability at least two full-time Permanent (in-house employee) software developers need to know everything about any particular system.

4.       Operationally Repeatable – Production controls must exist to allow for the restoration and re-running of a system from any arbitrary point.

5.       Archivals and Deletions – You need utilities to remove old data to prevent the gradual quicksand degradation of performance that will  suddenly cause a cascade of timeouts.

6.       Implementation Rollback – anytime a major software change gets implemented you need a way to roll it back.

Those are the biggies in general philosophical terms. Yeah this list probably doesn’t help much down in the weeds for what documents or gateways to put in place, but it’s a good start for group discussions to get there.

Artistic Simplicity

In an earlier post I pointed to some general principles you should abide by when developing software. Two of these, parsimony and perspicuity, could use a bit of further elaboration. Together they delineate the promotion of a simpler approach to design. Why is simpler better?

As the saying goes, "there's more than one way to skin a cat." Relax it's just a saying, I have nothing against cats. The point however is that given any coding challenge there's usually an algorithmic technical nifty way to get it done, and a more lengthy but more easily understood mechanical way around the problem. The technically elegant way will execute faster and may allow for some future bells and whistles that can be accomplished with just a couple of extensions. The more transparent, simpler to understand, lengthy code will take longer to execute and will require more lines of code when it eventually becomes time for a new feature.

But frankly the longer code is easier for a novice to understand. And in the modern corporate world of high employee turnover, constant technological change, and shifting management fads, the longer code -- because it can be maintained by newcomers -- realistically has a greater probability of staying in use.

Simplicity begets Longevity.

Artistic Specialty

To some extent folks adopt their software development specialization depending upon their own skills and tastes: do they feel more comfortable with a broader and lighter background or do they prefer to be deeper and more focused? Managers may also have a strong influence over how employees develop their skills simply by the tone and expectations that they set.

Obviously software development requires some specialization, since the technological complexity and the rate of change in our profession precludes a sole person from spending the time required to be a technical expert at all the myriad skills that span the available range of tools. If you wish to employ an expert a direct corollary will be that you therefore choose a specialist. If you plant a honeysuckle tree, you are going to get hummingbirds: that's the sort of specialist this tree desires.

Selecting specialists for software development presents some challenging issues. Unlike hiring engineers to design cars -- where once the model rolls off the production line the engineers move along to their next project -- hiring engineers for a software project is likely hiring them for the entire life of the system.

In this instance the problem that a manager faces is that quality, job security, and concurrency are in opposition. Asking your staff to specialize may give them greater job security and improve the quality of your operations, but they will lose their awareness of how to incorporate and apply new tools and hence lose their personal marketability.

If you hire a specialist to do the job then the result is going to be somewhat unmaintainable by anyone but that specialist. If you hire a generalist it will be far easier to maintain the resultant system, although the final product may well appear less slick and polished.

Ultimately then who you hire depends upon the "positioning" your company is trying to achieve. Are they looking for a short-lasting expensive best of breed solution? Or something long term, stable, and adequate? What sort of tree exactly are you planting?

Artful Choice

While sipping my morning coffee in a Starbucks I overheard a building contractor say, “Good fast or cheap… choose any two.” I recognize this aphorism as a common quandary that binds all designers together whether we work on architecting buildings, developing software, or drawing illustrations.

So how do you approach an intelligent method for making that choice? It seems to be a decision that definitely happens early in the software development cycle, usually during the initial architectural and interface planning.

In a earlier post I discussed the other half of the meta-surrounds of a project; here we look at how to go about approaching choices that affects the first half. I will begin with quite a concrete explanation — a description that makes the process sound precise and analytical. Implementing it is more an art than a craft, and hence it is easier to begin by explaining its craft-like aspects.

Imagine if you will a triangle with each corner labeled: one is quality, one is speed, and one is thriftiness. Your project will end up as a dot somewhere within this triangle, but for now you just want to be able to draw a circle identifying the target area for where you wish to land.

To draw this circle however you need a side step to another chart: the one that maps your company’s strategic plan based upon its competitive position and the growth stage of its industry. The proximity to the thrifty vertex depends on whether your company is “harvesting” and winding down or if it is growing explosively in a new industry.

The proximity to the quality vertex depends upon how you are positioning yourselves in relation to your competitors, the talent pool that you can draw upon, and if such quality provides a material advantage at this stage of industry development.

Finally the proximity to the speed vertex depends upon the urgency of implementing your strategic plan, typically higher at the industry start and end-points than in the middle.

So now that you’ve got your general target the art kicks in: typically none of this is something that you can explicitly relate to your staff. Instead you guide the developers toward the circle within your triangle by your plans, attitudes, and actions: you try to instill the habits in them that will land the project at its goal. Good, fast, or cheap? Choose the appropriate mix.

Artistic Ownership

The managers at many places I have worked confuse the fruits of our work (lines and lines of programming code) with “intellectual property” and with knowledge. It is a convenient deception, but it ultimately causes more problems than it solves. Let’s take a look at precisely what these concepts are, why code and knowledge get conflated, and the problems that ensue.

This is somewhat akin to confusing traintracks with a railroad. A manager walks in and says “oh, that guy is a direct path to Denver, that guy goes to our hub in Chicago.” But a railroad is a whole lot more than traintracks. It is also the locomotives, semaphore crossings, stations, leased rights of way, you get the idea.

“Lines of code” is a semi-fluid corpus of rules and instructions that implement business logic. These change as customer or management needs allow and they also get modified in reaction to bug fixes and refinements in technology. They are like traintracks. In nearly all cases the software code is the property of the owners of the company. So sure the railroad owns the traintracks. Yet there is so much more.

In the context of software development “Intellectual Property” is actually a narrowly defined term. Occasionally it may be a patent, but for commercially marketed software it is more likely a copyright. Most internal-use software (the bulk of what we create) is Intellectual Property by means of “trade secret” and employment work-for-hire agreements. So these are like the leases held on the rights of way.

The code isn’t an “asset” in the sense that a physical piece of property is an asset; it lacks a particularly liquid “market” and the rules and peculiarities it encodes keep little value when transferred to a non-related owner. Selling your traintracks to Australia doesn’t do them much good.

“Knowledge” in this context encompasses three overlapping concepts. First it is a domain-specific understanding of how things are supposed to work conceptually or legally. Second it is the cultural understanding of who is proficient at maintaining what. And third it is the historical insight into how things evolved to reach their current state. Software developers employ this knowledge (which resides primarily in their heads) to produce lines of code from the influx of rules, regulations, requirements, and bug reports. So the knowledge is the locomotives, semaphore crossings, and stations.

It is convenient for a manager to jump from the work-for-hire agreement to the (mistaken) impression that he “owns” the knowledge in developers’ heads. In this way he conflates the human “resources” through the midpoint of Intellectual Property to the endpoint of an “asset” that he owns and manages. He confuses the traintracks with the railroad. Although convenient, it is wrong in that it loses pieces in the translation, creating dangerous side effects. It is like thinking you can pick up the traintracks, move them to India, and suddenly have the Southern Indian & Pacific railway.

Knowledge isn’t transferable nor is it intellectual property. And if you move people from their areas of expertise the company effectively loses their knowledge. Sure it still owns their code, but after the company saddles somebody new with maintaining the code they have to reacquire the knowledge of how to do so. They have to relearn how to run the railroad.

Artistic Non

The non-functional requirements in software are rather like the things that we shop for when we go buy a car. We rather assume that it comes equipped with wheels, an engine, and brakes… these are functional requirements. Beyond that though we are looking for more: do we like the color, how it feels on the road, its gas mileage, its styling, whether to lease or to buy.

In substance the main non-functional requirements pretty much remain the same between large software projects:

- the software has to last ten years across a changing variety of display devices;
- it has to remain maintainable and enhanceable without it turning into spaghetti code;
- it should fail gently and has to recover fully;
- it needs a consistent interface that is intuitively easy to use.

Non-functional specs deal less with the “what” and more with the “how”. Hence the responsibility for defining their implementation tends to fall rather less upon the business analysts and more upon the IT management team.

- The philosophical battle over the non…

Just as folks can have finely honed sensibilities about their automobiles, a manager can argue the non-functional requirements from quite different perspectives depending on their personal philosophy.

Swaying the choice of styling is how the manager regards acts of creative destruction. If his experience suggests that leading edge regrowth ultimately compensates for the havoc caused by destroying legacy systems or jobs then he may bend toward less maintainability, static display devices, and higher levels of ease of use. He is focused more on the shorter term.

Some managers relish having a variety of platforms and tools yet others prefer a restrictive “vanilla” operation. Those tending toward variety may favor looser interface standards, but they may prefer a better corroborated sense of maintainability. They are focused more on organic growth.

Managers tend to sanction different turnover velocities, both with respect to employees and subsystems. It’s lease or buy in a different context. Those that favor a faster turnover may care less about maintainability but will tend toward common interface components. They are focused on punctuated evolution.

Finally some managers have a high aversion to risk. These will pay much greater attention to failure and recovery modes and request more thorough testing across the ever growing panoply of display devices. They are focused on safety. Generally then the vectors pointing the politics of “non” push along the four completely independent dimensions of creative destruction, acceptance of variety, turnover speed, and risk aversion.

What kind of car does your manager drive?

An Artistic Haircut

In the quest to stay ever current of technology, companies constantly face the challenge of deciding whether to further complicate old, existing systems, or whether to scrap the old technology entirely for something new. On first inspection it is often difficult to appreciate the extent of effort that has been invested in a legacy system. Whereas on the surface the risk here would seem to be a straightforward analysis of uncertainty versus potential reward, the deeper diagnosis actually revolves around customer service issues.

Most likely a high percentage of your existing customers use only a small contained core section of your legacy system. You want to target the 70 percent or so of these customers who you can move to the new system with the least amount of risk. Realistically however you should prepare both the management and you staff that the process will essentially give your revenue a 30% haircut: it is simply unprofitable to convert everyone away from the legacy system, as after the new system is close to code-complete it becomes too costly (primarily for your customer support and technical support staffs) to maintain the other 30% on the legacy.

Artistic Herding

At some point in your career you may be presented with the choice, opportunity or responsibility of joining the technical management team. Most folks who have an ambition toward this (and who are competent people with good technical skills) may find themselves hitting their early management strides by their middle 30s.

The appearance and draw of a management position looks radically different when you're on the climb up however, than when you are actually saddled with the responsibility. Sometimes you hear folks give the old warning "be careful of what you wish for." It's not however, actually as severe a misgiving as all that. It's more like crossing the razor's edge between two distinct and incompatible points of view.

While you're on the upward climb a management position appears as though it could be a good place to share your knowledge and discoveries about how to be a creative professional. Once you're actually there however you often find that your subordinates aren't particularly interested in your opinions, and secondly that you spend most of your time consumed by political positioning, "gaming", and preventing infighting.

What it all boils down to eventually though -- the real guts of the viewpoint canyon -- is a deep social and spiritual divide over the concept and purposes of "work." A simplified way to describe it is like the difference between being a successful feline and being the boarding caretaker for said herd of cats.

From the lone cat's (and lone programmer's) perspective, the objective is to keep your claws (your skills) sharp, to stay light on your feet, to be cautious and curious and aware of the changing environment. You are in a world where you need to hide the mice you discover so that other cats don't steal them. You stay coy and aloof and full of misdirection. Your concept of "work" involves a certain amount of territorialism.

From the perspective of the cat herder you just mostly want to ensure that people aren't spending all of their time picking cat fights. There is some sense of direction and movement you want to achieve, of course, but mostly you want to avoid having the cats bored to tears, sleeping on the job, or scheming for their next kill. You really just want to keep them out of trouble. Your concept of "work" is about the gestalt of your employer.

More about how a manager reconciles these viewpoints in later posts.