The Art of the Job

Every couple of years my mom asks "What exactly do you do?"

Exactly? Well you see she only knows about computers from her laptop and the internet. So what does the job of being a software developer actually consist of? The easiest way to describe it (the work-work part of the job that is, aside from the social or sociological part of the job) is in terms of all of the steps involved in battling through a software project. This is the so-called SDLC, the software development lifecycle:

Research current and near-future relevant technology
Interview or focus sessions with clients
Develop implementation alternatives
Research vendors
Review alternatives with management
Develop a project planDevelop functional specifications
Design data flow and data structures
Develop design specifications
Design draft of interface
Various programming tasks
Unit test programming
Software corrections from unit testing
System and stress testing Software
Corrections from system and stress testing
Alpha testing
Software corrections from alpha testing
Regression testing
Software corrections from regression testing
Technical documentation
Develop implementation plan
Develop training materials
Develop online help
Beta or parallel production testing
Training
Post implementation corrections
Production Support

Well yeah, it's a lot of stuff, and it's what makes the job always interesting and challenging.

Artistic Multitasking

Multithreading is a wonderful tool but you need to use it very carefully. First don't use it recklessly, as there are specific instances where it is of most benefit (see this post for examples). If you are not in a "cloud" environment that scales-out to add additional CPUs as required, then I have four tips to help you along your way.

1. Tip 1 is to be sure to assign the Name property an each thread you start. This is really the only way to tell which thread caused exceptions and is the only clue when you're debugging of how you got here. Use a name relevant to the instigator method and the serial count of that particular thread.

2. Think in terms of threads working on an instance of an object. This helps keep multiple threads from stepping on each others memory space. Put all the values for input into properties of your new object, create a method in that object that performs the "heavy lifting" and deep calculations, and then execute that method through the thread.Start (placing return values back into a property). As an added tip I like to build an array-list of all the objects I have actively submitted to threads; when the thread terminates I can pop its values out of the array and then null out its instance.

3. If you anticipate that you may ever need the capability to abort the thread you are starting (for example if the user cancels the operation) it is far far better to make provisions for this within the thread, in the method you are executing, rather than using thread.Abort. I usually create a boolean property in my main class called panicStop. All threads check this value within each iteration of a for-loop, and bail out appropriately.

4. Manage the quantity of threads you allow to run simultaneously. Beyond a certain threshold (depending on the number of CPUs on your box and other factors) firing more threads doesn't aid performance much. So manage it.

Multithreading works great as long as you remember to use it for appropriate tasks, and pass each thread a new object to work on. Remember that you are flowing parallel, but in separate "streams", so as to avoid conflating the instance of each object.

Artful Minimalism

Nowadays you can buy development tools that incorporate tons of form-behind widgets and code generation that can produce web pages with pop-out scroll overs, call-outs, and just about every kind of special effect imaginable. Is all of this necessary?

Unless you are building a data entry form (which has its own special challenges) you are probably better off just to make sure your web pages meet a certain set of minimum requirements. After you have thrown together your page, run it through a few simple tests:

1) Window Resizing
Open your page in a browser and resize the window to various heights and widths to see what happens. Did your layouts get mashed? Do you need to change column widths from fixed to a percentage?

2) W3C compatibility
Submit your page to the free W3C verifier, just as a courtesy check to assure that your tags are all balanced and that your links are all valid.

3) Powersave Mode
Fire up a browser on a smaller screen device and in the browser options turn off the setting that allows the web pages to "set their own colors." Also go into the properties tab on your desktop and change the color scheme to a power saving configurations, with a black background and white text. Now open up your web page to see what happened. Do you need to remove some "color" tags to keep it legible?

4) Just The Facts
Go back into the browser settings and turn off automatic image loading. Refresh your web page. Does it still make sense with just the facts? Did you provide the necessary alt tags so that the user can still figure out where to click for navigation?

Sure, spinning flashing widgets are fun to view on your site. For all of about ten seconds. What stays as important though is the actual informative content that you provide. Try to make it work under all conditions.

Artful Influence

The art of software development can be much more than accomplishing projects. Often the successful path begins by focusing beyond the project at hand -- you need to review everything you can think of: personalities, creative ideas, political issues, hidden agendas, insecurities, technical questions, doubts about the future direction of technology, everything.

You may find a need to be intensely aware of a company's culture. At other times it may appear that your real purpose is to change the culture, and your development project is just a means to achieve that end. You may also need to include a bit of an appraisal as to the strategic direction and positioning of your company: it might be wrong for you to try to turn your employer into the Nordstrom's of retailers if its intentions and strategy is to be the K-mart.

In some consulting environments your purpose is not to accomplish anything. Your project is actually an anti-project, the staff is against you, and you succeed by making the staff accomplish their own agenda. You must use grace to provide a backflow, purposefully pointing away from their objectives. Occasionally you can be successful overall (and act in the best interest of your retainer) even though your own personal accomplishments and monthly status reports might lack a certain luster.

Around one quarter of software design could be considered "art" in every sense of the word. Congruent to other creative arts a software designer encapsulates thought into both visual and written structures. Sometimes though the whole point of creating software is to go through the process of meeting with people and moving them from their preconceived positions to change the culture, nature, and very heart of your employer. So once in a while the parallels to art go deep down to its truer purpose: to move people's souls.

The Art in Value

One of the challenges in a creative profession is managing the perception of your “value” including the work that you perform, the objects that you create, and the temperament that you nourish. These are truly three independent metrics.

Due to the demands of the profession and its continually advancing technologies developers tend to dwell more on what they already know (or what they need to learn) rather than focusing on how others perceive them. As software development is both an art and a vocation however, it is important to pay attention to both your internal qualifications as well as the perceived external value that you add to your job.

The value of your physical software development opus is fourfold: it may provide a revenue stream by itself, it may tender “goodwill” to customers and draw them in to purchase other products, it may offer cost savings as the business operates, and finally it may give a strategic advantage by opening new markets that create unique services.

The value of the product you help design is a separate matter however from the value of the actual work that you perform. Somewhat impishly, you can lead a project that creates the next YouTube and still only get paid like a system designer. Your employer compares your base salary to that of any other developer they could hire off the street. The cost of a foot of electrical wire inside a jet plane is the same even if you only use it to connect your power outlet to your reading lamp.

But now let’s consider your actual value as viewed by your employer. This is a more nebulous quantity that relates to a host of factors, many of which are strictly outside of your vocation. Are you a healthy employee? Then you add value by your contributions to the company’s medical plan. Are you cheerful and easy to work with? Then you add value by improving the morale of the workplace. Are you a good mentor? Then you add value indirectly through improving the productivity of others.

As a professional remember not only to stay sharp on your skills, but also to pay attention to the art in your value.

The Art of Humor

A friend consulting with me on an Data Warehouse development effort (Lee Laniear) once told me an excellent metaphor that explains the three types of software development projects.

You are like an archer with a quiver full of arrows. In the first type of project, your user places the target in the distance, you grab and load your straightest arrow from your quiver, you draw back the bowstring, take aim, and you give it your best shot. The objective, of course, is to hit the bullseye. You may hit or you may miss, and certainly the aim is more difficult with increasing distance to the target or with increasing crosswinds. But the main point is that you can see where you want to go.

In the second kind of project, you are still the archer, but the user runs back and forth fifty yards away carrying the target. The objective, of course, is to hit the bullseye (not the client). He may stop, run one way, then suddenly stop and run another. He may stand still for brief periods of time, and then without warning start running around again. You grab your straightest arrow from your quiver, you draw back the bowstring, and you aim where you think the target is going to be by the time your arrow reaches it. Your success depends to a great extent on your historical observations of how the user changes his mind. And your self-control to aim safely for the target and not allow your emotions to divert your attention toward more animated ends.

In the third kind of project, you grab your straightest arrow from your quiver, you draw back the bowstring, and then leaning backward, you shoot your arrow straight up into the air. The object is to get the user, carrying the target on his head, to position himself directly underneath the falling arrow.

Successful project managers have an ability to separate themselves from their immediate feelings about the folks providing them with work. They need to be able to understand a larger picture of how others view their world, have a sunny disposition, and cultivate a fair amount of self-effacing graceful humor.

Artful Versions

When you focus on modular resilience, one item that helps a great deal is to pay attention to versioning. Version numbers are of use not only for change management but for testing and customer support as well. Apps developed with Microsoft tools keep their version numbers in a config file. You should standardize on a common meaning for the four subparts of version numbers.

Upgrades that produce a system that is incompatible with a prior release should increment the "major" version number. Upgrades that retain compatibility but that add significant functionality, should upgrade the "minor" version number. Bug fixes that get packaged into a distribution should increment the "release" version number. Finally the developers should increment the fourth component, the "build" number, with every compilation.

I like to present versions to the customer as major dot minor, for example this is versions 2.4 of the software. This version should display in a welcome pop-up and in the title bar of the main screen of your application. But for purpose of technical support I like to display (in a pop-up "About" box) something like version 2.4.3 build 118. Of course all of the version components are retrievable using the system reflection class.

Another helpful trick is to include a parallel matching version number within the backing database. When the client begins to run your software, make sure the version number in the data is compatible with the number in your software.

You should also version all of the modules within your software in a similar fashion. Although the component module versions aren't typically presented to the end user, reflect them to a called module (for debugging purposes) so that the called class can display the version of the invoker whenever it throws an error.

Sometimes it may seem superfluous, but it only takes about 30 seconds to update a version number, and with multiple developers on a project you will find that having this tracking available is invaluable.

Artfully Clean

Imagine buying a house and then never doing the laundry, vacuuming, taking out the trash, or washing the dishes. You just let stuff pile up until you run out of space and then you throw your up hands and go “oh, it must be time for a new house.” Sounds ludicrous eh?

In a highly competitive development environment though you build tables and files and directory folders, develop software, test a bit, and away you go. You patch up the bugs, optimize the queries, add a handful of indices, and you’re flying.

After a couple years though people are wondering why things take so long. Unsurprisingly the typical reason is a failure to archive and then remove the old data. Nobody is taking out the trash! Data retention and archival policies never get baked into the original specifications as the sponsors seldom see any immediate competitive value from them. Eventually however a clean system becomes less of a luxury and more a necessity. Folks start to notice the pile of dirty dishes.

The effort required to retrofit for cleanliness is about the same as the effort for planning for this up front. Strictly operationally however it is better to delay this expenditure until a system becomes a mess as you will then have a better understanding of the corners with the most fuzzballs.

Artful Vendor Relations

In any business, nobody does all the work themselves. We all rely on outside vendors from such mundane chores as providing the toilet paper up through providing phone lines, tax services, and legal assistance. And here in Information Technology we have our own set of vendors with wonderfully peculiar characteristics.

We group our unique compatriots into three worlds: the language vendors, the software package vendors, and the hardware vendors. We'll get to all three in due time, but today we will chat about the marvelous world of language vendors.

Selecting a language vendor is a lot like choosing a wife. You may not have as much actual contact with the vendor as you would like, but you will get to deal with what she cooks up, and her aesthetic tastes will affect you long after she is gone. Yeah and changing language vendors is as painful as going through a divorce, but that's a story for another post.

So what should you look for when selecting a language vendor? Probably the most important consideration is to perform an evaluation of their openness and how they admit to bugs and problems. Upgrades to language tools are major undertakings and a vendor doesn't take it lightly. So to meet ongoing challenges they should show resolve to provide workarounds.

Languages and development tools constantly change (and I've never seen a version of a language released with fewer verbs than the previous version). To some extent, the vendor is trying to remain competitive by adding the functionality that becomes available in competitors' products or other languages. A software vendor employs a large number of programmers, who need to keep releasing new versions in order to maintain their livelihoods.

So it turns out to be a bit of a balancing act managing your relationship with a language vendor: stability helps your programmers be productive, and yet staying current of tools allows you to incorporate new aesthetics and retain younger talent. If you happen to be in the position of selecting a vendor at the start of a large project you will partly need to use your intuition (and contacts in the industry) to get a better sense of the credibility of the press and of the vendors.

On the other hand, if one of the major vendors announces their plans for their next release of one of their market-leading programming tools, then you would be remiss to overlook its pending availability in your plans. So take your time and be very deliberate when choosing a language vendor, for you will be living with your choice for a long time.

Artistic Sparsity

Even though we know it in our hearts, we still fail to account for how a software system grows and becomes more complex over time. Designing interface screens is a lot like planning an informal garden; if you want it to look great once all the plants take hold and fill out you need to think a bit about the future. Instead folks tend to wireframe a product that looks “complete” with reasonable use of the menu real-estate and a felicitous spread of field-density on the screen at the initial planting, as if the product is somehow “finished” once you implement it.

In doing this we make things look good in the /present/, yet we fail to account for system growth. When we later add significant new functionality… surprise, our screens and menus appear too overgrown and cluttered. It’s as if you planted all the apple trees two yards apart because it looked better; now that they have branched and fruited they are all atop on another.

Therefore when you are creating a new system plan for expansion by deliberately under-designing menus and occupancy: make the interface artistically sparse. And then don’t worry about it; like a garden your interface will “fill out” as it matures.

The Art in Modularity

Good software developers tend to be organizing freaks: everything goes in a tiny little box in its proper place. You know the saying “there’s no limit to how complicated things can get, on account of one thing always leading to another.” This is especially true in programming.

To combat this creeping complexity we strive for modularity and maintainability. A small module that encloses a specific business rule is easier for testing and allows for greater flexibility of reuse. When modules become too small however, the sheer act of tracking them, organizing their use, maintaining consistent versions, and finding where a business rule gets implemented interferes with your productivity.

Somewhat counter-intuitively smaller modules also need greater external documentation, mainly to track the nature of the parameters passed between them.

What’s the right size for a module then? And how many modules should a system have? Although I like a module to be between a half and five printed pages it really depends upon two things: how many people are maintaining the software and the density of the development language.

The fewer the number of developers the larger modules can be. Each of you is baking your own cake. So if it’s just the two of you maintaining a legacy billing system written in an old dialect of Basic (that gets about as complicated as a GOSUB and a CALL) then sure, go with the 5-page modules.

But if a half-dozen of you are planning to maintain ASP web pages in C# (with overloads and inheritance) then you’d better veer closer to the halfpage module size. In that case you’re not each baking a cake; you’re all contributing to building a car. You need to be able to swap in new replacement parts when the old ones wear thin. Every once in a while step back to review how you are developing the code: are you using appropriately sized modules?