I started Elder Heinchez' bike the other day and it features something I don't usually incorporate on most builds....a short tube. 100mm. This is the shortest I've ever put on a bike....29ers included. I like to leave them as long as possible, and, well, this one is as long as possible. Longer head tubes just make sense from a physics and engineering stand point. They diffuse the stress on the headset cups and bearings from the leverage of the fork and the varying forces applied to it as the front wheel rolls over and impacts the terrain and braking forces are applied. Imagine the lower headset cup as the fulcrum of a lever in the simple machine sense of things and the upper cup as the object being moved by this machine. The closer the upper and lower cups are together the easier it is to stress this area (loose headsets, ovalized head tubes, etc.)
A number of factors will dictate the length of a head tube in a bike's design, but ultimately the build of the rider is going to be the limiting factor. A rider will have a maximum seat height and therefore a maximum or minimum handlebar height. For most mountain bikes I figure a handlebar height that is equal to the saddle, unless the rider knows he wants handlebar drop. The other considerations immediately affecting head tube length are:
fork length (axle to crown)...longer travel forks are, well, longer
bottom bracket height--raises/lowers the rider over the axles. A taller BB warrants a longer head tube.
seat tube angle--a slacker ST will lower rider position
component choices--a riser bar leaves less room for a head tube. Usually this is not critical but is definitely something to be considered when that space is going to be at a premium.
you could probably also argue that ETT length and head tube angle will affect head tube length, but it seems negligible and these are both design considerations that hold a more important function.
there is no reason to have an excessive stack of headset spacers on a custom bike unless a wide variety of positions is desired by the rider. To me over 20mm of spacers is excessive (think of the lever and fulcrum, your headset, being worked from the handlebars, too). Allow for dynamic adjustments for comfort within a range. Most people as they age, though, fancy comfort (higher handlebars) over speed (lower, more racy position).
The Elder's bike is interesting here. He is built stout with a long torso and long arms and carries a very good portion of his weight in his upper body. He wanted a bike that is stable and that gives him confidence on the descents. Until now he's often felt like the front end on the downhill is a precarious balance point. This bike will have slacker angles and a lower BB than most that I do, but given his build and experiences, I think this will suit him very well. He's also the type of rider to barrel through the hard parts, no compromise to parts, so stability is key. This was a tricky design to work out but the overall lower center of gravity left me with a 100 mm head tube and probably not much room for rise in either the stem or the handlebars. Now here's where I think this shorty HT is just cool: with the oversize tubes (31.8 TT, 34.9 DT) and giving myself enough room for a nice fillet on top and enough room to swing a Fox Fork around on the bottom, the top tube and down tube intersect. This is coming out to be the most precisely mitered bike so far. The pics below are of the build sequence so far. I ended up tacking the top tube and head tube and will clean it up as a sub assembly before tacking the down tube. The seat tube and BB are already tacked and sleeved as is pretty standard. All said I'm pretty stoked on it.