Thumb loading

I talked about thumb drives a while back. And, i'm at it again, downloading huge amounts of stuff and tossing it onto thumb drives.

Not only does it take awhile to fill a 2 GB kingston, but it also sends the load average through the roof. It's typical to take an otherwise unloaded system from a load of zero to a load of 5. And, it stays there for the duration. What's with that? The CPU is 99.7% idle. So, it's not like parallel ports where the CPU needs to do something every few bytes transfered. What are the other 4 tasks waiting to run? And despite the low CPU load, responsiveness during copies to the USB thumb devices is sluggish and jumpy.

This is under Linux.

Anyone have any idea why? Is it different under Windows? Mac? OS/X?

Canoe trip

It's late in the season, but the canoe trip experience has finally taken place. We're not talking about a shakeout run to make sure everything works. We're also not talking about days on the water, or some other big deal. But, a trip with passengers. It's taken nearly a month to get it together.

Since we're, at heart, city folk, the trip takes place in a city. The Huron River runs through Ann Arbor, MI. Since it was otherwise quiet and peaceful, we brought along a couple ten year olds to provide continuous noise. The ducks were unable to compete with this noise, and so, stayed quiet themselves.

Gorgeous day. This is, unfortunately, the best picture taken all day. It had to be modified to look even this good. Mallards, i think.

Internal Discipline

When we last left our hero the large organization was overrun by high latency task groups that were effectively obstacles to progress.

Another feature of large organizations, and by large i mean organizations with more than one person, is externally applied discipline. The problem to be solved is How does one motivate the employee to produce good work?. For the software developer, it's the wrong question.

One of the common management approaches to this problem is characterized by the carrot and stick paradigm. The idea was that to get a horse to pull the buggy, one had a carrot and a stick. The carrot was a reward for good behavior. The stick was punishment for bad behavior. Now, modern management has no experience with horses, or they'd know that the stick was not for punishment. The carrot was suspended by a string from the stick. But that isn't the modern interpretation.

Any parent will note that a promised reward is a much better motivator than punishment for children. In fact, punishment doesn't work very well, and not at all over the long haul.

But managers generally don't think that they have anything to offer an employee (which is false). They also don't think that they can get away with much in the way of punishment (which is also false). So, despite the above very common and widely talked about concept, it often plays almost no actual role in obtaining quality work.

What managers really know how to do is the paper trail. To get the developer to use source control, have a manager sign off that code is in the system. To get the developer to adhere to database standards, have a database administrator approve any database usage or changes. To get the developer to add user level help pages, have someone who only handles installing help pages on the system review the text and install it. To make sure that the customer is satisfied, have the customer sign off on the project before it is placed on the server. Each of these processes adds delays to the project, but do they ensure quality?

No. It turns out that the database administrator doesn't know anything about the application code. So even a simple query that uses only indexed keys might in fact be issued to the database millions of times. Only the developer would know that. The help text manager doesn't really know the application, and can easily miss omissions and outright errors in the text. Worse, help text usually concentrates on the details of how a function works, but never gives the end user any idea why one might want to use the function. That takes context. The customer signs off on the application, but does so at application launch. It may be a month before the customer is familiar enough with the product to be able to really evaluate it.

External discipline is inefficient and ineffective. The solution is to empower the developer to do the right thing. First, the developer should be aware of the kinds of quality that the customer wants. It may be consistency, repeatability, discoverability, ease of use, etc. Second, for those quality issues that only a developer could understand, review should be made by another developer. Code reviews have the side benefit of disseminating knowledge. Even the most senior developers learn something. Involve the database administrators early in the process, not late. The DBA should suggest good ways to do things up front, not stall a project nearing completion. To do that, the developer and the DBA need to work together. For best possible speed, they should be a single person. The customer should be able to give feedback at any time, not just before the launch.

Empowerment

Large organizations have a problem. It's a hard problem to solve. That is, how do you organize the division of labor when you have a large number of people? It's not a new problem. Sun Tzu's The Art of War covers it. The general was asked, "How large an army can you command?". His answer was "the larger the better". Old problem but solved.

People tend to learn new skills fairly slowly, so it is commonly thought that no one knows how to do everything. In any case, the large organization tends to discover, often by luck, that a person knows how to do some specific task, and then uses them for that one task exclusively. As the organization gets larger, groups of people form to perform specific tasks, and the organization tends to treat these the same as individuals, performing one task forever.

The large organization then takes these task groups and establishes processes that let the groups get things done together. When there is a complaint that the process isn't written down, it tends to get written, and becomes less flexible. When problems of quality arise, and let's face it, humans make mistakes, the reaction is to manage quality by requiring that a high level manager signs off on actions. But these people don't know how to do everything, so they don't know how to judge quality. They become a source of latency for projects, with no added value. Management thinks in terms of paper trails, and requires paper documents for everything.

The ebb and flow of work to do means that some task groups feel overloaded from time to time. They set up often elaborate queues where tasks are submitted, and 'customers' are given a minimum wait to get things done. For example, a database administration group may require a minimum of a week for any task - even if the task requires no analysis, no research, no diagnosis, just a simple command sent to the software. Often, these groups require that there is no face to face or even phone contact, and miscommunications are frequent, requiring resubmission of the task, with attendant latency.

So, let's say you're a developer, and the project is to project a web based application. Checking your source into the source tree requires sign off from the source code control group. Placing the application on the server requires sign off from the hardware and web server software groups. The database administrators need to sign off on the way the database is used, even if there are no new tables. The help text for your application is managed by yet another group. The application took a week to write, but takes an additional six weeks to deploy, if you're lucky.

How can this be fixed? Instead of attempting to manage quality by instituting sign offs by management enforced by restricted access to systems, the trend should be to manage quality by automated paper trails, with automated back-out systems for failure mitigation. The emphasis is on automation to reduce latency. Empower the developer to get things done by giving them the tools needed to get the entire job done. Trust that the developer will attempt to do the right thing, but don't be afraid of failure. Fear of failure causes paralysis.

There are a complicated set of issues. For brevity, this is just one of them.

Troll

Kirk vs Janeway

COBOL vs Java

Miata vs Jeep

Spaceship One vs Shuttle

Meede vs Orion

Discuss

Today is National Relaxation Day

End of joke.

Tom Swift

On the way home from Bible Camp, my ten year old son and i listened to a 4 hour book downloaded from Librivox. Tom Swift and the Visitor From Planet X, written by Victor Appleton II. It went over well. There was a cliff hanger every chapter, except the last. This could be serialized on radio easily. 20 Chapters, so they must have only been ten or fifteen minutes each. That's short enough for nearly any attention span.

The reader, Mark Nelson is very good. No memorable mistakes. Maybe there aren't any. He does it with a 1950's style, with gusto, WoWee! But should you want to read it yourself, the full text is available on Gutenberg.

It's science fiction, with an action/adventure feel, but i had difficulty placing just when it might have been written. It has a 1950's style, but seemed to have references to more modern inventions. Fortunately, one doesn't have to guess, since Wikipedia knows it was written in 1961. And, it knows who Victor Appleton II is, and what a busy guy Tom really is. He had something like a hundred such adventures. And those are just the ones that have been written up so far.

The story has it all. Hot sex - well - there was a quick kiss on the cheek. I wasn't paying attention. It might have been Tom's sister. And, it has violence. Well, there was an earthquake, and some property damage. Space aliens, including the Visitor from Planet X. I was hoping the space alien would have something to teach us about global warming or world peace. Not a word. The one named villain turned out to be a spy working for the good guys. In retrospect, it's hard to believe there were 19 cliff hangers.