Friday, May 16, 2008

Shadow IT vs Silo IT

IT departments in larger companies seem to have a mandate to find and stamp out Shadow IT whereever it can find it. First, a bit about Shadow IT. Imagine that your department actually does something for the company. Let's say that it's at headquarters, and manages sales. Let's say that there is a sales force spread across the country, and this department is responsible for getting the most from it. One thing it might attempt is to track how each sales person, each small group of sales people, and each region are doing. The idea is that a group manager writes up sales data for each sales person in the group. The data is tagged with what the group is and what the region is. Then, the data will be analysed back at the headquarters.

Doing this analysis by hand is going to be time consuming (expensive, remember, Einstein proved that time = money), tedious (which was never stops businesses) and error prone. So you hire a programmer, and tell them what you want. She says, we need a way for the managers to enter the data. We'll write a web application where they'll enter the data, and it will get stored back here in a database. There's no budget of any kind, so she finds a PC that's so old, slow and has so little memory that no one wants it. She loads Linux on it, installs an Apache web server, and a Postgres database server and configures them. Then she writes a form, and a Perl script that takes form data and stuffs it into the database. She announces that it's in production, send this URL by email to the managers, and let them at it. And despite using a four year old PC instead of a quarter million dollar server, the application is responsive and reliable.

Soon, data is coming in. By then, she's written a quick report that adds up sales by group and region for each month, in a table. The analysts say it's great. For one thing, they're no longer swamped in paper. While our intrepid programmer adds logins and real security, someone gets the idea that there should be a report that graphs the groups and regions over time so that one can pick out which ones are doing better and which ones are doing worse. She grabs some graphing routines from CPAN and puts something up.

What's wrong with this picture? Well, corporate says that having a server sit under someone's desk is bad. For one thing, there's no backup. She counters that she does have backup. There's a second PC with Linux, and it gets a copy of the primary's data every night. They also complain that Perl isn't supported, that Java is the corporate standard. Further, they say that they can save money by having the server administrated by the Unix group, the database administrated by the database group, the web server administrated by the Web group, security managed by the network security group.

Two years later, the application is rewritten in Java, it's hosted in the server room, and it follows all the company rules. Everything's great, right? Except that the original developer has been spending the past two years bringing the offshore people up to speed on the app, instead of adding the new features that the customer now knows they want, and the app still doesn't do everything that the original did, and it cost a million dollars, and nothing was saved, because the original developer is still on the payroll.

Why did the rewrite, with 30 developers take so long? For one, they had to use the Silo IT development model. What's Silo IT?

Corporate management has labeled the department model Shadow IT. It's a negative name. It casts the endeavor as murky, hard to follow, uncontrollable, and underhanded. So, it's only fair that the popular alternative also has a negative name. Silo IT refers to this idea that under the idea that pooled resources are efficient resources, the endeavor is split up into areas of responsibility. Typically, this is systems administration, web administration, database administration, network administration, architechture, web framework, mentors and application developers. Each of these groups forms their own sub-department.

If a developer wants to create an application, it needs to follow the rules laid down by architects, and it needs to fit in with the framework. Now, there's a considerable amount of information to go through, and typically there are documents, hundreds or thousands of pages of documents, that one needs to go through to absorb it. No problem - there are mentors that can help, right? Wrong. The mentor's job is to make sure that the rules are followed. The developer gets the application to work, then submits the code to the mentor, who comments one sections that don't follow the rules. It's like an exam. Mentors never consult with the developers to get them going. Their job is to slow the deployment process down by several days while they review the code.

What does the developer do while waiting? Well, perhaps they have another application to work on. If they do, they'll generally find the going slow and error prone, because each shift in focus takes several hours for their brains to catch up. So, as often as not, they do nothing.

So, let's say that the developer needs a new table in the database. They submit a ticket, and it enters the database administrator's queue. In ten days, an administrator takes the ticket out of the queue and creates the table. The developer then tests it to make sure it's what they needed, then submits a ticket to get the change made to the test servers. Another ten days. Then ten more days to get the change into production. Mind you, new tables can have no negative impact on production, so it could have been there at the start. And in Shadow IT, that's what happens. Except that it doesn't take thirty days. It takes maybe half an hour in Shadow IT. It's not hard to see where this is going. The developers are easily thirty times more productive in a Shadow IT organization than in a Silo IT system. So, it's not that the Shadow IT developer is brilliant, she just doesn't have the same ball and chain attached to her ankle while she swims the English Channel.

So what if you can't find a developer who knows how to install and administrate Linux, Apache, Postgres, and knows a web aware language? Well, according to Fred Brooks, the department should build a surgical team. Maybe it takes two or three people. But since they are a single team, they can still be responsive. In particular, the database administrator can spend time learning the current applications, and tune both the database and the queries to the real needs.


Berkay said...

I can understand where you're coming from. it is quite frustrating to see things done in such unproductive way.

Corporations do not like their systems to be dependent on any single person. Employees can leave anytime for whatever the reason.

If the developer leaves, who will maintain the application? have to find someone else who knows the developer's language of choice, db of choice, etc. and understand the code that may not be written in a way someone else can maintain (perl code can be notoriously hard to maintain for someone else for example)

What if you have dozens of developers, each writing apps in different languages, using different databases, apps and operating systems. How do the corporation ensure maintainability of the applications going forward with many different platforms, languages etc.

This is the fundamental problem that drives organizations to specify standard platforms and languages, and groups to support these standards. As a result the silos come about and causes the problems you've outlined in your post.
Silo IT is simply bad practice. Taking 10 days to fulfill a request is not acceptable. IT has to have much better service. If not, shadow IT emerges as a result. But I don't think the Shadow IT is the answer as it introduces other problems as stated above.

Stephen said...

Shadow IT isn't about having a single point of failure.

Let's get this straight. Shadow IT didn't name itself. It isn't evil.

I have firm evidence that Shadow IT is up to thirty times (30x) more efficient than Silo IT. More than one corporation. With that kind of efficiency, you can have double staff with each having double wages and still come out way ahead, not just in cost, but in responsiveness. You might even get used to it.

I've done maintenance on two half million line perl applications. I did it in the worst possible way both times - there was no hand off from previous maintainers. It wasn't that hard. I don't know where this misconception comes from. Picking up other people's code is difficult in any language.

The same people who set up Silo IT are the ones who enforce a minimum 10 day turnaround for simple services. It's a mind set. What is bizarre is what incentives they might have to make these decisions. They'll defend it to the last.

Shadow IT isn't about everyone just picking whatever tools they want to get the job done. Typically, the department has zero resources, so the developers use what is available for free. Perl is free. MySQL and Postgres are free. Apache is free. Linux is free. Python is free. So you get LAMP development. Go figure. But this isn't as complicated as Solaris (or much worse, Windows) Java, JSP, Javascript, pick one of Oracle, Sybase, MS SQL, DB2, UDB, etc., a zillion class libraries, and deployment to a server by script. The development server doesn't behave like production. When (not if) production goes down, it stays down for days or weeks. It's because no one knows enough to diagnose the problem.

I have more on this topic in future episodes.