Why Incompetence Spreads through Big Organizations

Promoting the people most competent at one job does not mean that they'll be better at another, according to a new simulation of hierarchical organizations.

There's a paradox at the heart of most Western organizations. The people who perform best at one level of an organization tend to be promoted on the premise that they will also be competent at another level within the organization. I imagine that most readers will have had personal experience at the way that this hypothesis fails in practice.

In 1969, a Canadian psychologist named Laurence Peter encapsulated this behavior in a rule that has since become known as Peter's Principle. Here it is:

"All new members in a hierarchical organization climb the hierarchy until they reach their level of maximum incompetence."

That's not as unfair as it sounds, say Alessandro Pluchino and buddies from Universita di Catania, who have modeled this behavior using an agent-based system for the first time. They say that common sense tells us that a member who is competent at a given level will also be competent at a higher level of the hierarchy. So it may well seem a good idea to promote such an individual to the next level.

The problem is that common sense often fools us. It's not so hard to see that a new position in an organization requires different skills, so the competent performance of one task may not correlate well with the ability to perform another task well.

Peter pointed out that in large organizations where these practices are used, it is inevitable that individuals will be promoted until they reach their level of maximum incompetence. The unavoidable result is the runaway spread of incompetence throughout an organization.

Now Pluchino and co have simulated this practice with an agent-based model for the first time. Sure enough, they find that it leads to a significant reduction in the efficiency of an organization, as incompetency spreads through it. That must have an uncomfortable ring of truth for some CEOs.

But is there a better way of choosing individuals for promotion? It turns out that there is, say Pluchino and co. Their model shows that two other strategies outperform the conventional method of promotion.

The first is to alternately promote first the most competent and then the least competent individuals. And the second is to promote individuals at random. Both of these methods improve, or at least do not diminish, the efficiency of an organization.

Interesting idea that would be fascinating to see in action. What would be a suitable prize for the first CEO to implement such a policy?

Ref: Technology Review: the physics arXiv blog

PHP object orientation - Separating concerns : Building more modular PHP applications
Translate Atom to RDF : From syndication to semantics with ease
The new role of XML in cloud data integration : Using XML to integrate Salesforce data with enterprise applications
An introduction to custom application development on the cloud using Force.com : Fundamentals and Workbook

Chocolate Lines

"Strength is the capacity to break a chocolate bar in four pieces with your bare hands- and then just eat one piece "
--JUDITH VIORST
"There are two kinds of people in the world: those who love chocolates and communists"
--LESLIE MOAK MURRAY
"It's not that chocolates are a substitute for love.Love is a substitute of chocolate. Chocolate is far more reliable thana man"
--MIRANDA INGRAM
"Nine out of ten people like chocolate. The tenth person always lies."
--JOHN Q. TULLIUS

Ours to question why

The first excerpt is written by Karan Thapar. I have always thought him to be a little stupid specially after his television programs and also his other articles where he had written about benazir bhutto as if she would have solved all the problems between india and pakistan if she was alive and at the helm of pakistan. I on the other hand think its probably the best that lady was killed or else pakistan would have by now used its entire terrorists against us while the americans would be eating out of her hands. I have not forgotten that while she was at the helm pakistan was the most vocal regarding kashmir.
So when I read this article, it came as a very nice surprise.----

At first, I was taken aback that the president of France should have spoken about the burqa and in an address to the parliament at that. But the more I read, the more sense it seemed to make. Presidential addresses ought to be about issues that transcend the daily struggle of politics and Sarkozy had framed the burqa in bigger, more important, terms.

“The issue of the burqa is not a religious issue, it is a question of freedom and of women’s dignity,” the French President said in his Versailles speech. “The burqa is not a religious sign, it’s a sign of the subjugation, of the submission, of women. I want to say solemnly that it (the burqa) will not be welcome on our territory.” Perhaps, more than any other, this sentence was received with rapturous applause. In fact, Sarkozy’s comments on the burqa captured the headlines, even though this was the first presidential address to the French parliament after Charles Louis Napolean Bonaparte 136 years ago and despite the fact that Sarkozy’s 45-minute address touched on many subjects, including the economic crisis.

Beyond his indisputably correct comments on the burqa, what struck me about Sarkozy’s speech was how different it was to the sort of fare we, in India, have grown used to. Neither our politicians, nor our president talk to us about issues other than politics. Either for reasons of misplaced political correctness or because they haven’t thought through the matter themselves — and I bet it’s the latter — they avoid moral issues. This, I might add, is both sad and a mistake.

Moral issues need to be questioned and debated. They must not be buried under the weight of custom or under fear of the controversy any comment could provoke. If politicians feel strongly about them they must speak out. Not just because silence would be deception but because that’s how a debate is started. And democratic societies need to question and debate.

Let me also add that just because a politician speaks doesn’t mean his or her point of view will be accepted. Sarkozy knows that only too well. So let not an exaggerated opinion of themselves become an excuse for timidity or reticence. However, because they are politicians and are practised speakers they could frame the issue intelligently and create a platform for equally thought-provoking responses. And that is important.

But will we ever hear Manmohan Singh, L. K. Advani or Sonia Gandhi speak to us about issues such as the right of women to drink in pubs, wear jeans in colleges and lead normal lives after widowhood? I hope so. But I doubt it.

The second one is written by Indrajit Hazra whose articles I really like probably because his thinking matches mine after all he is also a fellow bong.

Ever wondered what goes through the head of Ashok Srivastava each time a young woman in jeans walks past him? It’s demanding enough for the Convenor of the Uttar Pradesh Principals’ Association to stay composed when any jeans-wearing young woman walks by. But imagine the serious conflict raging inside Srivastava, a good man of the kind we don’t meet often enough these days, if a jeans-clad college girl with a dexterous figure — with the wind blowing through her hair — and humming ‘Jaadu hai nasha hai’ walks by in slo-mo.
Well, I can’t see his perfectly normal heterosexual reaction to a young, attractive lady wearing figure-hugging trousers being any different from yours (if you’re a man, that is) or mine — except, perhaps, in intensity, which in turn depends on the frequency of spotting women in jeans on a regular basis (not that much for Srivastava, I would presume) and one’s own hormonal balance.

What is different, though, is how Srivastava wants to deal with his biologically-driven affection for women in denim: by not having them anywhere near him. (In some societies, of course, a more effective method would be to punish women in jeans so as to make them stop existing altogether.) As far as I’m concerned, you don’t have to be a hick or a pervert or even the head priest of the Guruvayur Temple to be distracted by the ergonomic quality of jeans when fitted on to a charmingly-shaped lady. The nature of the limbs-hugging jeans, after all, is to highlight the physical attractiveness of the wearer. (Thus, the total pointlessness or more of, say, President Pratibha Patil wearing a pair of Levis 901s.)

No woman — or man, for that matter — wears clothes to look unattractive, not according to their own set of aesthetics, that is. Their objective may be to look ‘smart’, ‘traditional’, ‘radical’ or a permutation-combination of all three. But the basic premise, even of someone like Sushma Swaraj, is to present oneself as ‘attractive’, a diluted-by-evolution-and-social mores version of the original biological purpose of looking attractive: advertising one’s sexuality.

The woman-in-jeans, of course, elicits different reactions in different settings. A jeans-wearing girl walking along Flora Fountain in Bombay will be seen as a different entity from the same girl in jeans cycling along a Gorakhpur alley. It’s as different as an attractive lady in a sari in Delhi is from an attractive lady in a sari on the streets of, say, Melbourne.

So, much of everything that surrounds the business of women in jeans boils down to what men make of it — and what women make of what the men make of it. The pitch against women in jeans, of course, will never be in the following form: “I am reacting hormonally to those girls in jeans under that tree. Please ensure that they don’t wear such tight clothes and force me to think of things other than the price of plums!” Instead, the rationale is always on this line: “Other men — lascivious ones — are reacting hormonally to those girls in jeans under the tree. Please ensure that they don’t wear such tight clothes!” Here’s Ashok Srivastava’s version: “It has been seen that eve-teasers generally target girls wearing jeans or modern clothes.” The truth is that I don’t think he’s wrong. One man’s women in hip-hugging jeans can be another man’s women in bodice-hugging salwar-kameezes.

The latest jeans imbroglio won’t be the last jeans imbroglio. Men will — comfortably or uncomfortably — get turned on by this iconic, all-pervasive apparel that accentuates the wearer’s hips and buttocks.

Where the mind is with fear

Excerpts from two articles in hindustan times written by Manas Chakravarty and Indrajit Hazra.

We are a non-violent people. We hate it when people resort to violence. In Lalgarh, the tribals have all along been very peaceful. True, the primary health centres in their villages didn’t have any medicines and doctors from the towns rarely visited them. So what’s new? Many people saw their sick loved ones die as they made the long trek to the district hospital from their villages over the dirt tracks that pass for roads. But there was no violence.

Finding drinking water in the summer has always been a problem in the villages. Ponds have had to be used for both drinking water and for bathing. Children have often suffered from diseases as a result. But the tribals of Lalgarh are used to their children dying early. They never complained.

Most villagers in the region are caught in a vicious poverty trap. Malnutrition is rife. Doctors from Kolkata who recently visited the place said that what the people needed was not pills but food. Nobel Laureate Amartya Sen once said that hunger was “a quiet violence”. He meant that if a state can’t feed its people it’s guilty of violence towards them. But he was just twisting words to suit his theory.

Indians are malnourished not just in Lalgarh, but all over the country. A recent Unicef report said that 405 million people in South Asia suffered from chronic hunger. India’s rank in the Global Hunger Index of 88 countries is 66, below several African countries. So there’s nothing special about Lalgarh. Also, in spite of being hungry, the people were peaceful. Being peaceful is the most important thing.

Every election, the Lalgarh tribals voted the Left to power in the hope that these self-proclaimed friends of the poor would help them. But in spite of the promises, nothing happened. The money from the anti-poverty programmes never reached them, the police occupied the buildings that were supposed to be clinics and the irrigation canals dried up. They watched in silence as the local party bosses built mansions and businesses for themselves and their cronies.

For more than 60 years after independence, they patiently waited for better times. And it’s not that the country wasn’t doing well. Some of them went to the grand city of Kolkata and came back with wondrous tales of shining malls and air-conditioning and taps that never ran dry. They were right to wait. For as we all know, it’s just a matter of time. Once the Sensex goes up enough and CEOs start earning several crores a year and India becomes a world power, then money will trickle down and reach places like Lalgarh. True, generations may be destroyed before that happens. But that is not violence.

Some things do seem to suggest, at first glance, a hint of brutality. Take the routine manner in which the police pick up tribals for questioning and then torture them. But that’s required for the police to conduct their investigations. How else will they protect the people from the Maoists? True, tribals in Lalgarh lived in constant terror of the police and of the party thugs. But that is not terrorism.

Of late, though, the people of Lalgarh have been behaving very oddly. They drove the police and the party bigwigs out of the area and torched their houses. They have started digging wells, setting up schools and running health clinics, without any help from the state. They have formed a Committee against Police Atrocities which wants electricity in their villages and roads and bridges to be built. Worse, they even want the politicians to apologise! Very strangely, after all these decades, they seem to be running out of patience.

What on earth is going on? Outsiders must be inciting them to violence. We are a peace-loving people and must stop this violence at once. Don’t worry, our tribal brothers, our troops are on their way to save you.

--Manas Chakraborty

The conditions that have led people to fall for the seductive charms of violent revolt were being pressure-cooked for years. An administration had long forgotten to recognise, never mind keep, its part of the bargain with the very people who had given the CPI(M)-led front its generational power and the pelf that comes with it.

Take the case of Kuna Sabar, a resident of Darra village in West Midnapore’s Belpahari sub-district. On December 22, 2007 — when a million miles away in Calcutta, people were frantically speculating about the return of Sourav Ganguly in the Indian cricket squad — Sabar died of hunger. If his cause of death (confirmed by a doctor) wasn’t shameful enough for a government that took pride in prioritising the concerns of its rural masses, the subdivisional officer’s response to the death was horrific. He said that documents showed that Sabar had bought “8 kg rice, 2 kg wheat and 2.4 kg sugar” from the ration shop “between December 2 and December 16”. Effectively, he was telling Sabar’s widow that his death must have been her husband’s fault.

Sabar is just one statistic. During 2004-2005, a year before the Left Front won the 2006 assembly polls by a landslide, dozens of ‘hunger deaths’ across Bengal were recorded by the Asian Human Rights Commission. And these sordid deaths were overwhelmingly because of utter administrative failures. Till January 2008, only 34 people in West Midnapore, for instance, had received the minimum 100-day job and corresponding pay under the National Rural Employment Guarantee Scheme. The remaining earned wages for an average of 11.6 days.

In a universe where bureaucrats, academics,policemen, the cogs and wheels of administration and governance are deeply entrenched in ‘party affiliations’, accountability can only be a silly theological notion for bourgeois ‘management types’. It is this affliction of apathy — and of genuinely being stumped about why people might be enraged about pointless deaths, of living in life-defying poverty — that really makes for something rotten in the state of Bengal.

What applies to administrative ignorance (an evolutionary byproduct of administrative apathy) holds true for a police force that simply doesn’t know anything about crowd control or how to tackle a riotous mob. Either the police do nothing (as they did when the Maoist-goaded People’s Committee Against Police Atrocities in Lalgarh first marched to the CPI(M) zonal headquarters in Dharampur last week to destroy any signs of the CPI(M)/administration and attack party workers), or they shoot first and ask questions later (as they so memorably did in Nandigram on March 14, 2007).

Only in Left-ruled Bengal do you get armed partymen being regularly and openly sent to ‘capture and liberate’ towns and villages that have fallen in the ‘wrong hands’. The police arrive at the scene later, if the comrades and their local commissars have failed to do their job. As this is being written, the state government has finally let the police and security forces enter Lalgarh to ‘reclaim’ it from the Maoist ‘invaders’. It will remain unclear for a long while whether this reclamation is being conducted at the behest of the CPI(M) or the people of Lalgarh, considering that the concerns of the two are different and almost diametrically opposite.

--Indrajit Hazra

Leveraging Amazon Web Services for enterprise application integration : XML messaging with Amazon SQS
Connecting to the Cloud, Part 3: Cloud governance and security : Secure the HybridCloud application
Anti-Patterns To Avoid In N-Tier Applications
Build Better Data-Driven Apps With Distributed Caching
Using Mocks And Tests To Design Role-Based Objects
Building A Data-Driven Expense App with Silverlight 3
The ADO.NET Entity Framework Overview

Implement Web cut-and-paste using Atom XML and Firefox XUL
Using the Twitter REST API :Explore the Twitter REST API for automated Web 2.0

During Microsoft Teched at Hyderabad, there was a contest for the top architect which I contested.
The following images are of the slides I sent and the text is basically the audio I sent.

High Overview

UseCase

Microsoft Solutions Architecture

Deployment

The first slide is a high level overview just for illustration purposes, so I won't delve into that. Let me speak of the Use cases which is in the 2nd slide.Here as I state a user or rather an authenticated user can vote, Mark Self as Candidate, sponsor someone else as candidate, view candidate info and so on. Ialso have another actor Candidate who extends from user and can upload content. I also have a external system who gets data and functionality from the system. To do so it pays to the system which I have assumed is done via a Payment Partner. The system also takes money when a user marks self as candidate and sponsoring another person.This is another assumption i have made. Now to the solution Architecture, I have proposed a N tier application(in code) with complete seapration of UI, business logic, and data access logic. I follow the business facade pattern and will have a generated Data access layer where every method corresponds to one stored proc. Now one thing people might not like is the site having both a web client and a rich internet client. I like rich internet clients a lot and I think its usage could be a great plus in this site. I have suggested a web client to go with a rich internet app as well as it can then run on mobile browsers and on those browsers which do not have rich internet client framework.The rich internet client as well as the external services can connect directly to service layer to get the data. Now if we go to the microsoft technology based architecture slide you will see that I have replaced all the technologies spoken earlier with the Microsoft technologies, Rich Internet Client has become Silverlight, the web client app has become ASP.NET MVC ( to mobile enable it , you just need to add the mobile browser definition files).I'm a great believer in software factories and so I have used the web service software factory with WCF in the service and business layer, the ADO.NET entity framework takes up the job of data layer along with data access application block which is heavily being used along with other application blocks in enterprise library cross cutting all layers for exception management, validation, logging, security, etc. I have proposed SQLserver as the dbas we are using .net and Microsoft's SQL client provider is super optimized for SQL Server.MS is supposed to come out with a rich internet client technology for mobile but as it is still in the pipeline( not even a beta release), I would like to first build the ria and even after it is ready if ms does not have the ria client for mobile browsers, I would create the web client, and my experience tells me that it will not require a heavy resource as the business logic has already been created.Finally security in WCF, to connect to silverlight u need to use basic http binding , so the security will have to happen at transport-level, HTTPS, IIS-based auth for the entire application and authentication outside of silverlight, but for b2b scenarios I suggest consider using messagebased brokered authentication with X.509 certificates with certificates the certificate issued by a commercial certificate authority. Now for the deployment diagram, a little costly but I believe this is the best. First there is the hardware firewall which will handle DDOS attacks, TCP flood, Malformed Packets efficiently as there is a dedicated processor in Hardware Firewall that handles all the filtering. If i use just Windows firewall and too many malformed requests come in, my Web servers CPU will be too busy saving me from those attacks then doing the real job like running my .NET code.My fire wall connects to a router or switch which has load balancing capability that evenly distributes traffic to my web servers. I had thought of firewalls with load balancing capabilities built in and with enough NIC to connect all my web servers but finally decided on the present architecture as i feel every device should do its own wotk and also this is perfect if the site is a hit and requires scaling up. I have seen many networks and web sites which have gone down for hours even a day because they did not have a backup, so I suggest each device should have a backup. So if the firewall goes down or has to be patched up, my backup can do its work.Note, the physical architecture is two-tier,the decision to have two tier architecture is due to speed. Since we don't store super critical information, we don't need to worry about Security as much as we would do if we were building a financial application.The servers should always run 64 bit windows as otherwise you cannot fully utilize the 4 GB RAM or more than that.The 64bit version of .NET framework is stable enough to run heavy duty applications. Some people have had bad experience running 64bit Windows on their personal computers, but 64bit servers are pretty solid nowadays. Web server layer contains three web servers in load balanced mode. Each web server hosts the exact the same copy of the code and other artifacts of the application that we have.Also our main users are coming from RIA and external systems which are any way going to be connected to the service layer but even if we only had a web client application, I would still do the same as a separate application layer has been proven to be a bad practice for high performance websites particularly the ones developed using ms technologies.even if you say that the webserver has no idea of the sql server,in modern applications, almost all operations are exposed via services. There's very little ad-hoc SQL query. So, this means, if someone can compromise the web tier, all the service methods are exposed to hacker and calling those service methods are not more complicated than calling SQL Server. IIS generates humoungous sizes of logs, and also we have the application logs been genrated by logging application block, my suggestion is to keep a large amount of space in the drive where the app is located as might need to store several weeks worth of IIS logs incase our internal systems to move those logs to somewhere else for reporting gets broken. Now the webservers are also connected to the internal router. the db servers sit behind it. I know that some people would say that a firewall is required to keep the webservers in dmz but my exp is that this firewall becomes a bottleneck for all traffic between web and database servers. What I do is use a router and open only port 1433 to pass anything through the router from a web server to any DB server.I have been told by many security experts that if you can hack and get the web.config,everything else whether dmz or anything else is of no use.I have suggested windows clustering witha active /passive cluster but if the site scales up, we can make it 2 active /passive cluster. I have also suggested SAN for the main db data as windows clustering needs it(very costly i know, but if the site is big just worth it) but only local raids for backups and reporting data. Finally I have suggested MDFs and LDfs kept in spearate disks with RAID10 for storing MDFs where the read data is normally kept and RAID1 for LDFs which contain the high write scenarios.