Monday, April 23, 2012

Lies, Damn Lies, and (Shaikha Mai's Use of My) Statistics

For those familiar with my formal research on Bahrain, you will know that in 2009 I conducted a mass political survey of the country as part of my dissertation. Among the products of that survey is an estimate of Bahrain's demographic composition (i.e., Sunni-Shi'i "balance") that is far more accurate than those available previously, which are based on decades of extrapolation or simply pure conjecture.

Well, it has come to my attention that this result of the study has been put to use in an "academic" article by none other than Bahrain's Minister of Culture (and the subject of a recent run-in with Bahrain's Sunni parliamentarians) Shaikha Mai bint Muhammad. The paper in question is titled "Notions of Identity in the Bahrain Unrest of 2011" (.pdf here), and it appears in a journal with which I am unfamiliar, Turkish Policy Quarterly. According to the journal's website, the article has been downloaded more than 400 times since being published on March 3, 2012.


The statistical results of my survey make it into the third paragraph of the paper, framing indeed the entire discussion. The point of this introductory section is, apparently, to dispel the "simplistic media myth[s]" surrounding the issue of Bahrain's sectarian demographics. Through these "popular media presentations," Sh. Mai observes, Bahrain is portrayed as "a divided society in which a minority Sunni power base rules over a Shiite majority," whereas she notes that "this has never been statistically proven."


Enter the findings of "an American researcher" (i.e., me):
"Interestingly, however, a recent population sampling of 500 Bahraini households by an American researcher found that the Sunni-Shiite split was 57.6 percent-42.4 percent."
Thus, the larger context of the discussion as well as the presentation of the actual figures clearly suggest that my survey found, contrary to popular belief, that Sunnis in fact are a majority of the population in Bahrain, at 57.6%, with Shi'a comprising a 42.4% minority.

The only problem, then, is that this is exactly the reverse of my published findings, which is obvious enough from the graph below. The original blog post where this graph originally appeared notes explicitly (as does the full discussion in my dissertation),
"As for the findings regarding Bahrain's Sunni-Shi'i balance, then, we see this below. Bahraini Shi'is comprised 58% of my survey sample, Sunnis 42%."

Since Sha. Mai clearly did not (or chose not to) pay attention to the analysis of Bahrain's religious demographics found in my dissertation, it is likely too that she missed the following discussion, which incidentally involves her role in disseminating misinformation about Bahrain through the publication of "scholarly" books and articles.

An except from pp. 57-58, on Bahrain's suppression of scholarly works (e.g., Khuri) that conflict with the government's official "history":
The second most significant volume on the modern political history of Bahrain is undoubtedly the journals of Charles Belgrave, the British officer who served in the position of personal “advisor” to the Bahraini ruler for some thirty-one years between 1926 and 1957 and who eventually came to be known simply as “المستشار‎”—“the advisor” (Khuri 1980, 110). In the wake of British intervention just three years prior to replace recalcitrant ruler ‘Isa bin ‘Ali with his son Hamad, Belgrave’s appointment was meant to provide Bahrain with some measure of political continuity. At the same time, he was charged with finishing the task of modernizing the whole of the country’s outdated bureaucracy, an initiative that effectively spelled the end of the prevailing feudal estate system and one therefore strongly endorsed by ordinary Shi‘a but resisted by the ruling and wealthy elite. Belgrave’s diary, then, consists of detailed daily reports of meetings and conversations with the ruler and various state officials, observations on Bahraini society, and other quotidian affairs.

Like Khuri’s, this important work too is banned inside Bahrain. Or, more precisely, while selected excerpts from the diary were published in 1960 and again in 1972, the original papers are said to reside in the royal library and in any case have not been made available. Unauthorized copies of the diary somehow made their way onto the Internet in June 2009, however, and were thereafter translated, made to be published, and imported for distribution by the Bahrain Centre for Human Rights (BCHR), itself outlawed. The printed copies were subsequently confiscated by the Ministry of Information, which informed the publisher of the government’s decision to ban the book, barring any further imports. But the damage was already done. The leaked version has persisted in the form of a massive, 2,302-page electronic document that has become required reading for the country’s political opponents, having been viewed over 7,000 times in but three years. Indeed, upon hearing of my study of Bahraini politics, my contacts repeatedly directed me to the text as a sort of prerequisite to any understanding of the local political situation.

Yet for all the controversy surrounding the Belgrave diary, and despite its constituting perhaps an embarrassing intrusion into the private lives and court politics of the ruling family, there is nothing in it that could be considered a direct attack on the Al Khalifa or that substantiates any heretofore unknown wrongdoing that must be covered up at all costs. No, the papers of Belgrave, like Tribe and State in Bahrain, are banned not for their contents per se but for what they represent: information, and more importantly the contradiction of the “official,” sanitized version of Bahraini history that the regime has worked hard to construct. Curiously, the same Minister of Information [Sha. Mai] who enacted the Belgrave ban has herself published at least two separate editions of translated excerpts from the journals, a coincidence that has led the BCHR to speculate about the completeness and accuracy of the latter volumes, and ask whether this might not help explain the ban on its own, presumably less abridged Arabic translation.
So, Your Excellency, please stop manipulating the results of my research for your own purposes. Thanks. And, to the editors at Turkish Policy Quarterly, you might want to issue a correction (or retraction) for this dubious piece of "scholarship."

Update: Thanks to The Guardian.

16 comments:

  1. Its alright Justin. Nobody with any sense would ever take an article written by an AlKhalifa about the AlKhalifas seriously. It's rather amusing to watch them scramble. The Shia Sunni think IMO is less relevant than the main aspects of democracy - you know, the bit that says that a single family cant be in charge of a country

    ReplyDelete
  2. As an "academic scholar", you should probably be aware that a sample size of 500 is hardly represntative of Bahrain's 600,000+ population.

    ReplyDelete
  3. As an anonymous troll on the Internet unfamiliar with statistics or survey methodology, you should know that a sample to population ratio of 500 : 600,000 (or 1 : ~1,200) is extremely high for a mass survey. To achieve the same ratio in the United States, for example, you would have to interview ~250,000 people. Certainly, it would have been nice to interview even more, but given that it's the first-ever mass political survey of Bahrain, and had to overcome many practical difficulties, I'd say it's pretty good.

    In any case, I have an entire post devoted to the sample, where you'll find that it's quite representative. Why not post there if you still have complaints?

    http://bahrainipolitics.blogspot.com/2011/03/depicting-division-sample-design.html

    ReplyDelete
  4. I have to say after hearing this , and reading the paper presented above , I lost all respect I had to sh. Mai , I had read some of her books before and had some respect to her academic nature , but I guess this pretty much shoves it down the drain.

    ReplyDelete
  5. It seems to me pretty obvious that this is just a simple error. At no point does she press the point that the data is the reverse of the popular perception. The key point that she is making, is that the two numbers are reasonably close together.
    Easy with the histrionics.

    ReplyDelete
    Replies
    1. I fail to see how data demonstrating that Shi'a comprise around 58% of Bahraini citizens is "the reverse of the popular perception" that Bahrain is "a divided society in which a minority Sunni power base rules over a Shiite majority?"

      In that case, what irony that in endeavoring to correct "simplistic media myth[s]" and inaccurate "popular media presentations" the paper proceeds to do precisely the opposite, offering (at best) a highly ambiguous or (at worst) a deliberately misleading alternative.

      Delete
    2. Justin, you misunderstnd my point, my apologies. The popular (and no doubt correct) perception is that Shia are the majority group. In her writing she has mistakenly transposed the two figures. All I am saying is that it is clear that this was an accident because she has not sought to express (in words) this point - which would be contrary to public perception and worth remarking upon if that was her intention.

      Delete
    3. I can agree with your interpretation until the sentence following that containing the statistics: "Most importantly, this simplistic media myth ..."

      If, as you point out, the popular (and correct) perception is that Shi'a are the majority group; and these statistics were meant to demonstrate this (though the group names were accidentally inverted), then what exactly is the "simplistic media myth" alluded to in the very next sentence?

      You're correct in saying that the connection is never made explicitly, but not only the inversion of the two numbers but also the larger progression of the paragraph gives the same impression, namely that contrary to popular mythology in fact Sunnis are the majority in Bahrain. This would not be the first time that such a claim is put forward.

      I will add finally that the paper was pointed out to me by someone having arrived independently at the same conclusion, who sent me the following (unsolicited) e-mail:

      "See what they did to your stats..."

      Two idiots of course does not a genius make, but I think the conclusion that this was a "mistake" is much less obvious than you would have it.

      Delete
    4. Justin, I feel your angst and frustration but I just don't see it the way you do. The 'simplistic media myth' alluded to (in my opinion) is that of 'a divided society in which a minority Sunni power base rules over a Shiite majority'.
      Personally, I think the key word in the expression 'simplistic media myth' is 'simplistic' whereas you have focused on the other two words and have (not unreasonably) connected this with the use of your statistics as though they are being used to explode a population demographic 'myth'.
      We all know that there are more Shia than Sunni, I don't believe even Shaikha Mai would attempt to argue otherwise.
      Anyway, we're down to semantics now and I guess we'll never know one way or the other!
      It has been good to correspond with you, but I think our positions are established and I have to get back to work!

      Delete
  6. Indeed. Thanks for the comments in any case.

    ReplyDelete
  7. Hi, is there a way to get a copy of your dissertation? I would love to read it!
    - Alya

    ReplyDelete
  8. Justin, whether it's a "mistake" on Sh. Mai's part or not, it is definitely a significant misrepresentation of your findings. Have you considered requesting a correction from the editors of the journal in which her article appeared?

    ReplyDelete
  9. Just noticed that you did address the journal editors in the post, but I meant getting in touch directly and in formal/professional mode.

    ReplyDelete
  10. Even if "shaikha" Mai's point had been merely that the disparity between the Shia and Sunni sectors is negligible -- or at least less than commonly perceived --, transposing the two figures is a serious error. It is one of the most salient statistics in any discourse pertaining to Bahrain. Regardless of whether Mai's thesis was predicated on the actual ratio or not, making this kind of error is AT BEST extremely sloppy, negligent and unscholarly.

    I suspect, however, that this was far more deliberate, which should surprise no-one familiar with the Khalife propensity to disseminate outright falsehoods through the mass media.

    ReplyDelete
  11. This may or may not be an appropriate request, but I recently was doing some research on demographic changes, particularly as they relate to percentage changes. I had noticed some apparent discrepancies and as luck would have it, one of those discrepancies pertains to Bahrain.

    In short, various sources seem to provide conflicting numbers related to the number, and percentage, of non-Muslim religious minorities as they relate to the total population of Bahrain.

    Any thoughts?

    ReplyDelete
  12. I'll make this short..fifth try.

    Do you know why various sites seem to have conflicting numbers pertaining to the percentage of non-Muslim religious minorities as those numbers relate to Bahrain?

    Hope you consider this comment appropriate.

    Thank you.

    ReplyDelete