Fact-checking FactCheck: Antisemitism in the Labour Party

JVL Introduction

Channel 4’s Factcheck is generally highly respected.

So it is distressing to report that its analysis of antisemitism in the Labour Party is open to serious question.

Rory Allen, who has taught statistics at Goldsmiths, University of London, and is the author of a textbook on statistics for psychologists has recently submitted a complaint to Channel 4.

He is awaiting a reply.

Rory Allen writes:

In a report by Georgina Lee dated 31 October 2020, available here you analyze Jeremy Corbyn’s claim that “The numbers have been exaggerated […] the public perception in an opinion poll last year was that one third of all Labour party members were somehow or other under suspicion of antisemitism. The reality is, it was 0.3 per cent of party members had a case against them which had to be put through the process.”

In the fifties there was an amusing and popular book by Darrell Huff entitled “How to lie with statistics”. I was reminded of this book when I read your analysis. The problem, as Huff would be quick to point out, is that it makes one of the classic errors, that of using the wrong measure to summarize a set of data.

In your analysis of Corbyn’s comment on public perceptions, the key passage is where you look at the Survation report and conclude that it showed that a typical response to the question about the prevalence of antisemitism allegations in the Labour Party, was “0 to 9%”. You said this showed that the report was entirely consistent with Jeremy Corbyn’s claim that such allegations concerned only a small percentage of the Labour Party membership. Your verdict was that there was no exaggeration in public perceptions of antisemitism in Labour, and that Corbyn’s claims to the contrary were therefore factually wrong.

This conclusion rests on the assumption that “the most popular figure” in the Survation report tells us something important about the public response as a whole. I am surprised that any academic statistician – and you say you consulted them – did not put you right about this.

The point is a slightly technical one, but given that your stated purpose is to get the facts right, I am sure you will not mind a detailed explanation. There are three ways of calculating “typical” values for a set of numerical data: the mean, median and mode. Your comment referred to the mode, the “most popular” figure in a set of data.

Why is the mode a bad measure to use? I will illustrate the point by looking at the age distribution of the Mexican population. If you were to give a single figure for the ‘typical’ age of Mexicans, what would it be?

First look at the data below, which gives the numbers (in millions, and by gender) of Mexicans, in each five year age range, for 2018.

The mode, what you call the “most popular” value, for age, is the range from 0 to 4 years 11 months. So to apply your own logic, a typical Mexican would be somewhere between a babe in arms and a nursery school infant. This is clearly misleading. A much more informative figure would be the age such that half the population is younger, and half older than this age. This is what statisticians call the “median”. The median age, which works out to be around 28, gives a much better impression of the ‘typical’ Mexican. This is why statisticians don’t in general recommend using the mode for numerical data.

The Survation report quoted the mean estimate of antisemitism, a figure of 34%. You also cite this in your analysis, but suggest that it is unreliable. There is indeed a case against using the mean in data of this kind where the distribution is “skewed”. I suggest almost all statisticians would in such a case recommend using the median.

If you look at the spreadsheet giving the Survation data, which is available from Survation’s website, it shows that of the total of 1009 people sampled, a majority – 570 – either did not answer the relevant question or replied ‘don’t know’ to the question. 438 out of 1009 did attempt an estimate for the percentage of Labour Party members who had had complaints of antisemitism raised against them, which I will call the ‘prevalence percentage’.

The median attempts to answer the question: what is the percentage such that 219 out of the 438 estimated the prevalence percentage to be less, and 219 estimated it to be greater than this? From the Survation website data, we can see that 154 – significantly less than half – estimated the prevalence to be between 0-19%. 235 – just over half – estimated it to be between 0-29%. So the median lies somewhere between 20% and 29%, and closer to the latter. Using a standard linear extrapolation method, we can estimate the true median to be just over 28%.

The second half of Corbyn’s comment refers to the true figure for complaints of antisemitism against Labour members. You point out that there are no reliable figures for this, because Labour record-keeping under Ian McNicol was inadequate, and you do not provide any evidence from the EHRC report: indeed, your references to the report suggest that no data can be gathered from it. However, this is not quite correct.

It is true that the report only investigated complaints about Labour Party office holders, because the Party was only legally responsible for the actions of its office holders. However, the EHRC investigation had been public knowledge since May 2019. It gathered information for many months, ample time for anyone who wished to, to provide evidence and make a complaint about antisemitism concerning ordinary members of the Labour Party. And though the EHRC report did not examine such reports in depth, it did provide the number of them, as seen in this extract from page 16:

This investigation was prompted by complaints made to us by Campaign Against Antisemitism (CAA) and the Jewish Labour Movement (JLM) in the summer and autumn of 2018. These complaints provided evidence of acts of antisemitism in the Labour Party, and of the Party’s handling of antisemitism complaints. The documents they provided to us included information about more than 220 allegations of antisemitism within the Labour Party, dating back to 2011. Complaints focused mainly on the use of social media by members, but also included political discourse in the mainstream media and behaviour at events and meetings.

This is not conclusive, but it is not an unreasonable assumption that at least the majority of victims of antisemitic abuse would have registered their complaints with CAA or JLM. It is of course possible that one allegation might refer to multiple members of the Labour Party. On the other hand, the allegations referring to ‘the use of social media by members’ rely on the complainants being able to identify social media users as Party members. As many people comment anonymously on social media, we cannot be sure that these were, in fact, all legitimate members.

Anyone with the slightest knowledge of the internet will be aware that it is infested by “trolls”, of which a significant number are people whose main intention is to stir up dissent. Dame Margaret Hodge has written of being subjected to a “torrent of relentless abuse”, and of putting in “over 200 examples, some vile, where evidence suggested they came from Labour.” But we do not know how conclusive this “evidence” was, since Dame Margaret has not shared it publicly. It may simply have been that the comments were made on social media regularly used by Labour Party members, in which case troll activity is at least a possible alternative explanation.

In February 2019, by which time the Party’s record keeping had improved, the Labour Party reported that it had received 1106 specific complaints of antisemitism since April 2018. On investigation, it found that of these, many did not refer to Labour Party members. It reported that 673 reports did refer to Party members. Out of these 673 reports, we are not told how many of them concerned multiple complaints about the same person, so it is possible that the number of individuals complained against was rather less than 673.

So we have two figures for complaints against Party members as such: “more than 220” from the EHRC report, dating back to 2011, and a maximum of 673, covering a period from April 2018 to February 2019. Allowing for the fact that both these figures may be an underestimate, and that many incidents of antisemitism may have gone unreported, and that complaints made prior to April 2018 may have simply disappeared, they do at least allow for an order of magnitude estimate of the number of antisemitism complaints made against Party members. It seems reasonable to suppose that an upper bound for the number of individual members subject to complaints of antisemitism would be around 2000. During Corbyn’s leadership, total membership was around half a million. This would give a rough estimate for the percentage of complaints at 0.4% of the membership, as an upper limit.

So from these facts, it seems reasonable to conclude the following: “we can estimate that a large proportion of the population – nearly half – has heard of accusations of antisemitism against members of the Labour Party, and has some idea of their prevalence. A typical estimate for this prevalence is 28%, more than a quarter of the whole membership. The reality is that only around 0.4% of the membership, at the most, has been subject to such complaints. It appears that a large proportion of the population has an exaggerated view of the prevalence of antisemitism in the Labour Party.”

Allowing for his ignorance of statistics, which is after all shared with the overwhelming majority of his political colleagues in all parties, it appears that Corbyn’s comments were therefore justifiable, on the basis of such evidence as was available to him. Your conclusions, on the other hand, were incorrect. May I ask that you publish a retraction?



Comments (24)

  • goldbach says:

    I wish Rory well but would not expect him to get anywhere because:
    1. Virtually no journalists understand statistics.
    2. Most have no idea of the difference between statistics and data.
    3. They don’t understand that they don’t know what they are talking about.
    4. When a statistician explains things to them they can’t follow the explanation.
    5. Those who do understand often deliberately misuse data for political purposes.
    Remember, there are liars, damned liars and those who misuse statistics (or to be more accurate, those who misuse data).

  • Philip Ward says:

    I wish Rory well, but I feel that is is necessary to point out that it is likely that the majority of the complaints enumerated in his report were vexatious, coming as they did from ardent defenders of Israeli Apartheid, in the form of the CAA, JLM and Hodge. This is the meaning of the rest of Corbyn’s statement that the numbers were exaggerated by particular forces for political purposes. You only need to look at the CAA web site (if you can bear to) to see its vile abuse of opponents of Israeli oppression.

    I have just heard that tonight that the Charities Commission is going to come down like a tonne of bricks on the National Trust for pointing out that large sections of the aristocracy and bourgeoisie got its wealth through slavery and exploitation of the working class. This, apparently, is “rewriting history”. You are not allowed to tell the truth in this new, Orwellian world.
    The CAA, being a “charity” is seemingly under no such threat, despite having carried out a five-year-long campaign against the 500,000 members of the Labour Party.

  • Tim says:

    Well done for submitting the complaint and I hope you have a positive response. Channel 4 want to have their cake and eat it. Rory Allen & Prof Greg Philo have explained why the available numbers confirm Corbyn’s assertion of the exaggeration of the number of antisemitic cases within the Labour Party compared to the reality. Also, with the conflation of anti-Zionism and criticism of Israeli policies with real antisemitism, exacerbated by the unwise adoption by the Party of the IHRA working examples, the actual real number of cases could be even less. I am not aware of the mainstream media having any objective numbers to back up their hyperbolic and feverish reporting of the “antisemitic crisis” within the Labour Party, and it’s interesting that Channel 4 are now trying to prove that mainstream media is having no impact on public perception/belief – incredible!

  • Julian says:

    Yes, but matters are much worse than that: we have important and powerful people who completely understand, and carry on regardless because they have no interest in the ‘truth’ – there is a ‘special hell’, we might hope, for such people (but I don’t believe there is, of course, unless we make it so)

  • Naomi Wayne says:

    Elegant and beautiful! Perhaps basic stats should be included in Labour’s educational programme. . .

  • Sean O'Donoghue says:

    Thanks Rory, great analysis. Like goldbach above, I doubt if your rather excellent work will ever see the light of day…perhaps OFCOM

    One of the problems with this is that all we have is number of people “complained against”….I was one of those and would be amongst the 2000. But the number of zionist headhunters around can put as many complaints in as they like and the complainee is automatically disqualified and the number of complaints registered is increased, thereby increasing the public perception of actual antisemitism…the no smoke without fire theory

  • Linda says:

    But Fact Check’s public reputation rests on the programme reporting accurately what the facts are …. I think there’s a reasonably good chance the editor, presenter and researcher(s) would respond honestly and appropriately to Rory’s criticisms.

  • Interesting. But if you are a straight white member of the LP – you are probably one of 400,000. If 700 people are having a go at your group you probably don’t notice it that much. But, if you are a diverse member – say you’re LGBT+ you are probably one of 2,000. If 700 other party members are having a go at your group you will feel threatened and scared.
    The hate crime/phobias in the party appear to me to be seriously underpinned by misogyny: the trolls seem to target black, gay, Jewish, Muslim and trans women…

  • Hazel Seidel says:

    I imagine that the BBC Factcheck people also have a good grounding in statistics. I find it amazing that Rory Allen finds it necessary to teach them something as basic as the difference between mode, median and mean, even if the result of the analysis does come close to the figure used by Corbyn in his extremely ill-judged statement. Also that he ignores the fact that getting on for two thirds of people ignored the question or said they did not know. This surely suggests that the issue of Labour antisemitism had not penetrated the consciousness of some to any extent. The truth is that it is hard to draw any conclusions at all from this survey. Yes it seems likely that the media have caused people to overestimate the prevalence of proven cases of antisemitism in the party. There is a far larger group who persistently ignore or deny antisemitism within Labour, most of whom have not been dealt with, and who, if my own CLP is anything to go by, have been distressing to Jewish and Jewish-heritage members.

  • sean clarke says:

    excellent work and a forensic reply to the untruths perpetuated constantly by a biased media,keep up the good work.

  • David Bull says:

    An interesting and illuminating report showing, at least, sloppy journalism…..

  • Catherine Hutchinson says:

    Channel 4 fact check has for some time been a source of misinformation about Jeremy Corbyn. Glad to see a knowledgeable response which I hope (without much hope) they take seriously.

  • Eleanor Gordon says:

    Can we be updated on whether or not Channel 4 respond, or indeed, if there is anything that can be done to put pressure on Channel 4 to correct the misrepresentation?

  • Jan Brooker says:

    Oh. I fully expected this article to be by Dr Alan Maddison! Welcome Rory Allen. I have posted the above on widely, with Jonathan Cook’s observations, in an attempt to get some *facts* ‘out there’. Coupled with Chomsky’s extract about *shaping the narrative*: https://chomsky.info/consent01/

  • Dawn Smith says:

    Could a flow chart be done illustrating this article? It might help to clarify the point.

  • Jan Brooker says:

    Hazel Seidel
    “There is a far larger group who persistently ignore or deny antisemitism within Labour, most of whom have not been dealt with, and who, if my own CLP is anything to go by, have been distressing to Jewish and Jewish-heritage members”. *FACTUAL ANALYSIS vs. HERESAY*. JVL seems to be the largest representative group of left-wing Jews within the LP; this sort of fact-free contribution doesn’t really add anything to the issue or debate. I have yet to hear from LP Jewish members [without a factional axe to grind] that backs up your comment.

  • Christine Ladyman says:

    I strongly suspect, that channel 4 is now in defensive mode. Since they were threatened with license revoking for their honest reporting, they have, in recent times, become far less critical of the government, and unlikely to support such truths as we know exist in the Labour Party. There needs to be a very loud and clear exposure of the leaked report for a start.

  • Alasdair Mathers says:

    Thanks for the detailed analysis . So it seems right then that public perception and facts are at variance, which is what the former leader said . No justification in that for suspension , whatever other justifications there might be thought to be .

  • John Hall says:

    The simple riposte to exaggerated claims of anti-semitism is that many were simply that many such claims were merely about anti-Zionism, (a mainly Christian movement).
    Until this is acknowledged you can use all the clever explanations you want and it will just be water off a duck’s back

  • Glen Shakespeare says:

    I was suspended for alleged antisemitism. They sent me the evidence they had against me via email. All of this so-called evidence was valid criticism of Israeli actions against Palestine. So Southside is being manned by staff so stupid they don’t even understand the meaning of antisemitism.

  • goldbach says:

    I sympathise with the position taken by Hazel Seidel and with the position taken by, at least some of, those she identifies as having been in denial.
    There were, undeniably, some in the party who did post offensive comments. There was also, undeniably, a campaign by anti-socialists to create a climate where members would split into “camps” and internal warfare would break out. Those whom Hazel sees as having ignored or denied “antisemitism in the party” may, like me, simply have never witnessed it either in person nor in the antisocial media and their protestations would, likely be seen as ignoring or denying antisemitism. They would then be likely to see those who say that they are ignoring or denying antisemitism as doing so for their own factional purposes.
    “Camps” are formed. Internal warfare breaks out. The anti-socialists have achieved their objective and we are fighting each other.
    I had hoped that, once Mr Corbyn had resigned as leader and a biddable leader had been installed, everything would calm down and we could get on with putting forward Labour’s modest socialist policies. It was not to be. The right did not let up. There is now no alternative but to fight the good fight and be as awkward as possible. I hope Hazel is on board.

  • Julia says:

    Very helpful. Including generally about the difference between the median and the mode, which I realise I had been muddling up for years.

  • David M says:

    Some further information on the Survation survey. Although 0-9% was the estimate chosen by the largest number of individuals, those individuals made up only 20% of those who chose a figure. In other words, 80% thought the figure was above 10%. Even if the actual figure were 10 times higher than the estimate of 0.2%, that means 80% of the public have an exagerrated view.

  • Bernard Grant says:

    The fact that the public’s estimate was so high, is easily explained, the MSM were at fever pitch in their efforts to stop a Socialist Government, lead by a True Socialist. It started as soon as Corbyn became leader of the Party, lasting right up to the 2019 election.
    The accusation of endemic AS in the Party was their top propaganda issue, even lying to their readers ie, that Corbyn visited the grave of a terrorist, who was buried in a completely different cemetery.
    Even though AS has been shown to be more prevalent on the Right and less on the Left and even less in the LP.
    This nonstop propaganda in the Rightwing Press, would have persuaded the public at large that the Party must have a large number of antisemites in it, so the average guess of 28%+ of Members being AS shouldn’t be surprising. Add in the Panorama programme, which, if Starmer had let the trial go ahead, especially as the LP Lawyers said, those bringing the case would lose, that would have shown the charge of endemic AS in the Party to be nonsense.

Comments are now closed.