95 Pages Based on Nothing

The Prison Reform Trust and a charity called Unlock have produced a 95 page report about finances and prison. Basic idea seems to be that people in prison (or just released) find it harder to convince banks to trust them. Shock.

But what got me was that this long report is pretty much worthless. On page 81 in the appendix we are told how they gathered their results. Out of a prison population of 85,228 they interviewed 144 or 0.17%. Of those 97 responded through self-filled in surveys leaving just 47 who were actually interviewed or 0.06%. 24 former prisoners filled in on-line surveys and 29 families did likewise. These last two groups chose to do so after reading about it in the Unlock charity's e-newsletter.

OK so the sample sizes were far too small and obviously biased. Now get this from page 18 of the report in the Methodology section:

Statistical data were analysed on SPSS to produce frequencies and simple cross tabulations. The small size of the samples precluded tests for statistical significance.

Yet somehow enough to produce a 95 page report.

Stupidity in Published Scientific Literature

Pink News reports on a publication reporting that some adverts on Facebook are targetted at gay men exclusively even though they have nothing to do with sexual orientation. A quote from the published article [pdf] (emphasis mine):

Alarmingly, we found ads where the ad text was completely neutral to sexual-preference (e.g. for a nursing degree in a medical college in Florida) that was targeted exclusively to gay men. The danger with such ads, unlike the gay bar ad where the target demographic is blatantly obvious, is that the user reading the ad text would have no idea that by clicking it he would reveal to the advertiser both his sexual-preference and a unique identifier (cookie, IP ad- dress, or email address if he signs up on the advertiser’s site).

How does something get through peer-review stating that if someone signs up to a site with their email address they "would have no idea" that the people they are signing up to will have their email address?

One other point. The experiment run to find these conclusions involved three Facebook accounts, 2 of straight men and one of a gay man (with all other details the same). Surely that's far too small a sample for any meaningful conclusions to be drawn?

The paper is entitled "Challenges in Measuring Online Advertising Systems" published in  Proceedings of the 10th ACM SIGCOMM conference on Internet measurement conference

You Might Have Made a Mistake There

From the BBC:

More than 41,000 primary school boys (2%) have a statement of needs and 489,250 (23.4%) have unstatemented needs. This compares to 15,600 SEN girls (0.8%) with statements and 269,890 girls (13.5%) without a statement.

When you're definition of "special" includes almost 1 in 4, you've probably made a mistake.

Reminder - Scientists Are Often Wrong

Not really related to politics but I thought this article was interesting. It discusses a particular problem with statistics and inter alia reminds us that you can't always trust what a scientist tells you.

Statistics and Choices

At Left Foot Forward, Ranjit Sidhu, founder of Statistics into Decisions, discusses higher education funding in the UK. In his discussion he says:

this country is spending far lower than any other OECD country and actually half that of the USA on education.

This claim about spending half what the USA does comes from a report from the Russell Group[pdf]. That report states:

At 2.9% of GDP, the US spends more than twice what the UK does.

Earlier in the article Mr Sidhu has noticed the point about different population sizes pointing out that:

With about 5 per cent of the population of China it has an economy more than half its size.

Yet here he seems to have forgotten about that. Nor has he read much of the report, it seems. Just two lines earlier the Russell report states:

The UK spends approximately $15,447 (USD) per student, while the US spends $25,109.

By my reckoning that's 1.6 times as much not twice. But crucially, apart from not considering population sizes, Mr Sidhu has not considered the source of this funding. In fact, 66% of the funding in the US comes from private sources whereas in the UK it is 35%. Some quick maths and we find that the taxpayer pays $10,040.55 for each student in the UK but only $8,537.06 per student in the US.

So when the headline says:

We can afford to fund our universities, the fact is we choose not to 

he may be entirely correct. But he is not correct to suggest that in order to keep up with America the taxpayers should be forced to pay.