On 25th February 2021 the Office for Statistics Regulation (OSR) published some draft guidance on “Collecting and reporting data about sex in official statistics”. As regulators, it is their role to support public confidence in statistics and making sure that statistics serve the public good. Statistics are expected to meet standards of trustworthiness, quality, and value.
It’s important what they think. They decide which statistics get designated with the quality assurance mark of “National Statistics”. They can also remove or suspending National Statistics designation. It is OSR who will decide whether the Census 2021 achieves the status of “National Statistics”. They say on their website:
“The Office for Statistics Regulation (OSR) is carrying out an assessment of the UK Censuses to allow us to recommend whether the Census outputs should be designated as National Statistics when they are first released. National Statistics designation means the data and statistics are reliable and provide valuable insights; that they meet the standards of trustworthiness, quality and value outlined in the Code of Practice for Statistics”
Their intervention is also notable and timely, coming 13 days after the ONS published its controversial Guidance on how to answer “What is your sex” in the Census 2021 and just 6 days after Fair Play For Women initiated legal action against the ONS requesting it is withdrawn.
It also comes after persistent complaints from ourselves and others about the way information on sex is being collected and reported. Our most recent letter, sent on 4th February, reported the publication of National Statistics by the Home Office using misleading crime figures provided to them by the police. We specifically reported the document called “police powers and procedures, England and Wales, year ending 31 March 2020“. There is no way a data user would be expected to know or check whether the figures reported under ‘female’ also includes some males.
On 17th February we were very encouraged to receive this letter from OSR stating
“We agree that, from the information currently provided in the crime statistics bulletins, user guides or technical reports, users could not be expected to know (or determine) whether or not the figures for a given sex are likely to include transgender individuals. We think it would be helpful if this aspect of the statistics was highlighted prominently alongside the statistics to aid user interpretation. We will raise these issues with Home Office and Ministry of Justice statisticians”.
It’s OSR’s job to keep an eye on how the ONS have been setting up the Census for 2021. They’ve been alerted by data experts that the ONS was not taking their concerns seriously over sex question guidance. Back in September 2020 the OSR told the ONS to do better.
The questions to be asked in the Census in 2021 have been finalised. However, we are aware that some stakeholders have outstanding concerns with the guidance on the Census sex question. ONS has continued to engage with users and stakeholders about this and we recognise the efforts being made to provide answers or explanations on areas of concern. ONS should seek to address outstanding concerns raised by users within its further question testing and research on the guidance on the sex question. ONS should share the outcomes of this research in a transparent and open way.
Despite this, based on the new guidance published by the OSR today, the ONS and its Census appears to be failing on numerous counts. So do many other public bodies.
Here’s what the OSR draft guidance says:
As a producer, if you are currently collecting and reporting data on sex or are considering making changes to how you do so, it is important that you do so with clarity, being transparent about the reasons for your judgements and decision making throughout. You should explain to users what and how data are being collected and support the appropriate use of the statistics.
They go on to summarise their expectations of producers when collecting and reporting data about sex, under each pillar of the Code.
Trustworthiness
Producers need to understand what data they can legally collect about an individual’s sex and comply with relevant legislation, as well as considering any relevant nationally- and internationally-endorsed guidelines.
ONS have asserted that the asking people for their “sex registered at birth” in Census breaches a transgender person’s right to privacy. Woman’s Place commissioned a formal legal opinion from Aidan O’Neill Q confirming this is not the case.
Quality
- Producers should be clear about definitions or terminology they use, and these should be harmonised to be consistent and coherent with related statistics and data where possible. The terms ‘sex’ and ’gender’ should not be used interchangeably in official statistics.
Hallelujah! Something we have been shouting about for years! At last we are seeing this being taken seriously by regulators. (IPSO please take note!) This is hugely significant.
- Producers cannot always design or change administrative systems which enable public services to be delivered. But they should seek to understand the systems, including any risks and biases that may arise from the way the systems collect and categorise data. Uncertainty in the source data should be identified and the extent of any impact on or limitations of the statistics should be clearly reported. For data about sex this may be particularly relevant when considering data at smaller sub-group levels.
We and other stakeholders have made it very clear to the ONS that a small misclassification of males as female can have a large impact on some data sets. For example, if even if 0.5% of the 13,000 male sex offenders in prison were to be misclassified as female it would almost double the extremely small number of female sex offenders in prison.
- Producers must have a good understanding of, and clearly explain, the sources of data about sex they are using and how these are collected. This applies to survey and administrative data sources; both the question(s) used, and the way that these are completed can influence the exact nature of data collected and whether this is a mixture of sex registered at birth, self-identified sex, lived gender and others.
The police currently record reported crimes based on gender identity and not birth sex. They do not make this clear to users of the data by labelling figures simply “male” and “female”. This has led to the inevitable misunderstanding of what the data means and false claims asserted by the BBC and others. For example, a recent BBC documentary claims a rise in women sexually abusing children from the which the Daily Mail follows with the headline “Female paedophiles doubles”. At no point is it made clear that a significant number of these ‘females’ are likely to have been born male.
The OSR even give a worked example in their guidance using criminal justice statistics as important statistics which help the public understand the nature of incarceration in their country. They say OSR would expect producers of the statistics to:
Understand the nature of the data they are currently using to produce their statistics
Clearly state how data about sex are gathered
Clearly explain any terminology in the statistics
Clearly explain any limitations of the data and the impact these have on the statistics and their use.
Value
- Decisions about whether to continue, discontinue or adapt statistics about sex should be made in discussion with users and other stakeholders. If a change is made to data collection, or if information about a data collection practice emerges which makes it clear that the nature of the data may have been previously misunderstood, a clear explanation of the change should be published, with evidence of the rationale and, wherever possible, the analysis that informed the change.
- Collection and reporting of data about sex is a sensitive and potentially divisive topic and there may be times when producers are unable to meet the requests of everyone who has an interest in their statistics. In these cases, it is more important than ever to be transparent and open about the decision-making process and the evidence used to inform the choices that have been made, particularly in relation to any areas of contention.
It has been made very clear to the ONS that experts are concerned about the loss of data on birth sex in Census. Professor Alice Sullivan organised a letter from 80 social scientists, including 50 professors, in late 2019. The resulting engagement process has been lack-lustre and dismissive as described in this blistering public attack by data experts published in a letter to the Times.
The public servants leading this process have failed to engage in a serious and timely way with those outside government who have expertise on the collection and use of population data in the social sciences. No robust evidence or argument has been provided to support such a fundamental change. We are on the brink of losing robust, high quality data on sex in the UK. Once gone, we may never get it back.
The ONS had 10 years to get this right for the Census 2021. But they didn’t start any real engagement over the sex question Guidance until June 2020, and that was only after pressure from stakeholders to do so. They conducted last minute research in secret after telling us they weren’t doing it. Neither have they been transparent or open about the studies findings, publishing only the bare bones for peer review. And what has been shared suggests poor quality. All this culminating in the development of guidance published on 12th February 2021 which we now consider to be unlawful.
As policy analysts MBM wrote yesterday, their decision to publish a finalised version of the guidance for the sex question little more than five weeks before census day might have been designed to thwart the ability of campaigners to halt the process. They under-estimated our resolve and the strength of public feeling. They are now embroiled in a legal challenge to force them to withdraw their unlawful guidance made possible by a public crowd funder that has raised £60,000 in a week. Public trust that our national statistics are free from interference by ideological pressure groups risks being severely dented by adverse newspaper headlines, recent coverage of the legal challenge on BBC Radio 4’s Woman’s Hour, and even mention of the furore by several peers in the House of Lords this week.
Lord Hunt of Kings heath “The ONS has caved into bullying — pathetic. The head of the ONS was interviewed, I think on the “Today” programme, and said that it was not going down the path it has now chosen. However, it has caved in, as many public bodies do, because they come under aggressive attack on social media and are accused of transphobia. The Government remain absolutely silent.”
Baroness Noakes “Some organisations, deliberately or carelessly conflate sex and gender. The ONS for example has dug itself into this hole for the upcoming census. With the likely result that inaccurate statistical data will be coming from that about women.”
Lord Triesman “I note that the basis on which the ONS has decided to collect data on biological sex, or rather not collect data on biological sex in the census, now means that there are a number of leading quantitative social scientists who believe that we will have both inadequate data and an inadequate track back through data historically and we have been given almost no time to comment on the wording of the census and yet the wording of the census which guides so much social policy, and so much of our understanding of our country, and indeed should guide a great deal of our debates in this house. That will now be poorly defined and I suspect will be poorly used in policy making and I hope the minister will be able to comment on how we might rectify that problem.”
There will be an increasingly heavy price to pay for institutional policy capture. It’s a shame the OSR intervention hadn’t come sooner and with more force. If it had, perhaps the reputation of the National Statistician and his £1 billion Census could have been spared this embarrassment.
