Survey design problems torpedo Universidad de Oviedo's Faculty of Psychology ability to understand societal attitudes towards trans women / trans women / TIMs in Spain

Last week, the Universidad de Oviedo, a public university located in the capital of Spain’s region of Asturias, published a survey for participation by the public about society’s attitudes towards “trans women” or “mujeres trans” in Spanish and their involvement in the feminist movement.  Underlying design problems mean the survey will never be capable of answering the questions it asked, even if every person in Spain participated in the survey.

Unsurprising to many, the reason such a survey was likely written was the topic of trans women, the direct translation of the Universidad de Oviedo’s study title for a specific group, and their involvement in feminism is a highly contentious one, and one that may very well impact the outcome of regional Spanish elections taking place on 23 May 2023 and national elections that must be scheduled before 10 December 2023.  The issue has caused massive fractures inside the Spanish left, particularly within PSOE who currently led a coalition government but are expected to lose in December, with currently polling suggesting a Partido Popular led government coming into power in coalition with Vox.  These splits were most noticeable when openly abolitionist[1] feminist Vice President Carmen Calvo voted against party discipline and voted no to self-identity for gender/sex for biological males to identify as women.[2]  It is an important topic to explore, especially given the electoral implications and the flow on impact for women’s rights a whole.

Sadly, this survey appears to have been designed by people with little understanding of best practices in research methodologies and best practices in survey design.  Without going line by line through the seven demographic questions and the twenty-seven specific questions related to trans women / transwomen / TIMS, their major mistake is their survey took a specific ideological perspective, in this case queer feminism / transmaribollero feminism[3], used these definitions as their starting point, defined terms in ways that were controversial or not agreed upon by most people, did acknowledge that there is a gradient in people’s views in society despite existing research saying there is, asked questions from an ideologically loaded perspective and asked questions using a negative form, which also has a huge impact on how people answer survey questions even if the other problems did not already exist.

In an abortion rights context, another controversial topic, the questions asked by the Universidad de Oviedo would be akin to asking things like starting off your survey with the statement, “Babies are humans at any stage of development up until the second born” and “Gestational carriers are the biological vessel which houses a baby until the point small human is capable of breathing on its own outside biological material that created it”. If you’re pro life / opposed to abortion access / do not believe that women should have the ability to control their health decisions that may result in their death, sure, these are fantastic definitions that clearly define your beliefs.  If you do not, then such a survey is likely to offend the hell out of you to the point where you either will self-select against participating or you will be unable to answer a number of questions in such a way to match the author’s intent in asking because of basic problems understanding their language.  These problems were then compounded by asking like, “Do you not support murder of babies?” Ummmm.  I mean, murdering babies is bad.  I don’t think babies should be murdered.  Do I strongly agree with this statement, agree with this statement, neither agree nor disagree with this statement, disagree with this statement or strongly disagree with this statement?  If you’re someone who supports a woman’s right to choose, that question is very hard to answer and putting it in the negative, an atypical survey design question known to result in bad responses, is makes it harder, especially when you do not have the ability to explain something like, “I support the right of women to have elective abortions up until the final trimester, at which point I think women should only have the ability to have an abortion if her life is in danger.”

A good survey addresses these issues, tries to frame things in ways that do not result in leading answers[4], because it doesn’t want to confirm an answer but to understand a problem / concern / issue.  It fails on the very basics.

First, what is “trans woman” / “mujer trans”? What is a “cis woman” / “mujer cisgénero”? That’s a hugely loaded question, which is why I got into a small bit of a Twitter fight asking a few Spanish radfems the first one.  Why?  There is, according to a certain perspective, no such thing as a “trans woman”.  There are merely men, who claim to be women. This perspective says that these are not a special subset of men, and should be viewed no differently as a specific subset of men as men who drink cappuccinos.  Now, try to ask this particular group, a group who represent a historically important segment of PSOE’s voting block, to answer a question that consistently uses the phrase “trans woman” / “mujer trans” questions about access to women’s spaces. There is a huge difference between asking if you agree with these two statements in a survey about “trans women”:

· Women’s sports should accept only cis women who have a gender identity as a woman; and

· Women’s sports should exclude men.

In theory, these questions ask the exact same thing. There is also a huge different between asking if you agree or disagree with the following statements:

· Trans women should have fewer occupational opportunities than cis women.

· Men should be excluded from certain employment opportunities that involve providing intimate services to biological women and from jobs involving physically and intellectually vulnerable women in intimate care, such as bathing them, assisting them in using the toilet / bathroom and performing reproductive medical care.

A third example from this study:

· Health services for women should exclude trans women.

· Transwomen and men should be denied access to surrogacy / womb rental / child selling services, whether financed by the state through public health or done privately.

Again, from a research design point of view, these three sets of statements ask essentially the same thing but from specific ideological perspectives.  I can hear all my acquaintances with trans friends in the USA howling with rage over the second framing and all my rad fem acquaintances howling with rage over the first, with both sides insisting that neither is ideologically frame and both are based on fact, with no concession being required about the use of language.  That’s not particularly relevant, because this is a survey designed to capture everyone’s views and opinions, not just one particular ideological one[5].

With this particular topic of “trans women” or “biological men who identify as women”, like the abortion topic, there is also a gradient issue that has been well documented in research related to public opinion around this topic. That involves the state of transition. This, unsurprisingly, often involved people who are less ideologically minded and to a certain degree less informed about the topic. This issue involves the state of gender transition. A good example statement where this comes into play would be, “Trans women / transwomen / men who identify as women should be allowed to use publicly accessible toilets designated for women.”  The simple model of, “I agree” and “I disagree” do not quite work because we have not defined clearly what “Trans women / transwomen / men who identify as women” are.  The best responses to indicate feelings for such a statement would be something like this:

· I agree with the statement as written

· I agree only if the trans woman has had gender affirming surgery like removal of the penis and the creation of a neovagina

· I agree only if the trans woman takes hormones or has had gender affirming surgery

· I agree only if the trans woman identifies as a woman and has taken steps to socially present themselves based on how they dress, their hair, their makeup and through other use of symbolically female items

· I disagree with the statement as written

· I do not know / I have a different opinion

Such a response would allow for better understanding of how society is defining “Trans women / transwomen / men who identify as women” and if a person’s transition type or process or transition stage should be a factor in decision making related to these issues[6].  Studies need to account for differences in understanding of what a “trans woman / transwoman / man who identifies as a woman” are as they are not universal.  A survey is flawed when it does not ask this because of the lack of a shared definition, and “Trans woman is a man who identifies as a woman” is not good enough to capture the four major, generally understood possibilities related to this.

While on it, in addition to those six options, you also realistically need two more, “Trans women are women” and “Trans women are men” to cater specifically to ideologues who are taking part in the survey.  In addition, you also need to switch from a purely quantitative analysis to a mixed methods of quantitative and qualitative to allow for better understanding of these gradients and the impact this has on policies regarding this specific population.  The lack of commonly understood definitions and the ideological nature of the topic demand it.  The study by Universidad de Oviedo does none of this.

So how do you actually write good questions to genuinely capture public sentiment?  First off, you can’t and second off, you shouldn’t be looking to find out about everyone as if they were a monolithic population and as if Spain or the United Kingdom or the United States of America had a Borg like hive mind on the issue. You need to be looking at what specific subsets of the population are doing and thinking related to this topic, and then look at their views against a broader backdrop of their views on near social issues.  PSOE abolition feminists, radical feminist historically Democrat voters, female homosexuals may be in agreement with Vox voters, Donald Trump, Rick Desantis, Marjorie Taylor Greene, Riley Gaines, Posie Parker, the United Kingdom’s Conservative Party, neo-Nazis that biological sex exists but what that agreement means in actual practice is very different.  Feminism In New Terms has a great essay related to this from the perspective of left-wing feminists who reject affiliation with political conservatives; whether or not such alliances should exist is another huge point of contention within the community[7].  Back to my point and away from the digression, you cannot easily do so but you should damned well try.  There are three easier routes to try to create a study on that captures those diversity of opinions:

1. You create a survey where you shovel in multiple, conflicting perspectives and terminology into the same question, while asking for participant understanding they may not agree with all the terms used.  For example, a do you agree with this statement with the various trans related gender affirming surgery options and potential ideologue responses of, “Trans women / transwomen / TIMs should be included in formal feminist debates about legal and cultural policies that impact cis women.”

2. You create a create customized survey pathways that essentially ask the same question but from the ideological point of view and using language that survey participants are likely to be most receptive to.  For example, one group may get an agreement statement of “Trans women should be included in feminists debates on legal and cultural policies that impact women.”  Another group may get, “Post op transsexual women, but not men who merely / only identify as women without further steps towards transition, should be included in feminist debates about legal and cultural policies that impact cis women.”  A third group could get, “Men should be should be included in feminist debates about legal and cultural policies that impact biological women.”

3. You splurge a lot of cash to create focus groups or you use your contact group to develop a representative sample and do a quantitative approach similar to focus groups where answers and responses can be tailored for the specific respondent.

With all that in mind, I have rewritten the Universidad de Oviedo survey.  The English language version can be found at https://forms.gle/DcFkGoi5t59rrT3b9and the Spanish language versioncan be found at  https://forms.gle/futuadxbKeGrzux5A.    It uses the first approach described. This study is less about the results than about the process of trying to rewrite such an initially flawed survey.[8]

Endnotes

[1] In an English speaking context, this sometimes has different meanings but it the dominant “gender critical” / “radical feminist” framework in Spain, and means the abolition of treating women’s bodies as capitalist commodities, including in prostitution, pornography and surrogacy / womb rental / child selling.

[2] No similar measure existed for biological females to identify as men, because this would have entailed a loss of rights and legal protection under the law for females.

[3] These aren’t quite the same thing, but that requires a deep dive into veins of Spanish feminism this piece isn’t trying to do.

[4] Unless it their intention to publish biased results in order to further a specific political agenda, which one would hope would not be the case with the Universidad de Oviedo’s Facultad de Psicología intention.

[5] Unless of course, it actually is and that is the intent of the Universidad de Oviedo’s Faculty of Psychology to conduct a survey that shows everyone is in favor of trans rights / biological men having a say on feminism / women’s rights / biological women’s ability to have single sex accommodations to address historical discrimination based on sex and the very real reality of the particular vulnerability of women to sexual abuse that is born out by large numbers of statistics and why Spain’s Podemos and PSOE did not strip transmen / women of their rights as women even when transmen  / women chose to identify as men / view themselves as men and want to be viewed that way under the law.

[6] It demonstratable do this in studies.  It is why if you frame it as, “Should biological males with fully functional penises who have been convicted of sex crimes against biological women be housed in the same prison cell as a woman who has been sexually victimized by a male rapist?” as opposed to “Should transwomen be put into women’s prisons?” gets hugely different results.

[7] As someone primarily interested in lesbian related issues, it is a huge discussion within our own communities, because participation with groups joined along the idea that the major issue is trying to win back single sex spaces often means participating in deeply lesbophobic and often times racist spaces.  This can be uncomfortable, extremely so and each lesbian ends up making their own decision around their own priorities.

[8] And I owe major thanks to my more queer feminist aligned American friend with a background in research methods and survey design processes who helped me with the rewrite, adding her valuable insights to my own on both research methods, data analysis and survey design informed by experiences with British and Spanish feminists. Thanks much. You’re the best.  It is always great to have friends who you disagree with ideologically but can still have difficult conversations with… and man, do I bring some of my friends difficult conversations on such a regular basis.

Enter your email below to sign in or become a 4W member and join the conversation.
(Already did this? Try refreshing the page!)