Hidden agendas – do researchers and participants bring bias to studies?

The recent publication of our research on child health in same-sex families has gained a little media attention this week following my piece in the Conversation. I have had some wonderful feedback from members of the community who feel that this research provides valuable insights into the many diverse ways in which same-sex attracted people are raising children. But in the background the usual group of critics have set about trying to undermine our work.

One issue that has been raised is that of bias. There are many types of bias that can have an impact on research and this is one key area that academics like myself make every effort to account for. In clinical sciences the randomised control trial is the gold standard for eliminating bias, but in many population studies this is not always an achievable methodology. You can’t take a group of children, for example, and randomly allocate them to being raised by same-sex attracted parents or heterosexual parents. When bias cannot be completely omitted it is important to acknowledge its potential influence and this is something authors will highlight when considering the limitations of their work.

When researching child health in same-sex families the first problem is identifying, and then recruiting, such families. In Australia there is only one way that this can be achieved: through convenience sampling. Overseas, attempts have been made to draw representative samples from population surveys. While perhaps removing the volunteer nature of the sample these studies do suffer from small sample sizes, as the same-sex parent population within any population sample will be very small. This strategy is not an option in Australia where there are no population datasets that capture parent sexual orientation. Thus the volunteer sample is the only option. This is where the first criticisms of our study have been leveled.

It is argued that only parents with a vested interest in promoting positive health outcomes will volunteer. While this cannot be discounted as a possibility there is no evidence to suggest that this is the case. The underlying aim of the study was to characterise health and wellbeing and to understand the influence of stigma on child health in this context. As any parent will tell you what is paramount is ensuring that your children have the best health opportunities available. To intentionally corrupt research so that it cannot identify where problems may lie is doing yourself, and your children, a disservice. Only by providing an accurate picture of your child’s health will you know what can be done if there are issues. And in a context where stigma might be having a negative impact this is important to identify.

Further to this is the fact that significantly positive outcomes were only identified on a handful of scales. Overall 29 different measures of health and wellbeing were included in our study. Almost all showed no difference to the general population. Only 3 measures showed a significant difference, albeit a positive one for same-sex families. If same-sex attracted parents were systematically working to ensure an outcome that shows their children in the best light would there not be a more consistent skewing across all the scales? What is more likely is that there truly is something within same-sex families that shows a benefit on those three measures.

While the criticism has touched on the methodology, the limitations of which are clearly outlined in the published paper, there has also been an unfortunate focus on more personal aspects of the research. Some groups have highlighted the fact that I, the lead author on the paper, am a gay man raising children, suggesting that this is a conflict of interest and introduces inappropriate bias.

Family photo

The research team for this study is based within the Jack Brockhoff Child Health and Wellbeing Program at the University of Melbourne. This Program is led by Professor Elizabeth Waters, an internationally renowned child health researcher who leads work on many aspects of child health including obesity, oral health, disability and disaster recovery, to name a few. She is also a heterosexual parent and second author on the paper. The rest of the research team bring a range of research experiences and family backgrounds. But is there any relevance to their personal situations? All published papers go through a rigorous peer review process that is blind to such things, assessing work on its methodological merits alone. To suggest that my family situation is of relevance implies that no heterosexual researcher can produce unbiased work on heterosexual families. Or that any non-Caucasian researcher would be able to objectively conduct research on racial discrimination and child health.

Increasingly, across all areas of research, there is a move towards the publication of protocols. This establishes the theoretical underpinnings and methodological basis of studies and allows a continued process of transparency to be engaged from the outset. Our recently published findings on same-sex families took this approach and a protocol was published in the early phases of the project. By being up front and transparent at all times about the research we are allowing the work to be continually interrogated.

Research on same-sex families always draws critics. There is a small but vocal minority who do not accept family forms that fall outside the traditional mother-father construct. It is interesting to note that at the same time as criticising methodologies such as ours these same groups would baulk at including questions around sexual orientation on national surveys such as the Census. It is up to the scientific community to consider the merits of our work and place it in its appropriate context in terms of future health interventions and policy work.

It will never be possible to exclude bias completely from all research. But in the mean time it is up to us, the researchers, to maintain objectivity, be open about limitations, and be true to the evidence that our work provides.

Dr Simon R Crouch
Lead Investigator, the Australian Study of Child Health in Same-Sex Families (ACHESS)

Parent-reported measures of child health and wellbeing in same-sex parent families: a cross-sectional survey

Kids from same-sex families fare as well as peers – or better

2 thoughts on “Hidden agendas – do researchers and participants bring bias to studies?

  1. The religious anti-gay bigots who attacked your work have been lying to the public about how research is conducted on tiny national minorities.

    The truth is that no tiny national minority comparable in size to that of gay parents’ children is, or even practically can be, studied mainly if at all through large national random sample studies.

    Papers from Mark Regnerus and Loren Marks — both avowed religious anti-gay bigots — in June 2012 attempted to impose a standard for the study of gay parents’ child outcomes. According to Regnerus and Marks, only large random national sample studies are valid, when it comes to the study of the tiny national minority that is gay parents’ children. Regnerus and Marks ignored that no other comparably small national minority is studied chiefly by such means.

    Regnerus and Marks are violating the American Sociological Association’s Code of Ethics, section on Public Communications.

    On top of their general lie about how tiny national minorities are studied, Regnerus and Marks are lying to the public about the nature of Regnerus’s sample. The sample was drawn from an on-line panel, whose members receive puny incentives to take weekly surveys. The incentives are $5 to take a screener and $20 to take the full survey. For families who do not have computers and/or internet connections in their dwelling place, the company provides free laptops and internet connections for as long as the recipients continue taking surveys.

    Such an arrangement obviously cannot give you a random and representative sample. Consider, for instance, the son of a lesbian couple who was in the age group for Regnerus’s “study,” 18 – 39, but is working as a surgeon.

    Why would a busy surgeon spend time taking surveys every week to earn $20 for each survey? The less successful a person is in a career, the more likely they are to be taking online surveys every week. Regnerus’s data are not a random sample of Americans 18 to 39 raised by gay parents.

    And, we have documented beyond all doubt that the peer review for the Regnerus and Marks papers was corrupt. Moreover, we have documented beyond all doubt that the “Audit” published by Social Science Research editorial board member Dr. Darren Sherkat continues untruths and omissions. Sherkat is guilty of otherwise lying to the public about circumstances of the publication of the Regnerus paper. For example, he told an audience in CA that the peer reviewers’ conflicts of interest with Regnerus are “minimal.” Meanwhile in documented reality, one of the peer reviewers, W. Bradford Wilcox, as a Program Director at Regnerus’s main funder, the Witherspoon Institute recruited Regnerus to manage the study, colluded with him on its design, its data collection, its data analyses and its promotions in anti-gay-rights contexts. Before data collection occurred, Wilcox and Regnerus went to Colorado on Witherspoon’s expense account and met with the malicious anti-gay bigot Glenn Stanton of Focus on the Family. They spent a full day discussing NFSS promotions, before NFSS data collection occurred.

    The religious anti-gay bigots who promulgated this hoax have yet to be sufficiently punished by the professional groups to which they belong, for example, the American Sociological Association. Also, the Regnerus and Marks papers must be retracted, as at present they continue to contaminate the scholarly record. However it should be noted that on the one occasion when Regnerus was subject to significant scrutiny, in a court in Michigan, where he presented himself as an “expert witness,” Judge Bernard Friedman noted that the Witherspoon Institute wanted a desired outcome to the “study,” and then Regnerus complied. Judge Friedman also wrote that Regnerus’s testimony was “entirely unbelievable, and not worthy of serious consideration.”

  2. Pingback: What the Australian Study Really Tells Us About Gay Parenting | Care2 Causes

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s