Jonathan Haidt and Candice Odgers debate the relationship between social media and mental health

Does social media worsen mental health for young people, especially young women? It has become an article of faith for many that it does. And there is bountiful anecdotal and research evidence that supports the view. Take, for example, the furore that erupted back in 2021 around Frances Haugen's leaking of internal Facebook research showing the negative impacts of Instagram on young women.

I've written on this topic several times before (most recently here, but see the list of links at the bottom of this post as well). My take is that much of the research on social media and mental health, or social media and subjective wellbeing, shows correlation, but not causation. The challenge here is that perhaps people with mental health issues (or people with lower wellbeing) are more likely to use online social networks, in which case there is reverse causality (the causality runs from mental health to social media, not from social media to mental health).

So, I was interested to read this recent article in Nature by Candice Odgers, reviewing the new Jonathan Haidt book The Anxious Generation (which I have yet to read, but it is currently on my Amazon Wish List). Odgers really takes Haidt to task, claiming that all that Haidt is demonstrating is correlation, not causation:

The plots presented throughout this book will be useful in teaching my students the fundamentals of causal inference, and how to avoid making up stories by simply looking at trend lines.

Hundreds of researchers, myself included, have searched for the kind of large effects suggested by Haidt. Our efforts have produced a mix of no, small and mixed associations. Most data are correlative. When associations over time are found, they suggest not that social-media use predicts or causes depression, but that young people who already have mental-health problems use such platforms more often or in different ways from their healthy peers...

Odgers then suggests some alternative explanations:

There are, unfortunately, no simple answers. The onset and development of mental disorders, such as anxiety and depression, are driven by a complex set of genetic and environmental factors. Suicide rates among people in most age groups have been increasing steadily for the past 20 years in the United States. Researchers cite access to guns, exposure to violence, structural discrimination and racism, sexism and sexual abuse, the opioid epidemic, economic hardship and social isolation as leading contributors...

The current generation of adolescents was raised in the aftermath of the great recession of 2008. Haidt suggests that the resulting deprivation cannot be a factor, because unemployment has gone down. But analyses of the differential impacts of economic shocks have shown that families in the bottom 20% of the income distribution continue to experience harm... 

Haidt responded, initially on X, but then in thorough detail in this post on After Babel. He starts by pointing to the range of published evidence:

Zach Rausch, Jean Twenge, and I began to collect all the studies we could find in 2019, and we organized them by type: correlational, longitudinal, and experimental. We put all of our work online in Google Docs that are open to other researchers for comment and critique. You can find all of our “collaborative review” documents at AnxiousGeneration.com/reviews

The main document that collects studies on social media is here:
Social Media and Mental Health: A Collaborative Review

Then notes that:

In that document, we list dozens of correlational and longitudinal studies...

In that document, we also list 22 experimental studies, 16 of which found significant evidence of harm (or of benefits from getting off of social media for long enough to get past withdrawal symptoms)...

In that document, we also list nine quasi-experiments or natural experiments (as when high-speed internet arrives in different parts of a country at different times), eight of which found evidence of harm to mental health, especially for girls and women...

I am not saying that academic debates are settled by counting up the number of studies on each side, but bringing so many studies together in one place gives us an overview of the available evidence, and that overview supports three points about problems with the skeptics’ arguments.

First, if the skeptics were right and the null hypothesis were true (i.e., social media does not cause harm to teen mental health), then the published studies would just reflect random noise... and Type I errors (believing something that is false). In that case, we’d see experimental studies producing a wide range of findings, including many that showed benefits to mental health from using social media (or that showed harm to those who go off of social media for a few weeks). Yet there are hardly any such experimental findings. Most experiments find evidence of negative effects; some find no evidence of such effects, and very few show benefits. Also, if the null hypothesis were true, then we’d find some studies where the effects were larger for boys and some that found larger effects for girls. Yet that’s not what we find. When a sex difference is reported, it almost always shows more harm to girls and women. There is a clear and consistent signal running through the experimental studies (as well as the correlational studies), a signal that is not consistent with the null hypothesis.

Haidt supports this with a further footnote:

Yes, there could be a “file drawer problem” if researchers on one side are systematically discouraged from publishing, so the missing “positive” studies are all sitting in file drawers in researchers’ offices. But because findings of benefits would be unusual and newsworthy, I don’t believe that there is a strong or consistent bias against the skeptics. 

However, simply asserting that there is no file drawer problem is not the same as showing that there isn't. That's where meta-analysis comes in. Haidt could easily conduct a meta-analysis with these studies to demonstrate what the overall effect is, and whether there is evidence of publication bias. In fact, he even cites some meta-analyses that have already been conducted (such as this one, which found "mildly significant" publication bias in one of two tests of bias, with the other being statistically insignificant).

Haidt then goes on to address Odgers' suggested alternative explanations, focusing on her assertion that the Global Financial Crisis explains the sudden change in adolescent mental health. Haidt concludes that:

Odgers has pointed to an alternative causal explanation that A) does not fit the timing in the U.S., B) does not fit the social class data in the U.S., and C) does not fit the international scope of the crisis.

Having satisfied himself that he has rebutted Odgers' critique, Haidt then reiterates some solutions from the book:

In contrast, if leaders and change makers were to embrace my account of the “great rewiring of childhood,” in which the phone-based childhood replaced the play-based childhood, what policy implications follow? That we should roll back the phone-based childhood, especially in elementary school and middle school because of the vital importance of protecting kids during early puberty. More specifically, we’d try to implement these four norms as widely as possible: 

1. No smartphones before high school (as a norm, not a law; parents can just give younger kids flip phones, basic phones, or phone watches).
2. No social media before 16 (as a norm, but one that would be much more effective if supported by laws such as the proposed update to COPPA, the Kids Online Safety Act, state-level age-appropriate design codes, and new social media bills like the bipartisan Protecting Kids on Social Media Act, or like the state level bills passed in Utah last year and in Florida last month).
3. Phone-free schools (use phone lockers or Yondr pouches for the whole school day, so that students can pay attention to their teachers and to each other)
4. More independence, free play, and responsibility in the real world.

Note that these four reforms, taken together, cost almost nothing, have strong bipartisan support, and can be implemented all right now, this year, if we agree to act collectively.

Even if Haidt is wrong about the causal relationship here, I agree that these reforms are relatively low-cost, and the precautionary principle suggests that they might be appropriate. However, I have argued previously that we should be cautious about regulation that allows parents discretion over their children's social media use. Odgers even partially agrees at the end of her review:

Many of Haidt’s solutions for parents, adolescents, educators and big technology firms are reasonable, including stricter content-moderation policies and requiring companies to take user age into account when designing platforms and algorithms. Others, such as age-based restrictions and bans on mobile devices, are unlikely to be effective in practice — or worse, could backfire given what we know about adolescent behaviour.

It will be interesting to see how this debate progresses. Odgers clearly needs to step things up, because Haidt was very well-prepared for her critique, and had clearly anticipated the points that she (and other skeptics) would raise. I look forward to reading the book after I place my next book order.

[HT: Marginal Revolution for the Odgers article, and Haidt's initial response on X]

Read more:

• Quitting Facebook might make you happier
• The more you use Facebook, the worse you feel
• Does the Internet make people happier?
• What happens when you disconnect from Facebook?
• The latest NZ research on social media and mental health is nothing special
• Online social networks, social capital, and wellbeing

The impacts of home care vs. day care of young children

The second half of the 20th Century involved some massive social change in Western countries. One of those changes was the rapid increase in female labour force participation, including an increase in labour force participation among mothers of young children. As mothers have increasingly gone to work, fathers have in the main not compensating by decreasing their work time. So, childcare has become increasingly important over time. On top of that, there is a lot of tension (and judgment) associated with the decision by mothers to return to work or not.

A useful question to consider, then, is what is the impact of mothers working on their child's outcomes, compared to the mother staying out of work to care for the child. That is essentially the question addressed in this 2023 NBER Working Paper (ungated version here) by Jonathan Gruber (MIT), Tuomas Kosonen (VATT Institute for Economic Research), and Kristiina Huttunen (Aalto University School of Economics). They look at the case of Finland, where they look at the effects of:

...the Finnish Home Care Allowance program (HCA). This program provides substantial payments to mothers who stay home with their children from age ten months through 3 years old, rather than placing the children in formal child care, which is almost exclusively publicly-financed and of relatively high-quality in international comparison. The HCA program has a long tradition in Finland. It was introduced in 1985 and more than 80% of mothers in Finland utilize the HCA. As a result, the share of children in formal child care is much lower in Finland than in other Nordic countries...

Gruber et al. exploit variations in the value of HCA across municipalities, because each municipality could provide a supplement to the HCA, and many have done. The value of these supplements changes over time, and that is the variation that is key to assessing the impact on mothers and children. Essentially, they look at how differences in the amount of HCA assistance affect mothers' work, and children's outcomes, using linked data from the Finnish population register, birth registry, tax and benefits records, early childhood data from clinical assessments of children's readiness for school, and education and youth crime records for when the children were older. The early childhood data are interesting:

The individual tests we consider for four years olds (from 2010 onwards) are Cross (needing to draw a cross, where the two lines intersect), Ask (the child is able to ask following types of questions: when and where?), Details (the child is able to explain details from a specific picture), and Colors (the child is able to identify three out of four main colors from a color card). The tests for five years old (prior to 2010) are Circle (the child can cut a circle from a paper with scissors), Square (the child is able to draw a square on paper), Human (the child can draw human that has at least head, body and limbs come out of body, not from head), and Instruct (the child is able to follow three-part instructions).

Gruber et al. apply a difference-in-differences approach with a continuous treatment variable (the amount of HCA assistance received), which essentially estimates how a 100-Euro change in the HCA supplement amount affects outcomes between the years before and after the supplement changes. The results in terms of mothers' employment are summarised in Figure 1 in the paper:

Notice the big drop in employment that occurs at Time 0, which is when there is a 100-Euro increase in the HCA supplement. However, eyeballing the figure, while the point estimates are negative, it looks like they are not statistically significant (at the 5 percent level). When they move to a more standard difference-in-differences (rather than a dynamic DID approach), the results are statistically significant. Nevertheless, for this analysis Gruber et al. note that:

Maternal labor supply then falls by about 1.5% for each 100 euro increase in the homecare allowance and remains at that level in the municipalities that increased their supplement amount. So supplements are clearly reducing maternal work in favor of at home care, and the effect corresponds to about 5 percent reduction when compared with mean share of employment of mothers of one-year-old children.

Gruber et al. also show that the HCA supplement leads to a decrease in maternal labor income, but an increase in total income." Interestingly, they go on to show that their:

...estimates are large enough to explain the entire difference between the Finnish and (for example) Danish levels of short run child penalties of 20%.

Turning to children's outcomes, they find that:

...children become more likely to fail the cognition test at age four or five when their parents were eligible for higher HCA supplements at child’s age 1...

After an increase in supplement when one year old, enrolling to academic high school declines and committing a youth crime increases.

Needless to say, these are all bad outcomes for children. Gruber et al. then move onto a standard DID approach in order to better quantify the effects, and find that:

...the impact of receiving a 100 euros per month supplement when the eligible child is one year old is to reduce the employment of mothers by -1.27 percentage points, which is a roughly 5% decline in the odds of working... the impact on annual labor earnings is –194 Euros. Given the increase in HCA of 273 Euros... this suggests an almost three-quarters “crowdout” of the income benefits of HCA; that is, for every dollar of HCA received, mothers offset 72 cents through lower labor earnings... the effect of supplement on all income including earnings and taxable income transfers (including HCA and supplements). The effect on this outcome is 237 Euros.

So, mothers work less, earn less labour income, but receive higher total income as a result of the HCA supplement. In terms of child outcomes though:

...a 100 euro per month increase in the supplement leads to a statistically significant 1.78 percentage point increase in the odds of failing a test; the effect size represents about 7% increase from the baseline failing rate...

...higher HCA in form of supplements when the child is one year old leads to -.6 percentage point decline in the odds of enrollment in an academic high school, which is about 1 percent of the sample mean...

...each 100 euros per month of supplement leads to a rise in youth criminal sentencing of .22 percentage points, off a mean of 4 percentage points, a roughly 6% effect...

All of those effects are statistically significant. And they all point to the homecare supplement having negative impacts on children's short-term and long-term outcomes. Perhaps that is uniquely due to the homecare supplement? Gruber et al. go on to investigate a daycare reform in 1997 that unified daycare fees across the country. As a result, some families ended up paying higher daycare fees, while others paid lower fees. A 100-Euro increase in daycare fees should have a similar (but opposite direction) effect to a 100-Euro increase in the HCA supplement. Indeed, Gruber et al. find an effect of the daycare fee change that is of a similar magnitude, but in the opposite direction, to the HCA supplement results.

That they find similar effects based on changes in HCA supplement and daycare fees should provide some confidence in the results. However, we should treat them with a little bit of caution for at least a couple of reasons. First, as I noted in this post, the 'two-way fixed effects' approach that they have adopted has recently attracted a lot of criticism (which is nicely outlined in two posts on the Development Impact blog, here and here, as well as this post). The short version is that the two-way fixed effects approach is likely to lead to biased estimates of the treatment effect. Gruber et al. do try a few ways of dealing with this, and the results are robust to the approaches they adopt, but given that their dynamic DID results are statistically insignificant, this still leaves me concerned. Second, Finland is somewhat unique in the pervasiveness of homecare. As they note in the paper, 80 percent of mothers make use of the homecare allowance, which is a high take-up rate. It's not clear that similar effects would be observed in other countries.

This external validity problem is the biggest issue for me. Mothers are essentially choosing between homecare, where they look after the children and teach them the basics required to prepare them for school (with whatever resources and support they have available to them), or they send the children to a daycare service, which employs professionally-trained early childhood educators to perform the task. There are pros and cons either way, but in terms of school readiness, the daycare may have the edge. On top of that, Finland has a high-quality, publicly funded daycare system, which further tilts the balance in favour of daycare. In countries where the daycare system is of lower quality, the negative impacts on child outcomes are likely to be smaller, or perhaps absent entirely.

Gruber et al. conclude that:

...there may be limits to general international lessons from such policy analyses, and that conclusions are best drawn on a country-by-country basis.

Given the issues of external validity I've noted above, I'd say that further research in other countries is imperative, before we definitively conclude that homecare of children is bad for them.

[HT: Marginal Revolution, last year]

The gender of a doctor matters for medical evaulations

There is lots of evidence that there is gender bias in healthcare. This Medical News Today article summarises some examples and consequences. It seems plausible that at least some of the gender bias in healthcare arises when male doctors examine or treat female patients. A useful question to ask, then, is what would happen to bias if patients were examined by same-gender doctors?

That is essentially the research question underlying this recent article by Marika Cabral (University of Texas at Austin) and Marcus Dillender (Vanderbilt University), published in the journal American Economic Review (ungated earlier version here). Cabral and Dillender first outline the problem, being that:

...female patients, relative to male patients, receive less health care for similar medical conditions and are more likely to be told by providers that their symptoms are emotionally driven rather than arising from a physical impairment... Differences in doctors’ evaluations of medical issues for male and female patients may be a key factor contributing to observed differences in treatment. Beyond influencing the treatments patients receive, medical evaluations also impact benefit eligibility in social insurance programs. Recent evidence suggests there are large gender disparities in social insurance programs that rely on medical evaluations...

Cabral and Dillender make use of:

...comprehensive administrative data and random assignment of doctors to patients within the Texas workers’ compensation insurance system. Random assignment of doctors to patients occurs in this setting through the dispute resolution process. Insurers and injured workers may request independent medical evaluations to settle disputes over an injured worker’s impairment level... The random assignment of doctors to patients means that differences in assessments between male and female doctors stem from the doctors themselves rather than from differences in the types of patients assigned to doctors.

That last point is important. It is the random assignment of patients to doctors that means that the results from this study can be interpreted as causal evidence of the effect of doctor gender on patients' outcomes, and evaluate the difference in those outcomes between male and female patients. Essentially, this is a form of difference-in-differences analysis, looking at the difference in outcomes between male and female patients with a male doctor, and comparing that with the difference in outcomes between male and female patients with a female doctor.

The outcomes that Cabral and Dillender look at are whether the patient is evaluated as having a disability, and the amount of cash disability benefits they receive after the evaluation. Having controlled for patient characteristics such as the type of injury and the industry that the patient worked in, there should be no differences between male and female patients in either disability assessment or disability benefits, depending on whether they have a male or female doctor. Instead, Cabral and Dillender find that:

...patient-doctor gender match increases evaluated disability and subsequent cash disability benefits for female patients but has little impact on outcomes of male patients... Compared to differences among their male patient counterparts, female patients randomly assigned a female doctor rather than a male doctor are 3.1 percentage points more likely to be evaluated as having an ongoing disability and receive 8.6 percent more cash benefits on average, or $483 evaluated at the mean of $5,622. There is no analogous gender-match effect for male patients. We note the magnitude of these effects is sizable. The estimated 3.1 percentage point increase in the likelihood of being evaluated as disabled is nearly large enough to offset the entire observed gender gap in this outcome when male doctors evaluate claimants.

Cabral and Dillender then turn to explaining why this gender bias exists, and find that:

Controlling for available baseline patient information, the estimates indicate that female doctors evaluate female and male patients as similarly disabled while male doctors evaluate female patients as less disabled than male patients. While only suggestive, this evidence is consistent with male doctors evaluating female patients against a stricter standard than male patients and female doctors applying similar standards to male and female patients.

On that last point though, as Cabral and Dillender note in one of the footnotes in the paper, these results alone can't distinguish between whether it is male doctors who evaluate female patients to a higher standard, or female doctors who evaluate male patients to a lower standard. However, Cabral and Dillender report a range of survey evidence from a sample of over 1500 people that is consistent with the former, including:

...that women—relative to men—more often report having a negative experience where a doctor didn’t understand their concerns, had assumed something without asking, talked down to them, made them feel uncomfortable, or didn’t believe them. When asked about how a doctor’s gender influences the likelihood of having a positive interaction, women were much more likely than men to report an own-gender doctor would be more likely to treat them with respect, understand their concerns, believe them, provide needed testing and treatments, make them feel comfortable, and ask appropriate questions instead of making assumptions.

Cabral and Dillender also report on the intensity of preferences over doctor gender, showing that:

...48.5 percent of women are willing to pay an additional $5 copay to see an own-gender provider compared to only 29.3 percent of men—a 19.2 percentage point difference.

It would have been interesting if they had extended that analysis to an estimate of the female patients' average willingness-to-pay for having a female (rather than a male) doctor, but they didn't. Finally, Cabral and Dillender looked at the policy implications, noting that based on their results:

...increasing the share of independent medical evaluations performed by female doctors from 17 percent to 50 percent would cause a 0.88 percentage point increase in the share of female patients evaluated as disabled, closing approximately 41 percent of the gender gap conditional on observables among disputed claims.

Given that still less than half of medical school graduates in the US are female, there is a long way to go before we get to that point. For comparison, in New Zealand in 2019, over 58 percent of medical school graduates were female. I guess that is good news for New Zealand, in terms of reducing the gender bias in medical evaluations here.

This week in research #19

Here's what caught my eye in research over the past week (a really quiet week):

• Strozza et al. (open access) find that COVID-19 mortality disproportionately affected those of lower socioeconomic status and exacerbated existing social inequalities in Denmark, using Danish population register data
• Nguyen finds a positive association between temperature and women's exposure to intimate partner violence across 34 developing countries, and that women from rural areas, those from poor households, those having low education, and those living with low-educated partners are particularly vulnerable to intimate partner violence as temperatures increase

The recorded video of my Professorial Lecture, titled Beyond the Buzz: The Sobering Economics of Alcohol, is now available on YouTube. Watch it here if you weren't able to attend in person. Like and subscribe, or whatever.