Searching for a dataset about chess ELO rating distribution broken down by gender

Question

A friend and I had a friendly argument recently about the origin of the apparent gender gap in chess ratings, wherein the grandmaster list is dominated by males, and the best male player today is far better than the best female player. My suspicion is that this is an issue of sample size rather than any real difference: since there are so many more male players to choose from, the pool of male players will sample the tail of the distribution more often and be more likely to contain someone with a very high rating. Or in stats term, the expectation value of the maximum value of N samples randomly drawn from a distribution will grow with N.

Most of the articles I have read on the subject don't even consider this effect. They just say "We looked at the top 50 men and top 50 women and found that the men were better" or something along those lines, which would be exactly what you would expect to see if the distribution of chess ratings was gender-independent but the male pool had a larger sample size.

I've been poking around Kaggle trying to find a data set containing chess ratings and genders, but so far no luck. Does anyone have any idea where I could find a dataset with which to test my theory? I think it would be enough to have a list of chess players with a rating over some high value (2000, say?) broken down by gender.

Answer 1

FIDE rating lists are available to download on this page.
The standard rating list contains both rating and sex information.

Your hypothesis is interesting, but it seems not very easy to test it, because FIDE Elo rating distribution has many abnormalities.

Here below are comparative density plots with and without filtering by K-factor:

library(ggplot2)
library(grid)

p1 = ggplot(elo, aes(rating, fill = sex)) +
     geom_density(position = "stack") +
     labs (title = "any K")
p2 = ggplot(elo[elo$K != 40,], aes(rating, fill = sex)) +
     geom_density(position = "stack") +
     labs(title = "K ≠ 40")

grid.arrange(p1, p2, ncol = 2)

---

There are non-Elo rating systems and other Elo-based rating systems, e. g. USCF rating system.

Update

It was noticed in comments that analyzing local communities makes more sense.

P. S. Order of plots corresponds to this order. It seems that there is no great correlation between "Average" and "Total titled".

Answer 2

I'm posting here the results of my analysis so far. If this is not an appropriate place for this, let me know. The actual answer to my question is here, where you can find a very large database of mostly German players, with some international ones.

My analysis follows closely the analysis done in this paper, with a few tweaks that I'll discuss as needed.

First, we filter the data set. This data set has 2 ratings listed: FIDE and DWZ. These are very closely correlated, being equal for most players but the DWZ rating system has more entries and so we will use that one for the sake of data quality. Removing players without ratings leaves 68232 men and 4453 women. We then plot the full rating distribution, just to see what it looks like.

This appears to me more or less Gaussian at the high end, but has some wierd stuff happening at the low end. To find out what's going on there, we plot a heatmap of rating vs age, which clearly shows a bimodal distribution:

Since the strange peak at the low end appears to come predominantly from youth chess, we will ignore it in subsequent analysis, and remove any players from the dataset with a rating less than 1150 (the red line on the heatmap).

Finally, we filter out all international players and focus only on the German population, since this is the dominant population in the data set. Our final data set contains 51529 men and 2276 women, for a gender ratio of about 23:1.

The ratings distributions for men and women separately are shown below, along with Gaussian fits.

We find that the mean and standard deviation for the men are 1612 and 321, compared to 1406 and 353 for women. We'll revisit this difference later.

We now attempt to determine the effect of sample size on the top 100 players in each population. To do this, we assume as our null hypothesis that the underlying distribution for men and women is the same, and equal to the male population, and we consider the expected difference in ratings between the top 100 pairs from each population. This assumption is generous to the counter argument that there really is a difference between the two populations: if we draw samples for the men from the male population and samples for women from the women's population our expected ratings differences will be much larger than if we draw both samples from the male population.

We bootstrap some data by drawing 51529 random samples to represent a realization of male chess players, and 2276 samples to represent the women. From each population we take the 100 best samples, and consider the pairwise differences between them. We then average this over 1000 realizations of this process. The results are plotted below.

From his analysis, it seems that the top 20 or so female players are actually performing better than one would expect based on sample size differences. From this I'm comfortable concluding that there isn't anything biological at play, and that the differences between the top male and female chess players can be explained by sample size issues.

As to why there is a small difference in the mean between male and female players in the full population, I can't really go on statistics anymore. Clearly there is some selection bias against women in general because of the 23:1 sex ratio, but it's difficult to say exactly why. A possible clue lies in the first plot in the accepted answer, which shows that in the FIDE database, there is a dip in the female probability density around 1800-1900 which is not present in the male population. This suggests that women who are almost, but not quite master-level are more likely to drop out than their male counterparts, which would have the effect of suppressing the mean when considering the full population, but which would only have minimal effect, if any, on the top 100 players, consistent with the approximate skill parity we see there.

The drop in female players in not as evident in the German DWZ data. If we look at the fit residuals for the female distribution it's pretty even across the whole range. We do see a small dip between 2000 and 2200 where the fit overestimates the density consistently, but it's not enough to explain the difference by itself. That said, the difference is only about half a standard deviation, so it's not hard to argue that it is insignificant. to be begin with.

Why female players in this range drop out is a question for FIDE.

EDIT: in the comments it was pointed out that rating distributions are usually logistic. Below, I show the same plots, but with logistics fits. The data here doesn't support logistic fits very well, as the fit severely overestimates the high-rating tail of the distribution when using a logistic model. As was pointed out, there are likely a sum of a few distributions going into these things, which comes out approximately normal. Further data stratification might be necessary to get logistic fits working.