Thursday, 3 April 2014

Charting the Quantity of Harry Potter Fan Fiction

Ok, so I should probably start with the ‘why’…

As some of you will know, I’m currently concluding my BA in English Literature (don’t laugh). One of the courses available this year was ‘Children’s Literature’. Being an enormous snob I didn’t take it because it included a small section on Harry Potter, and I do not want to have to admit to having studied Rowling, however briefly, for my degree. However, many of my friends (who are not such massive snobs and consequently much nicer, better people) did take the course, and so every now and then the conversation turns Potter-ward. A couple of weeks ago the topic at hand was whether Potter had ‘run its course’, inspired by a remark to that effect from the lecturer (the esteemed and june Adam Roberts), and how one could possibly measure that.

Various ideas were put forward – sales figures, new editions, etc. I suggested fan engagement – in particular the ability of Potter to inspire new ‘literature’ in the form of fan fiction. It would be interesting, I said, to know whether the amount of Harry Potter fan fiction was increasing or decreasing over time.

So that’s what this is: an attempt to measure the subset of a subset of a pointless and frivolous question which benefits no-one. BA students, you’ll feel right at home.

Anyway, onto the ‘how’. is the largest such website by a considerable margin.1 At the time of capture it contained 517,847 non-pornographic Harry Potter stories.2 Its first, about which I will kindly say no more, was published in September 1999; its most recent appeared about 30 minutes ago. I downloaded the index pages (2.5 GB) using Httrack and then ran a script to sort the entries. Finally, I copied the data into Excel and produced this tasteful graph showing production by month:
Click to enlarge
The blue bars show the number of new stories each month. The red bars show thousands of active site users by year, based on data I found here; I couldn't find comparable data for after 2010 so I've just left it blank. I've also marked the release dates of the various films and books which fit the time-frame shown here; you'll notice that they perfectly fit the various spikes in activity.

So, what happens next? Well, the overall trend is up, but I don't think it's set to continue. You'll notice that we basically have a reverse sawtooth pattern, in which each new release sparks activity which slowly decreases until the next release - this is perhaps most noticeable in the two-year gap between the double release of Phoenix (film) and Hallows (book) in 2007 and the release of Prince (film) in 2009. At this point activity steadily increases - bucking the trend for the first time - and peaks at the release of the final film, since which activity has been steadily decreasing. The Fantastic Beasts trilogy doesn't have a release date yet - or a director, cast or even script - and it's unlikely that we'll see it before 2017. If activity continues to fall at its current rate (it almost certainly won't but hush) then by that time it will be back down to 2001/2 levels.

I'm not going to extrapolate from these results to say anything about the state of Harry Potter as a cultural force; there are too many variables not accounted for to allow any safe conclusion. Maybe everyone's migrating from to Tumblr; maybe they're all writing in private, ready to pounce if Kindle Worlds opens the door to them making money off it; maybe the core writing bloc has simply grown up and moved on. I leave that as an exercise for the reader.

1. A commonly accepted truth, but I went out and had a look around anyway. has, at the time of writing, over 654,000 Harry Potter stories. Sadly that’s not a typo. As a point of comparison, has almost 300,000 such stories and has about 11,000; after that there’s a host of small sites such as Ficwad (2585), Fictioncentral (111) and Fanworks (78) fighting over the scraps. The elephant in the room is Tumblr, which is extremely hard to analyse properly – it doesn’t even publish user numbers. In any case, most of the fan fiction there (by my observation) is ‘art’ rather than ‘literature’.

2. Observant readers will notice that this something of a drop from the 654,000 mentioned in the previous footnote; that's because there are over 120,000 pornographic Harry Potter stories which my script didn't collect at first due to a filtering error. I thought about going back to add them in, but there are some depths to which even I will not sink, and collecting Harry Potter porn is one of them. One other point worth mentioning is that due to a rather silly scripting error every fiftieth story was omitted from my data collection, a loss of 2%. However, as this loss is evenly distributed across all the data you don't need to worry about it affecting the results. 


  1. Oh my! That is a lot of fanfiction! Why can't we have this amount of fanfiction for DECENT works, like ASoIaF or anything by Brandon Sanderson?

    By the way, hpfanficarchive has about 800 stories, of which 90% are the dregs of the pornographic dregs. Although it may be hard to believe, there are some stories that are so exaggeratedly sexual that has refused to host them. The authors of these stories meander off to hpfanficarchive in a huff, and there freely publish as much smut as they please. Feel free to browse, but bring along a bottle of brain bleach. The site has story tags such as "Necrophillia", "Incest", "Beastiality", "Rape", and many stories are tagged with all of the above. *shudders*

    1. Heh, I missed that one - and from your description I'm rather glad that I did!