shivver13 (shivver13) wrote,
shivver13
shivver13

Stats analysis - "The Actor"

The problem with messing with statistics is that once I've done it, I want to do more! So, here is an analysis I did for the first time a year ago or so, and just redid with more recent data.


I think I've mentioned before that I work for a development studio that runs a Facebook game, and one of the things that I do is analyze our player data - how they play, how often they log in, what things they buy, how much they pay, etc. - and try to figure out what they like and what keeps them coming back, so that we can improve the game for them and keep them coming back. Well, it's something you can also do with writing, if you have adequate data. AO3 has some statistics, but it's not useful in my case, but ffnet actually tracks some useful data that I'm able to use to analyze my writing.

While analyzing game performance and player behavior, one of the statistics we track is called retention: how many first-time players come back to play the next day, or the next week, or the next month. For example, a pretty good number for "players who come back the next day", otherwise called "day-1 retention", is 25% - that means that 1/4 of people who try out your game for the first time come back and play again the next day. Once you know your day-1 retention number, you can then introduce new features and see if that number goes up or down.

If you're looking at a multi-chapter fic, you can do the same thing, more or less: see how many people, after reading your first chapter, go on to read the next, and you can extend this to all the chapters in your fic. I pulled the "views" numbers for the chapters in The Actor from ffnet, and this is what I found. I only pulled data from 2015, mostly because I'm doing this on my lunch hour and didn't want to pull more.

The analysis below is non-spoilerific.

Chapter Overall Ch-to-Ch Ch Note
1 Childhood, crash
2 57.26 57.26 Wake up in TARDIS.
3 51.99 90.80 It's really real.
4 51.41 98.87 Bedroom
5 43.44 84.51 Cambridge 1
6 46.49 107.01 Cambridge 2
7 44.38 95.47 Dream, talk with Amy
8 38.41 86.54 TARDIS life, talk with Rory
9 39.81 103.66 Polthite ship 1
10 40.05 100.59 Polthite ship 2
11 41.80 104.39 Slave ship
12 35.95 85.99 Talk with the Doctor
13 38.88 108.14 Dalek 1
14 40.28 103.61 Dalek 2
15 48.36 120.06 Last chapter


The first column is the overall retention, meaning, the percentage of people who viewed that chapter, of the people who viewed the first chapter. For example, the first number implies that 57.26% of the people who viewed the first chapter then went on to view the second chapter. This assumes that people moved in a linear fashion, but remember that on ffnet, you can jump to whatever chapter you want. I expect, though, that most people go linearly through the story unless they're revisiting the story and looking for something specific.

The second column is the chapter-to-chapter retention, meaning, the percentage of people who viewed the previous chapter who then went on to the next chapter. A number over 100 means that more people hit this chapter than the previous chapter, and a number below 100 means that less people hit this chapter than the previous chapter. For example, the second number in that column, 90.80, means that of the people who viewed chapter 2, 90.80% went on to view chapter 3; nearly 10% abandoned the fic at this point. Again, we're still assuming that people for the most part move linearly through the story.

The first interesting statistic (to me at least) is that 57.26% of readers visit the second chapter. That tells me that nearly 3 out of 5 people who open the fic are intrigued enough to turn the page to see what happens next. Now, I don't have standards to measure by, but that seems pretty good to me. I'd like to compare that statistics to my other long fics sometime; I'm pretty sure this one will be better than those because the end of the first chapter ends on an uncertain and tantalizing note, urging the reader to find out what happened. (Not to mention the fic hasn't gotten into the part promised in the summary, so you'd think that anyone who bothered to open it would read a little further to get to that part before deciding it wasn't worth it.)

The nice thing is that once people get into the story, they tend to stick with it. There is reader drop-off almost every chapter until chapter 9, which is to be expected as the true story hasn't quite gotten going yet and there's a lot of talking and not much action up through this point. I find it very interesting that readership improves with chapter 9, as I thought chapter 8 was the weakest of the entire story and would have figured that most readers would have abandoned the fic there. (More on the increase of readers later.) If I want to shore up the quality of the fic, these are the chapters that I should work on, and honestly, that's borne out by opinion. I've been told before that the earlier chapters are nowhere near as polished as the later ones.

The latter half of the fic has almost all 100 or more in chapter-to-chapter retention. The only one that doesn't is chapter 12, and there are two reasons for this, in my mind. First, an important event happens in chapter 11 (no spoilers!) that I know upset a number of readers, and I expect that some fraction of them refused to read on after that. Second, view numbers went up in chapter 11, and I think that there are probably some readers who, after finishing the fic, went back to re-read chapter 11, which would then cause the chapter-to-chapter retention for chapter 12 to decrease.

This is an important point, by the way. Since readers can choose to jump to any chapter they want, the numbers get thrown off as people revisit the fic and go only to the chapters they want to read. I know that people do this: I've had a number of readers tell me that they've gone back and forth over chapters 13-15 multiple times, because the three, taken as a whole, are the heart of the story, and that's demonstrated in the numbers. It's the only thing that explains why it looks like chapter 15 has 20% more readers than chapter 14, and why it seems that 1/2 of the readers finished the story when only 1/3 of them made it through chapter 12.

One consequence of this ability of people to return to the fic and look at whatever chapter they want is that the number of views of the first chapter is skewed: it's a total of people who opened the fic for the first time plus the people who returned to the fic to re-read a specific chapter (since ffnet makes you start on the first chapter). This means that the number of people who opened the fic for the first time is probably significantly lower, which means that the percentage of people who then went on to chapter 2, which here is 57.26%, is probably higher than that in reality. (And of course, all of the numbers in that column should be higher, too.) I might even estimate that the number of people who were interested enough to go on to the second chapter is higher than 60%, meaning, more than 3 out of 5. That's fantastic.

With this data, what conclusions can I draw about my writing? Well, first, the most popular chapters are the four adventures, and that's not surprising: that's the kind of thing I write best. There's some amount of drop-off after each adventure, which is to be expected, as the readers don't have as much incentive to turn the page as they do when the chapter switches in the middle of an adventure. The worst chapters, other than the first half, which I already know isn't as good as the latter half, are the ones where it's mostly character interaction. It's not that I'm really bad at that, but that I need to work harder on it. Well, at least I was already aware that the David / Rory chapter wasn't that good. Lastly, I spent a hell of a lot of time on that last chapter (rewrote part of it from scratch five times!), and it seems to have paid off. Oh, and that event in chapter 11? It may have driven people away, but it was so worth it.

All right, too much writing, time to get back to work. I hope this was an interesting read.
Tags: real life, stats
Subscribe

  • AO3 question

    Since some of you understand how this stuff works better than I do... I just got an email from AO3 saying that someone is requesting to include one…

  • Jigsaw Puzzles: Ye Saga Continues (and other things)

    So when we last left our heroes, Mother-in-Law had asked my husband for a suggestion for a birthday present for me. He suggested a jigsaw puzzle,…

  • Hanging in there

    It's been a while, hasn't it? Life's been plodding along, and now it's December. A lot of same-old, same-old: working from home, practicing…

  • Post a new comment

    Error

    Anonymous comments are disabled in this journal

    default userpic

    Your IP address will be recorded 

  • 6 comments