Log in

No account? Create an account

Previous Entry | Next Entry

Stats analysis - "The Actor"

The problem with messing with statistics is that once I've done it, I want to do more! So, here is an analysis I did for the first time a year ago or so, and just redid with more recent data.

I think I've mentioned before that I work for a development studio that runs a Facebook game, and one of the things that I do is analyze our player data - how they play, how often they log in, what things they buy, how much they pay, etc. - and try to figure out what they like and what keeps them coming back, so that we can improve the game for them and keep them coming back. Well, it's something you can also do with writing, if you have adequate data. AO3 has some statistics, but it's not useful in my case, but ffnet actually tracks some useful data that I'm able to use to analyze my writing.

While analyzing game performance and player behavior, one of the statistics we track is called retention: how many first-time players come back to play the next day, or the next week, or the next month. For example, a pretty good number for "players who come back the next day", otherwise called "day-1 retention", is 25% - that means that 1/4 of people who try out your game for the first time come back and play again the next day. Once you know your day-1 retention number, you can then introduce new features and see if that number goes up or down.

If you're looking at a multi-chapter fic, you can do the same thing, more or less: see how many people, after reading your first chapter, go on to read the next, and you can extend this to all the chapters in your fic. I pulled the "views" numbers for the chapters in The Actor from ffnet, and this is what I found. I only pulled data from 2015, mostly because I'm doing this on my lunch hour and didn't want to pull more.

The analysis below is non-spoilerific.

Chapter Overall Ch-to-Ch Ch Note
1 Childhood, crash
2 57.26 57.26 Wake up in TARDIS.
3 51.99 90.80 It's really real.
4 51.41 98.87 Bedroom
5 43.44 84.51 Cambridge 1
6 46.49 107.01 Cambridge 2
7 44.38 95.47 Dream, talk with Amy
8 38.41 86.54 TARDIS life, talk with Rory
9 39.81 103.66 Polthite ship 1
10 40.05 100.59 Polthite ship 2
11 41.80 104.39 Slave ship
12 35.95 85.99 Talk with the Doctor
13 38.88 108.14 Dalek 1
14 40.28 103.61 Dalek 2
15 48.36 120.06 Last chapter

The first column is the overall retention, meaning, the percentage of people who viewed that chapter, of the people who viewed the first chapter. For example, the first number implies that 57.26% of the people who viewed the first chapter then went on to view the second chapter. This assumes that people moved in a linear fashion, but remember that on ffnet, you can jump to whatever chapter you want. I expect, though, that most people go linearly through the story unless they're revisiting the story and looking for something specific.

The second column is the chapter-to-chapter retention, meaning, the percentage of people who viewed the previous chapter who then went on to the next chapter. A number over 100 means that more people hit this chapter than the previous chapter, and a number below 100 means that less people hit this chapter than the previous chapter. For example, the second number in that column, 90.80, means that of the people who viewed chapter 2, 90.80% went on to view chapter 3; nearly 10% abandoned the fic at this point. Again, we're still assuming that people for the most part move linearly through the story.

The first interesting statistic (to me at least) is that 57.26% of readers visit the second chapter. That tells me that nearly 3 out of 5 people who open the fic are intrigued enough to turn the page to see what happens next. Now, I don't have standards to measure by, but that seems pretty good to me. I'd like to compare that statistics to my other long fics sometime; I'm pretty sure this one will be better than those because the end of the first chapter ends on an uncertain and tantalizing note, urging the reader to find out what happened. (Not to mention the fic hasn't gotten into the part promised in the summary, so you'd think that anyone who bothered to open it would read a little further to get to that part before deciding it wasn't worth it.)

The nice thing is that once people get into the story, they tend to stick with it. There is reader drop-off almost every chapter until chapter 9, which is to be expected as the true story hasn't quite gotten going yet and there's a lot of talking and not much action up through this point. I find it very interesting that readership improves with chapter 9, as I thought chapter 8 was the weakest of the entire story and would have figured that most readers would have abandoned the fic there. (More on the increase of readers later.) If I want to shore up the quality of the fic, these are the chapters that I should work on, and honestly, that's borne out by opinion. I've been told before that the earlier chapters are nowhere near as polished as the later ones.

The latter half of the fic has almost all 100 or more in chapter-to-chapter retention. The only one that doesn't is chapter 12, and there are two reasons for this, in my mind. First, an important event happens in chapter 11 (no spoilers!) that I know upset a number of readers, and I expect that some fraction of them refused to read on after that. Second, view numbers went up in chapter 11, and I think that there are probably some readers who, after finishing the fic, went back to re-read chapter 11, which would then cause the chapter-to-chapter retention for chapter 12 to decrease.

This is an important point, by the way. Since readers can choose to jump to any chapter they want, the numbers get thrown off as people revisit the fic and go only to the chapters they want to read. I know that people do this: I've had a number of readers tell me that they've gone back and forth over chapters 13-15 multiple times, because the three, taken as a whole, are the heart of the story, and that's demonstrated in the numbers. It's the only thing that explains why it looks like chapter 15 has 20% more readers than chapter 14, and why it seems that 1/2 of the readers finished the story when only 1/3 of them made it through chapter 12.

One consequence of this ability of people to return to the fic and look at whatever chapter they want is that the number of views of the first chapter is skewed: it's a total of people who opened the fic for the first time plus the people who returned to the fic to re-read a specific chapter (since ffnet makes you start on the first chapter). This means that the number of people who opened the fic for the first time is probably significantly lower, which means that the percentage of people who then went on to chapter 2, which here is 57.26%, is probably higher than that in reality. (And of course, all of the numbers in that column should be higher, too.) I might even estimate that the number of people who were interested enough to go on to the second chapter is higher than 60%, meaning, more than 3 out of 5. That's fantastic.

With this data, what conclusions can I draw about my writing? Well, first, the most popular chapters are the four adventures, and that's not surprising: that's the kind of thing I write best. There's some amount of drop-off after each adventure, which is to be expected, as the readers don't have as much incentive to turn the page as they do when the chapter switches in the middle of an adventure. The worst chapters, other than the first half, which I already know isn't as good as the latter half, are the ones where it's mostly character interaction. It's not that I'm really bad at that, but that I need to work harder on it. Well, at least I was already aware that the David / Rory chapter wasn't that good. Lastly, I spent a hell of a lot of time on that last chapter (rewrote part of it from scratch five times!), and it seems to have paid off. Oh, and that event in chapter 11? It may have driven people away, but it was so worth it.

All right, too much writing, time to get back to work. I hope this was an interesting read.


( 6 comments — Leave a comment )
Sep. 21st, 2015 08:51 pm (UTC)
I sometimes jump around, if there's something I read in one chapter that seems to refer to something 3 chapters back, I definitely will go back to find it (won't stop until I do).

Statistics, I am told, are a funny thing, they can tell you whatever you want if you manipulate the numbers and methodology enough. Numbers are fun!
Sep. 23rd, 2015 08:36 pm (UTC)
Oh yes, I do that, too. I'll also return to favorite fics to read parts I particularly liked. I feel bad when I do that, because I know it throws off people's stats. Of course, I'm sure very few care about chapter stats like I do. :P
Sep. 21st, 2015 10:35 pm (UTC)
A VERY interesting read. I'm one of those who reads straight through. I have a tendency to drop very few fics, but my reading would look odd, because I might exit out, then come right back to the chapter I was last on before moving to the next.

I know after my first two chapters in my longer fictions, I get drop off. One or two of my fics have chapters which can be read as standalones, which would explain the jump in some of those as compared to previous chapters...

I ramble too much, lol!!

Sep. 23rd, 2015 08:40 pm (UTC)
Thanks! I'm glad you enjoyed this - I think it's not something people think about much, how you can tell how people approach your fics. Of course, everyone has different reading habits, but the stats show what the majority do. Hopefully I'm reading them right.

I think it's very difficult to get people to read past the first chapter - you have to really grab them to make them turn the page. There are just so many fics out there that people will move on if they don't immediately see what they want to read in the first chapter.
Sep. 22nd, 2015 05:02 pm (UTC)
I sometimes jump around in long fics for the somewhat less flattering reason of "does this get any better?" Like, I might be vaguely interested in something, but if after a few chapters it seems to be meandering and I kind of want to like it because it's got some good lines or hey this is a fun Ten or whatever, I might jump ahead to the middle and see if there's anything that grabs me, to help decide whether I want to invest the time actually reading the thing or not.
Sep. 23rd, 2015 08:41 pm (UTC)
Oh yes, I've done that, too! I especially do that if the writing is bad but the idea is good - I want to see if the idea develops well enough to weather through the bad writing. :)
( 6 comments — Leave a comment )

Latest Month

February 2019
Powered by LiveJournal.com
Designed by chasethestars