How I use rating systems

2024-11-15

Rating systems are obviously dumb, it's obviously dumb to point out how obviously dumb they are. You can't actually place all art on a numerical tier list of comparison. Everyone knows this but they're too convenient to leave behind. On the one hand they're useful for the person who issued a rating as a memory-jogging device. On the other hand, they are ostensibly useful for other people to get a sense of your taste. To quickly communicate your opinions on a piece of art without having to go in depth where that level of detail is not required.

The problem arises because although we all agree that 10 means really good and 1 means really bad, everything in between those numbers is a complete fucking free for all. Although 5 should mean average, the actual average score on any aggregating site is always around a 7. I have no idea why this is, maybe people don't bother to rate shows they dislike or drop so ratings tend upwards, maybe people have a bad intuitive sense of what a 7 is supposed to represent out of 10. There are people who never give anything a 10 because "nothing is perfect". If you tell me you thought a show was a 7/10, I have no way of knowing if you mean it was about average, or 2 points above average. Meanwhile, giving something a 3/10 would often be interpreted as you having hated that show, even though it's just the flip side of 7. Ok this discourse is ancient and I won't go on at length because I'm sure we all know the problems here.

I'm going to focus on myanimelist's rating system, because it's the system I interact with most often. When you go to rate something on MAL, it gives you a verbal definition of each number.

So my reaction is to treat the numbers as shorthand for these adjectives. When I go to rate a show I think something like "would I describe that as 'Good' or 'Very Good'?" I'm just going to lay out how I use this rating system.

We'll use 5 as a pivot point. If I give a show a 5, that means it's a 50/50 whether I would have dropped it or not. At 4, we're on "I feel comfortable dropping this". Anything below a 4 I would almost never actually sit through to the end. 3 is the minimum badness threshold where I would always logically drop it discounting, some extenuating circumstances which compel me to finish the show, external from the content of the show itself. Both 2 and 1 mean "among the worst things I've ever watched", but a 1 has to also personally offend me on some level.

On the flip side, just like I could still finish a 4, I might still drop a 6. It's a show which entertained me enough to finish or at least feel like I could finish, and that's it. At 7, this show was actively a positive impact on my life in some way, I am glad I spent my time watching it. An 8 is like a really good 7, not only was I entertained, but it also carried some meaning or memorability for me on a deeper level. To get a 9, this should be a show I regularly find myself thinking back on, something that sticks in my head as a reference point of quality. I very rarely give out 10s, it should mean something with no negative qualities. It should be something singular, and as MAL says, something I would feel comfortable describing as a "masterpiece".

Regarding never giving out 10s

The 1-10 rating system is lopsided with regard to extremes. Each number isn't just itself, but also every decimal until the next number, i.e. it's truncated at the decimal point. It has to be truncated rather than rounded to the nearest whole, because that scenario wouldn't make sense. Imagine your friend asks you how you would rate a show, you say "I'd give it a 6", they would always, if asking for more detail say something like, "6 point what?" No one's immediate reaction would be to assume that by 6 you really meant a 5.5. Therefore, even if a show has a 1, it could actually be as high as a 1.999 but still be truncated down to a 1. In other words, it could have some redeeming qualities while still appearing as a 1. On the other hand, since 10 is the upper bound, you can't have a 10.5 on the rating scale, just like you can't have a 0.5 as that would be out of bounds. So anything with any qualities that might reduce the rating automatically place you back in the 9 range.

That's the mathematical reasoning as to why I have not historically given 10s. However, I have come to realise that this is a highly autistic way of looking at things, and that it's more useful to just as I mentioned earlier, treat the numbers as shorthand for an adjective. This might be less mathematically accurate, but it's more useful for communication. So I've given out a few 10s.

Regarding rating dropped shows

Some people might have a problem with the fact that I rate shows even when I don't finish them. Since MAL tracks whether or not you've finished a show, and how many episodes you've watched, and always lists that information alongside your score, it's immediately obvious to anyone looking at my ratings what portion of the show I'm basing it off. If I didn't rate dropped shows, I would almost never rate anything negatively, since I'm not going to sit through a bad show. This would defeat the point of rating systems, since it's still useful to differentiate between shows I mildly disliked vs shows I absolutely hated. So the question of "how can you know that the show deserves that rating if you didn't even finish it?" is irrelevant, I'm obviously not basing my rating on the portion of the show I haven't seen, that would be impossible.