Discussion of movies and ReelThoughts topics

It is currently Wed Jul 23, 2014 5:12 am




Post new topic Reply to topic  [ 37 posts ]  Go to page 1, 2  Next
The Arbitrary Rating System 
Author Message
Second Unit Director
User avatar

Joined: Sat Sep 15, 2012 10:51 pm
Posts: 419
Location: Durham, NC
Post The Arbitrary Rating System
When I got big into films I decided to keep track of the ones that I had watched, and with this track list I rated each one, however I always had a hard time nailing down a system where I could go into a film, leave, think, and say that's a "___ star film" While I've gotten a bit more secure in rating films, I still have to say that it still comes off as extremely arbitrary and subjective. Somones' 3.5 could just as well be someones' 4, or one man's 3.5 a 3 and so on.

With this stated I would love to hear (for those of you who do rate films) how you go about doing it, what you look for, how you differentiate, etc.

_________________
"I have now come to claim that satisfaction."


Sun Apr 14, 2013 8:50 pm
Profile
Producer
User avatar

Joined: Sun Oct 07, 2012 6:26 pm
Posts: 2157
Post Re: The Arbitrary Rating System
I find linear ratings silly and personally don't give them, but I've been pondering the idea recently of a ratings compass... or maybe even a ratings triangle or cube or something. It would add some dimensions to it so it isn't so oversimplified and vague. I'd have to decide what those dimensions are and it would still demand objectively scoring a set of objective criteria, but it would be much more informative than a numerical or alphabetical scale.

_________________
The temptation is to like what you should like--not what you do like... another temptation is to come up with an interesting reason for liking it that may not actually be the reason you like it.


Sun Apr 14, 2013 9:04 pm
Profile
Second Unit Director
User avatar

Joined: Sun Oct 07, 2012 4:57 pm
Posts: 418
Post Re: The Arbitrary Rating System
1 - 10 scale, whole numbers. Has enough shades for each value to show every facet of my opinion.

1 - DEAR GOD WHY DOES IT HURT!
2 - Absolutely horrible
3 - Very bad
4 - A couple good ideas/scenes floating in a scene of crap
5 - Mediocre
6 - Almost good
7 - good
8 - very good
9 - great
10 - THE GREATEST!

With that, 1's and 10's from me are extremely rare


Sun Apr 14, 2013 9:20 pm
Profile
Second Unit Director
User avatar

Joined: Sat Sep 15, 2012 10:51 pm
Posts: 419
Location: Durham, NC
Post Re: The Arbitrary Rating System
Ken wrote:
I find linear ratings silly and personally don't give them, but I've been pondering the idea recently of a ratings compass... or maybe even a ratings triangle or cube or something. It would add some dimensions to it so it isn't so oversimplified and vague. I'd have to decide what those dimensions are and it would still demand objectively scoring a set of objective criteria, but it would be much more informative than a numerical or alphabetical scale.


I think that sounds like a plan Ken. Even though I'm pretty bound to my star rating scale, I can say that it comes across as really subjective. I can easily differentiate between bad and good in my opinion, even the very bad, but stamping something with greatness is hard sometimes, even something good for that matter. I guess it all boils down to singular opinion.

_________________
"I have now come to claim that satisfaction."


Sun Apr 14, 2013 9:27 pm
Profile
Second Unit Director
User avatar

Joined: Sat Sep 15, 2012 10:51 pm
Posts: 419
Location: Durham, NC
Post Re: The Arbitrary Rating System
patrick wrote:
1 - 10 scale, whole numbers. Has enough shades for each value to show every facet of my opinion.

1 - DEAR GOD WHY DOES IT HURT!
2 - Absolutely horrible
3 - Very bad
4 - A couple good ideas/scenes floating in a scene of crap
5 - Mediocre
6 - Almost good
7 - good
8 - very good
9 - great
10 - THE GREATEST!

With that, 1's and 10's from me are extremely rare


I'm a fan of the 1-10 and A,B,C,D,F scales. They do give much more room for differentiating.

_________________
"I have now come to claim that satisfaction."


Sun Apr 14, 2013 9:33 pm
Profile
Critic
User avatar

Joined: Wed Jul 29, 2009 9:35 am
Posts: 7260
Location: Easton, MD
Post Re: The Arbitrary Rating System
JackBurns wrote:
patrick wrote:
1 - 10 scale, whole numbers. Has enough shades for each value to show every facet of my opinion.

1 - DEAR GOD WHY DOES IT HURT!
2 - Absolutely horrible
3 - Very bad
4 - A couple good ideas/scenes floating in a scene of crap
5 - Mediocre
6 - Almost good
7 - good
8 - very good
9 - great
10 - THE GREATEST!

With that, 1's and 10's from me are extremely rare


I'm a fan of the 1-10 and A,B,C,D,F scales. They do give much more room for differentiating.


Why? 1-10 gives you exactly one more option than 0 stars to 4.

_________________
I'm lithe and fierce as a tiger


Sun Apr 14, 2013 9:35 pm
Profile
Second Unit Director
User avatar

Joined: Sat Sep 15, 2012 10:51 pm
Posts: 419
Location: Durham, NC
Post Re: The Arbitrary Rating System
JamesKunz wrote:
Why? 1-10 gives you exactly one more option than 0 stars to 4.


True, but it feels as if it does give you more room. The star system feels much stricter in this regard, whether this actual strictness exists or not.The difference between a 2.5 and a 3 is big--to me thats going from mediocrity to goodness-- its very hard sometimes to justify a ".5" difference.

I could more easily rate a film a 6 over a 5(mediocrity) if I liked it, but didn't find it quite there. With the star system I would have to choose between a 2.5 and a 3, which is going from mediocre to good. Going from 5 to 6 would just feel more comfortable in my opinion, it gives perhaps a little more perceived leeway.

_________________
"I have now come to claim that satisfaction."


Sun Apr 14, 2013 9:47 pm
Profile
Producer
User avatar

Joined: Sun Oct 07, 2012 6:26 pm
Posts: 2157
Post Re: The Arbitrary Rating System
JackBurns wrote:
I think that sounds like a plan Ken. Even though I'm pretty bound to my star rating scale, I can say that it comes across as really subjective. I can easily differentiate between bad and good in my opinion, even the very bad, but stamping something with greatness is hard sometimes, even something good for that matter. I guess it all boils down to singular opinion.

I think you're hitting right upon the greatest flaw. You can get as good as you can possibly get at gauging that singular opinion, but that's still all it can be: a statement that the movie was good, bad, or some gradation thereof. It contains basically the same information as a music video critique by Beavis and Butthead. It ruled or it sucked or it was okay.

Perhaps if we were to develop a rating compass, we might place the traditional linear scale (good to bad, four stars to no stars, A to F, or whatever) on the Y axis and a new, different scale on the X axis. Perhaps that second scale would be the spectrum between art and entertainment. Between spiritual fulfillment and guilty pleasure. Between deep complexity and visceral fun. Perhaps you guys can suggest a wording that better encapsulates this idea, but hopefully I'm getting the idea across. One end for respectable fare and one end for less respectable fare, perhaps.

With the simple addition of that one extra dimension, we're already perhaps verging on a rating system that gives a much, much better idea of the opinion being expressed, while not drastically adding to the complexity of the visual representation of that opinion.

_________________
The temptation is to like what you should like--not what you do like... another temptation is to come up with an interesting reason for liking it that may not actually be the reason you like it.


Sun Apr 14, 2013 10:00 pm
Profile
Critic
User avatar

Joined: Wed Jul 29, 2009 9:35 am
Posts: 7260
Location: Easton, MD
Post Re: The Arbitrary Rating System
JackBurns wrote:
JamesKunz wrote:
Why? 1-10 gives you exactly one more option than 0 stars to 4.


True, but it feels as if it does give you more room. The star system feels much stricter in this regard, whether this actual strictness exists or not.The difference between a 2.5 and a 3 is big--to me thats going from mediocrity to goodness-- its very hard sometimes to justify a ".5" difference.

I could more easily rate a film a 6 over a 5(mediocrity) if I liked it, but didn't find it quite there. With the star system I would have to choose between a 2.5 and a 3, which is going from mediocre to good. Going from 5 to 6 would just feel more comfortable in my opinion, it gives perhaps a little more perceived leeway.


I guess I see your point -- it leaves more room for mediocrity. Both star ratings and A-F systems are equally limiting in this regard because people like us only see the movies we want to see. Therefore the overwhelming majority of films we see fall between 2.5 and 4 stars. That leaves *five* different ways to describe a movie we really don't like (2, 1.5, 1, .5, 0) and only two ways to describe a movie we really like (3.5, 4). The same thing happens with A-F, when only B+, A-, and A+ work for really good films but C, C-, D+, D, D-, and F work for really bad ones.

So I guess for numbers you could have 1-3 as terrible, 4-6 as mediocre/okay, and 7-10 as good to great. But you're still going to run into the same problems. Once you draw arbitrary lines, they become, well, arbitrary. Suddenly the gap between 8 and 9 becomes the big divide and you wish you had an 8.5. You can't really do anything about it.

_________________
I'm lithe and fierce as a tiger


Sun Apr 14, 2013 11:05 pm
Profile
Producer
User avatar

Joined: Tue Jun 30, 2009 1:35 am
Posts: 2012
Post Re: The Arbitrary Rating System
patrick wrote:
1 - 10 scale, whole numbers. Has enough shades for each value to show every facet of my opinion.

1 - DEAR GOD WHY DOES IT HURT!
2 - Absolutely horrible
3 - Very bad
4 - A couple good ideas/scenes floating in a scene of crap
5 - Mediocre
6 - Almost good
7 - good
8 - very good
9 - great
10 - THE GREATEST!

With that, 1's and 10's from me are extremely rare


That's more or less my scale, although I have no 10's. I allow half points and the possibility of a zero rating. My ratings are a combination of how much I enjoyed the movie and how flawed I think it is. I like things like "Jesus Christ Vampire Hunter," "The Machine Girl" and "Repo! the Genetic Opera" while acknowledging that none are really a good movie. (As opposed to "Faster, Pussycat! Kill! Kill!, which actually is a good exploitation film.)

_________________
Evil does not wear a bonnet!--Mr. Tinkles


Mon Apr 15, 2013 12:45 am
Profile
Auteur
User avatar

Joined: Sun Oct 07, 2012 12:02 pm
Posts: 3352
Location: Zion, IL
Post Re: The Arbitrary Rating System
I don't use a rating system because a- it's too confusing, b-it's hard to pick an exact rating, c-I prefer letting my words speak for themselves


Mon Apr 15, 2013 1:10 am
Profile
Online
Assistant Second Unit Director

Joined: Mon Oct 29, 2012 1:49 am
Posts: 103
Post Re: The Arbitrary Rating System
I just use a random number generator. It's way easier.


Mon Apr 15, 2013 1:22 am
Profile
Assistant Director
User avatar

Joined: Sun Nov 18, 2012 2:37 am
Posts: 983
Location: Laurel, MD
Post Re: The Arbitrary Rating System
The tried-and-true 4 star scale is where it's at for me. Pretty much everyone understands that 3 means solid recommendation, 3.5 is a strong recommendation, and 4 is a must-see. A movie pretty much has to scream 4 stars by the time the end credits roll in order to earn that distinction. For the record, I saw two movies last year that way (Looper and Lincoln, two movies that couldn't possibly be any different).

It's the 2.5 star rating that gives the most people headaches. Dustin Putman equates 2.5 stars to a "thumbs up," a B or B- letter grade, and a "Fresh" tomato rating. For JB, it's thumbs down and slightly on the side of "rotten." In all honesty, there are some movies that yearn for a "sideways thumb" or just a lukewarm recommendation.

I've used letter grades (with pluses and minuses) and numbers before, but it became too complicated. If you obsess over the rating more than how strongly you recommend/don't recommend people seek out the movie, something is wrong.

_________________
https://www.facebook.com/ken.rossman.5


Mon Apr 15, 2013 10:53 am
Profile
Cinematographer
User avatar

Joined: Sun Oct 07, 2012 8:21 pm
Posts: 562
Location: Milwaukee, WI (USA)
Post Re: The Arbitrary Rating System
I would concur that a 2.5 is a slight recommendation. I never did a direct correlation on Ebert, but I thought a 2.5 in print from him was a "Thumbs Up" on his show.

For me, I try to follow the 4-star rating although, depending on the movie, I do find myself inconsistent at times. What point is the movie when the opening credits roll? 0 stars and what follows "adds points"; 4 stars and then I ding it for missteps; 3 stars and then reward and punish as necessary? I guess for something like "Twilight" (a series I've never seen) I would be pre-disposed to go in with "0 stars" and add and something like "Lincoln" I'd start at a prejudiced "4 stars" and subtract. In many cases I let pedestrian feelings influence me. One online reviewer I like doesn't generally comment on the "quality of a movie", but on whether or not the experience merits your 10 hard earned bucks. For example, while I recognize that something like "Citizen Kane" is a better movie than "Raiders of the Lost Ark", I personally would rate the latter higher than the former in my "enjoyability rating"; and that will influence my final "star assignment".

So basically, don't trust any star ratings from me you may see in the "Last Movie Watched" section. They could just as easily be influenced by what I had for dinner the night before as anything else.


Mon Apr 15, 2013 11:08 am
Profile
Assistant Director
User avatar

Joined: Sun Nov 18, 2012 2:37 am
Posts: 983
Location: Laurel, MD
Post Re: The Arbitrary Rating System
Johnny Larue wrote:
I would concur that a 2.5 is a slight recommendation. I never did a direct correlation on Ebert, but I thought a 2.5 in print from him was a "Thumbs Up" on his show.

For me, I try to follow the 4-star rating although, depending on the movie, I do find myself inconsistent at times. What point is the movie when the opening credits roll? 0 stars and what follows "adds points"; 4 stars and then I ding it for missteps; 3 stars and then reward and punish as necessary? I guess for something like "Twilight" (a series I've never seen) I would be pre-disposed to go in with "0 stars" and add and something like "Lincoln" I'd start at a prejudiced "4 stars" and subtract. In many cases I let pedestrian feelings influence me. One online reviewer I like doesn't generally comment on the "quality of a movie", but on whether or not the experience merits your 10 hard earned bucks. For example, while I recognize that something like "Citizen Kane" is a better movie than "Raiders of the Lost Ark", I personally would rate the latter higher than the former in my "enjoyability rating"; and that will influence my final "star assignment".

So basically, don't trust any star ratings from me you may see in the "Last Movie Watched" section. They could just as easily be influenced by what I had for dinner the night before as anything else.


I always watched Ebert & Roeper whenever I could, and for Ebert, anything less than 3 stars in print was thumbs down on the show. Unfortunately, Siskel died before my interest in movies skyrocketed, so I never once saw their show "fresh," if you will.

Being inconsistent with ratings sometimes just goes with the territory. Movies are an art, not a science, and so it goes with reviewing the arts.

_________________
https://www.facebook.com/ken.rossman.5


Mon Apr 15, 2013 11:26 am
Profile
Second Unit Director
User avatar

Joined: Sun Oct 07, 2012 4:57 pm
Posts: 418
Post Re: The Arbitrary Rating System
KWRoss wrote:
Johnny Larue wrote:
I would concur that a 2.5 is a slight recommendation. I never did a direct correlation on Ebert, but I thought a 2.5 in print from him was a "Thumbs Up" on his show.

For me, I try to follow the 4-star rating although, depending on the movie, I do find myself inconsistent at times. What point is the movie when the opening credits roll? 0 stars and what follows "adds points"; 4 stars and then I ding it for missteps; 3 stars and then reward and punish as necessary? I guess for something like "Twilight" (a series I've never seen) I would be pre-disposed to go in with "0 stars" and add and something like "Lincoln" I'd start at a prejudiced "4 stars" and subtract. In many cases I let pedestrian feelings influence me. One online reviewer I like doesn't generally comment on the "quality of a movie", but on whether or not the experience merits your 10 hard earned bucks. For example, while I recognize that something like "Citizen Kane" is a better movie than "Raiders of the Lost Ark", I personally would rate the latter higher than the former in my "enjoyability rating"; and that will influence my final "star assignment".

So basically, don't trust any star ratings from me you may see in the "Last Movie Watched" section. They could just as easily be influenced by what I had for dinner the night before as anything else.


I always watched Ebert & Roeper whenever I could, and for Ebert, anything less than 3 stars in print was thumbs down on the show. Unfortunately, Siskel died before my interest in movies skyrocketed, so I never once saw their show "fresh," if you will.

Being inconsistent with ratings sometimes just goes with the territory. Movies are an art, not a science, and so it goes with reviewing the arts.


I think the only time Ebert gave a thumbs-up to a 2 1/2 star review was for Sudden Death


Last edited by patrick on Mon Apr 15, 2013 6:18 pm, edited 1 time in total.



Mon Apr 15, 2013 4:03 pm
Profile
Cinematographer
User avatar

Joined: Sat Dec 15, 2012 9:35 pm
Posts: 639
Location: Puerto Rico
Post Re: The Arbitrary Rating System
I give more weight to a review than I give a rating. Still, I think ratings are a useful tool for both measurement and reference. That said, it's still a personal thing, so it's supposed to be subjective. The rating is a way to encapsulate your feelings about a film in a simple way.

Being a professor myself, I tend to favor the A-F grading system.

_________________
"Get busy living, or get busy dying"

Visit my site: Thief12 profile


Mon Apr 15, 2013 5:56 pm
Profile WWW
Cinematographer

Joined: Sat May 05, 2012 4:33 pm
Posts: 584
Post Re: The Arbitrary Rating System
The more I think about it, the more I love the simple Thumbs Up/Thumbs Down rating system devised by Gene Siskel and Roger Ebert. Simple, to-the-point, and it allows for greater elaboration in terms of the actual movie, as opposed to stressing over the appropriate rating. A terrible movie might receive Two Thumbs Down, while a great movie would earn Two Thumbs Up. A bad movie would garner a Thumbs Down, and a good film would receive a Thumbs Up. (Everyone should know this, by the way. Why am I explaining?) A movie that I absolutely hate receives the designation of Most Hated, and a classic earns the rating of Great Movie.

So, I essentially have six ratings to choose from. Pretty simple.


Mon Apr 15, 2013 9:22 pm
Profile
Cinematographer

Joined: Sat May 05, 2012 4:33 pm
Posts: 584
Post Re: The Arbitrary Rating System
Johnny Larue wrote:
I would concur that a 2.5 is a slight recommendation. I never did a direct correlation on Ebert, but I thought a 2.5 in print from him was a "Thumbs Up" on his show.

For me, I try to follow the 4-star rating although, depending on the movie, I do find myself inconsistent at times. What point is the movie when the opening credits roll? 0 stars and what follows "adds points"; 4 stars and then I ding it for missteps; 3 stars and then reward and punish as necessary? I guess for something like "Twilight" (a series I've never seen) I would be pre-disposed to go in with "0 stars" and add and something like "Lincoln" I'd start at a prejudiced "4 stars" and subtract. In many cases I let pedestrian feelings influence me. One online reviewer I like doesn't generally comment on the "quality of a movie", but on whether or not the experience merits your 10 hard earned bucks. For example, while I recognize that something like "Citizen Kane" is a better movie than "Raiders of the Lost Ark", I personally would rate the latter higher than the former in my "enjoyability rating"; and that will influence my final "star assignment".

So basically, don't trust any star ratings from me you may see in the "Last Movie Watched" section. They could just as easily be influenced by what I had for dinner the night before as anything else.

I personally consider **1/2 a Thumbs Down rating. On the other hand, I am not really sure what Ebert regarded it as. On the one hand, most of his **1/2 reviews are considered "Rotten" on Rotten Tomatoes. However, when I was watching Ebert Presents At the Movies one day, I noticed that he gave Country Strong, a movie he had given **1/2 to, a Thumbs Up. There are, of course, dozens of other examples of this from the Siskel/Ebert and Ebert/Roeper eras, but this is the most recent one that comes to mind for me.


Mon Apr 15, 2013 9:32 pm
Profile
Cinematographer
User avatar

Joined: Sun Oct 07, 2012 8:21 pm
Posts: 562
Location: Milwaukee, WI (USA)
Post Re: The Arbitrary Rating System
Sean wrote:
I personally consider **1/2 a Thumbs Down rating. On the other hand, I am not really sure what Ebert regarded it as. On the one hand, most of his **1/2 reviews are considered "Rotten" on Rotten Tomatoes. However, when I was watching Ebert Presents At the Movies one day, I noticed that he gave Country Strong, a movie he had given **1/2 to, a Thumbs Up. There are, of course, dozens of other examples of this from the Siskel/Ebert and Ebert/Roeper eras, but this is the most recent one that comes to mind for me.


I guess mathematically if you are giving 1/2 star ratings on a 4-star scale, and assuming zero stars is a valid rating point, then you have these potential ratings:
0, .5, 1, 1.5, 2, 2.5, 3, 3.5, 4

If you are then going to apply a binary system of "thumbs up", "thumbs down", then it would make sense that anything above the mid-point of 2 would be a "thumbs up". Having 6 values being a negative and only 3 being positive seems rather draconian. Even if "0" is not a valid ratings point, 2.5 is still on the upper half of the binary scale.

So if you see one of my pathetic "reviews" in the "Last Movie Watched" area, consider a 2.5 as a tepid endorsement.


Mon Apr 15, 2013 11:14 pm
Profile
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 37 posts ]  Go to page 1, 2  Next


Who is online

Users browsing this forum: AJR and 2 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group.
Designed by Vjacheslav Trushkin for Free Forum/DivisionCore.
Translated by Xaphos © 2007, 2008, 2009 phpBB.fr