The End of the Star System
Monday, March 12, 2012 at 05:55PM 
Last week, the Los Angeles Times stopped awarding “stars” in its restaurant reviews. I’ve decided to do the same, but with a twist.
Unlike the LAT, I am still going to rate restaurants—in my own way (see below). Ratings are still meaningful, and I believe that consumers both expect and value them. But the existing stars are too laden with baggage to be useful any more.
The Problem
Although the LAT’s decision precipitated mine, the underlying issue has been on my mind for several years. The LAT explained it this way:
Starting this week, The Times will no longer run star ratings with our restaurant reviews. There are a couple of reasons for this. First, star ratings are increasingly difficult to align with the reality of dining in Southern California — where your dinner choices might include a food truck, a neighborhood ethnic restaurant, a one-time-only pop-up run by a famous chef, and a palace of fine dining. Clearly, you can’t fairly assess all these using the same rating system. Furthermore, the stars have never been popular with critics because they reduce a thoughtful and nuanced critique to a simple score. In its place, we’ll offer a short summary of the review.
There are also thoughtful comments from Huffington Post and preciently, a couple of weeks earlier, from Toqueland’s Andrew Friedman. Personally, I do not think the stars are any more difficult to apply than they were five, ten, or twenty years ago. They’ve always suffered from several problems.
First: a luxury restaurant is usually a candidate for three stars, but a disappointing luxury restaurant gets two; perhaps one or zero if it’s really bad. Conversely, a small neighborhood place usually gets one star, but it can get two if it’s exceptional. So the two-star level is a collision point, where you could find anything from SHO Shaun Hergatt to Parm.
Second: in the age of Yelp, most people are conditioned to think that if a restaurant gets one star, there must be something pretty badly wrong with it. Sometimes that’s true. But there are also some really good restaurants that have received one star in The New York Times—restaurants that the critic very clearly liked (despite some limitations), such as The Spotted Pig and Imperial Palace.
Compounding this problem, a number of critics use the same system nominally, but apply it in very different ways. Bloomberg’s Ryan Sutton awards zero to four stars, but to him two stars is “good, reliable,” while one star is “fair.” At The Times, two stars is “very good,” while one star is “good.” Time Out New York awards one to five stars (never zero), so one star there is terrible.
Third: there is an unwritten rule that some types of restaurants just cannot get three stars, no matter how good they are. Pete Wells’s threespot for Il Buco A&V may be an attempt to change that—we’ll have to see—but for the most part only fairly luxurious, expensive restaurants even get the chance for three stars.
Finally: the star rating, at least as practiced by The New York Times, takes price into account. Restaurants sometimes get docked a star for being too expensive (in relation to perceived value); others get a “bonus star” for offering an exceptionally good deal. But this system is open to manipulation. Time and again, restaurants have raised their prices after receiving a rave review. The rating remains available years later on the paper’s website, even after the bargain prices that contributed to it are no longer offered. (I wrote a blog post decrying this practice a couple of years ago.)
These problems have been around for a very long time—perhaps forever. Because there are such heavily ingrained views about “what a three-star restaurant must be,” any attempt to redefine the system while still awarding stars, is doomed to fail. What’s needed is a different system entirely.
The New System
I am going to classify NYC restaurants in the following way:
Extraordinary: One of the best five to ten restaurants in the city; a restaurant that has it all. A transcendent experience, one of the world’s best. Worth a trip to New York in its own right.
Category Killer: A restaurant that aces its category, on its own terms, and without comparison to restaurants in totally different genres; the best, or very nearly the best, of its kind in NYC, without any serious weaknesses or omissions.
Critic’s Pick: the restaurant does something out of the ordinary, something that makes it better than the average place you can find in just about anywhere in town; a place worth traveling to—assuming the cuisine and ambiance fit your mood, tastes, and price point; a minor destination.
Neighborhood Spot: if you’re in the neighborhood, it’s nice to know it’s there. Worth considering if you’re in the area.
No Recommendation: Not recommended; I wouldn’t go back.
Simplistically, there are five levels, just as there were before, from four stars to zero. But these ratings no longer carry the same meanings. If you believe (as Sam Sifton did) that Motorino serves the city’s best pizza, then you can rate it a “Category Killer,” even though Sifton gave it just one star in the old system. (I have not reviewed Motorino.)
Even when I was awarding stars, I tended to think of restaurants in a hierarchy, as above. A two-star restaurant had to be a destination in some sense, while a three-star restaurant needed to be a destination in every sense. But time and again, I was frustrated by the need to maintain fidelity to what the stars had traditionally meant. Thus, I’ve assigned “Category Killer” status to The Spotted Pig and Minetta Tavern, even though I never would have considered giving them three stars.
The top rating—but only that one—retains its old meaning. Since there are no more stars, I’ve replaced it with its synonym, “Extraordinary.” This level has remained relatively pure over the years. Four-star restaurants have practically always been luxurious and very expensive, and there have never been many of them. I could envision a system where the city’s best hot dog stand gets four stars, because it’s the best that a hot dog stand can ever be. But no professional critic has ever come even close to doing that.
Over the years, I’ve gradually moved away from taking price into consideration. I am now making it explicit: ratings do not take price into consideration. I always state in my reviews what I paid for the food at the time. You can decide for yourself whether the restaurant is offering a good value. Price and value are dependent on too many factors that a critic can’t assess. Of course, if I think I overpaid or got a terrific deal, I’ll still say so. It just won’t affect the rating.
I’ve never made a distinction between rated restaurants and “$25 & Under,” as The New York Times does. But it is worth noting that this system can work at all price levels. Restaurants are rated against the Platonic Ideals of themselves. Shack Shack could be a Category Killer, if you believed it was the ideal burger stand. (That’s just a hypothetical; I haven’t reviewed Shake Shack, but the last guy who did wasn’t impressed.)
There can be more than one Category Killer of the same kind, but there can’t be too many. If you think that twenty sushi restaurants are Category Killers, then none of them are. That’s one of the reasons why I’ve given no steakhouses that status (unless you count Minetta Tavern as a steakhouse—which I don’t). There are a number of steakhouses that I recommend, but none that really stands sufficiently apart from the others.
This new system clearly does not eliminate subjectivity. No doubt more chefs think they are operating Platonic Ideals of their restaurants, than actually are. But at least this system articulates specific criteria for the ratings, eliminates price as a factor, and does not purport to measure wildly different establishments on the same numeric scale.
The Transition
RedFarm is the first review published on the new scale. It’s a Critic’s Pick.
Starting today, I will gradually convert my old reviews to this new system. I have hundreds of reviews accumulated, so this will take some time. I am not going to update the reviews of restaurants that have closed. And if I reviewed the same place multiple times, I am only going to update the most recent review (the one linked from my ratings page).
The Restaurant Index page now shows all the restaurants I’ve reviewed in approximately their final positions, but I am still adjusting them. (For reference, the old ratings are available here.)
It is possible that, after doing this for a while, I will find that this solution isn’t all it’s cracked up to be. But the ratings described above correspond reasonably well to the way I believe critics ought to think about restaurants. It’s a system that I believe can work, and that other critics could use—not that I am holding my breath.
Reader Comments (2)
Bravo, very thoughtful piece. Your site has long been one of the few food blogs that gave ratings that were not inflated, and I admired you for that. Bloggers are too worried about not pissing off restaurant owners (for whatever reason), but you do things your way, the right way.
However like you said, the general public are used to the Yelp-type polarities. 1 or 5, everything in between is ignored. Your new system is a logical next step for food blogs. I'm sticking with my scoring system for now, but will need to add words to describe what exactly that score means.
Get you: it is tricky to adopt a rating system. For my web blog, I analyzed many existing rating systems..only to go back to the basic: 0-10/10 system. And yet, I feel that this is a work in progress. For me, the way to go is the result of what I found missing in other people food reviews, so what I ended up doing is rating each dish, rating the overall meal and even adding a pros/cons sections as well as a 'what I think months later' section. That way, everything is there: the comments, the marks, the progression of the evaluation. Nothing will ever be perfect, and that is what keeps this whole thing interesting. Cheers.