2014 Review Contest Results / Discussion

[SDHawk]

This year Gizmog, JSH357, and Charbile completed the review challenge (in that order!). You might want to check out Charbile's final game review.

As far as contest trends go it's pretty clear from the charts that the contest is locked into a fairly steady pattern. There are people who enter it, and there are people who don't. Looking at the charts you might think OHR game releases are on a decline, but do recall that the requirements to enter were a little more lax in the first review contest.

Let's get into discussing future contest rule changes. I think it's safe to say that the selection thread addition was largely pointless this year, no one used it other than to be ashamed of their own games (unsurprising since people aren't really playing the games before the contest starts). On the flip side I think the compilation list was a big success. So much so that I'm thinking of expanding it next year to include every ohr thing on the game list that doesn't normally qualify: tech demos, minor updates, tools, etc.

There's only one big rule change question to discuss this year: How do we handle updated games? Up until now the general qualification rule is something like "there's roughly 30mins of new content (if the author has even specified changes), or it hasn't been in the review contest before". This caused some problems this year. So what do we do about it? Right now I'm leaning towards putting all games that were previously in a review contest on the compilation list. That's pretty harsh to updated games, though.

In short the questionnaire for this year is:

1. Should we keep the selection thread process or get rid of it?
2. Do you like the compilation list (everything too short is lumped into a list that is reviewed together as one compilation review)?
3. Should we expand the compilation list to include tech demos, minor updates, tools, etc?
4. What do we do about updated games? Put them all on the compilation list, have stronger conditions for main list placement, allow reviewers to use their older reviews if they feel not enough has changed, or something else?

[Gizmog]

In short the questionnaire for this year is:

1. Should we keep the selection thread process or get rid of it?
2. Do you like the compilation list (everything too short is lumped into a list that is reviewed together as one compilation review)?
3. Should we expand the compilation list to include tech demons, minor updates, tools, etc?
4. What do we do about updated games? Put them all on the compilation list, have stronger conditions for main list placement, allow reviewers to use their older reviews if they feel not enough has changed, or something else?

Quick thoughts before bed:

1. Selection thread process could maybe be changed to an "Opt-Out" thread, and simultaneously a "talk people out of opting-out" thread. I don't always play a lot of games before the review contest starts and don't feel comfortable being the guy to choose what's worth reviewing and what's not.

2. Absolutely, that is a huge improvement to the old system. Smaller games don't get the slime knocked out of 'em, people can still review them if they want, and you're free to make smaller comments on the parts you liked and didn't like without having to take 'em to the cleaners.

3. 100% yes. Until we're just FLOODED with tech demos and stupid slime, I think a quick "Yep, demo'd some tech" is fine and kinda keeps you up to date on OHR Technological developments. It also serves as encouragement for me to um... make my demos as good as possible since people might actually be looking at them. Minor updates are trickier but I think it's fine to say "Yeah, that's a good fix" to an 8 hour game rather than demanding you replay 8 whole hours just to see the typos have been fixed.

4. We've talked about this at length in IRC, but my opinion is still basically: If an old review is still relevant, an author should be allowed to use it again. If someone else plays the game and sees that points the author made previously have been addressed, that new reviewer is allowed to call the old reviewer a lazy bum and the next round of drinks is on him.

A *MAJOR UPDATE* should be on the main review list. This would constitute actual gameplay+story content. Motrya Chapter 2 would constitute a *MAJOR UPDATE*. Adding Lyte Snap to a Lyte Snap-less Motrya would not, nor would adding a post-game meet and greet with the surviving NPCs ala Earthbound. For this, we would be somewhat bound by the descriptions provided by the authors, and the selection thread would be a good place for them to plead their cases Major or Minor, review me again or not.

A minor update would be something like bug-fixes, typo-fixes, or the the addition of mini-games, and would go on the Compilation list. Mini-games being included because they're essentially tech demos in parasite form.

Something like this year's "Harley and Joker" controversy should be addressed in the review: If you were interested enough in the advertised content to want to see it, but were unable to slog through the prerequisite first game to get there and think the content should've been made available seperately, that's content for a good review.

Long-form game development should be encouraged at all costs, reviewers should try to save their save games as much as possible in case of *MAJOR UPDATES* and people developing long games should endeavor to make sure that older saves are compatible.

Even extensive graphical reworks should be considered minor updates. Doom RPG being the best example of this: No matter how much nicer the maptiles get, no matter how many games worth of level progression and equipment management you glue on top, you're still going through the same intro, the same mandatory textboxes and story beats and nothing's really changed (except, inexplicably, making it impossible to complete the intro. That was weird!)

I don't want the review contest to be this great boogeyman that people are afraid of. I want people to release games and not worry about it. At the same time, if the review contest encourages people to quit making the same handful of mistakes (not including game.exe, unskippable intros or cutscenes before risky boss battles, etc.) and through so doing makes the average OHR Game better, I think that'd be a triumph.

The role a review plays to a game can be reviewed. Should it be an advertisement, an endorsement by a pleased customer? Should it be professional analysis and critcism? I get the feeling that some people think there shouldn't be any negative reviews on the list and I don't agree with that. At the same time I've seen some scathing hatred dumped on people who didn't deserve it. It's a weird debate.

[charbile]

was fun to read and swap thoughts. want to thank hawk for putting together the list and all, know that stuff can be a thankless job.

questionnaire...

include all the games to the list regardless of what they are or if only a minor update. is neat to see entire year's worth of games.

compilation list is cool. no idea what to expand it to.

discussion thoughts...

i think it's too much work to make the average ohr game better. it's a lost cause.

if these reviews are meant for the work's creator, which a lot of these you can take as, then it's more like feedback or coaching. which is fine, can also fit as reviews. but it also can come across as pretentious, missing the point, ill-willed, etc.

the problem has always been treating these games as things they aren't. like a contest game as something that had the time to be tweaked and playtested, or as anything outside the context in which it was released. and it's often hard to figure out that context, especially to someone who doesn't keep up with updates or what's what with ohr games.

or i guess it's been these games are often very difficult to get into. can't stress enough having played the amount of ohr games i have that you have to stop taking things for granted and being lazy, expecting to be showered with praise in reviews. like everyone's all about wanting a good review source so they can be flattered and get the attention to feel like a real pro. i get that it's rough. the only way you're going to get that praise is make a game someone would like to play, and hope that that someone with the same interests actually plays it. the reverse boogeyman has always been in play too, giving games higher marks just to make people feel better.

what makes this interesting is having to review games nobody wants to play and talk about outside creator and friends or fellow-contestants, who were also forced to say something out of an obligation to whatever contest rules.

i like reading the commentaries about people's own works the most. it does feel like dumping on the games a lot, but also everyone here dumps on big deal commercial games all the time, do any of these have a chance? well yeah, on the level of things easy to compete on like the experience, narrative, play, etc. but that itself is no easy task--easier to dress a game up in pretty graphics and sound. it's why graphics have always been a big deal here and we're all about showing off screenshots.

would love to play more games like samanthuel's lovely home, or megaman sprite game. or even that taradiddle. do i want to play random battles and walking around maps for a few hours? not really. you don't have to use the ohr engine like mad-libs, fill in the blanks with whatever childhood character creation you've grown fond of. it's always asking too much, i know.

keep on doing your thing. it's easy to ignore reviews, just as easy as it is to ignore games.

[Pepsi Ranger]

Sorry I didn't get a chance to review anything this year. I had planned on reviewing the Heart of the OHR games, but my plate was too full this round. I'll have to get to them on my own time.

1. Selection thread process could maybe be changed to an "Opt-Out" thread, and simultaneously a "talk people out of opting-out" thread. I don't always play a lot of games before the review contest starts and don't feel comfortable being the guy to choose what's worth reviewing and what's not.

I don't remember even seeing this thread. I doubt it's necessary. All games are worth reviewing if someone's gonna gain from reading it.

4. We've talked about this at length in IRC, but my opinion is still basically: If an old review is still relevant, an author should be allowed to use it again. If someone else plays the game and sees that points the author made previously have been addressed, that new reviewer is allowed to call the old reviewer a lazy bum and the next round of drinks is on him.

I know I was a big contributor to the controversy that led to this, and I apologize if it got out of hand. Having said that, I think that if you're trying to get to new content but can't, then explaining why you stop playing (and where) is definitely valid. Part of my gripe this year was that no one who reviewed just the old content really explained why they couldn't get to the new content, or even acknowledged the new content they were electing to skip. At best, they might've mentioned that the game was updated this year, but that's kind of a given seeing as how the game wouldn't be on the list if it hadn't been. But in fairness to those reviewers who didn't know what's new, I think people releasing games should also make an effort to inform the community what new things they're presenting. That's not a condition for the review contest, but it should be required of the game's author if they do want the new stuff acknowledged. It would be harder to complain about a review's treatment of new content if the information isn't out there in the first place.

A *MAJOR UPDATE* should be on the main review list. This would constitute actual gameplay+story content. Motrya Chapter 2 would constitute a *MAJOR UPDATE*. Adding Lyte Snap to a Lyte Snap-less Motrya would not, nor would adding a post-game meet and greet with the surviving NPCs ala Earthbound. For this, we would be somewhat bound by the descriptions provided by the authors, and the selection thread would be a good place for them to plead their cases Major or Minor, review me again or not.

This is the main reason why I'm responding. I agree with this policy. I think it's fair. However, it runs the risk of becoming highly subjective. Like, you could argue that Tim-Tim's new world is good enough to constitute an update, but what if that new world is just a single stage, and not that big? I think it helps to know exactly what's been implemented since the last release (not just in terms of new gameplay or whatever, but how much does it add to the old game?). This means someone's gotta know what's in there before it can be considered major or minor. How many people will be willing to play a game just to evaluate its new content? What exactly is required to make it major?

Honestly, I think this could get complicated enough to cause additional controversy if it's handled too subjectively, and it might cause too many headaches if handled with too much itemizing.

A better solution might be to create a new category: Updated in 20XX: and let the reviewer decide if it's worthy of a total analysis or just a compilation style list of comments. If standard reviews and compilation reviews can fit in civilized societies, then updated games can fit somewhere in the badlands of the Wild West. Might make things more interesting. It might also give the author greater incentive to spell out what's new so the reviewer doesn't have to try figuring it out. If he wants a substantial review, then he better make sure the reviewer knows what to look for. And if the evidence of new content is there, and the reviewer chooses not to address it, even to say that he couldn't slog through the old stuff to get there, then we get to banish the reviewer from the Wild West, and he has to go back to the civilized world where only tidy reviews live.

Or, we could just do what we're gonna do anyway and see what plays out.

A minor update would be something like bug-fixes, typo-fixes, or the the addition of mini-games, and would go on the Compilation list. Mini-games being included because they're essentially tech demos in parasite form.

That's an interesting way of looking at them, and pretty accurate.

Something like this year's "Harley and Joker" controversy should be addressed in the review: If you were interested enough in the advertised content to want to see it, but were unable to slog through the prerequisite first game to get there and think the content should've been made available seperately, that's content for a good review.

Just because I was the biggest complainer in this controversy, I think it's fair to say again that I agree with this policy.

Long-form game development should be encouraged at all costs, reviewers should try to save their save games as much as possible in case of *MAJOR UPDATES* and people developing long games should endeavor to make sure that older saves are compatible.

I'm not sure I'm in as much agreement with this. If it's a straight-to-the-point continuation, then sure. But most updates aren't like that. Most include new areas and fixes or modifications to older areas. I think using old save files should be optional, but a text outlining a general list of changes should be what's required. Then, if the reviewer pulls an Okedoke and quits long before the majority of new content is even touched, then he can talk about the content he didn't see and get by with the "couldn't slog through it" defense. This also absolves the author from holding something he hates about the early game hostage for fear that the player might use an incompatible save file that completely breaks the remaining game.

Even extensive graphical reworks should be considered minor updates. Doom RPG being the best example of this: No matter how much nicer the maptiles get, no matter how many games worth of level progression and equipment management you glue on top, you're still going through the same intro, the same mandatory textboxes and story beats and nothing's really changed (except, inexplicably, making it impossible to complete the intro. That was weird!)

COMPLETELY AGREE!

I don't want the review contest to be this great boogeyman that people are afraid of. I want people to release games and not worry about it. At the same time, if the review contest encourages people to quit making the same handful of mistakes (not including game.exe, unskippable intros or cutscenes before risky boss battles, etc.) and through so doing makes the average OHR Game better, I think that'd be a triumph.

This is what I've always thought the point of reviewing was for. Published reviews for commercial products are generally for the consumer since most commercial products are as-is once they hit the shelves. In a niche community like ours, however, we're all designers who play games, so it makes more sense that our reviews focus on what the author can improve more than it does what the player can play. Figure, all the consumer review will accomplish is to let the player know whether a game is worth his time. You could honestly just say, "yes it is," or "no it isn't," and that's plenty for the player. Everything else is just pretentious analysis. When you review for the author, then you're providing details that are useful.

The role a review plays to a game can be reviewed. Should it be an advertisement, an endorsement by a pleased customer? Should it be professional analysis and critcism? I get the feeling that some people think there shouldn't be any negative reviews on the list and I don't agree with that. At the same time I've seen some scathing hatred dumped on people who didn't deserve it. It's a weird debate.

I don't see a reason to debate a review unless it somehow becomes a complete waste of time to read: as in, says nothing that helps anyone, or is written with the purpose to troll. We can't judge subjective viewpoints, but we can comment on anything that proves the reviewer hasn't actually played the game, or proves he or she is trying too much to convince the author to rewrite the game into something that only the reviewer could enjoy.

Scathing reviews are valid if the game deserves it. The reasons it may deserve it are subjective. We can't judge subjective. It teaches authors to develop thicker skins, which is important for life.

P.S. I voted for you, Giz. You did an awesome job this year. Sorry if that should've been anonymous.

[SDHawk]

P.S. I voted for you, Giz. You did an awesome job this year. Sorry if that should've been anonymous.

People are totally free to discuss what they liked and disliked about each reviewer's reviews in this thread if they want to. If there's demand for it we could even have a formal non-anonymous OHR contest style voting thread next year (or maybe just a less formal explicit invitation to comment on each reviewer's work).