Designer ranking his own scenarios?

[Poor Yorek]

One issue to consider, perhaps, when evaluating even a designers "play" and/or AAR, is that, in my opinion, the variance to be expected based on PG combat tables is "large" compared to the number of die rolls in even a big scenario. Simply put, a designer could play the same scenario dozens of times with the same strategy/tactics and approach and yet attain quite different results (that being said, some scenarios, even with wide variance, will end usually in the same result - albeit perhaps arrive there differently).

I'm suggesting therefore, that though a designer's play might be weighted as of more value than any single given player at large, it ought not to be weighed too heavily, that is, should not "sway" one's judgement in and of itself. Turning that around, the designer need and ought not to consider their tendered play/analysis as necessarily "canonical."

I think this is one of the greatest values of PG-HQ's AAR section as we can observe the variance for a given scenario play out over time as folks submit Actions.

[Shad]

My freshman year of college I had to take an English class. In that class the professor required us to grade our own essays with comments before turning them in. My final essay for the term I clearly remember grading myself a B. My reasoning at the time being that while I believed my thesis was sound, I was not able to fully support it with textual references and literary evidence. To me this is perfectly reasonable, especially when one considers that what was "a good idea" in 2008 may no longer - with the benefit of hindsight - be a good idea in 2012.

Do you people who object to designer self-ratings have performance reviews in your workplace that require personal performance ratings? What do you do? Fill in N/A on every blank?!?

[zovs]

I agree with Shad on this.

At work we grade our selves and then our team lead grades us and then our peers and then we see the feedback from all three and talk about it. It's interesting to see what the individual rates himself as, his peers and his boss.

It works.

[plloyd1010]

Yes, Drew, we do. I and my boss, and his boss think it is stupid. It also isn't really germane, because the audience is different. Alan & Vince get it. None of us are suggesting Jay, or any other designer, wouldn't give an honest evaluation of their work. Amongst the sea of self promoters, and with design expectations, it really is best if designers stay out of the ratings game.

BtW, that thesis, did you really think your grading the paper was about your writing & research?

[zovs]

it really is best if designers stay out of the ratings game.

I have to disagree with this opinion.

Designers are also game players so there is no reason they need to stay out of a rating of a game.

Where it can get bad and negative is at say Game Squad where Designer # 1 hates Designer # 2 and they both viciously attack each other at every chance, that gets tiresome and that may only be seen on ASL places, but stating that a Designer should stay out of a ratings game is at best just silly, in my opinion.

Heck look at some of the old Generals, Moves and S&T magazines, designers did just that. It was not until F&M came out where "purists" wanted to see the ratings for players come on more.

Designers Ratings are like all things, their opinions of something, it's not gospel, and I may even disagree with them, or the opposite be so intreged that I go out and buy said product to see if its that bad/good as Mr. Soandso says it is.

[Shad]

Yes, Drew, we do. I and my boss, and his boss think it is stupid.

As an old HR guy, I suppose I hold personal evaluations (& 360 degree evaluations) in a higher regard than most.

Amongst the sea of self promoters, and with design expectations, it really is best if designers stay out of the ratings game.

To me the crux of the issue is whether it is obvious that Rater/Reviewer A is actually the person behind Game B. If there is a disconnect or some deception there then we have a problem. But I maintain that to expect or even demand silence does nothing productive.

BtW, that thesis, did you really think your grading the paper was about your writing & research?

The professor graded us wholly independently of our own evaluations. The point was to learn to critically evaluate our own performance. As you older generations are so found of pointing out, us post-1980 kids grew up in the everyone-gets-a-trophy period and are often ill-equipped for the not-niceties of The Real World.

[Matt W]

I'm a bit late to the party I'm afraid but I don't feel designers rating their own work to be a problem. This is a factor of three things:

First, this is a small but active community where I have trust in the people who put in the ratings.

Second, there is the potential for substantial variation from play to play for each scenario. I have found myself really enjoying low rated scenarios and bored by highly rated scenarios but I find that the AARs are usually very helpful.

Last, we simply don't have enough plays yet to consider many of the ratings statistically valid so any input on a scenario can be useful.

I hope that someday we can have sufficient ratings where statistical validity will matter but with something shy of 4,000 plays and 2,000 AARs for a universe of around 1,700 scenarios we aren't anywhere near that point. I understand that this could bias the sample but with such a small sample at this point I see this as a theoretical problem, not a practical one.

[J6A]

I don't mind a designer doing AARs on his own game, however I think that rating ones game is dicey. If there is disclosure, that helps, as people can take things with a grain of salt. I'd prefer letting the players make their own judgments without what could be seen as 'shilling' involved.

[JayTownsend]

Ok then, it sounds like a designer rating his sceanrios and doing AARs on his own material is ok overall but ranking your own game is in the gray area?

So I guess I will contiunue doing AARs and ranking my own scenarios but not ranking my games on BBG. I think doing AAR's bring more interest to the scenarios.

[Blackcloud6]

My initial reaction is "no" for raring but "yes" for AARs. But thinking about it, rating is fine in my view. No actually looks at a scenario the same as someone else; we all have known or unknown biases. As more ratings add up, the biases start to cancel out. So yes, a designer can, and should rate his scenarios.