Why no love for Secret Weapons?

[Hugmenot]

I don't know of any wargame with its own set of rules that was sufficiently tested before release in the past 6 years.

When a game with yet to be published rules goes into the development queue, the playtesters test first with the alpha rules (assuming the publisher has done due diligence and ensure the game is not broken) and send feedback. That feedback is incorporated into the next iteration of rules. Thus, the next round of testing is performed under a different set of rule. Depending on a number of factors, this process goes on for a couple months to a coupe of years until the rules are entirely stable and the game is fairly balanced, or until the drivers are burned out. The testing is basically a process done by (mostly) volunteers, and said volunteers are only willing to give X amount of their time before they decide they will have more fun testing another game or playing one game from their own collections.

The X amount of hours is more problematic for longer games because that amounts to a lesser number of complete plays. An example. Matt and I tested the campaign for yet-to-be-released game by another publisher. The full campaign takes over 20 hours to play. The first thing I did is tried to break the campaign early, which I did by abusing the victory point allocation mechanic. Eight hours to discover that strategy and then a couple more hours back and forth to discuss potential solutions with the designer. We tried one and it worked well but next we discovered a campaign-only rule is not are not as clear as we first thought and the, depending upon the interpretation, favors one side much more than the other. I wrote the designer with our interpretations and asked him to provide the revised wording. Note that I did not ask for the correct interpretation because I should be able to determine that from my reading of the revised rule. i received the response the next day but for all intent and purposes, few more days would not matter because Matt and I can only test together on weekends. You get the picture. In the end, I played the campaign once in its entirety and specific days from "reasonable" starting positions a few times in the roughly 100 hours I spent on this project. There were lots of playtesters but they mostly played the scenarios and sent very little actionable feedback.

So, do I believe the campaign is well-balanced?

All I am willing to say is I believe both sides have a chance at victory and I expect results to fall within 75-25 to 50-50 for one side or the other. Yes, I am not even sure which side has the advantage. The 75% limit is just a hunch from my experience playing this game and similar systems.

Does that represent enough development and testing?

From my perspective, a definite yes to development. There were enough plays of the scenarios to test the mechanics. It does not mean the testers caught everything, just that we were not catching anything new at the end. Again, from my perspective, yes to testing. The scenarios were played many times and I am not a believer that playing a 4 to 6 hour long more than a dozen times will change anything; it will not prevent cases where someone finds a much better strategy for one side after the game is released. Sure, Roger Miller may play a game 40 times before he releases it but he has a strong financial incentive to do so as he is one of the two principals at Revolution Games. That decision has allowed him to leave his other job and become a full-time publisher. I am glad he did because Roger is a very good guy and Revolution Games is my favorite company that offers games under $40.

Now back to PG.

A small scenario may take one hour to pay, a very large one over 20 hours. Getting to a point at which we're very confident a very large scenario is well balanced is impossible. I have invested well over 20 hours on a few single scenarios but at the end, all I could say is that I believe both sides can win. I think that is sufficient because I doubt any one player will play them more than once and my experience as a player is always positive if I believe either side could have won if I had done something different.

The same with medium scenarios except that I developed some parameters to decide, before testing, if a scenario is too unbalanced. I may have invested up to 8 hours on those except in the rare case where I discovered something suspicious during a test.

The small scenarios can be better balanced but there is also the problem they are more fragile. More fragile that an early extreme result (say a 2X result ) has a much greater impact on the result than lager scenarios. This is especially true for tiny scenarios (less than a dozen units on each side). We generally tried to provide (fun) options to the players rather than focusing too much on the balance.

I will leave now by saying I spent roughly 1,600 hours developing PG games and supplements and I definitely reached my X early in 2017. I've helped Matt on occasions, but not much, preferring to play the unplayed games I accumulated these past few years.