One of the more annoying questions I ask on forums, in playtests, and in real life is this one:
"Has that problem come up in actual play?"
Sometimes I use a variant, like "How has that played out in your game?"
Why this gets annoying is that I bust this out early and often in discussions.
Someone: "I found this problem in GURPS that breaks the game."
Me: "Did that problem come up in actual play?"
If the answer is yes, well, let's get down to figuring out how to fix it, house rule it, change it, adjust it.
No, it hasn't?
Then I am not convinced it's a big problem.
When playtesting, when house-ruling, or when thinking about changes or non-changes to a game, I place a very high premium on actual play outcomes.
A good example of this is the Committed Attack option in GURPS Martial Arts. When the book was in development, I convinced my players to let me try out lots of the rules in actual play, on their actual characters, with results that would stand in the campaign. So we tried out Committed Attack in play.
During the alpha testing, it got used a lot by some of my NPCS (me, trying it out in play) and especially by one of the PCs (a player who thought it was awesome). During the playtest, a playtester commented that it was basically useless because it traded off too much defense for too little offense, and no one would use it. But I had actual play experience that said, yes, it's a potentially dangerous tradeoff, but sometimes that's what you need and some players will take that risk. So it stayed in unchanged. Had there been a wave of actual play results where people said, no, it never gets used, and then we tried this variation and it got used more and worked out well . . . I'd have been much more willing to tweak it or change it in some fashion. So it was "actual play" vs. "in theory." Play trumps theory, because it's a game.
Sometimes what is bad in theory - or bad for one group in play - isn't actually different from the rule's intention. At least in that case, you get what you aimed for even if not everyone wants that result.
Sometimes what happens in actual play with a rule or rules change is different than what you expected. This can be good, or bad.
The Good: Doug Cole has a great example of this, what he refers to as "emergent behavior." A set of rules meant to make grappling more realistic not only gave results in line with reality checking and actual grappling, but also with ass-kicking over-the-top movie scene grappling. It wasn't what was planned for, but it was what happened.
The Bad: The original writeup of Beat (also GURPS Martial Arts) just let you substitute ST for DX when making a Feint, and didn't require any actual contact with the defender - contact was assumed. The goal was to reflect real-world beats, and to allow people to leverage their strength to push a defender's weapon (or shield, or arm, or body) out of a good defensive line to open it up to a follow-on attack. On paper, it looked fine.
So in my campaign we had a massive, multiple-session fight involving three pirate ships, at least 150 combatants, zombies, an ifrit, invisible weapon masters, light siege engines, sharks in the water, and magically summoned critters all over the place. One of those critters was a Dodge 15, ST 1, DX 15 flying eye monster - which was simply killed outright with a Beat followed by an Attack. Nevermind the guy couldn't have laid a finger on the critter in most circumstances - he couldn't land a telling blow because it would Dodge with ease. But mystically he could push it around physically without making contact. He did a Beat, won the ST contest by a huge margin (ST 1, it's just a flying eye), and then whacked it. It made no sense . . . but it seemed fine on paper. In actual play, broken. So it got fixed. What emerged from having this rule was strong guys using Beat to make physical contact with guys they couldn't otherwise touch and easily trashing them - not fun and probably not realistic. It needed even more revision as new versions got tried out in play and also spat out odd results or strange results, until we got the now-canonical version that does what we were trying to have it do in the first place. Again, actual play mattered a lot.
I can be really annoying with that question, but it's always worth asking about a rule. "X is broken." "Broken in play?" Sometimes what comes out of a "broken" rule in play is fun or interesting. Sometimes what comes out isn't, in fact, broken. Sometimes what comes out of a fix isn't fun or interesting, either - the law of unintended consequences doesn't care much about why you changed something.
It's a game, so actual play feedback is critical when evaluation the rules. So, almost inevitably, I ask my annoying question whenever I'm confronted with a question about a rule or a house rule. I think the answer is valuable, and not asked often enough.