I recently read an excellent blog post about 'Overkill Penalties'. This is the idea of penalising min-max players to bring them in line with more casual players or players that want to concentrate more on the roleplaying of their characters instead of the roll-playing aspects.
In D&D, the type of person who builds a character around a concept instead of mechanics, for example, choosing on purpose to use a spear despite its d6 damage over a long sword that does d8, is at a distinct disadvantage to the player who only ever picks a bastard sword because it deals 2d4 damage. The post I've linked to talks about doing penalising these min-maxers, who are less interested in the game setting and only in generating massive numbers, in a more modern game setting or a futuristic one, but this general idea easily apply to fantasy ones too.
Here's a quick idea for how to implement it in D&D; anyone that takes an enemy to -10hp from a positive number of hitpoints, in one hit, unleashes the spirit of the fallen humanoid as a spectre that remains on the mortal coil because of the nature of its violent death. Feel free to tweak the 'spectre threshold' based on the edition of D&D you are using and so on. Why a spectre? It's not massively overpowered but is a hugely dangerous creature with its draining powers. It could always be swapped out with a wraith - or wight - at lower levels.
So why bother doing this or even considering it at all? It's about putting the emphasis on using the least possible amount of force to get the job done. It would help not just with min-maxers (i.e. people who try to game the system for purely mechanical advantage, often at the expense of everyone else) but also with situations that really should be handled with more delicacy. The party mage might be a bit more careful throwing his fireballs around, for instance and the paladin may not simply smite every enemy and then rest after his smites have all run out. You might even find players switching to subdual damage when a foe is running low on hitpoints, letting the enemies merely slip into unconsciousness so the party can continue on their way instead of slaying them outright.
What are the downsides? Depending on your rule set, critical hits could accidentally unleash vengeful spirits. You could always rule that they do not, they are exceptions, or leave it on, as an unexpected and unintended possible consequence of driving your sword right into the heart of an opponent and ending his life in a split second.