As a verb, this is what it's called when a mob suddenly clears its aggro list, switches to Evade mode and runs quickly back to its spawn point, where it resets.
Overland[]
Leashing is the primary means for players escaping combat. From an early level, all players learn that if you run from a mob in one direction for long enough, the mob will break off combat and reset. However, hitting a mob seems to reset the leash range. One way to pull a mob to a far off location is to periodically attack it (for example, walking backwards while meleeing it) to keep its attention.
Leashing is also one way to prevent training, in the sense of griefing somebody by luring mobs onto them. (Since the mob will be in evade mode while returning to its spawn, and will not aggro on anyone it runs past.)
Bosses[]
For boss mobs, particularly raid encounters, leashing is triggered in a slightly different way. Some boss mobs are intended to be fought in a certain area. If the boss is pulled outside of that area, it will automatically leash (clearing aggro and returning to its spawn in evade mode as above), regardless of how recently it has been hit. Note that there are many bosses who either do not leash or are fought in an area where escape is impossible.
EXAMPLES:
Most of the priests in Zul'Gurub (with the exception of Arlokk the Panther and Bloodlord Mandokir) can be leashed. If a pull is bad or the fight goes awry, it is a common strategy to simply have the remaining people run out of the area. Hakkar will then tell the priest to "not be lured into the mortals' trap," and the boss will reset. This strategy has been known to glitch however. Thekal once continued chasing raid members around the entire instance despite being told to go back home four times by Hakkar. He was persistent enough to swim through rivers, cross bridges, dash up hills, and even go into Arlokk's temple seeking out every last raid member and cutting them down.
Another example of the leashing mechanic that works in a different way is Iron Juggernaut in Siege of Orgrimmar; if the Iron Juggernaut is pulled outside of the fenced off area towards the friendly NPCs the boss will instantly despawn and the encounter will be reset, causing the boss to respawn about 30 seconds later. This behavior is common across multiple encounters in different raids in the game world.
Instances[]
Non-boss mobs in instances generally do not leash. The only way to escape them is to exit the instance, or to use an aggro-clearing ability like the hunter's [Feign Death]. Some instances however have spots that cannot be pathed to by NPCs. You can identify such a location by attempting to attack an NPC while standing at it; if the NPC faces you but evades, the NPC cannot get to the location. Most of these spots are accessed by jumping onto objects or different elevations that cannot be reached by walking. This is to prevent players from being able to kill NPCs without being attacked themselves. (Note that the new knockback effects being added in Wrath of the Lich King would create many problems with evading mobs, and thus it is reasonable to infer that there will be some game mechanic to prevent such evade locations from appearing.) At these evade-locations, a player need merely to stand there long enough. The mobs will give chase as far as they are able, then stand and stare at you for a while, probably wondering why it is nobody ever taught them to jump, then they will remove you from their aggro list. If the evade-location is within aggro range of their normal patrol route or spawn point though, the leashed NPCs may immediately attempt to re-engage you; although they will be unsuccessful, it will make you unable to leave combat for more than a second or less.