I think it's more dependant on stun than hp when it comes down to brawling. I gathered this info from doing a search on the brawl code, and that's how it seemed to me. So, if you bob the merchant a few times, or something else ate away their stun, my interpretation would be if you went and hit them it would've knocked them out, thus, brawl-code says they're too far gone to hit, it means you'll probably knock them out if the hit connects. So if you want to fight this person you have to use kill instead of hit. But, that'll getcha crim-flagged since it means you intend to do more than knock them around a little bit.