No, because those who wanted the headband had the honor to fight him one-on-one. Not once were the kids presented as being in danger of becoming casualties of that headband because none posed a threat to the those who fought for it. For example, when Afro's father was killed--the new number one could have killed Afro but there was no honor in doing so. As a matter of fact, he even invited Afro to a duel once he is truly ready. Those men that fought Afro's master had no honor. This is evident by the fact that they all ambushed him for something that only one man can possess. And since the children were there as witnessess, there's no doubt that they would be the next target for useless violence no matter what they did.