IMO, I think for Winter Soldier, at least, expressions
have to count since the poor guy doesn't get to say that much. I mean he has the line in Russian on the causeway, the "who the hell is Bucky?", about 3 lines during his 'repair' scene, and then the "You're my mission" (which he says twice, once clearly, the second time accuating it with each punch to Steve's face) - other than that it's grunting and yelling during his final fight with Steve. That's like 6 lines in the entire movie and all of them pretty short lines (so
his CBR interview comment of "I'm finally in a silent film!" is kind of true
) And he DOES have some great expressions. The one you said, the one after he tells Pierce the 2nd time "But I knew him" and everything that follows that line. Earlier in the film him being much more stoic/focused worked because he was still essentially the mindless assassin. But once those memories start jostling loose....Sebastian Stan just rocked everything he did in this movie.