To be honest, I'm not sure that makes that much difference.
I'm not trying to say that the general moviegoing public is stupid, or don't care. It's unwise to make blanket statements.
But it's actually very hard to tell how people not already familiar with comics are processing all the different heroes. Far from it being clear that they regard the movies by different studios as representing completely separate heroic "worlds"... I was sitting at a showing of "Thor", and a middle-aged guy behind me was talking about, "And isn't Green Lantern also going to be in The Avengers?" (Fortunately the person he was with corrected him before I had to.

)
So, sure -- I wouldn't assume that the G.A. knows that the X-Men and the Avengers are from the same "world". But I wouldn't assume they care that much, either.
Use "adamantium" in The Avengers, and comics geeks will know that they're connected, and I honestly think either the G.A. won't care, or might at this point even say, "huh, maybe that means this is connected with X-Men".