Indeed.
In due time, I have a feeling that O'Brien will fade away from the memories of a lot of fans. Personally speaking, I just get a really bad sense of 'fan-casting' when I try to picture O'Brien as Peter/Spidey. He physically resembles the character and might like to joke around like Spidey, but I don't think he'd be able to realistically hold his own against the likes of Robert Downey Jr. or the other big MCU leads.
Though, to be fair, I'm not entirely sure that Tom Holland can either. I mean, I'm sure he'd be comfortable working with Hemsworth again, but it looks like we won't be seeing that happening (if at all) until Infinity War. Having said that, Holland is a much better actor than O'Brien is, I think.
Asa though... while I can totally understand folks' concerns about how he would handle the character (his take would definitely be different from Holland's), I do think he could very easily go toe-to-toe with the other MCU leads. The kid's been up there with the likes of Harrison Ford and Martin Scorsese and has the acting chops to match Holland.