Please... it's all Fox can do just to manage a single franchise and its spin-offs, much less two more franchises and a crossover movie. And character development and screen time within ensembles is bad enough in the X-Men franchise without the added complication of adding MORE characters.
I would like to see X-Men and the Fantastic Four cross paths in a movie but I don't think thats going to happen anytime soon. And as for Daredevil, I don't think he would fit in. Fantastic Four and X-Men are superhero teams and Daredevil is just by himself. He would fit in the Avengers more.