*Membership spots not really limited!
i wouldn't because in the X-Films they would be horribly cast ciphers of the characters. Whereas in the Avengers they would be reasonably well cast fully fleshed out characters written by someone who clearly loves them.
But the thing is that Magneto is an important part of their lives and seeing as how Magneto is part of the x films at Fox it would be hard to not mention him if they put them in the Avengers film at Disney. However if Fox and Disney were to work out a deal where both the X-Men and Avengers were to meet that could work.