I have left my main guild of 2 years. I will miss them. However, they were a close-knit group of real life friends and families and naturally distrusted outsiders, some took to me, and some did not. There was a habit of staying silent when a request was made in guild chat. After a while this became actively getting blanked. I decided I'd be brave and step away from a situation that wasn't making me happy. Like a bad relationship. I actually now have. So I have 2 toons in one guild who are not so close knit and quite nice, and one toon who holds my guild/bank with 3 tabs.
Remember, it's a pass-time you play with others, if they make you unhappy, there are others out there who will, go find them instead, they'll appreciate it too. I'm going to be brave, you should too.
Hopefully, see you out there. Footloose and guildfree ;)