Anyone know where they keep the fun around here?

I'm at a loss for how to begin processing the last few days.
I decided I needed to stay here at my brothers for a few more weeks to try to distance myself from all the pain, and craziness going on at "home".
I am truly ready to begin moving forward but that feels like an impossibility when I still don't know the truth of the past. Husband still isn't speaking to me other than to send hateful and mildly threatening text messages, and those sure aren't helping me process.

When I told him I needed to stay here longer he threatened to "arrange for somebody to come pick Esme up and drive her back" that made me so frustrated. He's going to feign this great concern for his daughter but really he can't even pull that off. Every time he expresses concern for her it's concern that he might not get his time with her, even that isn't about her, but about him. And then when he takes this concern to the extreme instead of coming to collect her himself (which would demonstrate his ultimate concern) he intends to arrange for someone else to come.  Anyone else vote that he's probably sending "his best friend"?

Then when I make it clear that it is not an option for me to send her back to his world of lies and confusion his response is to ask how I would feel if he took her on an extended trip.
There was only one response I wanted to make to that statement. Since I know it wouldn't have helped I resisted...but I have to get it out of my system here.

How would he feel if I had been sleeping with someone other than him for the last 1 - 4 - 6 - 8 months?
Probably not as bad as I feel knowing he was and I guess that's the whole point.

I've truly done nothing throughout this process but show concern for his and Esme's well being. He refused to move out and really wanted the house...I didn't care because I can create a home anywhere (note the last 8 years of my life). I gave him the house.
He wanted to disappear and told me flat out he didn't care if he ever saw his daughter again.
I finally faced off with him, stuck my foot in the door when he tried to slam it in my face, and told him straight up that would be one of the worst decisions he could make (too bad I didn't already know about the worse decisions he had made).
I encouraged him to stay, by giving him the house he wanted and encouraged him to spend all the time he wanted, 2 days and 3 nights a week with his daughter. Even though all of you saw how much those days and nights killed me.

When his girlfriend moved in for her extended visit I put Esme's well being first by refusing to let her spend the night in the house with them, but still sent her there for two day long visits with them. I still wonder if I made the right choice since she came home from the first day and announced to a room full of friends that "daddy was having a sleepover just like she was. Daddy was sleeping with (insert "his best friends name here)".  Then she came to me in tears confused about why this person was sleeping in mommy's bed with daddy.

There are some things you just never anticipate having to explain to your 3 year old.

So through all of this I have explained patiently, and tirelessly that I am now trying to protect my daughter and I from this continuous stream of bad decisions.

And now I need a break! I need a few weeks of fun, a few weeks to think about what I want out of life rather than react to what he wants. A chance to take Esme to parks, play with her, love her and have fun.

And all he's willing to give me is the threat of a lawsuit. Boy did I pick a winner or what?