Hi BeckyLou
I know about anxiety, I have been there, so I understand ho wit is making you feel.
My opinion, for what it is worth, is that, really, you harldy know the man and so should be very careful to begin with. Obviously you have done the decent thing and kept your baby, whome which I am sure you love to bits and have no regrets.
He does sound like he is untrustworthy or reliable. It only sounds like you have seen him a couple of times since the birth and so I have to question whether you both have a relationship in his mind anyway. Sorry to be so blunt, but its your health and wellbeing we are interested in here.
Are you holding on to something that isnt there because of the thought of being a single parent? Do you love him? Do you really want to be with him?
Anxiety, as I am sure you know, stems mainly from the unknown or situations being up in the air, which they obviously are for you. I believe you need to know yourself what you want and then the rest can be dealt with accordingly.
What would you say to the above? I hope it has not come across to harsh.
Paul