Your new partner knows you have a child, and what you are going through. Is he in a position to do the travelling or to move?
Its a huge ask of someone to give up full custody for a relationship that is only 6 months old and from your post it seems your focus is making the relationship work as opposed to it being an pleasant optional extra once your custodial arrangements are done.
Relationships cone and go, but your child will always be your child and time spent away from them is time you can’t get back.
If this new partner is worth their salt they will understand and they will wait for you to ensure a good pattern of visitation, and take a back seat while you focus on your kids. If they don’t then they obviously aren’t that serious about you…if they do then they will have proven their commitment not only to you but also respecting your role as a mother.
In terms of forcing away, it sounds like your mind is already 300 miles away, and that may be how he is interpreting it.. Maybe you’d be in a stronger position with the house if you were to make it clear your focus is your kids, and you are wanting to stay put no matter what to be with them and everything else comes second.
.
.