I’d say your ex isn’t putting your son first. For a 10month old to travel that long hours and to be away from his mother at this stage would cause distress to him. As he does not know of his surroundings yet or why his mother isn’t there with him.
I am in quite a similar situation only I do not agree to the father to have him overnight at all. My son is 1.
Hope you’re okay.