Have you ever thought about going to mediation along with your son’s mother?
She really shouldn’t be dictating when you can see your son, especially as you seem to be trying to be actively involved in your son’s life. She might think it is in his best interests, but it might not be?
If you go to mediation, an impartial person can help you both agree the best way forward and it’s easier to have a conversation with someone who is not involved and can hopefully help you both see it from both sides.
Also, they might be able to help you personally with the financial side of things.