That sounds awful. Stressful & Impossible to settle into a routine.
Are you in the UK? if so, an access order is about when you must make your child available for the other parent to collect. Neither you nor the court can force them to turn up.
is the breakup recent? Is the ex angry that living arrangements have turned out as they have? Are they doing it intentionally to upset you? Are things likely to settle down? Is your child old enough to understand what is going on or still a baby?
I would Document the shouting & threats, and every time they don’t show and you are left struggling for cover at short notice. If it hasn’t settled down in a few weeks,I think I’d go back to court and ask for supervised contact at a contact centre. 💐