Unless he’s in prison, I’d set up an email address and tell him that all access arrangements must be organised by email in future. Block his phone. Block him on social media. Hang up immediately if he calls from another phone.
Access involves him collecting his child and taking him out. No other options. Does your ex drive? Can he get to you? If so wild horses wouldn’t keep any decent loving parent away.
If he’s going to abandon his son, better he does it now, while your son is still young enough to forget.
Also, if he does arrange to visit your son, don’t tell your little one his dad is due, in case your ex decides to let his son down.