My ex did something similar, I brought it up with CAFCASS during the court proceedings and he didn’t have a choice but to send it to me. So you can
A. You can bing it up with CAFCASS.
B. Save yourself the stress declare the passports lost/ missing and just get a replacement, if you do this, the ones in his possession will be useless anyways.