I'm able to reproduce.
I can confirm that when this happens, it's the OVH server that fails to send the response to my network.
35 9.862648672 MY_PUBLIC_IP_AT_HOME → 54.38.38.159 ICMP 78 Echo (ping) request id=0x4b30, seq=33150/32385, ttl=1
36 9.862704895 54.38.38.159 →
MY_PUBLIC_IP_AT_HOME ICMP 106 Destination unreachable (Port unreachable)
MTR from the OVH server to home:
Start: 2023-09-07T23:30:08+0200
HOST: rbx Loss% Snt Last Avg Best Wrst StDev
1.|-- 54.38.38.252 0.0% 10 0.6 0.5 0.3 0.7 0.1
2.|-- 10.162.250.98 0.0% 10 0.9 0.5 0.4 0.9 0.1
3.|-- 10.72.52.32 0.0% 10 0.5 0.5 0.4 0.7 0.1
4.|-- 10.73.17.42 0.0% 10 0.2 0.2 0.1 0.3 0.0
5.|-- 10.95.64.152 0.0% 10 1.1 1.2 1.1 1.5 0.1
6.|-- 54.36.50.226 0.0% 10 4.4 4.4 4.2 4.7 0.2
7.|-- 10.200.2.73 0.0% 10 78.0 11.6 4.1 78.0 23.4
8.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0