So when this is happening mtr works but http, ssh and ping don't?
Yes
If you run tcpdump on the server, do your http/ssh packets even
reach it?
I wasn't seeing any request coming from my IP last time I checked. I'll re-check.
If you do tcptraceroute on port 22/80/443 from your home, how far
does it get?
Will test it next time it's blocked.
Are there any firewall rules in place on your server?
No (no fail2ban, ...).
DO you have access to any other host so you can tell if it is just
your home IP that has problems?
Yes, I have multiple servers (in OVH and outside OVH) that can all reach the server on IPv4. Same if I switch to my 5G connection, it works. Only my home's main IPv4 is blocked. IPv6 is OK.
I was even able to ask people using the same ISP to confirm, they can reach the IPv4 of my server.