to have a skeleton in the cupboard

a skeleton in the cupboard (BrE) (also a skeleton in the closet NAmE, BrE)

(informal) something shocking, embarrassing, etc. that has happened to you or your family in the past that you want to keep secret