I can at least think of a few counterexamples:

• Cats live longer than dogs.
• Small stars live longer than large stars (by dramatic amounts).
• Black holes larger than the ~10^10 kg mark shown here live a lot longer than any other object with the same mass. A hypothetical moon mass black hole can most likely outlive every single star in the universe.
• Protons are smaller than anything in this chart yet are believed to decay over an extremely long time scale (>10^34 years).
• Even smaller particles like electrons likely live forever.