I am fairly sure ballistic gel rots relatively quickly, and putting it inside latex is unlikely to help matters.
I would suggest seeking out a different material. When Adam and Jamie were testing the airplane vacuum-toilet myth in season one of Mythbusters, they were looking for realistic skin/flesh simulant and ended up using Dragonskin silicone from Smooth-On. Dragonskin is used for things such as prosthetics and sex-toys because it is skin-safe and has a realistic feel.
FWIW, Latex also "rots" from exposure to UV light, it is a natural material and requires regular maintenance and proper storage to last.