An alternative would be to not use silicone at all. Use foam weather stripping.
Wrap a piece of weather stripping around the baffle, compress and replace baffle, when the weather stripping absorbs water it will swell locking the baffle in place.
I have used this method for a few years now on...