Are you sure it's glass? the material it's made of will determine how you try to remove the caulk. On glass surfaces, and because the caulk is probably silicone, basically nothing can dissolve it. After removing as much as possible with the razor blade, try a plastic scrubber pad and an ammonia based cleaner like Windex and gently scrub the surface. I have used this method before and it has worked for me. The plastic scrubber is rough enough to scrape off caulk residue but gentle enough not to scratch the glass, as lomg as you don't scrub too hard.
On plastic, you will have to be really careful not to scratch it, although in a sump it probably won't matter so much if the surface is scratched. On plastic, I would suggest using only hard plastic tools, like a credit card type of tool, to scrape the surface, and then use a gentle all purpose cleaner with a soft plastic scrubber pad to remove the rest. Ammonia based cleaners can damage plexi surfaces.