The easiest way to remove old caulk is by using caulk remover.  They sell this product at just about any home improvement store. As you might guess, it is typically located near the caulk and sealants. Here's what you'll need to get started:
- A putty knife
- A straight razor blade
- Caulk remover
- Rubbing alcohol
1. Start by squirting a bead of caulk remover over the caulk that you want to remove. Make sure the entire line is covered.
2. Allow the caulk remover to set. Typically the instructions on the bottle say to wait several hours. However, the longer you leave it on there, the easier the caulk will be to remove. If you are dealing with particularly old or hard caulk, you can leave it on for up to 24 hours to really loosen the bond between the caulk and the surface. 
3. Once the time is up, simply use a putty knife to start scraping away the caulk. It should come off in large chunks or strips, making the removal far easier than if you tried to peel it away without caulk remover.
4. Use the razor blade held at a flat angle to gently scrape away any bits of caulk that may still be stuck to the surface. Use caution not to damage the surface that you are working on.
5. Clean the area with rubbing alcohol to get rid of any last traces of the caulk. This will help ensure that the new caulk you put down dries correctly and bonds properly to the surface.
Alternatively, if you have really steady hands, you can also use a utility knife to cut the caulk out without the need for caulk remover. However, it is important to note that you can cause damage to the surfaces you are working on if your hand slips or goes off track using this method, so you should only try it if you are completely confident in your abilities. Here's a video that shows how: