Rob, the polarizer is a good filter to get - in this case I used it to darken the skies and make the clouds really stand out.
As you play with a polarizer, and rotate it around on your lens, it works like polarizing sunglasses, and cuts out glare.
Get the angle right, and the sky goes deep blue (or black in B&W) and the clouds really stand out nicely.
You can use it to take reflections off of surfaces like water or glass as well - as you rotate the filter, you'll find the sweet spot where reflected light gets blocked by the filter.
I used a polarizer on this shot to get both effects - dark skies and (mostly) clear water
Polarizers can also help keep colors saturated on bright days - so when shooting in sunshine I'll often use one to ensure that my deep colors stay that way in the final shot.
The final useful effect is that they reduce the light coming into the camera, from 1-2 stops.
This can be key if you want to use your widest apertures in full sun - as in it's basic configuration, the X100 cannot cope with that much light coming in, and the shutter cannot fire fast enough to get the exposure correct.
You could change you aperture to correct, but that effects your depth of field, and if I'm shooting wide open, I'm going for that super narrow depth of field, with everything in the background nicely out of focus (OOF).
I can buy a special Neutral Density filter to use to cut the amount of light down, and get to use my wide apertures that way - or, I can get the same effect with my polarizer (up to 2 stops).
If I need more light reduction, I can buy ND filters that give 3 stops of reduction, 6 stops, etc etc.
If you are looking at filters, make sure they are multi-coated to keep the optical quality up - look for makes like Hoya HMC or Tiffen multicoated.