There are only 3 areas where you can get cold: head (and neck), hands and feet. Cover them and you'll be fine.
Here is what I mostly wear:
- a warm cap and a scarf or
- a balaclava if I use my helmet (but wear it tight and don't block your view)
- different gloves depending on the temperature (fingers are the most sensitive part and get cold first): I have 3 pairs - a woolen one, a fleece one and skiing gloves. If the temperature is above +5 °C I just wear the fleece gloves, if it has -5 to +5 °C I wear the woolen and the fleece gloves together, if it is below that I wear my skiing gloves. I hardly ever wear the woolen gloves alone.
- since I lean forward on the bicycle I wear long shirts to cover my kidneys
- a soft shell, kind of neoprene, jacket (it's windproof and has colors that can be seen from afar) or my coat
- legwarmes if it's really cold or if I wear a skirt (yes, I do wear skirts and tights in the winter - it's not particularly colder than a jeans, but much more stylish ;-))
- if it's really cold I wear warm knee-highs, also designed for skiing
- waterproof shoes or boots (waterproof shoes are quite practical if you have to stop in the mud)
- I own overshoes, but I never use them since I have waterproof shoes anyway
- reflector bands around my ankles (unfortunately I don't have a chainguard)
- if it gets really dirty cause the cars created a lot of dirty mud on the streets I will wear waterproof trousers above