The same thing happened to me - I wrapped parcel tape around my hand, (sticky side outwards), and pressed it onto the pollen stains to remove them. The pollen grains stick to the tape and come off easily. Worked brilliantly on cream cloth.
Good luck!
Ali, December 2009