assuming you are using good quality brushes the comm on the end of the armature must be rough, you could try spinning it with a cordless drill while you have a very fine sandpaper 500/600 grit wrapped round it, the commutator is where the brushes touch the armature, if you strip it give it a good clean before re assembly
keymatic, June 2012