Simple job, get some aluminium Mag Wheel cleaner from any auto spares shop. Spray on the swing arm , softly rub it with a green scotchbrite pad and rinse off within 1 minute, try not to get on the rims , I normally just slot some cardboard in between the wheel and swingarm. It comes out brand new.
** Please read the mag wheel cleaner instructions and to be safe test on a small patch before doing the entire swingarm as the mag cleaner is quite acidic and can cause harm if left on too long **