Yes, they can be swapped. However, the dirt gets trapped between the ball and the rollers and popping the assembly out may not necessarily remove whatever is stuck inside the assembly. You'd have to disassemble the trackball assembly (which I've done but don't recommend to just anyone) to remove the dirt that way.