One thing I have learned about CA - it does not, for me, color anything. CA prevents color transfer. I do segments and when I use other finishes, some other finishes (lacquer, acrylic/Acetone, some urethanes) will leach some woods such as bloodwood and ebonies and transfer the colors to lighter colors.
IF the color transfer for you is grays, or blacks, it is usually present on the metal either before applying or occurs during sanding. It sounds like metal sanding dust. Are you sanding the metal before applying it? Are you sanding the metal and wood after it is glued up? This will certainly do what you said.
There are several ways to either prevent or remove and it is easier for me to prevent than to remove.
Some people use DeNatured Alcohol. Some use other liquids, still that is difficult to remove from light woods. I like working with holly and it is not easy to clean holly.
I discovered a technique that I later found that some others used - I turn the wood and metal segments smooth with a chisel, to the point that it does not need sanding. Hard woods are capable of turning with a scraper chisel until smooth. Turn the lathe on FAST and use a sharp sharp scraper and take small light bites. On softer woods, a skew does much better. A scraper does not work well on soft woods unless they have been stabilized well.
The problem with sanding segments with metal in it is that the metalic dust is plum ugly on beautiful wood. And removing it is a pain. Prevention is better.
IN the first pict below is a couple of pens that could not be sanded and therefore required metal dust prevention.
The second and third pict is a close up 4 years later. OF course you can not see any sanding dust because there was never any there.
http://www.penturners.org/photos/images/940/1_30_Pieces_of_Silver.jpg
IAP Home - Photos - Clip end close up
IAP Home - Photos - segments '08