Since one cannot see it from here, I may be all wet, but am wondering if the broken piece is in any way hollow. If so, one possibility might be to find a small but sharp screw, and see if you can screw it in to the end of the broken piece just enough to pull it out. The spring holding it is likely not too strong, so it should not take a great deal of pulling force to pull it out if you're very careful.
If the broken part is made of plated brass, as it might be, it might even be possible to drill a tiny bit into it, but you'd have to be very careful of chips and debris. One of the handiest things I've ever had was a kit of tiny twist drills made originally for cleaning the tips of welding torches. The kit was crazily cheap, and the drills very very small. I've used them for all sorts of non welding applications such as extraction, carburetor jets, and the like over the years. The kit has a hollow handled hand chuck, drills inserted in the chuck, stored in the handle.
