Sample variation and, as Bjørn notes, undetected internal damage are possibilities to watch out for in any used lens.
My copy of the 28/2.8 Ais is amazingly sharp, center to edge, and fully useable at f/2.8. Also features a fairly flat field of focus at wide open.
I haven't had the 28/2 since film, so I can't comment on it's sharpness on modern digital camera applications.
Based on my sample of the 24-70/2.8G (used one for a year), for 'sharpness' I'd pick the 28/2.8 Ais at f/2.8 vs. the 24-70/2.8G at f/2.8.
The 28/2 will show a bit more barrel distortion in the medium-close focus range than the 28/2.8 Ais.