Due to how Mechanical Engineering works it's always about tolerances,,, You can't judge such measurements based on Nominal values.
The interesting measurement from the strength point of view is the Edge measurement not the Size especially with the now almost obsolete hexagon shape, now splines are used instead since they are far superior especially because of the Edge being so critical but also since the resist wear much better - Anyway,,,
The Edge tolerances are the factor, if you look up these you will find that worst case Edge for Metric key S min is 3,95mm and in an Metric screw it is a difference of 4.58 - 4.44 mm = 0.14mm ie it wobbles a little but it works,,,
For an Inch key S=4mm, the minimum is 3.9mm and in an Metric screw the Edge measurement worst case is; 4.58 - 4.40mm = 0.2mm of missing material!
The difference is thus about 0.06mm