Actually the noise itself is higher for the larger format. The noise varies as the square root of number of photons which means that the signal-to-noise ratio (SNR) varies in the same way. So the larger format has a higher SNR and it is this which manifests itself in a "less noisy" picture.
I mention this not in the cause of pedantry but because Eric Fossum made the same point [...]
Another way of putting Eric Fossum's point is that you can treat the whole sensor as a single pixel, and the bigger the pixel the lower the noise. You can do that because it turns out that you can bin pixels into larger and larger units with no increase in noise. The problem with that analysis is that it does not tell you anything helpful about images with significant amounts of information content.
In any image there are two sources of variation in voltage between pixels: noise and true pictorial information (noise as false pictorial information caused by lens aberrations could be considered separately as well). The only practical way to identify noise is to eliminate pictorial information by photographing a uniformly illuminated screen, so you can assume that all pixel to pixel variation is noise. If you are not interested in pictures of uniformly illuminated screens the SNR is
not the square root of the number of photons. The signal includes information about the picture, and you
can't bin pixels without losing information about the picture. A single pixel has an
information signal to noise ratio of almost zero: the only information you get is the average scene luminance.
Smaller pixels allow you to collect more content information, because the sampling rate is higher. Smaller sensors have more noise than bigger sensors, but the pictorial information the smaller sensor collects may be less, the same or more, depending on the pixel density (and on the quality of the lens), so the signal to noise ratio
including pictorial information in "signal" can be anything.
In most photographs the pictorial information is overwhelmingly predominant as a component of "signal" compared to the average scene luminance. So sensor size has no important effect on the signal-including-pictorial-information to noise ratio,
except when there is little pictorial information so that average luminance is a large proportion of the "signal" (eg, pictures of blue sky or areas of still water reflecting the sky).
Of course, there is an additional component of "signal": aesthetic or, as the case may be, political or scientific information. If it is present, as it all too often is not, that is so much larger a component of "signal" than any other that sensor size becomes completely irrelevant.