As i said i do not use the SD card but so far I havent thought that inserting a card while not using it may be a delaying factor as well??
The SD in the slot sould only delay if you set the memory management to "backup". The buffer then clears at SD card speed not at XQD speed. I did that once. It was healed by emptying the SD card slot.
The fps and AF speed of the D850 and possibly the data rate too, depend on the voltage the camera is operated with. With the ENEL 15 it is 7 Volts, the D5 batteries work at 11.4 Volts. The buffer size stays the same but the buffer fills much faster with 9 fps. I shoot 14 bit RAW lossless compressed plus JPEG fine and did not fall into a full buffer yet. For birds I might fall back to NEF only.
Buffer fills faster with uncompressed than with lossless compressed of course.
PS: my D850 sometimes does not recognize the ENEL18 if an ENEL15 is present. Work around is to open the battery compartmemt, slide theD5 battery out and in again.
Never had that with an empty ENEL15 compartement.