Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


DNS over TLS and a weird behavior UDP vs TCP
New on LowEndTalk? Please Register and read our Community Rules.

DNS over TLS and a weird behavior UDP vs TCP

I spent quite some time to try to figure out why UDP DNS lookups (which are the default) were slower than TCP lookups on all my Macs in my home network, as confirmed by running dig @1.1.1.1 somedomain.com with dig @1.1.1.1 somedomain.com +tcp.

I still haven't figured that one out. TCP lookups are consistently faster than UDP ones, and it should be the opposite. I tried different MTU settings to no avail.

Since I was looking into DNS stuff, I switched to DNS over TLS using Stubby, and I was surprised because DNS lookups are MUCH faster now than regular lookups.

So I have two questions for any DNS experts here:

  1. What could cause UDP lookups to be slower than TCP lookups? Can it be a problem with the router/switch I was given by my ISP?
  2. Why are DoT lookups faster than regular lookups? I wasn't expecting that due to the overhead of the encryption.

Anyone? Thanks

Thanked by 1harrison

Comments

  • Shot2Shot2 Member

    Loss ?
    Service availability/priorization ?

  • @Shot2 said:
    Loss ?

    Do you mean packet loss? I don't see any

    Service availability/priorization ?

    I haven't configured any of this

  • jackbjackb Member, Host Rep

    Is your ISP intercepting the UDP request and answering it themselves? Wouldn't be the first time I've seen a residential ISP do that and wouldn't be unexpected for their server to be slow responding.

  • @jackb said:
    Is your ISP intercepting the UDP request and answering it themselves? Wouldn't be the first time I've seen a residential ISP do that and wouldn't be unexpected for their server to be slow responding.

    How would I verify that?

  • UDP traffic is often deprioritized by ISPs.

    @vitobotta said:

    @jackb said:
    Is your ISP intercepting the UDP request and answering it themselves? Wouldn't be the first time I've seen a residential ISP do that and wouldn't be unexpected for their server to be slow responding.

    How would I verify that?

    Usually you can do this by comparing results you get over TLS vs over a plaintext connection. Look at TTLs and other metadata, specifically.

  • @ehhthing said:
    UDP traffic is often deprioritized by ISPs.

    @vitobotta said:

    @jackb said:
    Is your ISP intercepting the UDP request and answering it themselves? Wouldn't be the first time I've seen a residential ISP do that and wouldn't be unexpected for their server to be slow responding.

    How would I verify that?

    Usually you can do this by comparing results you get over TLS vs over a plaintext connection. Look at TTLs and other metadata, specifically.

    I tried a few queries and I get the same results. But who knows what they might be filtering

  • jeghjegh Member
    edited January 18

    Interesting observation indeed! I tried this on my home network (Elisa FTTB), but this network is too fast so dig will show Query time: 0 msec regardless of TCP or UDP.

    Quick test on the LTE network (operator is Elisa here too) seems to possibly confirm your observation, but the difference is quite small and my sample size not very large
    UDP gives following results: min 18 ms, avg 43 ms, max 60 ms (n=7)
    While TCP gives: min 30 ms, avg 36 ms, max 43 ms (n=7)

    EDIT: I did a third test on DNA coax connection. Here results are between 10-13 ms regardless of UDP or TCP.

  • Interesting observation. As others have said, maybe UDP is de-prioritized by your ISP?
    Also, is this the same for Google and Adguard servers?

  • jackbjackb Member, Host Rep
    edited January 18

    @vitobotta said:

    @jackb said:
    Is your ISP intercepting the UDP request and answering it themselves? Wouldn't be the first time I've seen a residential ISP do that and wouldn't be unexpected for their server to be slow responding.

    How would I verify that?

    Does the problem go away if you enable a VPN but use the same nameservers? Are results for controversial or banned sites (e.g. pirate bay) different?

    Some residential ISPs implement blocking of things like the pirate bay by intercepting DNS.

  • emgemg Member

    I tried @vitobotta's tests. I assume the times reported are ";; Query time: xx msec" from the dig output. I am typing commands in the Terminal on an older Mac.

    At first I thought I had replicated the same results with Cloudflare (1.1.1.1). Then I tried it on Google (8.8.8.8) and got the opposite. With Google, UDP was much faster than TCP.

    Next, I started repeating an alternating pattern of UDP and TCP queries. (I pressed the up-arrow key twice then Return in Terminal. Repeat. Repeat. ...) When I did that, I noticed that the times changed a lot and TCP was not necessarily faster than UDP. Several times I got UDP times around 13-15 msec.

    I tested with all three command styles:
    dig @1.1.1.1 somedomain.com (UDP)
    dig @1.1.1.1 somedomain.com +tcp (TCP)
    dig @1.1.1.1 somedomain.com +notcp (UDP)

    ... and also "dig @8.8.8.8 ..." and "dig @[private network DNS address] ..."

    I tested on the local DNS server (my firewall/router). It is too fast to see a difference. I think it may be the DNS servers, and not all DNS servers are configured the same.

    -> I am curious to know if Vito can repeat the alternating tests that I performed to see what results he gets.

  • @jegh said:
    Interesting observation indeed! I tried this on my home network (Elisa FTTB), but this network is too fast so dig will show Query time: 0 msec regardless of TCP or UDP.

    Quick test on the LTE network (operator is Elisa here too) seems to possibly confirm your observation, but the difference is quite small and my sample size not very large
    UDP gives following results: min 18 ms, avg 43 ms, max 60 ms (n=7)
    While TCP gives: min 30 ms, avg 36 ms, max 43 ms (n=7)

    EDIT: I did a third test on DNA coax connection. Here results are between 10-13 ms regardless of UDP or TCP.

    Which operating system?

    @harrison said:
    Interesting observation. As others have said, maybe UDP is de-prioritized by your ISP?
    Also, is this the same for Google and Adguard servers?

    Yes, I tried a few public DNS, same thing. Like I said the only way I could improve DNS performance was to switch to DNS over TLS.

    @jackb said:

    @vitobotta said:

    @jackb said:
    Is your ISP intercepting the UDP request and answering it themselves? Wouldn't be the first time I've seen a residential ISP do that and wouldn't be unexpected for their server to be slow responding.

    How would I verify that?

    Does the problem go away if you enable a VPN but use the same nameservers? Are results for controversial or banned sites (e.g. pirate bay) different?

    Some residential ISPs implement blocking of things like the pirate bay by intercepting DNS.

    Weird, same thing with a VPN. I am not using any weird websites

    @emg said:
    I tried @vitobotta's tests. I assume the times reported are ";; Query time: xx msec" from the dig output. I am typing commands in the Terminal on an older Mac.

    At first I thought I had replicated the same results with Cloudflare (1.1.1.1). Then I tried it on Google (8.8.8.8) and got the opposite. With Google, UDP was much faster than TCP.

    Next, I started repeating an alternating pattern of UDP and TCP queries. (I pressed the up-arrow key twice then Return in Terminal. Repeat. Repeat. ...) When I did that, I noticed that the times changed a lot and TCP was not necessarily faster than UDP. Several times I got UDP times around 13-15 msec.

    I tested with all three command styles:
    dig @1.1.1.1 somedomain.com (UDP)
    dig @1.1.1.1 somedomain.com +tcp (TCP)
    dig @1.1.1.1 somedomain.com +notcp (UDP)

    ... and also "dig @8.8.8.8 ..." and "dig @[private network DNS address] ..."

    I tested on the local DNS server (my firewall/router). It is too fast to see a difference. I think it may be the DNS servers, and not all DNS servers are configured the same.

    -> I am curious to know if Vito can repeat the alternating tests that I performed to see what results he gets.

    I see the same behavior with Google's DNS

  • Shot2Shot2 Member

    If UDP issues occur somewhere (either throttling by the ISP, or loss-retransmit issues, or...), using a VPN (over UDP) may not help :D

  • dfroedfroe Member, Host Rep

    Can you gather and share a pcap/tcpdump/wireshark capture while doing dig @1.1.1.1 somedomain.com (first to populate the recursor's cache), again dig @1.1.1.1 somedomain.com then dig @1.1.1.1 somedomain.com +tcp and ping -c 4 1.1.1.1?

    Analyzing the packets on the wire might give a clue whether there are any retransmits, RTT of Ping, UDP, TCP handshake etc.

    Thanked by 1TimboJones
  • @dfroe said:
    Can you gather and share a pcap/tcpdump/wireshark capture while doing dig @1.1.1.1 somedomain.com (first to populate the recursor's cache), again dig @1.1.1.1 somedomain.com then dig @1.1.1.1 somedomain.com +tcp and ping -c 4 1.1.1.1?

    Analyzing the packets on the wire might give a clue whether there are any retransmits, RTT of Ping, UDP, TCP handshake etc.

    Ehm I am not familiar with whireshark etc. Can you clarify what I need to do? :p

    BTW in the meantime I asked 3 other people who are on Mac, and they see the same. UDP is slower for them too, so it's not just me

  • @vitobotta said: Ehm I am not familiar with whireshark etc.

    Install wireshark, select your active interface (likely en0 for WiFi on a Mac), start a packet capture and run the commands given.

  • @supriyo_biswas said:

    @vitobotta said: Ehm I am not familiar with whireshark etc.

    Install wireshark, select your active interface (likely en0 for WiFi on a Mac), start a packet capture and run the commands given.

    I have wireshark and have captured packets for both udp and tcp queries. The packets are different, so not sure what I need to look for? There is a lot of information

  • edited January 19

    @vitobotta said: I have wireshark and have captured packets for both udp and tcp queries. The packets are different, so not sure what I need to look for? There is a lot of information

    A full wireshark tutorial is difficult to provide in this thread, but ideally you'd look for any signs of retransmits and delays.

    If there's nothing particularly sensitive in the packet captures I guess you could toss a copy here for me or one of the other posters to look at.

  • dfroedfroe Member, Host Rep

    @vitobotta said: Ehm I am not familiar with whireshark etc.

    I'd say then it's time to change this. ;)

    In addition to what's already mentioned you may add the capture filter host 1.1.1.1 before recording to ensure there is no other unrelated traffic in the capture file.

    This should result in a rather tiny capture containing only the very few packets exhanged between your host and 1.1.1.1.

    You may share it through some file upload and folks over here will likely be happy to analyze it. :)

  • @dfroe said:

    @vitobotta said: Ehm I am not familiar with whireshark etc.

    I'd say then it's time to change this. ;)

    In addition to what's already mentioned you may add the capture filter host 1.1.1.1 before recording to ensure there is no other unrelated traffic in the capture file.

    This should result in a rather tiny capture containing only the very few packets exhanged between your host and 1.1.1.1.

    You may share it through some file upload and folks over here will likely be happy to analyze it. :)

    I think he can simply use the filter "dns". And then when saving, set the save as settings to only save displayed. But yeah, he should get some basic usage understanding for future benefit as well.

    It would also be helpful to show the times he's getting.

Sign In or Register to comment.