Introduction to DNS & How DNS Works with Twingate

📘

Looking for a primer on DNS?

Then this article is for you: the first part focuses on DNS as a general concept and the second part focuses on how DNS works within Twingate.

IPv4 (ex: 123.3.4.5) was, up until the arrival of IPv6, the only way any computer could speak to any other computer on a network.

Because it is impossible for us humans to remember IP addresses for all the common services we rely on, Domain Name Systems (DNS) became a thing: Instead of having to remember Google’s public IPs for its search engine, now you only need to remember “google.com”. Much easier.

The problem is that machines cannot understand those names.. so DNS was invented as a solution to this, and is essentially a Service which aims at translating machine readable addresses (IPv4, IPv6) into human readable names (and vice versa): You give DNS “www.google.com” and it returns an IP address (a publicly accessible one in this case).

The com in Google.com

There is agreed upon hierarchy when dealing with domain names: The Top Level Domain refers to the very last part of the root of a URL. com, net, gov, org, fr, us, etc. are all Top Level Domains.

Zonefiles

a Zonefile is a text file made of a variety of different records for an individual domain. A zonefile resides on a DNS Server.

Here is an example for domain example.com:

$ORIGIN example.com.     ; designates the start of this zone file in the namespace
$TTL 3600                ; default expiration time (in seconds) of all RRs without their own TTL value
example.com.  IN  SOA   ns.example.com. username.example.com. ( 2020091025 7200 3600 1209600 3600 )
example.com.  IN  NS    ns                    ; ns.example.com is a nameserver for example.com
example.com.  IN  NS    ns.somewhere.example. ; ns.somewhere.example is a backup nameserver for example.com
example.com.  IN  MX    10 mail.example.com.  ; mail.example.com is the mailserver for example.com
@             IN  MX    20 mail2.example.com. ; equivalent to above line, "@" represents zone origin
@             IN  MX    50 mail3              ; equivalent to above line, but using a relative host name
example.com.  IN  A     192.0.2.1             ; IPv4 address for example.com
              IN  AAAA  2001:db8:10::1        ; IPv6 address for example.com
ns            IN  A     192.0.2.2             ; IPv4 address for ns.example.com
              IN  AAAA  2001:db8:10::2        ; IPv6 address for ns.example.com
www           IN  CNAME example.com.          ; www.example.com is an alias for example.com
wwwtest       IN  CNAME www                   ; wwwtest.example.com is another alias for www.example.com
mail          IN  A     192.0.2.3             ; IPv4 address for mail.example.com
mail2         IN  A     192.0.2.4             ; IPv4 address for mail2.example.com
mail3         IN  A     192.0.2.5             ; IPv4 address for mail3.example.com

every line in the file is a DNS record of some kind (see below for more)

See the reference to SOA, NS, MX, A, AAAA, CNAME? those are DNS Records. Keep that in mind for now.. more on that later.

Root Servers

When you point your browser to www.google.com, the top level domain of your request is com. This top level requests are typically processed by Root Servers responsible for handling Top level domain queries.

Those Root Servers respond to your browser with a bit more information (on where to go to resolve the full address) but usually not an actual public IP.

TLD Servers

A Root Server responds to your browser with more information as to who would be able to resolve the address. TLD servers are the ones helping resolve the google part of www.google.com.

Domain Level nameservers

In the same way the Root Server responsible to resolve com returned information about the right TLD Servers to query next, the TLD Servers return more information for your browser to be able to send the resolution request to Domain Level nameservers:

Those are responsible to resolving the now complete address: www.google.com. Domain Level nameservers will return the actual public IP (or IPs) matching the full address.

👍

DNS Resolution

DNS Resolution is always done in a hierarchical way from top level down.

Back To DNS Records

While DNS is responsible for mapping human readable names to IP addresses, it is also used as a form of key/value database for the Internet.

In the early days of the internet, a common service provided by most domains was the Mail server(s).

This makes sense because DNS needs to resolve public IPs for many different types of services, not just web servers.

Record Type

Description

A

Responsible for mapping individual hosts to an IP address. For instance, myserver in the myserver.mydomain.com syntax.

AAAA

The IPv6 equivalent of an A record (see above)

CNAME

Canonical name. Used to alias one record to another. For example, nas.mydomain.com could be aliased to myserver.mydomain.com.

MX

Specifies mail servers responsible for handling mail for the domain. A priority is also assigned to denote an order of responsibility.

PTR

Resolves an IP address to an FQDN. In practice, this is the reverse of an A record. (see Reverse DNS below)

SOA

Specifies authoritative details about a zonefile, including the zonemaster’s email address, the serial number (for revision purposes) and primary nameserver.

SRV

A semi-generic record used to specify a location. Used by newer services instead of creating protocol-specific records such as MX.

TXT

Arbitrary human-readable information that needs to be stored in DNS. Examples include verification codes and SPF records.

Back to Zonefiles

Now take a look at the zonefile again, and look at all Records on it:

$ORIGIN example.com.     ; designates the start of this zone file in the namespace
$TTL 3600                ; default expiration time (in seconds) of all RRs without their own TTL value
example.com.  IN  SOA   ns.example.com. username.example.com. ( 2020091025 7200 3600 1209600 3600 )
example.com.  IN  NS    ns                    ; ns.example.com is a nameserver for example.com
example.com.  IN  NS    ns.somewhere.example. ; ns.somewhere.example is a backup nameserver for example.com
example.com.  IN  MX    10 mail.example.com.  ; mail.example.com is the mailserver for example.com
@             IN  MX    20 mail2.example.com. ; equivalent to above line, "@" represents zone origin
@             IN  MX    50 mail3              ; equivalent to above line, but using a relative host name
example.com.  IN  A     192.0.2.1             ; IPv4 address for example.com
              IN  AAAA  2001:db8:10::1        ; IPv6 address for example.com
ns            IN  A     192.0.2.2             ; IPv4 address for ns.example.com
              IN  AAAA  2001:db8:10::2        ; IPv6 address for ns.example.com
www           IN  CNAME example.com.          ; www.example.com is an alias for example.com
wwwtest       IN  CNAME www                   ; wwwtest.example.com is another alias for www.example.com
mail          IN  A     192.0.2.3             ; IPv4 address for mail.example.com
mail2         IN  A     192.0.2.4             ; IPv4 address for mail2.example.com
mail3         IN  A     192.0.2.5             ; IPv4 address for mail3.example.com

see the CNAME for example.com? this is why www.example.com and example.com resolve to the same host (and why you usually don't have to worry about NOT typing www in your browser).

Computer Specific DNS Configuration

If you are administering systems, specifically Unix systems, you should be aware of two pieces of host-side configuration which allow your machines to interface with DNS:

  • /etc/hosts/etc/resolv.conf
  • the Hosts File

Hosts File

The /etc/hosts file (C:\Windows\System32\drivers\etc\hosts on Windows) has the purpose of acting as a local alternative to DNS (and always takes precedence over DNS).

📘

Why is the hosts file a thing at all?

Because the hosts file IS the original DNS system.. When the Internet was invented, it started on a small network so it was much easier to maintain a single file referencing ALL hosts on the network and distribute it across computers.

You might use this when you want to override the record in place in DNS on a particular machine only, without impacting that record and its use for others - therefore, DNS can be overridden using /etc/hosts.

Alternatively, it can be used as a back-up to DNS: if you specify the hosts that are mission-critical inside /etc/hosts, then they can still be addressed by name even if the nameserver(s) holding your zonefile are down.

However, /etc/hosts is not a replacement for DNS - in fact, it is far from it: DNS has a much richer set of records that it can hold, whereas /etc/hosts can only hold the equivalent of A records. An /etc/hosts file might, therefore, look like:

127.0.0.1         localhost
255.255.255.255   broadcasthost
::1               localhost
fe80::1%lo0       localhost

192.168.2.2       sql01
192.168.2.3       sql02
192.168.1.10      puppetmaster puppet pm01

The first four lines of /etc/hosts are created automatically on a Unix machine and are used at boot: they shouldn’t be changed unless you really know what you’re doing! In fact, the last two lines of this section are the IPv6 equivalents of the first line. After these first four lines, though, we can specify a name and map it an IP address. In the above example, we’ve mapped sql01 to 192.168.2.2, which means that on a host with the above /etc/hosts configuration, we could refer to sql01 alone and get to the machine responding as 192.168.2.2. You’ll see a similar example for sql02, too. However, there is a slightly odd example for the box named puppetmaster in that multiple friendly names exist for the one box living at 10.0.0.2. When referenced in this way - with multiple space-separated names against each IP address - the box at 10.0.0.2 can be reached at any of the specified names. In effect, puppetmaster, puppet, and pm01 are all valid ways to address 10.0.0.2.

Resolvers

📘

What are Resolvers for, anyway?

A Resolver is a resource which points to one or more DNS services and can be used to resolve an address like example.com.

The Operating System you run on your computer maintains an ordered list of such Resolvers: all DNS queries go to the first resolver and, if resolution is unsuccessful, it goes to the second resolver, then third, etc. until resolution of the address is successful.

If you are on a mac, try running the following command in a Terminal window (with the Twingate Client not running): scutil --dns :

my-mbp16 git % scutil --dns
DNS configuration

resolver #1
  nameserver[0] : 192.168.1.1
  if_index : 14 (en0)
  flags    : Request A records
  reach    : 0x00020002 (Reachable,Directly Reachable Address)

resolver #2
  domain   : local
  options  : mdns
  timeout  : 5
  flags    : Request A records
  reach    : 0x00000000 (Not Reachable)
  order    : 300000

The default Resolver (when the Twingate Client is not on) is, in this case, the router (the router here helps deal with all DNS requests for all machines on a local network like a home network).

Now, let’s take a look at our Resolvers again but with the Twingate Client on:

my-mbp16 git % scutil --dns
DNS configuration

resolver #1
  nameserver[0] : 100.95.0.251
  nameserver[1] : 100.95.0.252
  nameserver[2] : 100.95.0.253
  nameserver[3] : 100.95.0.254
  if_index : 29 (utun7)
  flags    : Supplemental, Request A records
  reach    : 0x00000003 (Reachable,Transient Connection)
  order    : 103200

resolver #2
  nameserver[0] : 192.168.1.1
  if_index : 14 (en0)
  flags    : Request A records
  reach    : 0x00020002 (Reachable,Directly Reachable Address)
  order    : 200000

the first Resolver is now different: It maps to Twingate’s own DNS Resolver service (100.95.0.25[1-4]).

📘

Why does Twingate adds its own Resolver on top of the list?

Jump to the "Why Does DNS Matter for Twingate?" section below to find out.

DNS Caching

Imagine every machine on the internet having to constantly resolve complex names into IP addresses: this wouldn’t scale very well and would be very intensive for a network to handle. This is why DNS records can be cached (meaning: temporarily kept in memory so as not to require a network request).

The SOA record at the start of each zonefile specifies an expiry value: this tells clients for how long they can keep the zonefile before re-requesting it from the domain server.

$ORIGIN example.com.     ; designates the start of this zone file in the namespace
$TTL 3600                ; default expiration time (in seconds) of all RRs without their own TTL value
example.com.  IN  SOA   ns.example.com. username.example.com. ( 2020091025 7200 3600 1209600 3600 )

Caching.. at what level?

But what exactly caches DNS records? the OS? an application?

Well, it depends.. both are possible and it is largely a function of what OS you are running.

On Unix based systems (Linux, Ubuntu, etc.) caching is typically done at application level (ex: your own browser handles its own DNS cache).

On Windows, it is a bit more centralized and you can see it by running the command ipconfig /displaydns:

C:\Users\aUser>ipconfig /displaydns

Windows IP Configuration

    array801.prod.do.dsp.mp.microsoft.com
    ----------------------------------------
    Record Name . . . . . : array801.prod.do.dsp.mp.microsoft.com
    Record Type . . . . . : 1
    Time To Live  . . . . : 406
    Data Length . . . . . : 4
    Section . . . . . . . : Answer
    A (Host) Record . . . : 40.91.80.89


    Record Name . . . . . : a.root-servers.net
    Record Type . . . . . : 1
    Time To Live  . . . . : 406
    Data Length . . . . . : 4
    Section . . . . . . . : Additional
    A (Host) Record . . . : 198.41.0.4


    Record Name . . . . . : b.root-servers.net
    Record Type . . . . . : 1
    Time To Live  . . . . : 406
    Data Length . . . . . : 4
    Section . . . . . . . : Additional
    A (Host) Record . . . : 199.9.14.201


    Record Name . . . . . : c.root-servers.net
    Record Type . . . . . : 1
    Time To Live  . . . . : 406
    Data Length . . . . . : 4
    Section . . . . . . . : Additional
    A (Host) Record . . . : 192.33.4.12

📘

DNS Caching & Propagation of DNS Records

DNS Caching is the reason why some DNS Record updates take a while to get propagated to all machines on a network.
If the SOA record at the beginning of the zonefile is set to 24hrs, it means it will take at most 24 hrs for all machines on a network to see the updated DNS records.

TTLs or Time To Live

TTLs, or ‘time to live’ values in Zonefiles allow you to force the expiry of individual records, thus bypassing the expiry time referenced in the SOA record on a per-record basis.

For instance, let’s say that opsschool.org has moved to a new web host but it needs to ensure that the service is available as much as possible. By reducing the TTL for the www and * records in the opsschool.org zonefile, the switch between previous and new vendor should be relatively pain-free because it will instruct clients to request an updated zonefile frequently enough to catch the update quickly.

TTLs and caching work well together - with a suitably high TTL and suitable caching in place, the time for a request to be responded to and the time for updated records to exist on caches are both dramatically reduced.

$ORIGIN example.com.     ; designates the start of this zone file in the namespace
$TTL 3600                ; default expiration time (in seconds) of all RRs without their own TTL value
example.com.  IN  SOA   ns.example.com. username.example.com. ( 2020091025 7200 3600 1209600 3600 )

Reverse DNS

DNS translates a human readable name to an IP address. Guess what Reverse DNS does? yup, it returns the human readable name when given an IP address.

In order for this to be possible, a zonefile must contain PTR Records.

PTR Records

Domain names follow a specific syntax - mydomain.com, where .com is set by ICANN and chosen by you when they register your own domain name (against a fee..).

So how do you “resolve” an IP address to a name?

With reverse DNS, a similar ordered syntax exists as with forward DNS.

Let’s assume that we want to know which hostname responds at 22.33.44.55. We do this as follows:

  1. Reverse the octets of the IP address -ex: 22.33.44.55 becomes 55.44.33.22
  2. Add in-addr.arpa to the end of the reversed address - we now have 55.44.33.22.in-addr.arpa
  3. The root nameserver tells queries to find the arpa nameserver (same mechanism as forward DNS)
  4. The arpa nameserver directs the query to in-addr.arpa’s nameserver
  5. The in-addr.arpa nameserver then responds with details of 22.in-addr.arpa, and so on…
  6. In the zonefile, the IP address matching the query is then found and the relevant hostname is returned

Private DNS

Resolving www.google.comfrom anywhere on the internet is functionally the same as resolving, say, [nas.home.int](http://nas.home.int) when I am at my house connected to my own local network however I do not want nas.home.int to be publicly resolvable and most companies do not their private resources to be exposed and visible to the internet either.

This is why companies typically leverage DNS servers that are only accessible within their private network (sometimes referred to as “Private DNS”): it allows companies to define their own domain & their own records / hosts so that they can all be accessed by name when connected to the private network yet remain invisible to the outside world.

👍

Made It This Far?

Great! Now let's check how this knowledge applies to Twingate!

Why Does DNS Matter for Twingate?

Because most of our customers use Resources as FQDNs (Fully Qualified Domain Names aka DNS-style resources like nas.home.int) as opposed to IP addresses (like 192.168.1.50) or CIDR ranges (a CIDR Range is simply a range of consecutive IPs):

487487

So the central question around how the Twingate Client works now becomes:

“How does the Twingate Client resolve a FQDN like nas.home.int which is only resolvable by my internal DNS at home, when I am in fact NOT at home?”

This is an essential component of how we make it seamless for people to work remotely just as if they were in the office without configuring anything at all besides installing the Twingate Client so let’s dive deeper.

The Twingate Client knows what Resources it can access

Let’s assume [nas.home.int](http://nas.home.int) is a Twingate Resource our Twingate Client & User have legitimate access to.

Our local Twingate Client is given a copy of all the FQDNs and Addresses it has legitimate access to (including nas.home.int); our client uses this list locally to intercept the right traffic (it will intercept everything destined for [nas.home.int](http://nas.home.int) but will ignore anything for, say, [server.home.int](http://server.home.int) (because it isn’t a Twingate Resource)).

How the Twingate Client Resolves nas.home.int

In order to check the IP address the Client resolves for it, let’s use dig and find out. (dig is a command you can use on Macs & Linux to resolve the IP from the FQDN).

Let’s first look at the dig command with the Twingate Client OFF and resolve our local resource nas.home.int while on our home network:

my-mbp16 git % dig nas.home.int

; <<>> DiG 9.10.6 <<>> nas.home.int
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 12309
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 1, ADDITIONAL: 2

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;nas.home.int.          IN  A

;; ANSWER SECTION:
nas.home.int.       86400   IN  A   192.168.1.50

;; AUTHORITY SECTION:
home.int.       86400   IN  NS  ns.home.int.

;; ADDITIONAL SECTION:
ns.home.int.        86400   IN  A   192.168.1.1

;; Query time: 153 msec
;; SERVER: 192.168.1.1#53(192.168.1.1)
;; WHEN: Mon Jun 27 19:05:27 EDT 2022
;; MSG SIZE  rcvd: 90

Great! it resolves to 192.168.1.50 which is indeed a local IP on my home network.

Now let's run the same command but with the Twingate Client ON:

my-mbp16 git % dig nas.home.int

; <<>> DiG 9.10.6 <<>> nas.home.int
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 57867
;; flags: qr aa rd ra ad; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;nas.home.int.          IN  A

;; ANSWER SECTION:
nas.home.int.       15  IN  A   100.108.194.142

;; Query time: 83 msec
;; SERVER: 100.95.0.251#53(100.95.0.251)
;; WHEN: Mon Jun 27 19:01:55 EDT 2022
;; MSG SIZE  rcvd: 46

Hmm, the address resolved is no longer a local IP but a public IP within the CGNAT range..

In fact, you will notice 2 big differences once the Twingate Client is switched on:

  1. [nas.home.int](http://nas.home.int) now resolves to 100.108.194.142 which is NOT a local IP address on our home network (it’s a public IP in the CGNAT Range)
  2. the DNS Server is no longer 192.168.1.1 but is now 100.95.0.251

📘

What’s a CGNAT IP?

For the purpose of this article, a CGNAT IP is a public IP within a range reserved for public use by Carriers.
If you want to know more, head over here and enjoy the read 🙂

So.. Why does a local resource resolve to a public IP?

Because The Twingate Client needs to intercept all traffic meant for nas.home.int; and to do this, the Client does a few things on your device:

  1. The Twingate Client sets up its own Network Interface, visible on Macs under Network:
574574

And on Windows via the Control Panel:

972972

See the Twingate Interface (next to last)

  1. Upon start up the Twingate Client adds a Primary DNS Resolver (on top of the resolver list for my computer) that points to its own DNS servers (100.95.0.25[1-4])
my-mbp16 ~ % scutil --dns
DNS configuration

resolver #1
  nameserver[0] : 100.95.0.251
  nameserver[1] : 100.95.0.252
  nameserver[2] : 100.95.0.253
  nameserver[3] : 100.95.0.254
  if_index : 29 (utun7)
  flags    : Supplemental, Request A records
  reach    : 0x00000003 (Reachable,Transient Connection)
  order    : 102400

resolver #2
  nameserver[0] : 192.168.1.1
  if_index : 14 (en0)
  flags    : Request A records
  reach    : 0x00020002 (Reachable,Directly Reachable Address)
  order    : 200000
  1. Twingate reconfigures its own DNS so as to map internal FQDN-like resources like nas.home.int to IP Addresses in the CGNAT Range (in this case 100.108.194.142)
  2. The Twingate Client modifies the local routing table on the device to ensure all traffic addressed to IPs in the CGNAT range (100.96/12)is dealt with by the local Twingate Network Interface (utun7 in this case, see screenshot below).

(modifying the Routing Table basically tells the OS “if you see traffic meant for an IP in the CGNAT range, it needs to be handled by the Twingate Network Interface”)

my-mbp16 ~ % netstat -rn | grep utun7
default            link#29            UCSIg           utun7       
10.0.0.5/32        link#29            UCS             utun7       
10.0.1.25/32       link#29            UCS             utun7       
10.0.1.102/32      link#29            UCS             utun7       
10.0.1.244/32      link#29            UCS             utun7       
10.128.0.5/32      link#29            UCS             utun7       
10.128.0.8/32      link#29            UCS             utun7       
10.128.0.112/32    link#29            UCS             utun7       
100.95.0.251       172.16.30.1        UGHS            utun7       
100.95.0.252       172.16.30.1        UGHS            utun7       
100.95.0.253       172.16.30.1        UGHS            utun7       
100.95.0.254       172.16.30.1        UGHS            utun7       
100.96/12          link#29            UCS             utun7       
172.16.30.1        172.16.30.1        UH              utun7       
172.16.62.242/32   link#29            UCS             utun7       
224.0.0/4          link#29            UmCSI           utun7       
255.255.255.255/32 link#29            UCSI            utun7

Altogether, this ensures that:

  1. [nas.home.int](http://nas.home.int) resolves to a CGNAT IP
  2. all traffic for nas.home.intgets handled by the Twingate Network Interface (because it resolves to a CGNAT IP and because the Routing Table tells the OS that all CGNAT IPs need to be handled by the Twingate Network Interface)

What happens to Network Traffic once it reaches the Twingate Network Interface?

The Twingate Client has now successfully intercepted traffic for nas.home.int, now what?

This is where the 3 proxies that are in the Twingate Client intervene. (The Twingate Client has 3 proxies, one for each of the 3 protocols TCP, UDP and ICMP (ping only)).

How do network packets go from the Client to the Connector?

Before we answer this question, let’s take a look at what Network Packets actually contain.

To keep things simple, conceptually, a Network Packet contains:

  • data / a payload (actual information that needs to reach the destination)
  • a Destination IP (where the payload should be delivered)
  • a Source IP (where response packets should be sent to)

In short, think of it as a single Network Packet as:

[SOURCE IP][PAYLOAD][DESTINATION IP]

Back to Proxies

The role of the proxy in the Twingate Client is to simply take the payload from the network traffic (without source or destination), and forward it from the Client to the Connector’s proxy via the network.

📘

What can Twingate see?

The packets are encrypted client side so Twingate cannot decrypt network traffic in anyway.

What does the Connector do with the Payload once it receives it?

At this point, the Connector locally resolves the original FQDN nas.home.int(which from the perspective of the Connector will correctly resolve to 192.168.1.50 via the private DNS on our home network) and reassembles the Payload in a new Network Packet with:

  • as a source: the Connector’s own IP (in our case 192.168.1.88)
  • as a destination: the resolved IP (192.168.1.50)
  • as a payload: the payload sent by the Twingate Client’s proxy

The Connector then sends the packets to the right destination (192.168.1.50) which responds with new network traffic now destined for the Twingate Client.

How does Traffic flow back from the Resource to the Client?

The traffic from the destination back to the Twingate Client works the exact opposite way:

  • the application responds with a payload to the Connector (which it gets from the “source IP” in the original packets sent by the Connector)
  • the Connector’s proxy strips the network packets from Source and Destination, forwards them to the Twingate Client’s proxy which reassembles the payload into network packets with the same payload and:
    • as a source: the local IP of the Twingate Network Interface (in this case 172.16.30.1)
    • as a destination: local IP of the Client
    This will ensure the traffic from the application flows back to my Twingate Client and therefore my computer.

📘

Still With Us?

If you have made it this far down, thank you for giving this article a read!

Next, you might want to check our Careers page, we are often looking for people as interested in this stuff as we are :).


Did this page help you?