How Not To Let DNS Bite You In The Ass
Recently, a site I frequent went down for an extended period. I waited patiently for it to return (it said it was upgrading). A week later, I hopped on Twitter and asked the author WTF? I learn that the site has been back up for days. Only I, and probably half of the internet, had no idea. Why? DNS. Done wrong, it’s a bitch.
I think there’s an object lesson here.
So, let’s say that you’ve decided that your webhost sucks rocks, and you want to change hosts for your domain from your existing server called junkhost to a much better and new fangled one called goodrack.
How To Do DNS The Wrong Way
(or why the site is currently totally missing from the internet as I see it)
Step 1: Your domain’s nameservers are currently set to dns1.junkhost.com and dns2.junkhost.com.
Step 2: Your new hosting company tells you to set them to dns1.goodrack.com and dns2.goodrack.com. You do so.
Step 3: You wait a day or two or possibly flush your home router’s DNS and voila, your host is now visible to you.
Why this is the wrong way: DNS is a caching system. When I try to lookup the IP address for yoursite, then I don’t ask your nameserver for it. I ask my nameserver for it. My nameserver then asks you, but only if it doesn’t already know.
See, you communicate with your site regularly. So you are doing lots of queries and flushing the DNS on a regular basis. Home router DNS systems tend to flush a lot, as do cable modems and cable companies, etc. So you might see that nameserver switch in just a day or so. Maybe less.
Me, on the other hand, am sitting here at BigCompany, which has weird fiber and satellite links to all sorts of places. If my DNS has your nameservers cached, it could be cached for weeks. And you don’t have a whole hell of a lot of control over what is called “TTL”, or “Time-To-Live”.
Time-To-Live is what tells DNS caching servers when to drop the info and get fresh data. If the TTL for, say, a normal DNS lookup is 2 hours, then my nameserver isn’t constantly querying yours whenever I look at your site. It only queries it once every two hours, tops. More or less.
Sitting here looking at the thing in debug mode, I see that there’s another 46 hours left to go before the old data I now know exists gets flushed out of the DNS server. By then, I won’t have been able to access his site for a bit under 2 weeks. And I’m not the only one.
How To Do DNS The Right Way
(or why I can change hosts in under a couple hours)
Step 1: My domain’s nameservers are currently set to dns1.myregistrar.com and dns2.myregistrar.com. My registrar offers DNS hosting for free, as part of my having bought my domain through them. If they didn’t, I could use a free service like ZoneEdit, or similar.
Step 2: My new hosting company tells me to set the nameservers to dns1.goodrack.com and dns2.goodrack.com. I look them right in the eye, tell them to “get bent”, and ask what the IP address of my new server will be. If they put up a fight, I tell them I can always find a better web host. That usually shuts them up.
Step 2 (alternate): I say “sure, whatever you say”, then do the following at a command line:
C:\otto\>nslookup Default Server: dns.mycompany.com Address: 1.2.3.4 > server dns1.goodrack.com Default Server: dns1.goodrack.com Address: 5.6.7.8 > set type=any > mydomain.com Server: dns1.goodrack.com Address: 5.6.7.8 mydomain.com internet address = 111.222.111.222 (root) nameserver = dns1.goodrack.com (root) nameserver = dns2.goodrack.com dns1.goodrack.com internet address = 5.6.7.8 dns1.goodrack.com internet address = 5.6.7.9
Now I know my new server’s IP address from there (it’s 111.222.111.222).
Step 3: I go to myregistrar.com’s site, edit my DNS settings (specifically, I change the A record for mydomain.com to point to that new IP address), and then I wait for a couple hours or so, max (since my registrar lets me set my own TTL, I can do what I like with it. Two hours is a decent tradeoff of time vs. functionality). Within 2 hours, every site on the whole blamed internet sees my new host.
Why is this better? Well, note that my nameservers never actually changed. Nameservers are set in the root domain lookup systems. Changes there not only take a while to propagate, but those servers are overloaded so caching values for nameservers are usually quite high. Individual host lookups though, not so much.
How a DNS request works:
- I ask my nameserver for example.com.
- It doesn’t know, so it goes and ask the root nameservers “Hey, who is the nameserver for example.com?”
- One of the 13 root nameservers says “Yes, example.com is serviced by dns1.somebody.com and dns2.somebody.com and here’s their IP address’s. Now leave me alone, I’m busy.”
- So my name server now goes and asks dns1.somebody.com, “Hey, do you know the IP for example.com?”
- And then dns1.somebody.com says “Sure, it’s 1.2.3.4”.
Lot of work. But note that my nameserver talked to TWO nameservers here. The root, and the actual nameserver for example.com. The TTL on the root lookup is not something I can adjust, usually. Or if I can, then it’s very difficult to do for most people. The TTL on the second is not nearly as hard, and doesn’t require all the root nameservers to be updated.
If you own a domain, then you need to own your DNS. Use a nameserver that you have control over, and which you will almost never change. You can change the addresses, you can change the MX records (for email), you can add TEXT records for SPF, and other DNS tricks all you want. But keep the nameserver location itself unchanged for as long as possible. Because when you do change it, the internet takes a long, long time to adjust to that fact. And you are then “offline” for quite a while.
To think of it another way: He who controls the DNS, controls the domain, as least for a while. Well, you’re going to change hosts a lot more often than you change domain names, right? So don’t give them your DNS control. No webhost should ever have your nameserver pointed to them unless you bought the domain name from them in the first place.
[…] wptavern RT @Otto42: @kwbridge @jeffr0 Here’s a quickie on the topic: http://ottopress.com/2010/how-not-to-let-dns-bite-you-in-the-ass/ […]
Thanks for the write-up. I had no idea this was an issue – I thought you *had* to change the nameservers to the ones for the new host.
Nice article, except it gets lost in the Step 2 (alternate). You might clarify dns.mycompany.com
When you run nslookup, it defaults to using your local DNS server for lookups. I change the server I’m using immediately after that prints out.
I’m not trying to be a jerk but you are writing a technical article and some of your audience isn’t going to understand what you are saying in step 2. May I suggest that you define dns.mycompany.com as you did in Step 1. Maybe place it in context. It’s a great article and I was just thinking you could make it just that much better.
See how important DNS is? Wikipedia disappeared today due to their DNS going out on them.
http://techblog.wikimedia.org/2010/03/global-outage-cooling-failure-and-dns/
Hey Otto, I use GoDaddy.com which does have the ability for me to add or change name servers for the domain. I’m not sure of the proper steps to take in order to avoid what happened last time and to avoid switching my domains name servers every time I need to switch hosts. Do you use Godaddy at all or can you specify what exactly I would need to do? I’m a bit lost with step 2.
Yes, I use GoDaddy for my site as well. Here’s what you do:
First, get your existing domain’s information from the host’s nameservers. You need the IP information for the A records, the CNAME records, etc. Basically you want to know everything on your domain’s nameserver space. You can get this yourself with nslookup if you want. Here’s an example of my DNS records (this doesn’t include CNAME subdomains, you’ll have to query for them individually):
C:\>nslookup Default Server: dns.example.com Address: 111.222.333.444 > set type=any > ottodestruct.com Server: dns.example.com Address: 111.222.333.444 Non-authoritative answer: ottodestruct.com primary name server = ns01.domaincontrol.com responsible mail addr = dns.jomax.net serial = 2010022500 refresh = 28800 (8 hours) retry = 7200 (2 hours) expire = 604800 (7 days) default TTL = 86400 (1 day) ottodestruct.com internet address = 64.202.163.10 ottodestruct.com nameserver = ns02.domaincontrol.com ottodestruct.com MX preference = 30, mail exchanger = ALT2.aspmx.l.google.com ottodestruct.com MX preference = 20, mail exchanger = ALT1.aspmx.l.google.com ottodestruct.com text = "v=spf1 include:aspmx.googlemail.com -all" ottodestruct.com MX preference = 40, mail exchanger = ASPMX2.GOOGLEMAIL.com ottodestruct.com nameserver = ns01.domaincontrol.com ottodestruct.com MX preference = 10, mail exchanger = aspmx.l.google.com ottodestruct.com MX preference = 50, mail exchanger = ASPMX3.GOOGLEMAIL.com(As you can see, I use Google Apps on my domain.)
Next, you go to your host (GoDaddy) and go to Manage Domains, then go to Total DNS Control. This *should* be available even though you have your nameservers pointing elsewhere. Don’t quote me on that though.
Using this, you set up your settings to match the existing one you have on the webhosts nameserver. That way, during the transfer, you lose nothing and no traffic. Both your old and new nameservers will basically serve up the same information.
After you’ve done thing, you change your nameservers for the domain to point back to GoDaddy’s servers. Mine are “NS01.DOMAINCONTROL.COM” and “NS02.DOMAINCONTROL.COM” on my GoDaddy domains, because that was what they defaulted to when I created the domains. I do not know if this is universal across their domains by default. Their help documentation says to use ns25 and ns26: http://help.godaddy.com/article/664 . You may want to ask them before changing.
Anyway, once you’ve changed nameservers, you wait for a couple weeks. Ideally, you lose no traffic in this time, because again, both your old and new nameservers say the same things. And now, when you’re wanting to change hosts next time, you can do it yourself just by editing your DNS settings and not by repointing your domain’s nameservers.
Otto.
I have just discovered you and now following you on Twitter.
Great post, I had no idea about this problem with the DNS. But unfortunately your solution drescription is too technical for me, I don’t understand it. I have no idea what “A record” or what “nslookup” is or how to find it or use it.
I am just about to change webhosts from myjunkhost to mynewgoodrack. But I have not found “mynewgoodrack” yet.
I am looking for a webhost and web account that does not constantly return “500 Internal Server Error” for my WordPress sites every time their server gets busy. I would like a dedicated server as I am intending to host my own autoresponder (which almost everybody recommends me not to do) but can not spend that much money yet so I am intending on taking out a reseller account but then using only for my own domains (not reselling) to hopefully increase both my disk space and my bandwidth (but I am concerned about percentage of CPU limits). I am confused about VPS. I don’t really understand what a “virtual private server” really is or what benefits it has over a reseller account.
Clearly you would understand this stuff a whole lot better than me.
Would you have any recommendations for which type of hosting account I would be better to us?
Do you have any recommendations for hosting companies?
I am very confused!
I would be very grateful for any recommendations you may have!
Wayne.
To help provide clarity for future readers, there is no need for dedicated hosting, and/or virtual private servers for the sole sake of using an email auto responder. When regards to using the services of GoDaddy (or their Reseller Company Wild West Domains, Inc.) all you need to setup an auto responder is simply an email address. This feature can be set up by clicking the “edit” button within the email control center next to the respective email address of interest.
Virtual Food for Thought: When setting up email for your website, [eg: _yourname_@yoursite.com]; instead, I recommend using either of the below options:
1. _yourname_@email.yoursite.com ~or~
2. _yourname_@yourothersite2.com
Let me explain. Since Google’s Penguin update, email associated with your website can actually work against your site’s search engine rankings. When people are checking their email, certain people mark emails as spam.. *even if they opted to receive an email (in the case, it could be an auto responder), and *independent of whether or not the email is actual spam.
Google is now using indicators, such as feedback through google+, gmail, and chrome (to name a few) to determine which sites end up on a blacklist, as identifiable by a websites IP address.
By using either a subdomain (option 1, or an entirely separate domain (option 2), you can help circumvent the possibility of ending up on a blacklist **assuming your online activity does not warrant such placement. Just something that might be worth considering.
I hope future readers (2013 and beyond) have found this to be helpful. To address the specific issue in the above post, purchasing wordpress hosting may be the solution to the 5xx errors that are showing in the browser for certain pages.
Disclosure- I’m roughly 3 years late on anything that would be considered a timely comment to this page, but considering the high prominence in Google search that this page still has, I believe that expounding on this specific email/seo topic for future Otto-dwellers like myself could prove to be helpful, especially when landing on this page with a query involving email setup/email servers.
[…] According to the Splashpress Media tech team, the problem might be related to DNS updates. For example, if you’re accessing Freelance Writing Jobs from a computer at work, you might be stuck behind a firewall which only updates the URL DNS one time per month. The Freelance Writing Jobs DNS has been updated correctly, so the firewall issue would definitely cause a problem on the user’s end. You can read more about DNS issues when changing nameservers here. […]
[…] stuff, and I just discovered how prevalent this practice was. I should have guessed it when I wrote a post about it earlier, but I didn’t know everybody was doing it this way. Most people I talked to […]
Greatly appreciate you writing this article! I had an unusual problem with Namecheap and my webhost whereby my mobile phone, friends’ devices and computers all returned my website, but my laptop returned Namecheap’s landing page instead. Despite flushing my dns with /ipconfig flushdns a couple of times, the problem still persisted.
I had no idea what to look for since I did not even understand what the problem was (being a first-time web developer) and if not for your article I would not have managed to fix this problem. Furthermore I actually found out what DNS meant and am grateful to have learnt how to set up my own dns in case I switch webhost in future.
Thanks a lot! If any other budding web developers have questions, feel free to comment below my comment and I will try to help you guys!
Forgot to toggle the ‘Notify me of follow-up comments by email.’
It seems bizarre to me that it would take more than a couple of days for DNS changes to replicate around the world, unless there is a problem. I have very little experience in this area but my experience is that if there are no problems then replication occurs within a couple of hours. Replication does not occur or occurs slowly when there is a problem.
For most hosting companies, at least most large ones, it is possible to use name servers that point to the host system and the A and CNAME records are then in the host system, not the registrar. I don’t know if that helps with the problems described here but I prefer doing that.
That’s exactly the problem I’m describing here. Changing your NS records to point to a new host takes a couple days, and in some cases, can cause outages for up to 2 weeks. That is not something you can control. So, the way to avoid it is to not change your NS records. Don’t point your NS at the hosting service. Avoid that at all costs. Use nameservers that you will not change, and just change the records there. Then you control the TTL.