This past weekend, I was cleaning up my ~/Sites folder, and I started thinking, “There’s got to be a better way!” And I wasn’t referring to baby hammock. I was talking about having to set up all those virtual hosts and whatnot. Well, a little research quickly brought me to the “how did I not know this before?!” point. And now, I will share it with you.
(Editor’s Note: This article turned out to be a little longer than I expected. The whole process is really quite simple. It should only take about 10 minutes, at the most.)
The first thing I didn’t realize is that Apache supports something called dynamic virtual hosts. That means you don’t have to create a virtual host in your conf files and restart Apache every time you want to create a new site. This is incredibly handy for development. Combine that with a real DNS server (BIND), and you have a much smarter system. I can now create a whole new site on my MacBook just by creating a new folder! Much simpler than the old way.
The Old Way?
To put this in context, let’s quickly review the old way. For years, I’ve developed websites on Mac OS X using the standard ‘MAMP’ setup: Mac/Apache/MySQL/PHP. When starting a new site, the first thing I do is set up a virtual host for Apache. If you’re like me, you know this as a 4-step process:
Create a directory for the site’s files. Something like:
mkdir -p ~/Sites/domain.com/public
Create a line in /etc/hosts for the domain:
127.0.0.1 domain.dev
Add a few lines to my Apache conf file to configure the virtual host:
<VirtualHost *> ServerName domain.dev DocumentRoot /Users/jason/Sites/domain.com/public </VirtualHost>
Restart Apache:
sudo apachectl graceful
The New Way!
Create a directory for the site’s files. Something like:
mkdir -p ~/Sites/domain.com/public
Done.
Not only that, but I also decided to set up a few TLDs to separate my sites. I use .ppm for my personal and freelance sites, .dev for my experimental stuff, and .bsi for my company work. This allows me to separate sites like so:
Sites
|-- bsi
| |-- this
| |-- that
| `-- theother
|-- dev
| |-- experiment1
| |-- experiment2
| |-- youget
| `-- thepoint
`-- ppm
|-- postpostmodern
`-- littlebeestudio
These would be:
- this.bsi
- that.bsi
- theother.bsi
- postpostmodern.ppm
- littlebeestudio.ppm
- experiment1.dev
- experiment2.dev
- youget.dev
- thepoint.dev
First, let’s talk about the hosts file.
The first thing you need for an Apache virtual host is a unique hostname. As can be seen in step 2, above, this is usually accomplished via the /etc/hosts file. The only problem is the hosts file doesn’t support any wildcards. So, you can’t say:
127.0.0.1 *.dev
Instead, you have to have this:
127.0.0.1 postpostmodern.dev
127.0.0.1 littlebeestudio.dev
...
...
…and eventually, you end up with a hosts file a mile long.
Enter BIND — the built-in, but inactive DNS server
BIND (named) comes with Mac OS X. We just need to configure it and turn it on.
I should mention here that DNS server stuff falls outside of my comfort zone. I just followed a few articles (on macosxhints and Ubuntu Forums) to get this working. I’m not sure how it impacts the vulnerability of your Mac from a security standpoint. All I know is that it works and provides a few advantages over the hosts file:
- You can set up TLDs to resolve to your local IP address. So, anything.dev and anything.test will automatically stay local.
- Apparently, since BIND will be caching DNS info, it will make web browsing faster. I haven’t formally tested this, but if it’s true, it’s a nice bonus. It does seem a little quicker.
Setting up rndc
This creates a configuration file and key for rndc, which controls named.
Get into sudo, and make it stick.
sudo -s
Generate the conf file.
rndc-confgen > /etc/rndc.conf
Copy the key to the key file.
head -n 6 /etc/rndc.conf > /etc/rndc.key
Exit sudo.
exit
Creating your DNS zone files
DNS zones are created via files in /var/named. Create a new file in there called dev.zone and fill it with this:
;
; BIND data file for dev sites
;
$TTL 604800
@ IN SOA dev. root.dev. (
2008101920 ; Serial
604800 ; Refresh
86400 ; Retry
2419200 ; Expire
604800 ) ; Negative Cache TTL
;
@ IN NS dev.
@ IN A 127.0.0.1
*.dev. 14400 IN A 127.0.0.1
Repeat the above for each TLD you want to set up, replacing ‘dev’ of course.
Configuring named.conf
Now, open /etc/named.conf. The first thing you want to do here is to show named where to get its DNS info (for the rest of the internet), i.e. forwarding servers. Let’s use OpenDNS. Add these lines in the options section of named.conf (after ‘directory “/var/named”;’):
forwarders {
208.67.222.222;
208.67.220.220;
};
Now, we just need to let named know about those zone files we created a minute ago. For each of the zone files, create a section like this:
zone "dev" IN {
type master;
file "dev.zone";
};
You’ll see where to put it. There’s a ‘localhost’ section already there. Just put yours below that.
Configuring and loading the LaunchDaemon
Okay. One last thing. Tell Mac OS X to activate named. Open the LaunchDaemon plist file for named (I had to use Textmate because Property List Editor didn’t let me save a file belonging to root.):
/System/Library/LaunchDaemons/org.isc.named.plist
Change ‘disabled’ from true to false, and save the file.
Now, load it:
sudo launchctl load /System/Library/LaunchDaemons/org.isc.named.plist
If everything went well, your DNS server should be up and running, and your personalized TLDs should resolve to your local machine. Try visiting something.dev and see if it resolves correctly. You might want to comment out all those custom lines from your /etc/hosts file too.
Now for the Apache Magic!
Now that the DNS stuff is out of the way, it’s just a matter of setting that magic directive in your Apache conf file. The directive is called VirtualDocumentRoot.
I don’t know how you have your Apache configured, but personally, I like to keep all of my custom configuration in my own file (the one in /etc/apache2/users). Here is what my /etc/apache2/users/jason.conf file looks like:
DocumentRoot "/Users/jason/Sites/default/public/"
NameVirtualHost 127.0.0.1
<VirtualHost 127.0.0.1>
VirtualDocumentRoot /Users/jason/Sites/%-1/%-2+/public
</VirtualHost>
The DocumentRoot directive is for the default site — the site that comes up when you visit http://localhost. The NameVirtualHost should be the reverse IP for your local machine. Finally, the VirtualDocumentRoot is an interpolated path for finding your sites. ‘%-1’ means the last part of the domain name (the TLD). ‘%-2+’ means everything before that. So, http://example.dev/ would load files from /Users/jason/Sites/dev/example/public. If you want a different scheme, read more about directory name interpolation on Apache’s web site.
Now, restart Apache!
sudo apachectl graceful
You should be good to go!
Please let me know your thoughts and corrections in the comments.
Update – 28 Oct, 2008
As Brian Toth mentioned in the comments, it’s probably also necessary to add 127.0.0.1 in your DNS settings in the Network System Pref pane.
Oct 22nd, 2008 at 1:36 pm Beau
I did a similar thing with my company’s DNS so that all the machines plugged into the LAN can access all of the development sites as domain.dev.
I like your folder hierarchy by TLD setup though compared to the Ubuntu one conf per site for Apache.
One problem to think about in the future is naming collisions when they open up TLD’s (*.dev, *.mycompany, *.whatever) for anyone to purchase/register.
Oct 22nd, 2008 at 1:40 pm Bradford C.
What about custom log locations? Do those work using the wildcards used in the VirtualDocumentRoot directive?
For example:
I’ll give this a shot when I have more time, but just wondering if you or anyone might know the answer to this.
Oct 22nd, 2008 at 3:09 pm Jason Johnson
@Bradford- According to the Apache site:
So, that might not work. I rarely have a need for separate log files on my development machine, but if you do, you might want to look into the pipe option.
Oct 22nd, 2008 at 4:02 pm Oliver
I’m deeply impressed: simple, elegant, easy. I wish I’d known about this a long time ago!
Is there a similarly simple way to create subdomains?
Oct 22nd, 2008 at 4:12 pm Oliver
Oh dear. It would appear I can’t read.
Oct 22nd, 2008 at 5:04 pm Brian Toth
Great tip, thanks!
I had to add 127.0.0.1 (instead of my router) as my DNS server in my network preferences before the domains would resolve.
Oct 22nd, 2008 at 5:33 pm Adam Norwood
Wow, that’s awesome! That just took my vhost config file from 120+ lines down to a fixed 10! I love blog posts like these, thanks for sharing the discovery.
I sidestepped the whole BIND thing though, because I’m happy enough adding new dev sites to my HOSTS file by hand, and don’t feel like fiddling around with named. I just set my VirtualDocumentRoot to /www/%0 and now I can point to arbitrary names and have Apache figure out the correct directory. Not fully-automated and elegant as your solution, but it’s still much easier than the “old way”.
Oct 22nd, 2008 at 10:34 pm Trey Piepmeier
I got it to work, but I did need to add this to my Apache config (which is
/etc/apache2/extra/httpd-vhosts.conf
):Oct 22nd, 2008 at 10:35 pm Trey Piepmeier
Well, that didn’t quite work. You get the idea, though.
Oct 22nd, 2008 at 10:43 pm Trey Piepmeier
Now I take that back—I’m not able to get this to work without having a line in the hosts file for every VirtualHost. I wonder what I’m doing wrong?
Oct 23rd, 2008 at 12:22 am jimeh
Awesome tip :)
I’ve just spent the last 2 hours reorganizing my ~/Sites folder.
I did a few things differently tho. I put all the sites and projects in ~/Sites/domains/ and then created ~/Sites/dev, wrk, and tmp. Within the dev, wrk, and tmp folders i created symlinks to the folders in question in ~/Sites/domains.
Also, some of my local development sites needs to be accessible from the rest of the internet for other members or projects and such. For that, I created a zone file called “local.mydomain.com.zone”, and I simply replaced “dev” within the file with “local.mydomain.com”. After which I created ~/Sites/com/ and within symlinks named “project.local.mydomain”.
Once done, I could use http://project.local.mydomain.com/ to access the project locally, and paste the URL to other project members and they’d also see my local dev site. This is accomplished by configuring the DNS on my host. I’ve created a CNAME DNS entry for *.local.mydomain.com which goes to the DynDNS domain I use for my laptop.
P.S. Please excuse the sloppy explanations here, it’s 8:21am, and I’ve soon been up for 24 hours… lol
Oct 23rd, 2008 at 1:04 am jimeh
Also, for two reasons I had to configure the Virtual Hosts a bit differently too.
The first is that I already had a bunch of virtual hosts which I wanted to keep. Like http://mysql/ being PHPMyAdmin for example.
The second was that I wanted people outside of my machine to access the dynamic hosts (see my above comment).
Hence my virtual hosts config looks like this:
NameVirtualHost *:80 NameVirtualHost *:3000 NameVirtualHost *:8080
I’m not a virtual host expert or anything tho, all I know is that this works perfectly for what I wanted it to do.
Oct 25th, 2008 at 8:26 am johnw
Great tip.
Followed the instructions verbatim to a T (had to because I’m a bit of a novice with this sort of thing) and it works perfectly — save for the fact that I can’t access my dev sites when I’m offline, as I frequently am.
I plugged in 127.0.0.1 into my System Preferences->Network->Ethernet->DNS but that didn’t seem to work.
In other words, when I’m not on a network although nslookup resolves “sandbox.dev” to 127.0.0.1 but I am unable to view my sites using http://sandbox.dev/ for instance.
Any ideas on how I can tweak my settings so I can view my dev sites when I’m offline?
Nov 4th, 2008 at 11:04 am Kent
Hey man- I met you awhile back on TTU’s campus. I’m in a band named Ribbonpigeon from Murfreesboro.
I had a question for you but I can’t find any email on this blog to send it to you.
Holla at me when you get a chance: kenteugene@gmail.com
-Kent
Nov 4th, 2008 at 2:21 pm Jason Johnson
@johnw - Good question. I’m not sure. I’ll post an update if I figure anything out.
Nov 14th, 2008 at 11:22 pm Bradford C
Have you been able to get this working with rails apps? Is there any specific configuration required? I was able to see the default rails-generated homepage but controllers created wouldn’t display. I didn’t do anything special to try getting this to work, but figured it’s worth asking.
Nov 15th, 2008 at 2:45 pm Jason Johnson
@bradford - With a Rails app, you need more than just Apache. I use Mongrel or Thin for development. In which case, you won’t be going through Apache at all. So, none of the above article will actually apply. You’ll be accessing the app server via your IP address on a different port (usually 3000).
I suppose you could use the Apache/BIND scenario with Passenger, but I haven’t tried that.
Dec 9th, 2008 at 6:08 am Chris
Is there a way to get the BIND part working with changing network locations and DHCP? I use my MB Pro in different environments (i.E. at work/home, while commuting using 3G and within the local networks of different clients). This means of course that I can’t simply add static forwarders in named.conf. Any suggestions?
Chris
Jan 10th, 2009 at 3:18 pm sajaki
anyone knows if the PHP in MAMP can be configured with —enable-debug ? like here http://bugs.php.net/bugs-generating-backtrace.php ?
Feb 9th, 2009 at 12:02 am Jason Johnson
@Chris – Sorry for the delay in responding. I’m not sure. I haven’t had any trouble changing networks. Using the OpenDNS servers as forwarders seems to work okay for me. I might be misunderstanding your question though.
I am, however, still trying to figure out how to get this to work with Windows via VMWare Fusion, since my MacBook doesn’t have a static IP.
Feb 9th, 2009 at 12:06 am Jason Johnson
@sajaki – I assume you’re talking about the MAMP app. The ‘MAMP’ I’m referring to in this article is just the acronym for Mac/Apache/MySQL/PHP. I’ve never used the MAMP app.
Mar 10th, 2009 at 4:30 pm Eric Clemmons
The only issue I’ve found with this is when I used “RewriteEngine On” in the site’s .htaccess, which returns a 500 error for the site.
I created a entry for my vhost with the settings Trey posted, which simply gave me a basic auth page instead.
Has anyone else been successful with using .htaccess files & this setup?
Mar 11th, 2009 at 2:06 am Tony Crockford
I found a couple of applications that handle Virtual Hosts and DNS so I don’t have to go through all the complications:
http://clickontyler.com/virtualhostx/ for virtual hosts and http://cutedgesystems.com/software/DNSEnablerForLeopard/index.html to make BIND much easier.
just thought I’d mention them.
Mar 11th, 2009 at 9:54 am Eric Clemmons
@Tony MAMP PRO is an excellent addition to MAMP for administering simple virtual hosts, similar to VirtualHostX.
And apparently, as a user of MAMP PRO, there’s a “template” that MAMP uses for its VHOSTS, which was conflicting with my rewrite’s path.
May 19th, 2009 at 11:57 am james
Thanks very much for this excellent writeup. I’d add that by default bind (named) when enabled listens for request from ANY IP address, so if you’re setting this up for just a single user development machine you might want cut down some on the potential for someone trying to exploit your homegrown DNS server.
Add listen-on and allow-query directives inside the options area of /etc/named.conf like so:
If you indeed wanted to setup your DNS for the local network so other machines could get to the dev sites on your machine, then you might uncomment the ‘localnets;’. If you do that, don’t forget to then add your dev machine’s local network IP address (eg 10.0.1.1, 192.168.0.1, or whatever it may be) to the list of DNS hosts in the Network Settings of the other computers on your network.
Also, when mucking around with named.conf always run named-checkconf on the command line after editing. Its too easy to forget a ‘;’! You might add this to your bash aliases, so you don’t forget to always check it…
May 19th, 2009 at 9:14 pm james
Unfortunately I’ve hit the same problem as johnw mentioned above. When I’m not connected to a network… the homegrown DNS server seems to just stop working. My gut says somehow named doesn’t listen for queries if there is no valid network interface (eg eth0). Wish I new more about network layer to troubleshoot this… hmmf :/
May 21st, 2009 at 3:48 pm Jason Johnson
@james - Thanks for the additional info on the listen-on and allow-query.
Regarding the offline problem – I have the same problem as you and johnw. I still haven’t been able to figure that one out.
May 31st, 2009 at 3:03 am james
Eureka!
I’ve solved my last two problems. the first one will probably be most important to you:
I got to thinking, ok so how does OS X serve the “machinename.local” TLDs on a localnetwork even when BIND is disabled? And the answer is ZeroConf Multicast DNS, (aka, Bonjour). I figured out that the answer to serving your own local domains is SOOO much more simple than using NAMED. After reading a bit about how this all worked, I found the answer on how to configure additional TLDs to use this Multicast DNS in the manual entry for resolver(5).
1) As super user, create the folder /etc/resolve
2) for each TLD domain you want to host on your machine, create a file in that folder that’s named after the tld. EG, if you want to host “.dev” locally you’d create a file named /etc/resolve/dev
3) inside each file you create in /etc/resolve file add two lines:
4) There is no step four, disconnect your internet connection and watch your local sites resolve once again! Like Butter.
The other problem i was having was that I was having trouble with mod_rewrite, and apparently AllowOverride All wasn’t setup explicitly in /etc/apache2/httpd.conf so I added it to my /etc/apache2/users/username.conf file…
Here is my final apache conf file:
And my project directory like this:
So now I can access the sites like thus:
http://projectname.dev/ http://beta.projectname.dev/
Without any external internet connection!
Jul 9th, 2009 at 1:56 am Juliendsv
I Love it,
I confirm you must add 127.0.0.1 at the first position in your System Prefences/Network/ DNS Server
Jul 21st, 2009 at 4:29 pm earph
Not working for me.
I followed everything to a T and even included the addition in quote #27, but can’t connect.
I’ve typed “host test.dev” in terminal and get this response.
Host test.dev not found: 2(SERVFAIL)
What ma I missing?
Aug 7th, 2009 at 7:21 am Jason Johnson
@earph - Sorry for the delayed response. Check comment #30, if you are not connected to a network. Also, check to make sure that the exact directory for which Apache is looking exists. Review your Apache error log (probably at /var/log/apache2/error_log) to see if Apache is trying to access a non-existent directory.
Beyond that, I’m afraid I can’t be of much help. It’s been a while since I went through this process myself, and I am no expert on Apache or bind. Hopefully other commenters will be able to continue to provide extra insight into the details of the setup.
Speaking of which, thanks to everyone who has added their discoveries and tweaks. They are very welcome here.
Aug 13th, 2009 at 6:33 pm Jo Dassler
I just wandered here looking for stuff on mamp app.This stuff’s really deep.hopefully one day I can do stuff like this at the moment it gives me the shivers!Always thought somethings are better left in the cupboards(apache configs etc)but i guess you guys blaze paths for future web dev jedis.The article’s a wonderful read though.For me its still a case of i understand but i don’t.Cheers!One day…
Aug 16th, 2009 at 10:23 pm em hr
This is brilliant. I just wanted to note that Johns comment (#30) has a typo in it. The proper directory for ZeroConf/Bonjour to work is not /etc/resolve it is /etc/resolver
Oct 22nd, 2009 at 11:30 pm james
oops, sorry for the typo! Thanks for noting that.
Also, the link to the manual page in comment #30 appears to no longer work. Heres an updated link to resolver(5):
http://developer.apple.com/mac/library/documentation/Darwin/Reference/ManPages/man5/resolver.5.html
Nov 9th, 2009 at 2:32 am james
I recently lost my hard drive, and had to go through this configuration by hand again. And so I decided to put all this into a script as I did it.
I haven’t had a chance to test it yet on a real clean install, so feedback on the snipt below is welcome.
http://snipt.net/jrguitar21/mac-os-x-apache-development-setup
Nov 10th, 2009 at 12:02 pm Paul
Editing the /System/Library/LaunchDaemons/org.isc.named.plist file isn’t necessary.
//To enable named sudo launchctl load -w /System/Library/LaunchDaemons/org.isc.named.plist sudo launchctl start org.isc.named
//To disable named sudo launchctl stop org.isc.named sudo launchctl unload -w /System/Library/LaunchDaemons/org.isc.named.plist
The “-w” flag tells launchctl to save the disabled status to the plist file.
Dec 30th, 2009 at 9:26 am Chris
I set this scheme up — thanks! Can you think of a change to this that would allow one-off departures? For example, is there a way to modify the user.conf file so that “special.dev” could go somewhere other than Sites/dev/special? Thanks again.
Feb 6th, 2010 at 9:34 am Aaron Nigelfeld
What about custom log locations? Do those work using the wildcards used in the VirtualDocumentRoot directive?
Aug 6th, 2010 at 4:32 am Aziz Light
Thanks for this article. It didn’t work as expected unfotunately. When I go to {whatever}.dev it brings me to localhost instead of localhost/dev/{whatever}/public…
Aug 17th, 2010 at 11:48 am Jason Hummel
One frustrating thing about using VirtualDocumentRoot is the DOCUMENT_ROOT env var will remain set to the main apache document root. This obviously causes problems when trying to reference files for a specific domain. After doing some searching I found this:
http://jbenner.net/blog/quick-tip-get-proper-document-root-when-using-mod-vhost-alias
Which completes the setup for me.
Apr 27th, 2011 at 3:30 am Christine
Jason - we were having a similar issue with DOCUMENT_ROOT when referencing from individual domains on the same serve - been pulling our hair out for a while trying to figure it out - that link was vert helpful
Jun 6th, 2011 at 4:49 pm Karen
I was dreading doing a spring clean of my ~/Sites folder. I feel I can take off my marigolds now - thanks.
Jun 12th, 2011 at 3:24 pm newnomad
So does the bonjour setup here with correction below actually complements or replaces the bind setup?
Oct 17th, 2011 at 5:09 am Andy
I followed James’ instructions to get the sites working with an internet conection. This worked when I turn off the WIFI once web sharing is turned on. But if web sharing is off when I have no internet connection it wont start.
Andy ideas?
[Mon Oct 17 11:09:22 2011] [crit] (22)Invalid argument: make_sock: for address [::]:80, apr_socket_opt_set: (IPV6_V6ONLY) (48)Address already in use: make_sock: could not bind to address 0.0.0.0:80 no listening sockets available, shutting down
Jan 3rd, 2012 at 9:46 am Nadeem
Can you suggest a similar solution for windows based computer.
Jan 3rd, 2012 at 12:14 pm Jason Johnson
Nadeem: I have no experience with DNS on Windows.
Everyone: I actually don’t use this method anymore for my Mac. I now use the DNS server built into Pow. I have been meaning to update this post accordingly. I will write more about how I use it with Apache soon.
Nov 26th, 2012 at 10:46 am Stu
Hi I’ve posted an alternative way of modifying hosts entry instead of using a DNS server. I’m using ‘Ghost’. It may be of use to somebody. http://caracaldigital.com/automatic-apache-virtual-hosts/
Apr 8th, 2013 at 9:53 pm Er1c
Pow uses rails… I like php… I heard ruby and rails is a different server language. Why would I learn all that?
I think I’ll do some other solution aside from pow.
Apr 9th, 2013 at 10:30 am Jason Johnson
Er1c: Pow is a web server and DNS server. You would not have to learn Ruby/Rails to use it as a DNS server. However, if you do not need the web server part of Pow, you might as well use some other DNS solution.