Avoid Duplicate Content In Redesigns

The big G has a strong negative stance on dupliate content.  If you keep your sites in production all on a staging site (example.com/sites/site-1) and then when the site is finished you move it to site-1.com.  The problem with this is that G can spider example.com, and find the new content on the staging site before it is moved over to the end domain.  It can show staging site results before the site is launched, it can flag the new site as duplicate content, and in general can just ruin your day.

There is a simple way to avoid this from happening: robots.txt

You need a robots.txt file to be on the top level of the staging domain (example.com/robots.txt).

A sample syntax might look something like this:

User-agent: *
Disallow: /sites/

What this syntax says is: hey.. any crawler bot, don't go do /sites/.  That's really all there is to it, but not doing this can cause you a headache down the road.

Take a look at this comic strip.. :D