This will tell them what they should crawl and what to ignore on your site. This is an example of robot HTML instructions

<META NAME="ROBOTS" CONTENT="NOINDEX">
in your HTML document, that document won't be indexed.
If you do:
<META NAME="ROBOTS" CONTENT="NOFOLLOW">
This snippet of HTML will instruct Robots to ignore certain parts of your site.

Add some HTML to your site that instructs Googlebots to visit on a regular basis for example: Meta Name="ROBOTS" Content="index,follow">
<Meta Name="Revisit-after" Content="5 Days"> this will instruct the search engine bots to check your site every 5 days.

As you can see you are able to control how often the Googlebots will crawl or check your site. Also you can have some control over what parts of your site they will collect information on. Once you do this you can then work on adding new quality content to your site. You can also work on gathering good relevant links that will act like resources to your visitors.

Once you do this you can train Googlebots to crawl your site and make new content for them to pick up. Be sure that you interlink your pages with anchor text to be sure that Googlebots can crawl and gather information from your entire site. They will also pick up these links and store them as valuable resources in their database.

When you do all this and take good care to tailor your site for the Googlebots you will see your Page Rank rise and increase your visitors. Always bear in mind that Google’s customers are your potential customers in that surfers (searchers) are your visitors. This means that when you make your website user friendly, full of good relevant information and resources you will also be making it Googlebot friendly.