Tek-Tips is the largest IT community on the Internet today!

Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!

Register Log in
  • Congratulations Chriss Miller on being selected by the Tek-Tips community for having the most helpful posts in the forums last week. Way to Go!

Robots Text

Status
Not open for further replies.
spartiums

spartiums

Technical User
Joined
Feb 22, 2005
Messages
44
Location
GB
Do you need Robots text on your site - will my site be spidered and indexed with the search engines if I don't include it?

Thanks, Tina
 
Sort by date Sort by votes
If you haven't got a robots.txt file then bots will still index your site.
 
Upvote 0 Downvote
robots.txt is an exclusion method you only need it to tell the bots that obey it NOT to index something.

Though having a blank one will cut down the error log entries.


Chris.

Indifference will be the downfall of mankind, but who cares?
Woo Hoo! the cobblers kids get new shoes.
People Counting Systems

So long, and thanks for all the fish.
 
Upvote 0 Downvote
Status
Not open for further replies.

Similar threads

HankReadon
  • Locked
  • Question Question
Graphics co users to input text, upload JPEGs and preview their signs
Replies
0
Views
117
HankReadon
HankReadon
melodyr
  • Locked
  • Question Question
Change COD text
Replies
0
Views
96
melodyr
melodyr
kainickara
  • Locked
  • Question Question
Gentran 850 Mapping - MSG segment - how to split text
Replies
0
Views
103
kainickara
kainickara
spewn
  • Locked
  • Question Question
is the robots meta tag a guarantee that the page will/won't be indexed
Replies
2
Views
116
johnwm
johnwm
jstreich
  • Locked
  • Question Question
Text and background
Replies
9
Views
239
jstreich
jstreich

Part and Inventory Search

Sponsor

Back
Top