Domain Name Forum  

Go Back   Domain Name Forum > Search Engine > Google > Google Sitemaps

Reply
 
Thread Tools Display Modes
  #1  
Old 07-26-2009, 08:58 AM
SEO SEO is offline
Senior Member
 
Join Date: Jun 2009
Posts: 34,439
Default What is Robots.txt?

What is robots.txt in sitemap? What exactly are their function?
__________________
Search Engine Optimization
Domain Name Forum
Reply With Quote
  #2  
Old 04-09-2012, 02:52 AM
sujith34 sujith34 is offline
Member
 
Join Date: Sep 2011
Posts: 71
Default

Robot.txt files tell your instructions to a search engine robot. it depends on what you what it to do. if you dont have the robot.txt file then when a robot visits your site and if it doesnt find the file, it will start to visit all the pages and contents of your site. these files are like walls which helps in preventing some files like java scripts, some images or any other files from indexing. it is also important to make a note that it is not a method that prevents search engines from crawling your site. so it is important to create a robot.txt file. if you need this file to be created you can get it from XnYnZ.com
Reply With Quote
  #3  
Old 09-17-2018, 04:14 AM
richarddang911 richarddang911 is offline
Junior Member
 
Join Date: May 2018
Location: Vietnam
Posts: 25
Default robot.txt

Robots.txt is a text (not html) file you put on your site to tell search robots which pages you would like them not to visit. Robots.txt is by no means mandatory for search engines but generally search engines obey what they are asked not to do.


-----------------------------------------------------------------------------------------------------
dedicated server in Vietnam
Reply With Quote
  #4  
Old 10-25-2018, 09:01 PM
kakakaoo kakakaoo is offline
Senior Member
 
Join Date: Feb 2018
Posts: 105
Default

cc20181026
http://www.coachfactoryoutlet-online.us.org
http://www.outlet-canadagoose.us
http://www.tiffanyandcojewelry.in.net
http://www.michaelkors-outlet.in.net
http://www.pandoracharms.name
http://www.pandorajewelry.in.net
http://www.coachhandbags.us.com
http://www.miksbg.us
http://www.nikeshoes.us.com
http://www.coachoutletonlinestore.in.net
http://www.christianlouboutinshoesuk.org.uk
http://www.nike-shoes.in.net
http://www.uggoutletstores.us
http://www.drmartensboots.name
http://www.asicsoutlets.us.com
http://www.coach.us.com
http://www.michael-korsoutlets.eu.com
http://www.uggsoutletofficial.com
http://www.uggs--canada.ca
http://www.uggscanadaugg.ca
http://www.nikereactshoes.com
http://www.longchampoutletonline.in.net
http://www.nikeoutletonline.us.com
http://www.canadagoose-coats.com
http://www.lacostepoloshirts.name
http://www.salvatoreferragamoshoes.com
http://www.canadagooseoutlet-stores.us
http://www.canadagoosejacketss.us.com
http://www.canadagoosesaleoutlet.us.com
http://www.birkenstocksandalsuk.co.uk
http://www.outletcoachstore.us.com
http://www.chicagowhitesoxjersey.us
http://www.michael-korshandbags.us.org
http://www.cristianoronaldojerseys.com
http://www.katespadeoutletonline.us
http://www.pittsburghpiratesjerseys.us
http://www.valentino-shoes.us.com
http://www.pandora-charms.us.org
http://www.supremeclothing.us.org
http://www.moncleroutletstores.com
http://www.coachfactoryoutletonlinesale.us.com
http://www.ugg-boots.us.org
http://www.pandorajewelryoutlet.in.net
http://www.outletuggs.us
http://www.michaelkors-outlet.com.co
http://www.uggsoutlet-store.com
http://www.nikeoutletshoes.us.com
http://www.pandoracharms.us.org
http://www.coachoutletcanada.ca
http://www.newyorkmetsjerseys.us
http://www.pandoracharmssale-clearance.us.com
http://www.outlettoms.us.com
http://www.nike-store.us.com
http://www.outlet-moncler.us.com
http://www.oakley--sunglasses.us.org
http://www.adadassuperstar.com
http://www.coach-outletonlines.us.com
http://www.nikefreerun.name
http://www.coachoutletsaleonline.us.com
cc20181026
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -4. The time now is 06:23 PM.

| Webmaster Forum | | Domain Name Forum |



Powered by vBulletin®
Copyright ©2000 - 2018, Jelsoft Enterprises Ltd.