Warning: Can't synchronize with repository "(default)" (Unsupported version control system "svn": No module named svn). Look in the Trac log for more information.

Changes between Version 3 and Version 4 of RobotsTxt


Ignore:
Timestamp:
05/12/08 08:13:35 (11 years ago)
Author:
Chris Arndt
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • RobotsTxt

    v3 v4  
    11User-agent: * 
    2 Disallow: /browser   # Hide SVN browser 
    3 Disallow: /changeset # Hide changesets 
    4 Disallow: /log       # Hide log 
    5 Disallow: /milestone # Hide milestones 
    6 Disallow: /search    # Hide search 
    7 Disallow: /report    # Hide reports 
    8 Disallow: /query     # Hide ticket queries 
    9 Disallow: /newticket # Hide newticket 
     2Disallow: /browser 
     3Disallow: /changeset 
     4Disallow: /log 
     5Disallow: /milestone 
     6Disallow: /search 
     7Disallow: /report 
     8Disallow: /query 
     9Disallow: /newticket 
     10User-agent: Slurp 
     11User-agent: msnbot 
     12Crawl-delay: 120 
    1013#Disallow: /          # Hide the base folder (no indexing) 
     14#Disallow: /browser   # Hide SVN browser 
     15#Disallow: /changeset # Hide changesets 
     16#Disallow: /log       # Hide log 
     17#Disallow: /milestone # Hide milestones 
     18#Disallow: /newticket # Hide newticket 
     19#Disallow: /query     # Hide ticket queries 
     20#Disallow: /report    # Hide reports 
     21#Disallow: /roadmap   # Hide roadmap 
     22#Disallow: /search    # Hide search 
     23#Disallow: /ticket    # Hide tickets 
    1124#Disallow: /timeline  # Hide timeline 
    1225#Disallow: /wiki      # Hide wiki 
    13 #Disallow: /roadmap   # Hide roadmap 
    14 #Disallow: /ticket    # Hide tickets 
    15 # Settings to prevent bots from indexing trac folders 
    16 # Simply change the path prefix to be the path to your trac URL 
    17 # For example if trac is located at http://www.site.tld/source/trac/ 
    18 # the rules would begin with /source/trac/ 
     26