Loading...
No monthly payments or hidden costs.
Websites from R1 650 ex vat
Hosting and e-mails for a year.
Websites with Hosting
from R2 460 ex vat
Online store. Platform allows unlimited products.
E-commerce website
from R13 500 ex vat
Directories, property websites and more...
Custom and advance websites
from R25 000 ex vat
By the only website design company first on google in SA.
Website Marketing
from R815 ex vat
Local and International solutions.
Hosting, E-mails and Domains
from R62.50 p/m ex vat

Robots.txt

A robots.txt file is used to tell search robots which pages on your website you want indexed and which pages to disregard when crawling the website. Thus pages with sensitive content can be excluded from search results at the behest of the client.

When implementing an SEO campaign for a website it is important to review and revise the robot.txt file once the site has gone live and the SEO campaign has been in action for a while. A robot.txt file is always located in the main directory of a website and when a search engine robots comes across the website, they look to this file for instructions. New instructions will make them aware of the fact that the website content has been updated and that there are new files to be indexed.

Secondly the robot.txt file may be excluding content that needs to be indexed, and a revision of this file can correct the problem. A review will also highlight any other potential problems that can arise as a result of the robot.txt file.

The chief aim of a robot.txt file review is to ensure that nothing on the main website is being excluded for any reason other than the fact that the client asked for it.


Contact Form

Please complete the reCAPTCHA to help us reduce spam and to complete the form sending process. If you struggle to complete this form please email sales@websitedesign.co.za with your enquiry.

Contact Us

Sales
Development
086 111 5596 pm@websitedesign.co.za
Hosting