Robots.txt file can improve the ability of search engine bots to crawl your website. Crawlers read the file before crawling and indexing. So make sure all the information in the robots.txt file is correct. Otherwise your website will not be indexed properly.

Another factor to consider is site structure. What kind of content is on your site and how are the pages organized? Some significant aspects of sitewide SEO are given below:

1. What are SEO-friendly URLs?

There are ways you can make the URLs of your websites better optimized for search engines.  

  • The URL should be focused. Avoid using words that do not add value or meaning such as “a”, “of”, and “the”. Even the use of words such as “is” and “have” is not required.
  • The length of the URL matters too. Avoid extremely short or long URLs. Be concise but descriptive. 
  • Don’t include underscores in your URLs. This is because the underscore will combine separate words into one. Use dashes instead.

2. What are XML Sitemaps for SEO?

An XML sitemap is the list of all the URLs on your website. This list helps search engines to find all the pages on your site. XML sitemaps help the search engine spider crawl and index your content. It is a good idea to include a last modified date in your XML sitemap. This tells crawlers which pages should be crawled and which have not changed since the last crawl. The Yoast SEO plugin can help you create an XML sitemap for the pages on your site. Google Search Console has a section that lets you analyze your sitemap and alerts you to errors in it. 

Sitemap best practices

  • Google will crawl the URLs as you listed them. So use consistent and fully qualified URLs. For instance, If your website’s address is https://www.websitetoon.com/ then write it as it is. Do not specify the URL as https://websitetoon.com
  • Use Hreflang annotations so that Google can have ideas about the alternative language versions of the URLs.
  • If your page has two versions, then list only the canonical version of the page in sitemap.
  • For the additional media types like videos, images and news use sitemap extensions. 
  • Maximum sitemap size is 50,000 URLs or 50 MB uncompressed. Break up the large sitemaps into small sitemaps. Use a sitemap index file to list all the individual sitemaps and submit a single file to google rather than submitting the individual sitemaps. 
  • Google reads lastmod value so make sure you add this value for every single page.
  • Sitemaps are recommendations for Google that which page of your site is most important. Google does not promise to crawl every URL listed in the sitemap.

Site Speed and SEO

Site speed measures how quickly searchers are able to access the site and its content. It affects not only user experience but also search engine optimization, which is why you should focus on improving load time for your site. 

Why is site speed important for SEO?

  • UX and conversion: If your site takes too much time to load, it has a negative impact on your user experience. People like sites that load quickly. If your site speed is fast, customers are happier, which increases the probability that they will buy your products. On the other hand, slow sites make customers feel dissatisfied, and they may not visit your site again. 
  • SEO: Site speed influences the ranking of a website. A fast loading site ranks higher in search engines than a slow site. 

What are tools to test SEO site speed? 

You can use the following tools to check site speed:

  • Google PageSpeed Insights
  • Pingdom Tools
  • Yslow
  • WebPageTest