Sitemap.xml & Robots.txt

If you have created custom Twig templates through the theme code editor, these pages will not be automatically included in an automatically generated sitemap.xml file. Instead, you'll need to add these pages manually so that they are included.

1. Navigate to Website settings > Sitemap.xml & Robots.txt
2. Locate the sitemap.xml section
3. Use the Additional page URLs section to enter the path of each custom page, selecting the add another+ option to add multiple pages
   • You can add up to 50 custom pages

Custom pages added to the sitemap are given a priority of 0.5 and changefreq of daily.

ShopWired provides a simple feature for you to upload your own sitemap file in place of ShopWired’s automatically generated file, although this functionality is not available if your automatically generated sitemap.xml file contains over 50,000 page URLs.

To upload your own sitemap.xml file:

1. Read Google’s guidance on the sitemap.xml file
2. Navigate to Website settings > Sitemap.xml & Robots.txt
3. Locate the section labelled Sitemap.xml file
4. For the setting Sitemap.xml file generation select manual
5. Enter the contents of the sitemap file

Your custom sitemap file will be immediately displayed on your website at https://www.yourdomain.com/sitemap.xml Follow the instructions provided by Google to help test the contents of your sitemap file.

Where the automatic sitemap.xml file would contain more than 50,000 individual URLs, ShopWired automatically generates multiple sitemap.xml files and displays an index of those files at https://www.yourdomain.com/sitemap.xml.

This sitemap.xml file then becomes a sitemap index file, ShopWired generates this in accordance with Google’s specification.

If your account is generating a sitemap index file, you should still provide Google with the URL of this file at https://www.yourdomain.com/sitemap.xml - you do not need to give Google (or other search engines) each individual sitemap file contained within the sitemap index file.

ShopWired automatically generates a news-sitemap.xml file for your ShopWired website containing all blog posts published within the last 48 hours. The file is generated in accordance with Google’s specification

Your website's news sitemap file is available at https://www.yourdomain.com/news-sitemap.xml and is generated every 24 hours. You cannot customise the contents of the news-sitemap.xml file.

ShopWired sets the following information in your news-sitemap.xml file:

• news:name as your ShopWired account’s company name
• news:genres as blog
• news:publication_date as the date/time the blog was created (this is not updated if your blog post is edited)
• news:title as the blog post title
• news:keywords as the blog post’s category and any tags assigned to it

ShopWired automatically generates a robots.txt file for your ShopWired account and displays it on your domain name. You can, however, configure your own robots file if you’d like to.

Your robots file is displayed on your domain name at https://www.yourdomain.com/robots.txt.

When your ShopWired account setting is to Allow search engines to crawl your website, ShopWired’s default robots file provides simple instructions to search engines:

• It tells bots not to crawl your website’s /wishlist page and not to follow any links on your website generated by the multi-currency app
• It tells bots that it can crawl all other pages on your website
• It tells bots the location of your website’s sitemap and news sitemap

1. Read Google’s guidance on the robots file
2. Navigate to Website settings > Sitemap.xml & Robots.txt
3. Locate the section labelled Robots.txt file
4. For the setting labelled Robots.txt file generation select manual
5. Customise the contents of the robots file

Your custom robots.txt file will be immediately displayed on your website at https://www.yourdomain.com/robots.txt. You should follow the instructions provided by Google to help test the contents of your robots.txt file

If you would like to block your website from search engines by asking search engine bots (such as Google bot) not to crawl your website:

1. Navigate to Website settings > Sitemap.xml & Robots.txt
2. Locate the section labelled Robots.txt file
3. For the setting Allow search engines to crawl your website select no