This may appear lots for a little site, but it really isn’t Considerably for large web pages. On the flip side, if you will get next to no site visitors from these serps, it would be a good way to save some bandwidth. Decrease the chance of specified web pages https://rankingseotoolz.com/robots-txt-generator