fureunoir/schon
fureunoir/schon
1
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

Features: 1) Update robots.txt to include improved crawler directives for /profile while maintaining SEO sitemap and host details;

Browse source 
Fixes: 1) Remove redundant `robots_frontend.txt` file and replace with consolidated `robots.txt` from `storefront/public`;

Extra: 1) Standardize formatting of crawler directives in `robots.txt`.
This commit is contained in:
Alexandr SaVBaD Waltz 2025-09-13 12:59:53 +03:00
parent 9877633a2c
commit e78d2fc652
2 changed files with 10 additions and 13 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

11
core/static/robots_frontend.txt
View file

@ -1,11 +0,0 @@
User-agent: *
Disallow: /admin/
Disallow: /static/
Disallow: /media/
Disallow: /cart/
Disallow: /account/
Allow: /
Sitemap: https://evibes.com/sitemap.xml
Host: evibes.com

12
storefront/public/robots.txt
View file

@ -1,2 +1,10 @@
User-Agent: *
Disallow:
User-agent: *
Disallow: /admin/
Disallow: /static/
Disallow: /media/
Disallow: /profile/
Allow: /
Sitemap: https://evibes.com/sitemap.xml
Host: evibes.com