Fixes: 1) Remove redundant `robots_frontend.txt` file and replace with consolidated `robots.txt` from `storefront/public`; Extra: 1) Standardize formatting of crawler directives in `robots.txt`.
10 lines
No EOL
156 B
Text
10 lines
No EOL
156 B
Text
User-agent: *
|
|
Disallow: /admin/
|
|
Disallow: /static/
|
|
Disallow: /media/
|
|
Disallow: /profile/
|
|
|
|
Allow: /
|
|
|
|
Sitemap: https://evibes.com/sitemap.xml
|
|
Host: evibes.com |