Fix/jekyll sitemap 2026 02 08

[chengjiahao1234]

Type of Change

Please select the type of change (uncomment one):

✅ Bug Fix

Summary

Briefly describe what this PR changes:

Fixes sitemap generation for Google Search Console by switching to jekyll-sitemap, and updates robots.txt to improve indexing while blocking known heavy scraper bots.

Changes Made

List specific files or sections changed:

• Gemfile, _config.yml: enabled jekyll-sitemap plugin for automatic sitemap.xml generation
• sitemap.xml: removed custom/manual sitemap file
• *.md: removed sitemap: false so key pages are included in the generated sitemap
• robots.txt: removed crawl-delay rules, explicitly allow major search engines, block heavy scraper/SEO bots, and point to sitemap.xml
• DEVELOPMENT.md, README.md: updated documentation to match the latest sitemap/robots behavior and policy
• favicon.ico: replaced incorrect favicon asset

Testing Checklist

Please confirm you've tested the following:

• Changes display correctly on desktop (1920x1080 or similar)
• Changes display correctly on tablet (768x1024 or similar)
• Changes display correctly on mobile (375x667 or similar)
• Tested in light theme
• Tested in dark theme
• No broken links or images
• No console errors
• Ran bundle exec jekyll serve locally without errors

Screenshots (if applicable)

Related Issues

Additional Notes

1. Googlebot ignores Crawl-delay, so it’s omitted to avoid Search Console warnings; crawler load is managed via targeted bot blocking plus GitHub Pages/Cloudflare protections.
2. After merge/deploy, sitemap should be available at https://atlasanalyticslab.github.io/sitemap.xml (auto-generated by jekyll-sitemap).

---

By submitting this PR, I confirm:

• I have tested these changes locally
• I have followed the code standards in DEVELOPMENT.md
• I have updated documentation if needed
• I am ready for this to be reviewed and merged