v1.1 - Enhanced Link Detection and Custom Exclusion Paths

What's New:

• Improved Link Detection: The crawler now captures more comprehensive links, ensuring better coverage of inner pages, especially those dynamically generated or deeply nested within the website's structure.
• Exclude Non-HTML Resources: Automatically excludes URLs pointing to non-HTML resources such as .css, .js, .png, .jpg, and other media files, ensuring the sitemap focuses on actual web pages.
• Custom Exclusion Paths: Added a prompt for users to specify paths to exclude from the crawl. This feature allows users to filter out irrelevant or unnecessary sections of the website, like /feed/, /admin/, etc.
• Enhanced XML Sitemap Formatting: The sitemap XML is now pretty-printed for improved readability, making it easier to review and edit in text editors like VS Code.

Initial Release

Initial Release

This release includes the initial implementation of the Sitemap Crawler and Generator tool. The key features of this release are:

• Website Crawling: Collects URLs from a given website up to a specified limit.
• Sitemap Generation: Creates an XML sitemap following Google's guidelines.
• Priority Setting: Automatically assigns priority to URLs based on their depth in the website structure.
• Directory Management: Saves the generated sitemap in a subdirectory named after the website's domain.

Please refer to the README.md for more detailed information on installation and usage.