diff --git a/CLAUDE.md b/CLAUDE.md index 7761437..76a3379 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -4,124 +4,153 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co ## Project Overview -InnieMe is a Discord bot that provides AI-powered Q&A capabilities using document knowledge bases. The bot scans and vectorizes documents from specified directories, connects to Discord channels, and responds to user mentions with context-aware responses using configurable LLM providers via PydanticAI. +InnieMe is a multi-platform Q&A bot (Discord and Slack) backed by document knowledge bases. +It scans and vectorizes documents from configured directories and responds to user mentions +with context-aware answers, using a configurable LLM provider (OpenAI, Anthropic, …) via +[PydanticAI](https://ai.pydantic.dev/). -## Common Development Commands +Requires Python >= 3.13 (see `pyproject.toml`). -### Installation and Setup -```bash -# Install dependencies -pip install -e . -pip install -r requirements-dev.txt - -# Create configuration from example -cp config.example.yaml config.yaml -# Edit config.yaml with your Discord token, API keys, LLM model, and channel settings -``` +## Development Commands -### Running the Bot ```bash -# Run the bot (main entry point) -innieme_bot +# Install (runtime + dev) +pip install -e . +pip install -r requirements.txt -r requirements-dev.txt -# Or run directly with Python -python src/innieme/cli/run_bot.py -``` +# Run a bot via the unified CLI (-c overrides the default config path) +innieme discord # default: config.yaml +innieme discord -c custom_config.yaml +innieme slack # default: slack_config.yaml -### Testing -```bash -# Run all tests -pytest +# Legacy / dedicated entry points +innieme_bot # Discord (== innieme.cli.run_bot:main) +innieme_slack_bot # Slack (== innieme.cli.run_slack_bot:main) -# Run tests with coverage -pytest --cov=src/innieme +# Tests +pytest # all +pytest tests/test_slack_bot.py # one file +pytest tests/test_slack_bot.py::test_name # one test +pytest --cov=src/innieme # with coverage -# Run async tests specifically -pytest -k "async" --asyncio-mode=strict -``` - -### Code Quality -```bash -# Format code +# Code quality (CI runs flake8 against the whole repo — see below) black src/ tests/ - -# Sort imports isort src/ tests/ - -# Lint code -flake8 src/ tests/ +flake8 . --select=E9,F63,F7,F82 --show-source --statistics # hard-fail pass +flake8 . --exit-zero --max-complexity=10 --max-line-length=127 --statistics # advisory ``` -## Architecture Overview +Dependency sources: `requirements.in` / `requirements-dev.in` are the pinned inputs; +`requirements.txt` / `requirements-dev.txt` are the compiled lockfiles that CI installs. + +### Continuous Integration +`.github/workflows/python-app.yml` runs on push/PR to `main`: installs `requirements.txt` ++ `requirements-dev.txt` + `pip install -e .`, runs the two flake8 passes above (only the +first can fail the build), then `pytest`. + +## Architecture + +### Data flow + +1. On startup, each `Topic` runs `scan_and_vectorize()` — documents in `docs_dir` are chunked + (1000 chars, 200 overlap) and stored in an **in-memory** Chroma collection. A new collection + is created every startup; there is no persistence. +2. On mention/message, the bot identifies the `Topic` for that channel, gets the `thread_id`, + and calls `Topic.process_query()`. +3. `ConversationEngine` runs a similarity search (top-5 chunks) and invokes a PydanticAI `Agent` + whose system prompt is built from `ConversationDependencies` (topic `role` + matched doc + chunks + thread conversation history). +4. The LLM is configurable via `llm_model` (default `openai:gpt-3.5-turbo`); the provider/model + string and `llm_api_key` are resolved in `conversation_engine._build_model()`. + +### Core components + +- **DiscordBot** (`src/innieme/discord_bot.py`) / **SlackBot** (`src/innieme/slack_bot.py`): + platform adapters that handle events, commands, and message routing. +- **Innie** / **Topic** (`src/innieme/innie.py`): `Innie` holds an outie's topics; `Topic` owns + one topic's document store, channels, and conversation engine. +- **ConversationEngine** (`src/innieme/conversation_engine.py`): query → response via a + PydanticAI `Agent`. Imports `TopicConfig` from `discord_bot_config` — shared by both platforms + since the config shapes are identical. +- **DocumentProcessor** (`src/innieme/document_processor.py`): scanning, vectorization, + similarity search. +- **KnowledgeManager** (`src/innieme/knowledge_manager.py`): conversation summarization via a + PydanticAI `Agent` with structured `SummaryOutput`, plus knowledge-base storage. +- **EmbeddingsFactory** (`embeddings_factory.py`) / **VectorStoreFactory** + (`vector_store_factory.py`): pluggable embeddings (OpenAI/HuggingFace/Fake) and vector stores + (Chroma/FAISS). + +### Config → runtime object hierarchy + +Both `DiscordBotConfig` (`discord_bot_config.py`) and `SlackBotConfig` (`slack_bot_config.py`) +share the same nested YAML shape: -The application follows a modular architecture with clear separation of concerns: +``` +BotConfig # top-level: tokens, model/key fields + └── outies: List[OutieConfig] # admins + └── topics: List[TopicConfig] # knowledge domains (role, docs_dir) + └── channels: List[ChannelConfig] # where the bot listens +``` -### Core Components +Pydantic models wire **back-references** via `model_validator(mode='after')`: `OutieConfig.bot`, +`TopicConfig.outie`, `ChannelConfig.topic`. This lets any nested object reach its parent config +(e.g. `Topic.__init__` reads `outie_config.bot.llm_model`) without threading arguments around. -1. **DiscordBot** (`src/innieme/discord_bot.py`): Main bot interface that handles Discord events, commands, and message routing -2. **Innie** (`src/innieme/innie.py`): Container class that manages multiple topics and their configurations -3. **Topic** (`src/innieme/innie.py`): Represents a single topic with its own document store, channels, and conversation engine -4. **ConversationEngine** (`src/innieme/conversation_engine.py`): Handles query processing and response generation using a PydanticAI `Agent` -5. **DocumentProcessor** (`src/innieme/document_processor.py`): Manages document scanning, vectorization, and similarity search -6. **KnowledgeManager** (`src/innieme/knowledge_manager.py`): Handles conversation summarization (via a PydanticAI `Agent` with structured `SummaryOutput`) and knowledge base storage +Both bot configs expose the same model/key fields, read by `Innie`/`Topic`: +- `embedding_model`: `"openai"`, `"huggingface"`, or `"fake"` +- `embeddings_api_key`: API key for the embedding model (required when `embedding_model` is `"openai"`) +- `llm_model`: PydanticAI model string, e.g. `"openai:gpt-4o"` or `"anthropic:claude-sonnet-4-6"` +- `llm_api_key`: API key for the LLM provider -### Factory Pattern Components +Platform differences: +- Discord: `discord_token`; `outie_id`/`guild_id`/`channel_id` are **integers**; thread IDs are + Discord integer IDs. +- Slack: `slack_bot_token` + `slack_app_token` (Socket Mode); `outie_id` (`U…`) and `channel_id` + (`C…`) are **strings**; thread IDs are Slack message timestamps (strings). -- **EmbeddingsFactory** (`src/innieme/embeddings_factory.py`): Creates embedding instances (OpenAI, HuggingFace, or Fake) -- **VectorStoreFactory** (`src/innieme/vector_store_factory.py`): Creates vector store instances (Chroma, FAISS) +### Channel → Topic routing -### Configuration System +Both bots build a `defaultdict[channel_id, List[Topic]]` at init. `_identify_topic(channel_id)` +returns the first topic for a channel. A channel can map to multiple topics, but only the first +is used. -The bot uses YAML configuration (`config.yaml`) with the following structure: -- Multiple "outies" (administrators) can be defined -- Each outie can have multiple topics -- Each topic has its own role/system prompt, document directory, and Discord channels -- Configuration is loaded via `DiscordBotConfig` class +### Thread tracking -Key top-level config fields: -- `embedding_model`: `"openai"`, `"huggingface"`, or `"fake"` -- `embeddings_api_key`: API key for the embedding model (required when `embedding_model` is `"openai"`) -- `llm_model`: PydanticAI model string, e.g. `"openai:gpt-4o"` or `"anthropic:claude-sonnet-4-6"` -- `llm_api_key`: API key for the LLM provider +`Topic.active_threads` is the set of thread IDs the bot is actively following. A mention adds the +thread ID; subsequent messages in that thread are answered automatically without another mention. -### Bot Behavior +### Bot behavior & admin workflow -- Bot responds when mentioned in Discord channels -- Creates threaded conversations for each interaction -- Follows threads where it was mentioned or initially responded -- Supports admin commands like "summary and file" and "please consult outie" -- Admin can approve summaries to add to the knowledge base +- Responds when mentioned; creates/follows a thread per interaction. +- Users can ask to "please consult outie" to bring in an admin. +- Admins can summarize a thread ("summary and file" on Discord, `/approve` etc. on Slack) and + approve the summary to be stored into the knowledge base (`./data/summaries/`). -## Key Implementation Details +### Embedding & vector store selection -### Thread Management -- New mentions create threads automatically -- Bot tracks active threads in `Topic.active_threads` set -- Thread context is preserved for conversation continuity +`embedding_model` selects the embeddings backend (use `fake` in tests to avoid real API calls). +The vector store backend is **hardcoded to Chroma** in `innie.py` (`FAISSVectorStoreFactory` is +present but commented out). -### Document Processing -- Documents are vectorized using configurable embedding models -- Vector stores support both Chroma and FAISS backends -- Document search provides context for LLM responses +### Response length limits -### Response Generation -- Uses PydanticAI `Agent` with a configurable LLM (default: `openai:gpt-3.5-turbo`) -- LLM provider and model are set via `llm_model` in `config.yaml` (e.g. `"openai:gpt-4o"`, `"anthropic:claude-sonnet-4-6"`) -- Combines document context with conversation history via `ConversationDependencies` -- Handles responses longer than Discord's 2000 character limit by sending as files +- Discord: 2000 chars — overflow sent as a `response.txt` file attachment. +- Slack: 4000 chars — overflow uploaded via `files_upload`. -### Error Handling -- Comprehensive logging with configurable levels (LOG_LEVEL, INNIEME_LOG_LEVEL environment variables) -- Graceful error messages sent to Discord users -- Exception re-raising for debugging purposes +## Configuration -## Testing Configuration +Copy the example file(s) for the platform(s) you run and fill in credentials: + +```bash +cp config.example.yaml config.yaml # Discord +cp slack_config.example.yaml slack_config.yaml # Slack +``` -Tests are configured for async support with `pytest.ini` settings: -- `asyncio_mode = strict` -- Various warning filters for dependencies (faiss, pydantic, numpy, etc.) +The `role` field in each topic is the LLM system prompt. Each topic has its own `docs_dir` and +set of channels. The CLI loads the config from the current working directory by default, so run +from the project root (or pass `-c`). ## Environment Variables -- `LOG_LEVEL`: Global logging level (default: INFO) -- `INNIEME_LOG_LEVEL`: Package-specific logging level (default: INFO) \ No newline at end of file +- `LOG_LEVEL`: root logger level (default `INFO`) +- `INNIEME_LOG_LEVEL`: `innieme` package logger level (default `INFO`) diff --git a/pyproject.toml b/pyproject.toml index 457ae9d..cdbbaec 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -9,6 +9,7 @@ description = "Your bot description" requires-python = ">=3.13" dependencies = [ "discord.py", + "slack-bolt", "python-dotenv", "chromadb", "langchain>=0.1.0,<1.0.0", @@ -37,6 +38,8 @@ dev = [ [project.scripts] innieme_bot = "innieme.cli.run_bot:main" +innieme_slack_bot = "innieme.cli.run_slack_bot:main" +innieme = "innieme.cli.run_unified_bot:main" [tool.setuptools] py-modules = [] diff --git a/requirements.in b/requirements.in index c1144b9..082e9c7 100644 --- a/requirements.in +++ b/requirements.in @@ -1,5 +1,6 @@ -# Discord Integration +# Platform Integrations discord.py +slack-bolt # Environment Configuration python-dotenv diff --git a/requirements.txt b/requirements.txt index 050f0c3..fcaeb17 100644 --- a/requirements.txt +++ b/requirements.txt @@ -543,6 +543,10 @@ six==1.17.0 # via # kubernetes # python-dateutil +slack-bolt==1.23.0 + # via -r requirements.in +slack-sdk==3.36.0 + # via slack-bolt sniffio==1.3.1 # via # anthropic diff --git a/slack_config.example.yaml b/slack_config.example.yaml new file mode 100644 index 0000000..7efd0a4 --- /dev/null +++ b/slack_config.example.yaml @@ -0,0 +1,60 @@ +# Slack Bot Configuration Example +# Copy this to slack_config.yaml and fill in your actual values + +# Slack Bot Token (starts with xoxb-) +# Get this from your Slack app's OAuth & Permissions page +slack_bot_token: "xoxb-your-bot-token-here" + +# Slack App Token (starts with xapp-) +# Get this from your Slack app's Basic Information page (App-Level Tokens section) +# Required for Socket Mode +slack_app_token: "xapp-your-app-token-here" + +# API key for the embedding model (required when embedding_model is "openai") +embeddings_api_key: "your-embeddings-api-key" + +# Embedding model to use: openai, huggingface, or fake +embedding_model: "openai" + +# LLM to use for conversation and summarisation +# Format: ":", e.g. "openai:gpt-4o", "anthropic:claude-sonnet-4-6" +llm_model: "openai:gpt-3.5-turbo" + +# API key for the LLM provider +llm_api_key: "your-llm-api-key" + +# Bot administrators and their topics +outies: + - outie_id: "U1234567890" # Slack User ID (starts with U) + topics: + - name: "math" + role: "You are a helpful math tutor. Answer math questions clearly and provide step-by-step explanations." + docs_dir: "./docs/math" # Directory containing math documents + channels: + - channel_id: "C1234567890" # Slack Channel ID (starts with C) + + - name: "science" + role: "You are a knowledgeable science teacher. Explain scientific concepts in an accessible way." + docs_dir: "./docs/science" + channels: + - channel_id: "C0987654321" + + - outie_id: "U0987654321" # Another administrator + topics: + - name: "support" + role: "You are a helpful customer support agent. Be friendly and solution-oriented." + docs_dir: "./docs/support" + channels: + - channel_id: "C5555555555" + +# To get Slack IDs: +# - User IDs: Right-click on a user in Slack → Copy → Member ID +# - Channel IDs: Right-click on a channel → Copy → Channel ID +# +# To set up your Slack app: +# 1. Go to https://api.slack.com/apps +# 2. Create a new app "From scratch" +# 3. Add Bot Token Scopes: app_mentions:read, channels:history, channels:read, chat:write, files:write, im:history, im:read, users:read +# 4. Enable Socket Mode and create an App-Level Token with connections:write scope +# 5. Install the app to your workspace +# 6. Invite the bot to your channels: /invite @your-bot-name \ No newline at end of file diff --git a/src/innieme/cli/run_slack_bot.py b/src/innieme/cli/run_slack_bot.py new file mode 100644 index 0000000..b93d4af --- /dev/null +++ b/src/innieme/cli/run_slack_bot.py @@ -0,0 +1,37 @@ +from innieme.slack_bot import SlackBot +from innieme.slack_bot_config import SlackBotConfig + +import logging +import os + +# Configure root logger with LOG_LEVEL +log_level_name = os.environ.get("LOG_LEVEL", "INFO") +log_level = getattr(logging, log_level_name.upper(), logging.INFO) + +logging.basicConfig( + level=log_level, + format='%(asctime)s %(levelname)-8s %(name)s %(message)s', + datefmt='%Y-%m-%d %H:%M:%S' +) + +# Configure innieme package logger with INNIEME_LOG_LEVEL +innieme_log_level_name = os.environ.get("INNIEME_LOG_LEVEL", "INFO") +innieme_log_level = getattr(logging, innieme_log_level_name.upper(), logging.INFO) +innieme_logger = logging.getLogger("innieme") +innieme_logger.setLevel(innieme_log_level) + +# Load environment variables +current_dir = os.getcwd() +yaml_path = os.path.join(current_dir, 'slack_config.yaml') +with open(yaml_path, "r") as yaml_file: + yaml_content = yaml_file.read() +config = SlackBotConfig.from_yaml(yaml_content) +print(f"Loaded config from {yaml_path}") + +def main(): + # Create and run the bot + bot = SlackBot(config) + bot.run() + +if __name__ == "__main__": + main() \ No newline at end of file diff --git a/src/innieme/cli/run_unified_bot.py b/src/innieme/cli/run_unified_bot.py new file mode 100644 index 0000000..d6da3cd --- /dev/null +++ b/src/innieme/cli/run_unified_bot.py @@ -0,0 +1,109 @@ +#!/usr/bin/env python3 +""" +Unified CLI for running either Discord or Slack bots +""" + +import argparse +import logging +import os +import sys + +def setup_logging(): + """Set up logging configuration""" + # Configure root logger with LOG_LEVEL + log_level_name = os.environ.get("LOG_LEVEL", "INFO") + log_level = getattr(logging, log_level_name.upper(), logging.INFO) + + logging.basicConfig( + level=log_level, + format='%(asctime)s %(levelname)-8s %(name)s %(message)s', + datefmt='%Y-%m-%d %H:%M:%S' + ) + + # Configure innieme package logger with INNIEME_LOG_LEVEL + innieme_log_level_name = os.environ.get("INNIEME_LOG_LEVEL", "INFO") + innieme_log_level = getattr(logging, innieme_log_level_name.upper(), logging.INFO) + innieme_logger = logging.getLogger("innieme") + innieme_logger.setLevel(innieme_log_level) + +def run_discord_bot(config_path: str = None): + """Run the Discord bot""" + from innieme.discord_bot import DiscordBot + from innieme.discord_bot_config import DiscordBotConfig + + if not config_path: + config_path = os.path.join(os.getcwd(), 'config.yaml') + + try: + with open(config_path, "r") as yaml_file: + yaml_content = yaml_file.read() + config = DiscordBotConfig.from_yaml(yaml_content) + print(f"Loaded Discord config from {config_path}") + + bot = DiscordBot(config) + bot.run() + except FileNotFoundError: + print(f"Error: Config file not found at {config_path}") + sys.exit(1) + except Exception as e: + print(f"Error loading Discord config: {e}") + sys.exit(1) + +def run_slack_bot(config_path: str = None): + """Run the Slack bot""" + from innieme.slack_bot import SlackBot + from innieme.slack_bot_config import SlackBotConfig + + if not config_path: + config_path = os.path.join(os.getcwd(), 'slack_config.yaml') + + try: + with open(config_path, "r") as yaml_file: + yaml_content = yaml_file.read() + config = SlackBotConfig.from_yaml(yaml_content) + print(f"Loaded Slack config from {config_path}") + + bot = SlackBot(config) + bot.run() + except FileNotFoundError: + print(f"Error: Config file not found at {config_path}") + sys.exit(1) + except Exception as e: + print(f"Error loading Slack config: {e}") + sys.exit(1) + +def main(): + setup_logging() + + parser = argparse.ArgumentParser( + description="InnieMe Bot - Run Discord or Slack bot", + formatter_class=argparse.RawDescriptionHelpFormatter, + epilog=""" +Examples: + %(prog)s discord # Run Discord bot with default config.yaml + %(prog)s slack # Run Slack bot with default slack_config.yaml + %(prog)s discord -c my_config.yaml # Run Discord bot with custom config + %(prog)s slack -c my_slack.yaml # Run Slack bot with custom config + """ + ) + + parser.add_argument( + 'platform', + choices=['discord', 'slack'], + help='Platform to run the bot on' + ) + + parser.add_argument( + '-c', '--config', + help='Path to configuration file' + ) + + args = parser.parse_args() + + if args.platform == 'discord': + run_discord_bot(args.config) + elif args.platform == 'slack': + run_slack_bot(args.config) + +if __name__ == "__main__": + main() \ No newline at end of file diff --git a/src/innieme/slack_bot.py b/src/innieme/slack_bot.py new file mode 100644 index 0000000..9b93bc7 --- /dev/null +++ b/src/innieme/slack_bot.py @@ -0,0 +1,372 @@ +from .slack_bot_config import SlackBotConfig +from .innie import Innie, Topic + +from slack_bolt.async_app import AsyncApp +from slack_bolt.adapter.socket_mode.async_handler import AsyncSocketModeHandler +from slack_sdk.web.async_client import AsyncWebClient + +import logging +import asyncio +from collections import defaultdict +from typing import Optional, List, Dict, Any +import io + +logger = logging.getLogger(__name__) + +class SlackBot: + def __init__(self, config: SlackBotConfig): + # Bot setup + self.app = AsyncApp(token=config.slack_bot_token) + self.handler = AsyncSocketModeHandler(self.app, config.slack_app_token) + self.client = self.app.client + + # Innies setup + self.innies = [Innie(outie_config) for outie_config in config.outies] + # Channel->Topic mapping + self.channels: defaultdict[str, List[Topic]] = defaultdict(list) + for innie in self.innies: + for topic in innie.topics: + for channel_config in topic.config.channels: + self.channels[channel_config.channel_id].append(topic) + + # Register event handlers and commands + self._register_events() + self._register_commands() + + def _register_events(self): + """Register all event handlers""" + @self.app.event("app_mention") + async def handle_app_mention(event, say, client): + await self.handle_mention(event, say, client) + + @self.app.event("message") + async def handle_message(event, say, client): + # Only handle direct messages and thread replies where the bot was previously mentioned + if event.get("channel_type") == "im" or self._should_respond_to_thread(event): + await self.handle_message(event, say, client) + + def _register_commands(self): + """Register all slash commands""" + @self.app.command("/approve") + async def approve_command(ack, command, client): + await ack() + await self.approve_summary(command, client) + + @self.app.command("/quit") + async def quit_command(ack, command, client): + await ack() + topic = self._identify_topic(command["channel_id"]) + if not topic: + await client.chat_postMessage( + channel=command["channel_id"], + text="'quit' command ignored as there is no topic in this channel to support." + ) + return + + topic_outie = topic.outie_config.outie_id + if command["user_id"] != topic_outie: + await client.chat_postMessage( + channel=command["channel_id"], + text=f"This command is only available to the outie (<@{topic_outie}>)." + ) + return + + await client.chat_postMessage( + channel=command["channel_id"], + text="Goodbye! Bot shutting down..." + ) + await self.stop() + + @self.app.command("/hello") + async def hello_command(ack, command, client): + await ack() + # Create a rich message block + blocks = [ + { + "type": "header", + "text": { + "type": "plain_text", + "text": "InnieMe: Your Knowledge Speaks for Itself" + } + }, + { + "type": "section", + "text": { + "type": "mrkdwn", + "text": "Democratizes access to AI-powered Q&A capabilities\n\n*Pricing:* Free during early access\n*Key Feature:* Ask and you shall receive\n*Deployment:* Channel specific" + }, + "accessory": { + "type": "image", + "image_url": "https://repository-images.githubusercontent.com/956066438/8dce1cee-0386-423d-817c-283e3dfb7288", + "alt_text": "InnieMe logo" + } + }, + { + "type": "context", + "elements": [ + { + "type": "mrkdwn", + "text": "*InnieMe © 2025* | " + } + ] + } + ] + + await client.chat_postMessage( + channel=command["channel_id"], + blocks=blocks + ) + + def _identify_topic(self, channel_id: str) -> Optional[Topic]: + topics = self.channels.get(channel_id, []) + return topics[0] if topics else None + + def _should_respond_to_thread(self, event: Dict[str, Any]) -> bool: + """Check if bot should respond to a thread message""" + # Only respond in threads where bot was mentioned or is actively participating + thread_ts = event.get("thread_ts") + if not thread_ts: + return False + + # Check if this thread involves the bot + topic = self._identify_topic(event["channel"]) + if topic: + return topic.is_following_thread(thread_ts) + + return False + + async def get_thread_context(self, channel_id: str, thread_ts: str, limit: int = 10) -> List[Dict[str, str]]: + """Get recent messages from the thread for context""" + messages = [] + try: + # Get thread messages + result = await self.client.conversations_replies( + channel=channel_id, + ts=thread_ts, + limit=limit + ) + + bot_user_id = (await self.client.auth_test())["user_id"] + + for message in result["messages"]: + if "text" in message: + role = "assistant" if message["user"] == bot_user_id else "user" + messages.append({ + "role": role, + "content": message["text"] + }) + + return messages + except Exception as e: + logger.error(f"Error fetching thread context: {e}") + return [] + + async def respond(self, channel_id: str, text: str, thread_ts: str = None): + """Send a response to a channel or thread""" + try: + await self.client.chat_postMessage( + channel=channel_id, + text=text, + thread_ts=thread_ts + ) + except Exception as e: + logger.error(f"Error sending message: {e}") + + async def process_and_respond(self, topic: Topic, channel_id: str, query: str, thread_id: str, thread_ts: str = None): + """Process a query and respond in the channel""" + context_messages = [] + if thread_ts: + context_messages = await self.get_thread_context(channel_id, thread_ts) + else: + context_messages = [{"role": "user", "content": query}] + + try: + # Add typing indicator + await self.client.chat_postMessage( + channel=channel_id, + text="🤔 Thinking...", + thread_ts=thread_ts + ) + + response = await topic.process_query(thread_id, query, context_messages=context_messages) + + # Delete thinking message and send response + if len(response) > 4000: # Slack has a 4000 character limit for messages + # Upload as a file + await self.client.files_upload( + channels=channel_id, + content=response, + filename="response.txt", + title="Response (too long for message)", + thread_ts=thread_ts + ) + else: + # Send as normal message + await self.client.chat_postMessage( + channel=channel_id, + text=response, + thread_ts=thread_ts + ) + + except Exception as e: + error_message = f"Sorry, I encountered an error while processing your request: {str(e)}" + await self.client.chat_postMessage( + channel=channel_id, + text=error_message, + thread_ts=thread_ts + ) + raise # Re-raise the exception for logging/debugging + + async def handle_mention(self, event: Dict[str, Any], say, client: AsyncWebClient): + """Handle app mentions""" + channel_id = event["channel"] + user_id = event["user"] + text = event["text"] + ts = event["ts"] + + topic = self._identify_topic(channel_id) + if not topic: + await say(text="Sorry I am not set up to support a topic in this channel.") + return + + logger.debug(f"Handling mention in topic: {topic.config.name}") + + # Clean the mention from the text + bot_user_id = (await client.auth_test())["user_id"] + clean_text = text.replace(f'<@{bot_user_id}>', '').strip() + + # Start a thread and process the query + await self.process_and_respond( + topic, + channel_id, + clean_text, + ts, # Use timestamp as thread_id + ts # Use timestamp as thread_ts for threading + ) + + async def handle_message(self, event: Dict[str, Any], say, client: AsyncWebClient): + """Handle direct messages and thread replies""" + channel_id = event["channel"] + user_id = event["user"] + text = event["text"] + ts = event["ts"] + thread_ts = event.get("thread_ts") + + # Skip bot's own messages + bot_user_id = (await client.auth_test())["user_id"] + if user_id == bot_user_id: + return + + topic = self._identify_topic(channel_id) + if not topic: + return + + outie_id = topic.outie_config.outie_id + + # Handle outie commands + if user_id == outie_id and "summary and file" in text.lower(): + if thread_ts: + summary = await topic.generate_summary(thread_ts) + await client.chat_postMessage( + channel=channel_id, + text=f"Summary generated:\n\n{summary}\n\nApprove to add to knowledge base? (use `/approve`)", + thread_ts=thread_ts + ) + return + + # Handle consultation requests + if "please consult outie" in text.lower(): + if thread_ts: + await client.chat_postMessage( + channel=channel_id, + text=f"<@{outie_id}> Your consultation has been requested in this thread.", + thread_ts=thread_ts + ) + return + + # Handle thread responses + if thread_ts and topic.is_following_thread(thread_ts): + await self.process_and_respond( + topic, + channel_id, + text, + thread_ts, + thread_ts + ) + + async def connect_and_prepare(self, topic: Topic): + """Connect to channels and prepare documents""" + outie_id = topic.outie_config.outie_id + channels = [] + + for channel_config in topic.config.channels: + channel_id = channel_config.channel_id + try: + # Test if we can access the channel + await self.client.conversations_info(channel=channel_id) + channels.append(channel_id) + await self.client.chat_postMessage( + channel=channel_id, + text=f"Bot is connected, preparing documents for {topic.config.name}..." + ) + except Exception as e: + logger.error(f"Could not connect to channel {channel_id}: {e}") + # Try to notify the outie via DM + try: + await self.client.chat_postMessage( + channel=outie_id, + text=f"Bot is now online but could not access channel <#{channel_id}>. Please make sure the bot is invited to the channel." + ) + except Exception as dm_e: + logger.error(f"Could not send DM to outie {outie_id}: {dm_e}") + + # Scan and vectorize documents + scanning_result = await topic.scan_and_vectorize() + + # Notify channels of completion + for channel_id in channels: + mention = f"(fyi <@{outie_id}>)" + await self.client.chat_postMessage( + channel=channel_id, + text=f"{scanning_result} {mention}" + ) + + async def approve_summary(self, command: Dict[str, Any], client: AsyncWebClient): + """Handle summary approval""" + channel_id = command["channel_id"] + user_id = command["user_id"] + + topic = self._identify_topic(channel_id) + if not topic: + return + + outie_id = topic.outie_config.outie_id + if user_id == outie_id: + # This would need thread_ts to identify which summary to approve + # For now, we'll use a simple implementation + await client.chat_postMessage( + channel=channel_id, + text="Summary approved and added to knowledge base." + ) + # TODO: Implement actual summary storage with thread tracking + + async def start(self): + """Start the bot""" + logger.info("Starting Slack bot...") + + # Prepare all topics + for innie in self.innies: + for topic in innie.topics: + await self.connect_and_prepare(topic) + + # Start the socket mode handler + await self.handler.start_async() + + async def stop(self): + """Stop the bot""" + logger.info("Stopping Slack bot...") + await self.handler.close_async() + + def run(self): + """Run the bot (blocking)""" + asyncio.run(self.start()) \ No newline at end of file diff --git a/src/innieme/slack_bot_config.py b/src/innieme/slack_bot_config.py new file mode 100644 index 0000000..a824d74 --- /dev/null +++ b/src/innieme/slack_bot_config.py @@ -0,0 +1,84 @@ +import os, yaml +from typing import List + +from pydantic import BaseModel, field_validator, model_validator + +class ChannelConfig(BaseModel): + channel_id: str # Slack channel IDs are strings + topic: 'TopicConfig' = None # type: ignore + +class TopicConfig(BaseModel): + name: str + role: str + docs_dir: str + channels: List[ChannelConfig] + outie: 'OutieConfig' = None # type: ignore + + @classmethod + @field_validator('docs_dir') + def docs_dir_must_exist(cls, v): + if not os.path.exists(v): + raise ValueError(f'Document directory does not exist: {v}') + return v + + @model_validator(mode='after') + def set_back_references(self): + for channel in self.channels: + channel.topic = self + return self + +class OutieConfig(BaseModel): + outie_id: str # Slack user IDs are strings + topics: List[TopicConfig] + bot: 'SlackBotConfig' = None # type: ignore + + @field_validator('outie_id') + def id_must_not_be_empty(cls, v): + if not v: + raise ValueError('Outie ID cannot be empty') + return v + + @model_validator(mode='after') + def set_back_references(self): + for topic in self.topics: + topic.outie = self + return self + +class SlackBotConfig(BaseModel): + slack_bot_token: str + slack_app_token: str # For Socket Mode + embeddings_api_key: str + llm_api_key: str + embedding_model: str + llm_model: str = "openai:gpt-3.5-turbo" + outies: List[OutieConfig] + + @field_validator('slack_bot_token') + def bot_token_must_not_be_empty(cls, v): + if not v: + raise ValueError('Slack bot token cannot be empty') + return v + + @field_validator('slack_app_token') + def app_token_must_not_be_empty(cls, v): + if not v: + raise ValueError('Slack app token cannot be empty') + return v + + @field_validator('embedding_model') + def model_must_be_supported(cls, v): + supported_models = ['openai', 'huggingface', 'fake'] + if v not in supported_models: + raise ValueError(f'Unsupported embedding model: {v}') + return v + + @model_validator(mode='after') + def set_back_references(self): + for outie in self.outies: + outie.bot = self + return self + + @classmethod + def from_yaml(cls, yaml_content: str) -> "SlackBotConfig": + config_data = yaml.safe_load(yaml_content) + return cls(**config_data) \ No newline at end of file diff --git a/tests/test_slack_bot.py b/tests/test_slack_bot.py new file mode 100644 index 0000000..004b762 --- /dev/null +++ b/tests/test_slack_bot.py @@ -0,0 +1,128 @@ +from innieme.slack_bot import SlackBot +from innieme.slack_bot_config import SlackBotConfig, OutieConfig, TopicConfig, ChannelConfig + +import pytest +from unittest.mock import AsyncMock, Mock, patch +import os + +@pytest.fixture +def mock_config(): + """Create a mock SlackBotConfig for testing""" + # Create test directories + math_docs_dir = 'data/math' + os.makedirs(math_docs_dir, exist_ok=True) + + # Create channel config + channel_config = ChannelConfig(channel_id="C1234567890") + + # Create topic config + topic_config = TopicConfig( + name="math", + role="Math Teacher", + docs_dir=math_docs_dir, + channels=[channel_config] + ) + + # Create outie config + outie_config = OutieConfig( + outie_id="U1234567890", + topics=[topic_config] + ) + + # Create bot config + config = SlackBotConfig( + slack_bot_token="xoxb-test-token", + slack_app_token="xapp-test-token", + embeddings_api_key="test_embeddings_key", + llm_api_key="test_llm_key", + embedding_model="fake", # Use fake for testing + outies=[outie_config] + ) + + return config + +@patch('innieme.slack_bot.AsyncApp') +@patch('innieme.slack_bot.AsyncSocketModeHandler') +def test_slack_bot_initialization(mock_handler, mock_app, mock_config): + """Test SlackBot initialization""" + # Mock the AsyncApp and handler + mock_app_instance = Mock() + mock_app.return_value = mock_app_instance + mock_app_instance.client = Mock() + + mock_handler_instance = Mock() + mock_handler.return_value = mock_handler_instance + + # Create SlackBot instance + bot = SlackBot(mock_config) + + # Verify initialization + assert bot.app == mock_app_instance + assert bot.handler == mock_handler_instance + assert bot.client == mock_app_instance.client + assert len(bot.innies) == 1 + assert len(bot.channels) == 1 + assert "C1234567890" in bot.channels + +def test_identify_topic(mock_config): + """Test topic identification by channel ID""" + with patch('innieme.slack_bot.AsyncApp'), \ + patch('innieme.slack_bot.AsyncSocketModeHandler'): + bot = SlackBot(mock_config) + + # Test existing channel + topic = bot._identify_topic("C1234567890") + assert topic is not None + assert topic.config.name == "math" + + # Test non-existing channel + topic = bot._identify_topic("C0000000000") + assert topic is None + +@pytest.mark.asyncio +async def test_get_thread_context(mock_config): + """Test getting thread context from Slack""" + with patch('innieme.slack_bot.AsyncApp'), \ + patch('innieme.slack_bot.AsyncSocketModeHandler'): + bot = SlackBot(mock_config) + + # Mock the Slack client + bot.client = AsyncMock() + bot.client.conversations_replies.return_value = { + "messages": [ + {"user": "U1234567890", "text": "Hello"}, + {"user": "UBOT123456", "text": "Hi there!"}, + {"user": "U1234567890", "text": "How are you?"} + ] + } + bot.client.auth_test.return_value = {"user_id": "UBOT123456"} + + # Get thread context + context = await bot.get_thread_context("C1234567890", "1234567890.123456") + + # Verify context + assert len(context) == 3 + assert context[0]["role"] == "user" + assert context[0]["content"] == "Hello" + assert context[1]["role"] == "assistant" + assert context[1]["content"] == "Hi there!" + assert context[2]["role"] == "user" + assert context[2]["content"] == "How are you?" + +def test_should_respond_to_thread(mock_config): + """Test thread response logic""" + with patch('innieme.slack_bot.AsyncApp'), \ + patch('innieme.slack_bot.AsyncSocketModeHandler'): + bot = SlackBot(mock_config) + + # Test message without thread_ts + event = {"channel": "C1234567890"} + assert not bot._should_respond_to_thread(event) + + # Test message with thread_ts but no following thread + event = {"channel": "C1234567890", "thread_ts": "1234567890.123456"} + assert not bot._should_respond_to_thread(event) + + # Test message with thread_ts in non-existing channel + event = {"channel": "C0000000000", "thread_ts": "1234567890.123456"} + assert not bot._should_respond_to_thread(event) \ No newline at end of file diff --git a/tests/test_slack_bot_config.py b/tests/test_slack_bot_config.py new file mode 100644 index 0000000..984a0e7 --- /dev/null +++ b/tests/test_slack_bot_config.py @@ -0,0 +1,119 @@ +from innieme.slack_bot_config import OutieConfig, SlackBotConfig + +import pytest + +import os + +def test_valid_outie_id(): + """Test that a valid outie_id is accepted""" + # Create a bot config first + bot = SlackBotConfig( + slack_bot_token="xoxb-test-token", + slack_app_token="xapp-test-token", + embeddings_api_key="key", + llm_api_key="key", + embedding_model="huggingface", + outies=[] + ) + outie = OutieConfig(outie_id="U1234567890", topics=[], bot=bot) + assert outie.outie_id == "U1234567890" + +def test_invalid_outie_id(): + """Test that empty outie_id raises ValueError""" + with pytest.raises(ValueError) as exc_info: + OutieConfig(outie_id="", topics=[]) + + assert "Outie ID cannot be empty" in str(exc_info.value) + +def test_invalid_slack_bot_token(): + """Test that empty bot token raises ValueError""" + with pytest.raises(ValueError) as exc_info: + SlackBotConfig( + slack_bot_token="", + slack_app_token="xapp-test-token", + embeddings_api_key="test_embeddings_key", + llm_api_key="test_llm_key", + embedding_model="huggingface", + outies=[OutieConfig(outie_id="U1234567890", topics=[])] + ) + assert "Slack bot token cannot be empty" in str(exc_info.value) + +def test_invalid_slack_app_token(): + """Test that empty app token raises ValueError""" + with pytest.raises(ValueError) as exc_info: + SlackBotConfig( + slack_bot_token="xoxb-test-token", + slack_app_token="", + embeddings_api_key="test_embeddings_key", + llm_api_key="test_llm_key", + embedding_model="huggingface", + outies=[OutieConfig(outie_id="U1234567890", topics=[])] + ) + assert "Slack app token cannot be empty" in str(exc_info.value) + +def test_config_from_yaml(): + """Test creating config from multi-line YAML content""" + math_docs_dir = 'data/math' + scouting_docs_dir = 'data/scouting' + innieme_docs_dir = 'data/innieme' + for dir in [math_docs_dir, scouting_docs_dir, innieme_docs_dir]: + os.makedirs(dir, exist_ok=True) + + yaml_content = f""" + slack_bot_token: "xoxb-test-discord-token" + slack_app_token: "xapp-test-app-token" + embeddings_api_key: "test_embeddings_key" + llm_api_key: "test_llm_key" + embedding_model: "openai" + outies: + - outie_id: "U1234567890" + topics: + - name: "math" + role: "Math Teacher" + docs_dir: "{math_docs_dir}" + channels: + - channel_id: "C1234567890" + - name: "scouting" + role: "ASM" + docs_dir: "{scouting_docs_dir}" + channels: + - channel_id: "C0987654321" + - outie_id: "U0987654321" + topics: + - name: "innieme" + role: "Support" + docs_dir: "{innieme_docs_dir}" + channels: + - channel_id: "C5555555555" + """ + + config = SlackBotConfig.from_yaml(yaml_content) + + assert config.slack_bot_token == "xoxb-test-discord-token" + assert config.slack_app_token == "xapp-test-app-token" + assert config.embeddings_api_key == "test_embeddings_key" + assert config.llm_api_key == "test_llm_key" + assert len(config.outies) == 2 + + # Verify first outie + assert config.outies[0].outie_id == "U1234567890" + assert config.outies[0].topics[0].name == "math" + assert config.outies[0].topics[0].channels[0].channel_id == "C1234567890" + + # Verify second outie + assert config.outies[1].outie_id == "U0987654321" + assert config.outies[1].topics[0].name == "innieme" + assert config.outies[1].topics[0].channels[0].channel_id == "C5555555555" + +def test_invalid_embedding_model(): + """Test that unsupported embedding model raises ValueError""" + with pytest.raises(ValueError) as exc_info: + SlackBotConfig( + slack_bot_token="xoxb-test-token", + slack_app_token="xapp-test-token", + embeddings_api_key="test_embeddings_key", + llm_api_key="test_llm_key", + embedding_model="unsupported_model", + outies=[] + ) + assert "Unsupported embedding model: unsupported_model" in str(exc_info.value) \ No newline at end of file