diff --git a/docs/experiments/content-classification.md b/docs/experiments/content-classification.md new file mode 100644 index 000000000..146aa7e24 --- /dev/null +++ b/docs/experiments/content-classification.md @@ -0,0 +1,402 @@ +# Content Classification + +## Summary + +The Content Classification experiment adds AI-powered tag and category suggestions to the WordPress post editor. It analyzes post content and suggests relevant taxonomy terms directly within the Tags and Categories sidebar panels. The experiment registers a WordPress Ability (`ai/content-classification`) that can be used both through the admin UI and directly via REST API requests. + +## Overview + +### For End Users + +When enabled, the Content Classification experiment adds "Suggest Tags" and "Suggest Categories" buttons to their respective panels in the post editor sidebar. Users can click these buttons to generate a list of AI-suggested terms based on the current post content. Suggestions appear as clickable pills that can be accepted (adding the term to the post) or dismissed. + +**Key Features:** + +- One-click tag and category suggestions from post content +- Suggestions shown as clickable pills within the existing taxonomy panels +- New terms are visually distinguished with a "new" badge +- Support for parent/child category relationships +- Configurable strategy: suggest only existing terms or allow new ones +- Configurable maximum number of suggestions (1-10, default 5) +- Regenerate suggestions for different results +- Requires approximately 150 words of content before enabling + +### For Developers + +The experiment consists of two main components: + +1. **Experiment Class** (`WordPress\AI\Experiments\Content_Classification\Content_Classification`): Handles registration, asset enqueuing, settings, and UI integration +2. **Ability Class** (`WordPress\AI\Abilities\Content_Classification\Content_Classification`): Implements the core suggestion logic via the WordPress Abilities API + +The ability can be called directly via REST API, making it useful for automation, bulk processing, or custom integrations. + +## Architecture & Implementation + +### Key Hooks & Entry Points + +- `WordPress\AI\Experiments\Content_Classification\Content_Classification::register()` wires everything once the experiment is enabled: + - `wp_abilities_api_init` → registers the `ai/content-classification` ability (`includes/Abilities/Content_Classification/Content_Classification.php`) + - `admin_enqueue_scripts` → enqueues the React bundle and stylesheet on `post.php` and `post-new.php` screens for post types that support the editor + +### Assets & Data Flow + +1. **PHP Side:** + - `enqueue_assets()` loads `experiments/content-classification` (`src/experiments/content-classification/index.tsx`) and localizes `window.aiContentClassificationData` with: + - `enabled`: Whether the experiment is enabled + - `strategy`: The configured taxonomy strategy (`existing_only` or `allow_new`) + - `maxSuggestions`: The configured maximum number of suggestions + +2. **React Side:** + - The React entry point (`index.tsx`) uses the `editor.PostTaxonomyType` filter via `addFilter` to wrap the native taxonomy selector components + - `SuggestionPanel` component renders a generate button and suggestion pills for each supported taxonomy + - `useContentClassification` hook: + - Gets current post ID and content from the editor store + - Checks word count using `@wordpress/wordcount` (minimum 150 words) + - Calls the ability via `runAbility()` when the button is clicked + - Manages suggestion state (accept, dismiss, regenerate) + - Adds accepted terms to the post via `editPost()` and REST API + +3. **Ability Execution:** + - Accepts `content`, `post_id`, `taxonomy`, `strategy`, and `max_suggestions` as input + - If `post_id` is provided, fetches post context using `get_post_context()` + - Normalizes content using `normalize_content()` helper + - Fetches all existing terms for the taxonomy to encourage consistency + - Builds a dynamic system instruction with strategy rules, existing terms, and JSON output format + - Sends content to AI client and parses the structured JSON response + - Returns an array of suggestions with term name, confidence score, new/existing flag, and optional parent + +### Input Schema + +The ability accepts the following input parameters: + +```php +array( + 'content' => array( + 'type' => 'string', + 'description' => 'Content to generate taxonomy suggestions for.', + ), + 'post_id' => array( + 'type' => 'integer', + 'description' => 'Post ID to generate suggestions for. Overrides content if both provided.', + ), + 'taxonomy' => array( + 'type' => 'string', + 'default' => 'post_tag', + 'description' => 'The taxonomy to generate suggestions for (e.g., post_tag, category).', + ), + 'strategy' => array( + 'type' => 'string', + 'default' => 'existing_only', + 'description' => 'The suggestion strategy: existing_only or allow_new.', + ), + 'max_suggestions' => array( + 'type' => 'integer', + 'default' => 5, + 'minimum' => 1, + 'maximum' => 10, + 'description' => 'Maximum number of suggestions to generate.', + ), +) +``` + +### Output Schema + +The ability returns a structured object: + +```php +array( + 'suggestions' => array( + array( + 'term' => 'machine learning', // string - the suggested term name + 'confidence' => 0.95, // float - relevance score (0-1) + 'is_new' => true, // bool - whether term exists on site + 'parent' => 'technology', // string - optional parent for categories + ), + // ... more suggestions + ), +) +``` + +### Permissions + +The ability checks permissions based on the input: + +- **If `post_id` is provided:** + - Verifies the post exists + - Checks `current_user_can( 'edit_post', $post_id )` + - Ensures the post type has `show_in_rest` enabled + +- **If `post_id` is not provided:** + - Checks `current_user_can( 'edit_posts' )` + +## Settings + +The experiment registers two settings on the AI Experiments settings page: + +- **Taxonomy strategy** (`wpai_experiment_content-classification_field_strategy`): + - `existing_only` (default) — Only suggest terms that already exist on the site + - `allow_new` — Allow suggestions for new terms based on content + +- **Maximum suggestions** (`wpai_experiment_content-classification_field_max_suggestions`): + - Integer between 1 and 10, default 5 + +## Using the Ability via REST API + +### Endpoint + +```text +POST /wp-json/wp-abilities/v1/abilities/ai/content-classification/run +``` + +### Authentication + +You can authenticate using either: + +1. **Application Password** (Recommended) +2. **Cookie Authentication with Nonce** + +See [TESTING_REST_API.md](../TESTING_REST_API.md) for detailed authentication instructions. + +### Request Examples + +#### Example 1: Suggest Tags from Post ID + +```bash +curl -X POST "https://yoursite.com/wp-json/wp-abilities/v1/abilities/ai/content-classification/run" \ + -u "username:application-password" \ + -H "Content-Type: application/json" \ + -d '{ + "input": { + "post_id": 123, + "taxonomy": "post_tag", + "strategy": "allow_new", + "max_suggestions": 5 + } + }' +``` + +**Response:** + +```json +{ + "suggestions": [ + {"term": "artificial intelligence", "confidence": 0.95, "is_new": true}, + {"term": "machine learning", "confidence": 0.9, "is_new": true}, + {"term": "technology", "confidence": 0.85, "is_new": false} + ] +} +``` + +#### Example 2: Suggest Categories from Content String + +```bash +curl -X POST "https://yoursite.com/wp-json/wp-abilities/v1/abilities/ai/content-classification/run" \ + -u "username:application-password" \ + -H "Content-Type: application/json" \ + -d '{ + "input": { + "content": "This article discusses the latest advances in renewable energy technology, including solar panel efficiency improvements and wind turbine innovations.", + "taxonomy": "category", + "strategy": "existing_only" + } + }' +``` + +#### Example 3: Using WordPress API Fetch (in Gutenberg/Admin) + +```javascript +import apiFetch from '@wordpress/api-fetch'; + +async function suggestTags( postId, taxonomy = 'post_tag' ) { + try { + const result = await apiFetch( { + path: '/wp-abilities/v1/abilities/ai/content-classification/run', + method: 'POST', + data: { + input: { + post_id: postId, + taxonomy, + strategy: 'allow_new', + max_suggestions: 5, + }, + }, + } ); + return result.suggestions; + } catch ( error ) { + console.error( 'Error generating suggestions:', error ); + throw error; + } +} +``` + +### Error Responses + +The ability may return the following error codes: + +- `invalid_taxonomy`: The specified taxonomy does not exist +- `post_not_found`: The provided post ID does not exist +- `content_not_provided`: No content was provided and no valid post ID was found +- `no_results`: The AI client did not return any suggestions +- `invalid_response`: The AI response could not be parsed as valid JSON +- `insufficient_capabilities`: The current user does not have permission + +## Extending the Experiment + +### Filtering Content Before AI Processing + +Use the `wpai_content_classification_content` filter to modify the content string before it is sent to the AI model: + +```php +add_filter( 'wpai_content_classification_content', function( $content, $taxonomy, $strategy ) { + // Add custom context to improve suggestions. + if ( 'category' === $taxonomy ) { + $content .= "\n\nSite focus: technology and science news."; + } + return $content; +}, 10, 3 ); +``` + +### Filtering Suggestions After AI Processing + +Use the `wpai_content_classification_suggestions` filter to modify the parsed suggestions before they are returned: + +```php +add_filter( 'wpai_content_classification_suggestions', function( $suggestions, $taxonomy, $strategy ) { + // Remove suggestions with low confidence. + return array_filter( $suggestions, function( $s ) { + return $s['confidence'] >= 0.7; + } ); +}, 10, 3 ); +``` + +### Filtering Strategy and Max Suggestions + +```php +// Override the strategy programmatically. +add_filter( 'wpai_content_classification_strategy', function( $strategy ) { + return 'allow_new'; +} ); + +// Override the max suggestions count. +add_filter( 'wpai_content_classification_max_suggestions', function( $max ) { + return 7; +} ); +``` + +### Filtering Preferred Models + +You can filter which AI models are used for suggestion generation: + +```php +add_filter( 'wpai_experiments_preferred_models_for_text_generation', function( $models ) { + return array( + array( 'openai', 'gpt-4' ), + array( 'anthropic', 'claude-haiku-4-5' ), + ); +} ); +``` + +### Customizing Content Normalization + +The `normalize_content()` helper function processes content before sending it to the AI: + +```php +add_filter( 'wpai_experiments_pre_normalize_content', function( $content ) { + // Custom preprocessing. + return $content; +} ); + +add_filter( 'wpai_experiments_normalize_content', function( $content ) { + // Custom post-processing. + return $content; +} ); +``` + +## Testing + +### Manual Testing + +1. **Enable the experiment:** + - Go to `Settings > AI Experiments` + - Toggle **Content Classification** to enabled + - Configure the taxonomy strategy and max suggestions + - Ensure you have valid AI credentials configured + +2. **Test in the editor:** + - Create or edit a post with at least 150 words of content + - Scroll to the Tags or Categories panel in the sidebar + - Click the "Suggest Tags" or "Suggest Categories" button + - Verify suggestions appear as clickable pills + - Click a suggestion to add it to the post + - Click the X on a suggestion to dismiss it + - Click "Regenerate" for new suggestions + - Click "Dismiss all" to clear all suggestions + +3. **Test panel toggle behavior:** + - Close and reopen the Tags/Categories panel + - Verify the button appears correctly after reopening + +4. **Test with different strategies:** + - Set strategy to "Only suggest existing terms" and verify only existing terms are suggested + - Set strategy to "Suggest new terms" and verify new terms appear with "new" badges + +5. **Test REST API:** + - Use curl or Postman to test the REST endpoint + - Verify authentication works + - Test with different input combinations + - Verify error handling for invalid inputs + +### Automated Testing + +Tests are located in: + +- `tests/Integration/Includes/Abilities/Content_ClassificationTest.php` +- `tests/Integration/Includes/Experiments/Content_Classification/Content_ClassificationTest.php` + +Run tests with: + +```bash +npm run test:php +``` + +## Notes & Considerations + +### Requirements + +- The experiment requires valid AI credentials to be configured +- The experiment only works for post types that support the editor (`post_type_supports( $post_type, 'editor' )`) +- The experiment does not load for attachment post types +- Users must have `edit_posts` capability (or `edit_post` for specific posts when using post ID) +- Post content must contain approximately 150 words before suggestions can be generated + +### AI Model Selection + +- The ability uses `get_preferred_models_for_text_generation()` to determine which AI models to use +- Models are tried in order until one succeeds +- Temperature is set to 0.5 for consistent, relevant results + +### Content Processing + +- Content is normalized before being sent to the AI (HTML stripped, shortcodes removed, etc.) +- The `normalize_content()` function handles this processing +- Additional context from post metadata (title, categories, tags) is included when using post ID +- All existing terms for the taxonomy are included in the prompt to encourage consistency + +### Limitations + +- Suggestions are generated in real-time and not cached +- The ability processes one taxonomy per request +- Generated suggestions should be reviewed before publishing +- The experiment requires JavaScript to be enabled in the admin +- The `is_new` flag is determined server-side by comparing against existing terms, not from the AI response + +## Related Files + +- **Experiment:** `includes/Experiments/Content_Classification/Content_Classification.php` +- **Ability:** `includes/Abilities/Content_Classification/Content_Classification.php` +- **React Entry:** `src/experiments/content-classification/index.tsx` +- **React Components:** `src/experiments/content-classification/components/` +- **Styles:** `src/experiments/content-classification/index.scss` +- **Types:** `src/experiments/content-classification/types.ts` +- **Tests:** `tests/Integration/Includes/Abilities/Content_ClassificationTest.php` +- **Tests:** `tests/Integration/Includes/Experiments/Content_Classification/Content_ClassificationTest.php` diff --git a/includes/Abilities/Content_Classification/Content_Classification.php b/includes/Abilities/Content_Classification/Content_Classification.php new file mode 100644 index 000000000..b47975315 --- /dev/null +++ b/includes/Abilities/Content_Classification/Content_Classification.php @@ -0,0 +1,590 @@ + The input schema of the ability. + */ + protected function input_schema(): array { + return array( + 'type' => 'object', + 'properties' => array( + 'content' => array( + 'type' => 'string', + 'description' => esc_html__( 'Content to generate taxonomy suggestions for.', 'ai' ), + ), + 'post_id' => array( + 'type' => 'integer', + 'description' => esc_html__( 'Content from this post will be used to generate taxonomy suggestions. This overrides the content parameter if both are provided.', 'ai' ), + ), + 'taxonomy' => array( + 'type' => 'string', + 'default' => 'post_tag', + 'description' => esc_html__( 'The taxonomy to generate suggestions for (e.g., post_tag, category).', 'ai' ), + ), + 'strategy' => array( + 'type' => 'string', + 'default' => Content_Classification_Experiment::STRATEGY_EXISTING_ONLY, + 'description' => esc_html__( 'The suggestion strategy: existing_only or allow_new.', 'ai' ), + ), + 'max_suggestions' => array( + 'type' => 'integer', + 'minimum' => 1, + 'maximum' => 10, + 'default' => Content_Classification_Experiment::DEFAULT_MAX_SUGGESTIONS, + 'description' => esc_html__( 'Maximum number of suggestions to generate.', 'ai' ), + ), + ), + ); + } + + /** + * Returns the output schema of the ability. + * + * @since x.x.x + * + * @return array The output schema of the ability. + */ + protected function output_schema(): array { + return array( + 'type' => 'object', + 'properties' => array( + 'suggestions' => array( + 'type' => 'array', + 'description' => esc_html__( 'Generated taxonomy term suggestions.', 'ai' ), + 'items' => array( + 'type' => 'object', + 'properties' => array( + 'term' => array( + 'type' => 'string', + 'description' => esc_html__( 'The suggested term name.', 'ai' ), + ), + 'confidence' => array( + 'type' => 'number', + 'description' => esc_html__( 'Confidence score between 0 and 1.', 'ai' ), + ), + 'is_new' => array( + 'type' => 'boolean', + 'description' => esc_html__( 'Whether this is a new term or an existing one.', 'ai' ), + ), + 'parent' => array( + 'type' => 'string', + 'description' => esc_html__( 'Parent term name for hierarchical taxonomies.', 'ai' ), + ), + ), + ), + ), + ), + ); + } + + /** + * Executes the ability with the given input arguments. + * + * @since x.x.x + * + * @param mixed $input The input arguments to the ability. + * @return array{suggestions: array}|\WP_Error The result of the ability execution, or a WP_Error on failure. + */ + protected function execute_callback( $input ) { + // Default arguments. + $args = wp_parse_args( + $input, + array( + 'content' => null, + 'post_id' => null, + 'taxonomy' => 'post_tag', + 'strategy' => Content_Classification_Experiment::STRATEGY_EXISTING_ONLY, + 'max_suggestions' => (int) Content_Classification_Experiment::DEFAULT_MAX_SUGGESTIONS, + ), + ); + + // Validate taxonomy. + if ( ! taxonomy_exists( $args['taxonomy'] ) ) { + return new WP_Error( + 'invalid_taxonomy', + /* translators: %s: Taxonomy name. */ + sprintf( esc_html__( 'Taxonomy "%s" does not exist.', 'ai' ), sanitize_key( $args['taxonomy'] ) ) + ); + } + + $assigned_terms = array(); + + // If a post ID is provided, ensure the post exists before using its content. + if ( $args['post_id'] ) { + $post = get_post( (int) $args['post_id'] ); + + if ( ! $post instanceof WP_Post ) { + return new WP_Error( + 'post_not_found', + /* translators: %d: Post ID. */ + sprintf( esc_html__( 'Post with ID %d not found.', 'ai' ), absint( $args['post_id'] ) ) + ); + } + + // Get the post context. + $context = get_post_context( (int) $args['post_id'] ); + + // Default to the passed in content if it exists. + if ( $args['content'] ) { + $context['content'] = normalize_content( $args['content'] ); + } + + // Get terms already assigned to this post for the taxonomy. + $assigned = wp_get_object_terms( (int) $args['post_id'], $args['taxonomy'], array( 'fields' => 'names' ) ); + if ( ! is_wp_error( $assigned ) ) { + $assigned_terms = (array) $assigned; + } + } else { + $context = array( + 'content' => normalize_content( $args['content'] ?? '' ), + ); + } + + // If we have no content, return an error. + if ( empty( $context['content'] ) ) { + return new WP_Error( + 'content_not_provided', + esc_html__( 'Content is required to generate taxonomy suggestions.', 'ai' ) + ); + } + + // Generate the suggestions. + $result = $this->generate_suggestions( + $context, + $args['taxonomy'], + $args['strategy'], + (int) $args['max_suggestions'], + $assigned_terms + ); + + // If we have an error, return it. + if ( is_wp_error( $result ) ) { + return $result; + } + + // If we have no results, return an error. + if ( empty( $result ) ) { + return new WP_Error( + 'no_results', + esc_html__( 'No taxonomy suggestions were generated.', 'ai' ) + ); + } + + return array( + 'suggestions' => $result, + ); + } + + /** + * Returns the permission callback of the ability. + * + * @since x.x.x + * + * @param mixed $args The input arguments to the ability. + * @return bool|\WP_Error True if the user has permission, WP_Error otherwise. + */ + protected function permission_callback( $args ) { + $post_id = isset( $args['post_id'] ) ? absint( $args['post_id'] ) : null; + + if ( $post_id ) { + $post = get_post( $post_id ); + + // Ensure the post exists. + if ( ! $post instanceof WP_Post ) { + return new WP_Error( + 'post_not_found', + /* translators: %d: Post ID. */ + sprintf( esc_html__( 'Post with ID %d not found.', 'ai' ), $post_id ) + ); + } + + // Ensure the user has permission to edit this particular post. + if ( ! current_user_can( 'edit_post', $post_id ) ) { + return new WP_Error( + 'insufficient_capabilities', + esc_html__( 'You do not have permission to generate taxonomy suggestions for this post.', 'ai' ) + ); + } + + $post_type_obj = get_post_type_object( $post->post_type ); + if ( ! $post_type_obj instanceof WP_Post_Type || empty( $post_type_obj->show_in_rest ) ) { + return false; + } + } elseif ( ! current_user_can( 'edit_posts' ) ) { + // Ensure the user has permission to edit posts in general. + return new WP_Error( + 'insufficient_capabilities', + esc_html__( 'You do not have permission to generate taxonomy suggestions.', 'ai' ) + ); + } + + return true; + } + + /** + * Returns the meta of the ability. + * + * @since x.x.x + * + * @return array The meta of the ability. + */ + protected function meta(): array { + return array( + 'show_in_rest' => true, + ); + } + + /** + * Generates taxonomy term suggestions from the given content. + * + * The LLM generates suggestions based purely on content analysis + * and the currently assigned terms. Post-processing then matches + * suggestions against existing terms and applies the strategy. + * + * @since x.x.x + * + * @param string|array $context The context to generate suggestions from. + * @param string $taxonomy The taxonomy to suggest terms for. + * @param string $strategy The suggestion strategy. + * @param int $max_suggestions The maximum number of suggestions. + * @param array $assigned_terms Terms already assigned to the post. + * @return array|\WP_Error The generated suggestions, or a WP_Error if there was an error. + */ + protected function generate_suggestions( $context, string $taxonomy, string $strategy, int $max_suggestions, array $assigned_terms = array() ) { + // Convert the context to a string if it's an array. + if ( is_array( $context ) ) { + $context = implode( + "\n", + array_map( + static function ( $key, $value ) { + return sprintf( + '%s: %s', + ucwords( str_replace( '_', ' ', $key ) ), + $value + ); + }, + array_keys( $context ), + $context + ) + ); + } + + // When using existing_only strategy, send the top terms to the LLM + // so it can select from actual terms rather than guessing. + $available_terms = array(); + if ( Content_Classification_Experiment::STRATEGY_EXISTING_ONLY === $strategy ) { + $available_terms = $this->get_top_terms( $taxonomy ); + } + + // Piece together the various prompt parts. + $prompt_parts = array(); + + $prompt_parts[] = '' . $taxonomy . ''; + $prompt_parts[] = '' . $context . ''; + + // If we have currently assigned terms, add them to the prompt to avoid redundant suggestions. + if ( ! empty( $assigned_terms ) ) { + $prompt_parts[] = '' . implode( ', ', $assigned_terms ) . ''; + } + + // If we're using the existing_only strategy, add the top 100 terms to the prompt. + if ( ! empty( $available_terms ) ) { + $prompt_parts[] = '' . implode( ', ', $available_terms ) . ''; + } + + $prompt = implode( "\n", $prompt_parts ); + + /** + * Filters the prompt string before it is sent to the AI model for taxonomy suggestion generation. + * + * Allows developers to modify, augment, or replace the prompt that the AI analyzes + * when generating taxonomy term suggestions. + * + * @since x.x.x + * + * @param string $prompt The prompt string to be sent to the AI model. + * @param string|array $context The context to generate suggestions from. + * @param string $taxonomy The taxonomy slug being suggested for (e.g., 'post_tag', 'category'). + * @param array $assigned_terms Terms already assigned to the post. + * @param array $available_terms Available terms to suggest from. + */ + $prompt = (string) apply_filters( 'wpai_content_classification_prompt', $prompt, $context, $taxonomy, $assigned_terms, $available_terms ); + + $builder = $this->get_prompt_builder( $prompt ); + + if ( is_wp_error( $builder ) ) { + return $builder; + } + + // Generate the suggestions using the AI client with structured output. + $result = $builder->as_json_response( $this->suggestions_schema() )->generate_text(); + + if ( is_wp_error( $result ) ) { + return $result; + } + + // Parse, match against existing terms, filter, and limit. + $suggestions = $this->parse_suggestions( $result, $strategy, $assigned_terms, $taxonomy, $max_suggestions ); + + if ( is_wp_error( $suggestions ) ) { + return $suggestions; + } + + /** + * Filters the parsed taxonomy suggestions before they are returned to the client. + * + * Allows developers to modify, reorder, add, or remove suggestions after the AI + * has generated them and they have been parsed into structured data. + * + * Each suggestion is an associative array with the keys: + * - 'term' (string) The suggested term name. + * - 'confidence' (float) Confidence score between 0 and 1. + * - 'is_new' (bool) Whether the term is new or already exists on the site. + * - 'parent' (string) Optional. Parent term name for hierarchical taxonomies. + * + * @since x.x.x + * + * @param array $suggestions The parsed suggestions. + * @param string $taxonomy The taxonomy slug (e.g., 'post_tag', 'category'). + * @param string $strategy The suggestion strategy ('existing_only' or 'allow_new'). + */ + return (array) apply_filters( 'wpai_content_classification_suggestions', $suggestions, $taxonomy, $strategy ); + } + + /** + * Get the prompt builder for generating taxonomy term suggestions. + * + * @since x.x.x + * + * @param string $prompt The prompt to use for generating taxonomy term suggestions. + * @return \WP_AI_Client_Prompt_Builder|\WP_Error The prompt builder, or a WP_Error on failure. + */ + private function get_prompt_builder( string $prompt ) { + $builder = wp_ai_client_prompt( $prompt ) + ->using_system_instruction( $this->get_system_instruction() ) + ->using_temperature( 0.5 ) + ->using_model_preference( ...get_preferred_models_for_text_generation() ); + + // Return a more specific error if there isn't a model that supports text generation. + if ( ! $builder->is_supported_for_text_generation() ) { + return new WP_Error( + 'unsupported_model', + esc_html__( 'Term generation failed. Please ensure you have a connected provider that supports text generation.', 'ai' ) + ); + } + + return $builder; + } + + /** + * Returns the JSON schema for structured output from the AI model. + * + * @since x.x.x + * + * @return array The JSON schema for structured output. + */ + protected function suggestions_schema(): array { + return array( + 'type' => 'object', + 'properties' => array( + 'suggestions' => array( + 'type' => 'array', + 'items' => array( + 'type' => 'object', + 'properties' => array( + 'term' => array( 'type' => 'string' ), + 'confidence' => array( 'type' => 'number' ), + ), + 'required' => array( 'term', 'confidence' ), + 'additionalProperties' => false, + ), + ), + ), + 'required' => array( 'suggestions' ), + 'additionalProperties' => false, + ); + } + + /** + * Parses the AI response into structured suggestions. + * + * Matches LLM suggestions against existing terms (case-insensitive), + * filters out assigned terms, applies the strategy, sorts by confidence, + * and limits to the requested number of suggestions. + * + * @since x.x.x + * + * @param string $response The raw AI response. + * @param string $strategy The suggestion strategy ('existing_only' or 'allow_new'). + * @param array $assigned_terms Terms already assigned to the post. + * @param string $taxonomy The taxonomy to suggest terms for. + * @param int $max_suggestions The maximum number of suggestions to return. + * @return array|\WP_Error Parsed suggestions or error. + */ + private function parse_suggestions( string $response, string $strategy, array $assigned_terms, string $taxonomy, int $max_suggestions ) { + $decoded = json_decode( $response, true ); + + if ( ! is_array( $decoded ) || ! isset( $decoded['suggestions'] ) || ! is_array( $decoded['suggestions'] ) ) { + return new WP_Error( + 'invalid_response', + esc_html__( 'Could not parse AI response as valid suggestions.', 'ai' ) + ); + } + + // Only fetch existing terms when we need them for post-processing (existing_only strategy). + $existing_terms = Content_Classification_Experiment::STRATEGY_EXISTING_ONLY === $strategy + ? $this->get_existing_terms( $taxonomy ) + : array(); + + // Build a lowercase → original name lookup for existing terms. + // We don't use slugs here because the LLM may generate terms that don't match the taxonomy slug. + if ( ! empty( $existing_terms ) ) { + $existing_terms = array_combine( array_map( 'strtolower', $existing_terms ), $existing_terms ); + } + + // Build a lowercase set of assigned terms for filtering. + $assigned_terms = array_map( 'strtolower', $assigned_terms ); + $suggestions = array(); + foreach ( $decoded['suggestions'] as $item ) { + if ( ! is_array( $item ) || empty( $item['term'] ) ) { + continue; + } + + $term = sanitize_text_field( trim( $item['term'] ) ); + $term_lower = strtolower( $term ); + $is_new = ! isset( $existing_terms[ $term_lower ] ); + $confidence = isset( $item['confidence'] ) ? (float) $item['confidence'] : 0.5; + + // Skip terms already assigned to the post. + // The agent should avoid suggesting these, but just in case we'll check here as well. + if ( in_array( $term_lower, $assigned_terms, true ) ) { + continue; + } + + // For existing_only strategy, skip terms that don't exist. + if ( Content_Classification_Experiment::STRATEGY_EXISTING_ONLY === $strategy && $is_new ) { + continue; + } + + // Use the original capitalized name for existing terms. + if ( ! $is_new ) { + $term = $existing_terms[ $term_lower ]; + } + + $suggestion = array( + 'term' => $term, + 'confidence' => max( 0.0, min( 1.0, $confidence ) ), + 'is_new' => $is_new, + ); + + // Only preserve parent for hierarchical taxonomies, and strip it + // when the AI returns the taxonomy slug itself as the parent. + if ( + ! empty( $item['parent'] ) + && is_taxonomy_hierarchical( $taxonomy ) + && strtolower( trim( $item['parent'] ) ) !== strtolower( $taxonomy ) + ) { + $suggestion['parent'] = sanitize_text_field( trim( $item['parent'] ) ); + } + + $suggestions[] = $suggestion; + } + + // Sort by confidence descending. + usort( + $suggestions, + static function ( $a, $b ) { + return $b['confidence'] <=> $a['confidence']; + } + ); + + // Limit to max suggestions. + return array_slice( $suggestions, 0, $max_suggestions ); + } + + /** + * Gets existing terms for a taxonomy. + * + * @since x.x.x + * + * @param string $taxonomy The taxonomy to get terms for. + * @return array List of existing term names. + */ + private function get_existing_terms( string $taxonomy ): array { + $terms = get_terms( + array( + 'taxonomy' => $taxonomy, + 'hide_empty' => false, + 'fields' => 'names', + ) + ); + + if ( is_wp_error( $terms ) ) { + return array(); + } + + return (array) $terms; + } + + /** + * Gets the top terms for a taxonomy, ordered by usage count. + * + * Used to provide the LLM with a set of existing terms to select from + * when using the existing_only strategy, improving match quality. + * + * @since x.x.x + * + * @param string $taxonomy The taxonomy to get terms for. + * @param int $limit Maximum number of terms to return. + * @return array List of term names ordered by count descending. + */ + private function get_top_terms( string $taxonomy, int $limit = 100 ): array { + $terms = get_terms( + array( + 'taxonomy' => $taxonomy, + 'hide_empty' => false, + 'fields' => 'names', + 'orderby' => 'count', + 'order' => 'DESC', + 'number' => $limit, + ) + ); + + if ( is_wp_error( $terms ) ) { + return array(); + } + + return (array) $terms; + } +} diff --git a/includes/Abilities/Content_Classification/system-instruction.php b/includes/Abilities/Content_Classification/system-instruction.php new file mode 100644 index 000000000..1cbc4d98f --- /dev/null +++ b/includes/Abilities/Content_Classification/system-instruction.php @@ -0,0 +1,29 @@ + tags, with any already-applied terms in ) and suggest relevant terms for the taxonomy (wrapped in tag). + +Rules: +- The taxonomy to suggest terms for is wrapped in the tag. +- Suggest as many relevant terms as you can identify from the content. +- The "term" field must contain ONLY the human-readable tag or category name (1-3 words), in Title Case (e.g., "Machine Learning", not "machine learning"). +- Confidence should reflect relevance: 1.0 = perfect match, 0.5 = somewhat relevant. Only suggest terms with confidence >= 0.5. +- Do not suggest duplicate or near-duplicate terms. +- Do not suggest terms listed in — they are already applied to this post. +- When the tag is provided, strongly prefer selecting from those terms. +- Prioritize specificity and relevance over breadth. +- Ensure the terms you suggest match the language of the content you are given. For example, if the content is in English, suggest English terms. If the content is in Spanish, suggest Spanish terms. +INSTRUCTION; diff --git a/includes/Experiments/Content_Classification/Content_Classification.php b/includes/Experiments/Content_Classification/Content_Classification.php new file mode 100644 index 000000000..3f5925be2 --- /dev/null +++ b/includes/Experiments/Content_Classification/Content_Classification.php @@ -0,0 +1,301 @@ + __( 'Content Classification', 'ai' ), + 'description' => __( 'AI-powered suggestions for post tags and categories based on content analysis.', 'ai' ), + 'category' => Experiment_Category::EDITOR, + ); + } + + /** + * {@inheritDoc} + */ + public function register(): void { + add_action( 'wp_abilities_api_init', array( $this, 'register_abilities' ) ); + add_action( 'admin_enqueue_scripts', array( $this, 'enqueue_assets' ) ); + } + + /** + * Registers any needed abilities. + * + * @since x.x.x + */ + public function register_abilities(): void { + wp_register_ability( + 'ai/' . $this->get_id(), + array( + 'label' => $this->get_label(), + 'description' => $this->get_description(), + 'ability_class' => Content_Classification_Ability::class, + ), + ); + } + + /** + * Enqueues and localizes the admin script. + * + * @since x.x.x + * + * @param string $hook_suffix The current admin page hook suffix. + */ + public function enqueue_assets( string $hook_suffix ): void { + // Load asset in new post and edit post screens only. + if ( 'post.php' !== $hook_suffix && 'post-new.php' !== $hook_suffix ) { + return; + } + + $screen = get_current_screen(); + + // Load the assets only if the post type supports categories or tags and is not an attachment. + // Also check if the user can manage categories. + if ( + ! $screen || + ! current_user_can( 'manage_categories' ) || + in_array( $screen->post_type, array( 'attachment' ), true ) || + ( + ! is_object_in_taxonomy( $screen->post_type, 'category' ) && + ! is_object_in_taxonomy( $screen->post_type, 'post_tag' ) + ) + ) { + return; + } + + Asset_Loader::enqueue_script( 'content_classification', 'experiments/content-classification' ); + Asset_Loader::enqueue_style( 'content_classification', 'experiments/content-classification' ); + Asset_Loader::localize_script( + 'content_classification', + 'ContentClassificationData', + array( + 'enabled' => $this->is_enabled(), + 'strategy' => $this->get_strategy(), + 'maxSuggestions' => $this->get_max_suggestions(), + ) + ); + } + + /** + * Registers experiment-specific settings. + * + * @since x.x.x + */ + public function register_settings(): void { + register_setting( + Settings_Registration::OPTION_GROUP, + $this->get_field_option_name( 'strategy' ), + array( + 'type' => 'string', + 'default' => self::STRATEGY_EXISTING_ONLY, + 'sanitize_callback' => array( $this, 'sanitize_strategy' ), + ) + ); + + register_setting( + Settings_Registration::OPTION_GROUP, + $this->get_field_option_name( 'max_suggestions' ), + array( + 'type' => 'integer', + 'default' => self::DEFAULT_MAX_SUGGESTIONS, + 'sanitize_callback' => array( $this, 'sanitize_max_suggestions' ), + ) + ); + } + + /** + * Renders experiment-specific settings fields. + * + * @since x.x.x + */ + public function render_settings_fields(): void { + $strategy_option = $this->get_field_option_name( 'strategy' ); + $max_suggestions_option = $this->get_field_option_name( 'max_suggestions' ); + $current_strategy = get_option( $this->get_field_option_name( 'strategy' ), self::STRATEGY_EXISTING_ONLY ); + $current_max = get_option( $this->get_field_option_name( 'max_suggestions' ), self::DEFAULT_MAX_SUGGESTIONS ); + ?> +
+ + + + + + + + + + + +
+ get_field_option_name( 'strategy' ), self::STRATEGY_EXISTING_ONLY ); + + /** + * Filters the strategy to use for content classification. + * + * @since x.x.x + * + * @param string $strategy The strategy to use. + * @return string The filtered strategy. + */ + $strategy = apply_filters( 'wpai_content_classification_strategy', $strategy ); + + // Return the sanitized strategy value. + return $this->sanitize_strategy( $strategy ); + } + + /** + * Gets the maximum number of suggestions to generate for content classification. + * + * @since x.x.x + * + * @return int The maximum number of suggestions to generate. + */ + public function get_max_suggestions(): int { + $max_suggestions = (int) get_option( $this->get_field_option_name( 'max_suggestions' ), self::DEFAULT_MAX_SUGGESTIONS ); + + /** + * Filters the maximum number of suggestions to generate for content classification. + * + * @since x.x.x + * + * @param int $max_suggestions The maximum number of suggestions to generate. + * @return int The filtered max suggestions. + */ + return $this->sanitize_max_suggestions( + apply_filters( 'wpai_content_classification_max_suggestions', $max_suggestions ) + ); + } +} diff --git a/includes/Experiments/Experiments.php b/includes/Experiments/Experiments.php index 27c8058f1..3a9e8dcef 100644 --- a/includes/Experiments/Experiments.php +++ b/includes/Experiments/Experiments.php @@ -28,6 +28,7 @@ final class Experiments { */ private const EXPERIMENT_CLASSES = array( // phpcs:ignore SlevomatCodingStandard.Classes.DisallowMultiConstantDefinition -- This is used as an array const. \WordPress\AI\Experiments\Abilities_Explorer\Abilities_Explorer::class, + \WordPress\AI\Experiments\Content_Classification\Content_Classification::class, \WordPress\AI\Experiments\Excerpt_Generation\Excerpt_Generation::class, \WordPress\AI\Experiments\Alt_Text_Generation\Alt_Text_Generation::class, \WordPress\AI\Experiments\Image_Generation\Image_Generation::class, diff --git a/package-lock.json b/package-lock.json index 107030e01..696ae694e 100644 --- a/package-lock.json +++ b/package-lock.json @@ -25,6 +25,7 @@ "@wordpress/notices": "^5.35.0", "@wordpress/plugins": "^7.34.0", "@wordpress/url": "^4.38.0", + "@wordpress/wordcount": "^4.41.0", "react": "^18.3.1" }, "devDependencies": { @@ -11555,9 +11556,9 @@ } }, "node_modules/@wordpress/wordcount": { - "version": "4.37.0", - "resolved": "https://registry.npmjs.org/@wordpress/wordcount/-/wordcount-4.37.0.tgz", - "integrity": "sha512-Uyl9aR4Tpr/AVoTcqQjkvGE8FE1jXOOooUwcEWCxxe4OLyyKDBs/uJJ2afXzgxc/gNI/QGHArdOA0UArywcnng==", + "version": "4.41.0", + "resolved": "https://registry.npmjs.org/@wordpress/wordcount/-/wordcount-4.41.0.tgz", + "integrity": "sha512-/22Fon1owjl4VnSD6ewReUZ+wEcNUsxEcZoKF/ew0CqOaprMwog3kPWq3Jub6Mt0aBWmjtrJ7Nhuo+wVkQ3Kog==", "license": "GPL-2.0-or-later", "engines": { "node": ">=18.12.0", diff --git a/package.json b/package.json index 208f5db57..cab879cba 100644 --- a/package.json +++ b/package.json @@ -64,6 +64,7 @@ "@wordpress/notices": "^5.35.0", "@wordpress/plugins": "^7.34.0", "@wordpress/url": "^4.38.0", + "@wordpress/wordcount": "^4.41.0", "react": "^18.3.1" }, "overrides": { diff --git a/src/admin/settings/index.scss b/src/admin/settings/index.scss index 654181234..d83db8a6f 100644 --- a/src/admin/settings/index.scss +++ b/src/admin/settings/index.scss @@ -128,4 +128,22 @@ margin-left: 1.5rem; } } + + /* Element: settings table (two-column label/field layout) */ + &__settings-table { + border-spacing: 0 0.5rem; + + td:first-child { + padding-right: 1rem; + white-space: nowrap; + } + + label { + font-weight: 600; + } + + input[type="number"] { + padding-right: 0; + } + } } diff --git a/src/experiments/content-classification/components/SuggestionPanel.tsx b/src/experiments/content-classification/components/SuggestionPanel.tsx new file mode 100644 index 000000000..e436fd84c --- /dev/null +++ b/src/experiments/content-classification/components/SuggestionPanel.tsx @@ -0,0 +1,166 @@ +/** + * Suggestion panel component for displaying AI-generated taxonomy suggestions. + */ + +/** + * WordPress dependencies + */ +import { Button, Flex, FlexItem } from '@wordpress/components'; +import { select } from '@wordpress/data'; +import { store as coreStore } from '@wordpress/core-data'; +import { __, sprintf } from '@wordpress/i18n'; +import { close as closeIcon, update } from '@wordpress/icons'; + +/** + * Internal dependencies + */ +import { useContentClassification } from './useContentClassification'; +import type { TagSuggestion } from '../types'; + +interface SuggestionPanelProps { + taxonomy: string; +} + +/** + * SuggestionPanel component. + * + * Displays a button to generate suggestions and renders suggestion pills + * that can be accepted or dismissed. + * + * @param props Component props. + * @param props.taxonomy The taxonomy to generate suggestions for. + * @return The suggestion panel component. + */ +export default function SuggestionPanel( { + taxonomy, +}: SuggestionPanelProps ): JSX.Element | null { + const { + isGenerating, + suggestions, + hasEnoughContent, + handleGenerate, + handleAccept, + handleDismiss, + handleDismissAll, + } = useContentClassification( taxonomy ); + + const taxonomyObject: any = select( coreStore ).getTaxonomy( taxonomy ); + const taxonomyLabel: string = taxonomyObject?.name ?? taxonomy; + + const hasSuggestions = suggestions.length > 0; + + return ( +
+ { ! hasSuggestions && ( + + ) } + + { ! hasEnoughContent && ! hasSuggestions && ( +

+ { __( + 'Add more content to enable AI suggestions (approximately 150 words).', + 'ai' + ) } +

+ ) } + + { hasSuggestions && ( +
+

+ { sprintf( + /* translators: %s: Taxonomy label (e.g., "Tags" or "Categories"). */ + __( 'Suggested %s', 'ai' ), + taxonomyLabel + ) } +

+
+ { suggestions.map( ( suggestion: TagSuggestion ) => ( + + +
+ + + + + + + + +
+ ) } +
+ ); +} diff --git a/src/experiments/content-classification/components/useContentClassification.ts b/src/experiments/content-classification/components/useContentClassification.ts new file mode 100644 index 000000000..1c7513ce3 --- /dev/null +++ b/src/experiments/content-classification/components/useContentClassification.ts @@ -0,0 +1,305 @@ +/** + * Shared hook for content classification logic. + */ + +/** + * WordPress dependencies + */ +import { dispatch, select } from '@wordpress/data'; +import { store as coreStore } from '@wordpress/core-data'; +import { store as editorStore } from '@wordpress/editor'; +import { useState, useCallback } from '@wordpress/element'; +import { store as noticesStore } from '@wordpress/notices'; +import { count as wordCount } from '@wordpress/wordcount'; +import { addQueryArgs } from '@wordpress/url'; +import apiFetch from '@wordpress/api-fetch'; + +/** + * Internal dependencies + */ +import { runAbility } from '../../../utils/run-ability'; +import type { + ContentClassificationAbilityInput, + ContentClassificationResponse, + TagSuggestion, + ContentClassificationData, +} from '../types'; + +const MINIMUM_WORD_COUNT = 150; +const NOTICE_ID = 'ai_content_classification_error'; + +const getSettings = (): ContentClassificationData => + ( window as any ).aiContentClassificationData ?? { + enabled: false, + strategy: 'existing_only', + maxSuggestions: 5, + }; + +/** + * Generates taxonomy suggestions for the given post. + * + * @param postId The post ID. + * @param content The post content. + * @param taxonomy The taxonomy to suggest terms for. + * @param strategy The suggestion strategy. + * @param maxSuggestions The maximum number of suggestions. + * @return A promise that resolves to the generated suggestions. + */ +async function generateSuggestions( + postId: number, + content: string, + taxonomy: string, + strategy: string, + maxSuggestions: number +): Promise< TagSuggestion[] > { + const params: ContentClassificationAbilityInput = { + content, + post_id: postId, + taxonomy, + strategy, + max_suggestions: maxSuggestions, + }; + + const response = await runAbility< ContentClassificationResponse >( + 'ai/content-classification', + params + ); + + if ( response?.suggestions && Array.isArray( response.suggestions ) ) { + return response.suggestions; + } + + return []; +} + +/** + * Gets the lowercase names of terms currently assigned to the post for a taxonomy. + * + * @param taxonomy The taxonomy slug. + * @return A promise that resolves to an array of lowercase term names. + */ +async function getAssignedTermNames( taxonomy: string ): Promise< string[] > { + const taxonomyObject: any = select( coreStore ).getTaxonomy( taxonomy ); + const restBase = taxonomyObject?.rest_base ?? taxonomy; + const { getEditedPostAttribute } = select( editorStore ); + const termIds: number[] = getEditedPostAttribute( restBase ) ?? []; + + if ( ! termIds.length ) { + return []; + } + + try { + const terms: any[] = await apiFetch( { + path: addQueryArgs( `/wp/v2/${ restBase }`, { + include: termIds.join( ',' ), + per_page: termIds.length, + } ), + } ); + + return terms.map( ( t: any ) => t.name.toLowerCase() ); + } catch { + return []; + } +} + +/** + * Hook for content classification functionality. + * + * @param taxonomy The taxonomy to generate suggestions for. + * @return Object with generation state, suggestions, and handlers. + */ +export function useContentClassification( taxonomy: string ): { + isGenerating: boolean; + suggestions: TagSuggestion[]; + hasEnoughContent: boolean; + handleGenerate: () => Promise< void >; + handleAccept: ( suggestion: TagSuggestion ) => void; + handleDismiss: ( suggestion: TagSuggestion ) => void; + handleDismissAll: () => void; +} { + const postId = select( editorStore ).getCurrentPostId() as number; + const content = select( editorStore ).getEditedPostContent(); + const [ isGenerating, setIsGenerating ] = useState< boolean >( false ); + const [ suggestions, setSuggestions ] = useState< TagSuggestion[] >( [] ); + const { removeNotice, createErrorNotice } = dispatch( noticesStore ) as any; + + // Check if content has enough words. + const hasEnoughContent = + wordCount( content || '', 'words' ) >= MINIMUM_WORD_COUNT; + + const handleGenerate = useCallback( async () => { + const settings = getSettings(); + setIsGenerating( true ); + setSuggestions( [] ); + + // Remove any existing error notices. + removeNotice( NOTICE_ID ); + + try { + // Generate suggestions. + const result = await generateSuggestions( + postId, + content, + taxonomy, + settings.strategy, + settings.maxSuggestions + ); + + // Filter out terms already assigned to the post. + const assignedNames = await getAssignedTermNames( taxonomy ); + const filtered = result.filter( + ( s ) => ! assignedNames.includes( s.term.toLowerCase() ) + ); + + // Update the suggestions state. + setSuggestions( filtered ); + } catch ( error: any ) { + // Create an error notice. + createErrorNotice( error?.message || error, { + id: NOTICE_ID, + isDismissible: true, + } ); + } finally { + setIsGenerating( false ); + } + }, [ postId, content, taxonomy, removeNotice, createErrorNotice ] ); + + // Remove a suggestion from the list. + const removeSuggestionFromList = ( suggestion: TagSuggestion ) => { + setSuggestions( ( prev ) => + prev.filter( ( s ) => s.term !== suggestion.term ) + ); + }; + + // Handle accepting a suggestion. + const handleAccept = useCallback( + ( suggestion: TagSuggestion ) => { + removeSuggestionFromList( suggestion ); + addTermToPost( taxonomy, suggestion ); + }, + [ taxonomy ] + ); + + // Handle dismissing a suggestion. + const handleDismiss = useCallback( ( suggestion: TagSuggestion ) => { + removeSuggestionFromList( suggestion ); + }, [] ); + + // Handle dismissing all suggestions. + const handleDismissAll = useCallback( () => { + setSuggestions( [] ); + }, [] ); + + return { + isGenerating, + suggestions, + hasEnoughContent, + handleGenerate, + handleAccept, + handleDismiss, + handleDismissAll, + }; +} + +/** + * Adds a term to the current post. + * + * @param taxonomy The taxonomy slug. + * @param suggestion The suggestion to add. + */ +async function addTermToPost( + taxonomy: string, + suggestion: TagSuggestion +): Promise< void > { + const { editPost }: any = dispatch( editorStore ); + const { getEditedPostAttribute } = select( editorStore ); + + const taxonomyObject: any = select( coreStore ).getTaxonomy( taxonomy ); + const restBase = taxonomyObject?.rest_base ?? taxonomy; + + // Resolve parent term ID for hierarchical taxonomies only. + let parentId: number | undefined; + if ( suggestion.parent && taxonomyObject?.hierarchical ) { + const resolvedParent = await findOrCreateTerm( + taxonomy, + restBase, + suggestion.parent + ); + if ( resolvedParent ) { + parentId = resolvedParent; + } + } + + const currentTerms: number[] = getEditedPostAttribute( restBase ) ?? []; + const termId = await findOrCreateTerm( + taxonomy, + restBase, + suggestion.term, + parentId + ); + + if ( termId && ! currentTerms.includes( termId ) ) { + editPost( { + [ restBase ]: [ ...currentTerms, termId ], + } ); + } +} + +/** + * Finds an existing term by name or creates a new one. + * + * @param taxonomy The taxonomy slug (e.g., 'category'). + * @param restBase The REST base for the taxonomy (e.g., 'categories'). + * @param termName The term name. + * @param parentId Optional parent term ID for hierarchical taxonomies. + * @return The term ID, or null if not found and could not be created. + */ +async function findOrCreateTerm( + taxonomy: string, + restBase: string, + termName: string, + parentId?: number +): Promise< number | null > { + try { + // Search for existing term via REST. + const searchResults: any[] = await apiFetch( { + path: addQueryArgs( `/wp/v2/${ restBase }`, { + search: termName, + per_page: 100, + } ), + } ); + + // If we have a direct match, return its ID. + if ( Array.isArray( searchResults ) ) { + const match = searchResults.find( + ( t: any ) => t.name.toLowerCase() === termName.toLowerCase() + ); + if ( match ) { + return match.id; + } + } + + // Create new term + const data: Record< string, unknown > = { name: termName }; + if ( parentId ) { + data[ 'parent' ] = parentId; // eslint-disable-line dot-notation + } + + const newTerm: any = await ( + dispatch( coreStore ) as any + ).saveEntityRecord( 'taxonomy', taxonomy, data ); + + return newTerm?.id ?? null; + } catch ( error: any ) { + const { createErrorNotice } = dispatch( noticesStore ) as any; + createErrorNotice( + error?.message || + `Could not add term "${ termName }". Please try again.`, + { + id: `${ NOTICE_ID }_term_${ termName }`, + isDismissible: true, + } + ); + return null; + } +} diff --git a/src/experiments/content-classification/index.scss b/src/experiments/content-classification/index.scss new file mode 100644 index 000000000..39f67c5d4 --- /dev/null +++ b/src/experiments/content-classification/index.scss @@ -0,0 +1,87 @@ +.ai-content-classification { + margin-top: 12px; + padding-top: 12px; + border-top: 1px solid var(--wp-admin-border-color, #c3c4c7); + + &__generate-button { + width: 100%; + + /* Override core button styles to center the text and icon. */ + &.has-icon.has-text { + justify-content: center; + } + } + + &__hint { + margin-top: 4px; + } + + &__loading { + display: flex; + justify-content: center; + padding: 8px 0; + } + + &__pills { + display: flex; + flex-wrap: wrap; + gap: 8px; + margin-bottom: 8px; + } + + &__pill { + display: inline-flex; + align-items: center; + border: 1px solid var(--wp-admin-theme-color, #007cba); + border-radius: 2px; + background: rgba(var(--wp-admin-theme-color--rgb, 0, 124, 186), 0.04); + overflow: hidden; + } + + &__pill-accept { + padding: 2px 4px 2px 10px !important; + min-height: 28px !important; + font-size: 12px !important; + border: none !important; + background: transparent !important; + cursor: pointer; + + &:hover { + color: var(--wp-admin-theme-color, #007cba); + } + } + + &__pill-parent { + opacity: 0.6; + } + + &__pill-badge { + display: inline-block; + margin-left: 4px; + padding: 1px 5px; + border-radius: 2px; + background: var(--wp-admin-theme-color, #007cba); + color: #fff; + font-size: 10px; + line-height: 1.4; + vertical-align: middle; + } + + &__pill-dismiss { + min-height: 28px !important; + min-width: 24px !important; + padding: 0 4px !important; + border: none !important; + background: transparent !important; + border-left: 1px solid var(--wp-admin-theme-color, #007cba) !important; + border-radius: 0 !important; + + &:hover { + color: #cc1818; + } + } + + &__actions { + font-size: 12px; + } +} diff --git a/src/experiments/content-classification/index.tsx b/src/experiments/content-classification/index.tsx new file mode 100644 index 000000000..dada593cd --- /dev/null +++ b/src/experiments/content-classification/index.tsx @@ -0,0 +1,51 @@ +/** + * Content classification experiment plugin registration. + */ + +/** + * WordPress dependencies + */ +import { addFilter } from '@wordpress/hooks'; + +/** + * Internal dependencies + */ +import SuggestionPanel from './components/SuggestionPanel'; + +/** + * Styles + */ +import './index.scss'; + +const SUPPORTED_TAXONOMIES = [ 'post_tag', 'category' ]; + +/** + * Wraps the taxonomy selector component with the AI suggestion panel. + * + * @param OriginalComponent The original taxonomy selector component. + * @return The wrapped component. + */ +function withContentClassification( + OriginalComponent: React.ComponentType< any > +) { + return function ContentClassificationWrapper( props: any ) { + const { slug } = props; + + if ( ! SUPPORTED_TAXONOMIES.includes( slug ) ) { + return ; + } + + return ( + <> + + + + ); + }; +} + +addFilter( + 'editor.PostTaxonomyType', + 'ai/content-classification', + withContentClassification +); diff --git a/src/experiments/content-classification/types.ts b/src/experiments/content-classification/types.ts new file mode 100644 index 000000000..8657b27e0 --- /dev/null +++ b/src/experiments/content-classification/types.ts @@ -0,0 +1,41 @@ +/** + * Type definitions for content classification experiment. + */ + +/** + * Input parameters for the ai/content-classification ability. + */ +export interface ContentClassificationAbilityInput { + content: string; + post_id: number; + taxonomy: string; + strategy: string; + max_suggestions: number; + [ key: string ]: string | number | undefined; +} + +/** + * A single taxonomy term suggestion from the AI. + */ +export interface TagSuggestion { + term: string; + confidence: number; + is_new: boolean; + parent?: string; +} + +/** + * Response from the ai/content-classification ability. + */ +export interface ContentClassificationResponse { + suggestions: TagSuggestion[]; +} + +/** + * Localized data from the PHP side. + */ +export interface ContentClassificationData { + enabled: boolean; + strategy: string; + maxSuggestions: number; +} diff --git a/src/experiments/review-notes/block-editor-augmentation.d.ts b/src/experiments/review-notes/block-editor-augmentation.d.ts index 091293db1..6d981259e 100644 --- a/src/experiments/review-notes/block-editor-augmentation.d.ts +++ b/src/experiments/review-notes/block-editor-augmentation.d.ts @@ -13,7 +13,9 @@ declare module '@wordpress/block-editor' { onClose: () => void; } - export const BlockSettingsMenuControls: (props: { - children: ( fillProps: BlockSettingsMenuControlsFillProps ) => ReactNode; - }) => ReactNode; + export const BlockSettingsMenuControls: ( props: { + children: ( + fillProps: BlockSettingsMenuControlsFillProps + ) => ReactNode; + } ) => ReactNode; } diff --git a/tests/Integration/Includes/Abilities/Content_ClassificationTest.php b/tests/Integration/Includes/Abilities/Content_ClassificationTest.php new file mode 100644 index 000000000..f16f49fe4 --- /dev/null +++ b/tests/Integration/Includes/Abilities/Content_ClassificationTest.php @@ -0,0 +1,967 @@ + 'Content Classification', + 'description' => 'AI-powered suggestions for post tags and categories.', + ); + } + + /** + * Registers the experiment. + * + * @since x.x.x + */ + public function register(): void { + // No-op for testing. + } +} + +/** + * Content_Classification Ability test case. + * + * @since x.x.x + */ +class Content_ClassificationTest extends WP_UnitTestCase { + + /** + * Content_Classification ability instance. + * + * @var \WordPress\AI\Abilities\Content_Classification\Content_Classification + */ + private $ability; + + /** + * Test experiment instance. + * + * @var \WordPress\AI\Tests\Integration\Includes\Abilities\Test_Content_Classification_Experiment + */ + private $experiment; + + /** + * Set up test case. + * + * @since x.x.x + */ + public function setUp(): void { + parent::setUp(); + + $this->experiment = new Test_Content_Classification_Experiment(); + $this->ability = new Content_Classification( + 'ai/content-classification', + array( + 'label' => $this->experiment->get_label(), + 'description' => $this->experiment->get_description(), + ) + ); + } + + /** + * Tear down test case. + * + * @since x.x.x + */ + public function tearDown(): void { + wp_set_current_user( 0 ); + remove_all_filters( 'wpai_content_classification_content' ); + remove_all_filters( 'wpai_content_classification_suggestions' ); + parent::tearDown(); + } + + /** + * Test that category() returns the correct category. + * + * @since x.x.x + */ + public function test_category_returns_correct_category() { + $reflection = new \ReflectionClass( $this->ability ); + $method = $reflection->getMethod( 'category' ); + $method->setAccessible( true ); + + $result = $method->invoke( $this->ability ); + + $this->assertEquals( 'ai-experiments', $result, 'Category should be ai-experiments' ); + } + + /** + * Test that input_schema() returns the expected schema structure. + * + * @since x.x.x + */ + public function test_input_schema_returns_expected_structure() { + $reflection = new \ReflectionClass( $this->ability ); + $method = $reflection->getMethod( 'input_schema' ); + $method->setAccessible( true ); + + $schema = $method->invoke( $this->ability ); + + $this->assertIsArray( $schema, 'Input schema should be an array' ); + $this->assertEquals( 'object', $schema['type'], 'Schema type should be object' ); + $this->assertArrayHasKey( 'properties', $schema, 'Schema should have properties' ); + $this->assertArrayHasKey( 'content', $schema['properties'], 'Schema should have content property' ); + $this->assertArrayHasKey( 'post_id', $schema['properties'], 'Schema should have post_id property' ); + $this->assertArrayHasKey( 'taxonomy', $schema['properties'], 'Schema should have taxonomy property' ); + $this->assertArrayHasKey( 'strategy', $schema['properties'], 'Schema should have strategy property' ); + $this->assertArrayHasKey( 'max_suggestions', $schema['properties'], 'Schema should have max_suggestions property' ); + + // Verify taxonomy property. + $this->assertEquals( 'string', $schema['properties']['taxonomy']['type'], 'Taxonomy should be string type' ); + $this->assertEquals( 'post_tag', $schema['properties']['taxonomy']['default'], 'Taxonomy default should be post_tag' ); + + // Verify strategy property. + $this->assertEquals( 'string', $schema['properties']['strategy']['type'], 'Strategy should be string type' ); + $this->assertEquals( 'existing_only', $schema['properties']['strategy']['default'], 'Strategy default should be existing_only' ); + + // Verify max_suggestions property. + $this->assertEquals( 'integer', $schema['properties']['max_suggestions']['type'], 'max_suggestions should be integer type' ); + $this->assertEquals( 1, $schema['properties']['max_suggestions']['minimum'], 'max_suggestions minimum should be 1' ); + $this->assertEquals( 10, $schema['properties']['max_suggestions']['maximum'], 'max_suggestions maximum should be 10' ); + } + + /** + * Test that output_schema() returns the expected schema structure. + * + * @since x.x.x + */ + public function test_output_schema_returns_expected_structure() { + $reflection = new \ReflectionClass( $this->ability ); + $method = $reflection->getMethod( 'output_schema' ); + $method->setAccessible( true ); + + $schema = $method->invoke( $this->ability ); + + $this->assertIsArray( $schema, 'Output schema should be an array' ); + $this->assertEquals( 'object', $schema['type'], 'Schema type should be object' ); + $this->assertArrayHasKey( 'properties', $schema, 'Schema should have properties' ); + $this->assertArrayHasKey( 'suggestions', $schema['properties'], 'Schema should have suggestions property' ); + $this->assertEquals( 'array', $schema['properties']['suggestions']['type'], 'Suggestions should be array type' ); + $this->assertArrayHasKey( 'items', $schema['properties']['suggestions'], 'Suggestions should have items' ); + + // Verify suggestion item properties. + $item_props = $schema['properties']['suggestions']['items']['properties']; + $this->assertArrayHasKey( 'term', $item_props, 'Item should have term property' ); + $this->assertArrayHasKey( 'confidence', $item_props, 'Item should have confidence property' ); + $this->assertArrayHasKey( 'is_new', $item_props, 'Item should have is_new property' ); + $this->assertArrayHasKey( 'parent', $item_props, 'Item should have parent property' ); + } + + /** + * Test that execute_callback() returns error when content is missing. + * + * @since x.x.x + */ + public function test_execute_callback_without_content() { + $reflection = new \ReflectionClass( $this->ability ); + $method = $reflection->getMethod( 'execute_callback' ); + $method->setAccessible( true ); + + $input = array(); + $result = $method->invoke( $this->ability, $input ); + + $this->assertInstanceOf( WP_Error::class, $result, 'Result should be WP_Error' ); + $this->assertEquals( 'content_not_provided', $result->get_error_code(), 'Error code should be content_not_provided' ); + } + + /** + * Test that execute_callback() returns error when post_id points to non-existent post. + * + * @since x.x.x + */ + public function test_execute_callback_with_invalid_post_id() { + $reflection = new \ReflectionClass( $this->ability ); + $method = $reflection->getMethod( 'execute_callback' ); + $method->setAccessible( true ); + + $input = array( + 'post_id' => 99999, // Non-existent post ID. + ); + $result = $method->invoke( $this->ability, $input ); + + $this->assertInstanceOf( WP_Error::class, $result, 'Result should be WP_Error' ); + $this->assertEquals( 'post_not_found', $result->get_error_code(), 'Error code should be post_not_found' ); + } + + /** + * Test that execute_callback() returns error for invalid taxonomy. + * + * @since x.x.x + */ + public function test_execute_callback_with_invalid_taxonomy() { + $reflection = new \ReflectionClass( $this->ability ); + $method = $reflection->getMethod( 'execute_callback' ); + $method->setAccessible( true ); + + $input = array( + 'content' => 'Test content for taxonomy suggestions.', + 'taxonomy' => 'nonexistent_taxonomy', + ); + $result = $method->invoke( $this->ability, $input ); + + $this->assertInstanceOf( WP_Error::class, $result, 'Result should be WP_Error' ); + $this->assertEquals( 'invalid_taxonomy', $result->get_error_code(), 'Error code should be invalid_taxonomy' ); + } + + /** + * Test that parse_suggestions() handles valid JSON correctly. + * + * @since x.x.x + */ + public function test_parse_suggestions_with_valid_json() { + $reflection = new \ReflectionClass( $this->ability ); + $method = $reflection->getMethod( 'parse_suggestions' ); + $method->setAccessible( true ); + + $response = '{"suggestions": [{"term": "development", "confidence": 0.9}, {"term": "plugins", "confidence": 0.8}]}'; + + $result = $method->invoke( $this->ability, $response, 'allow_new', array(), 'post_tag', 5 ); + + $this->assertIsArray( $result, 'Result should be an array' ); + $this->assertCount( 2, $result, 'Should have 2 suggestions' ); + $this->assertEquals( 'development', $result[0]['term'], 'First suggestion should be development' ); + $this->assertTrue( $result[0]['is_new'], 'Term should be marked as new in allow_new strategy' ); + $this->assertTrue( $result[1]['is_new'], 'Term should be marked as new in allow_new strategy' ); + } + + /** + * Test that parse_suggestions() returns error for invalid JSON. + * + * @since x.x.x + */ + public function test_parse_suggestions_with_invalid_json() { + $reflection = new \ReflectionClass( $this->ability ); + $method = $reflection->getMethod( 'parse_suggestions' ); + $method->setAccessible( true ); + + $response = 'This is not valid JSON'; + + $result = $method->invoke( $this->ability, $response, 'allow_new', array(), 'post_tag', 5 ); + + $this->assertInstanceOf( WP_Error::class, $result, 'Result should be WP_Error' ); + $this->assertEquals( 'invalid_response', $result->get_error_code(), 'Error code should be invalid_response' ); + } + + /** + * Test that parse_suggestions() limits results to max_suggestions. + * + * @since x.x.x + */ + public function test_parse_suggestions_limits_results() { + $reflection = new \ReflectionClass( $this->ability ); + $method = $reflection->getMethod( 'parse_suggestions' ); + $method->setAccessible( true ); + + $response = '{"suggestions": [ + {"term": "a", "confidence": 0.9, "is_new": false}, + {"term": "b", "confidence": 0.8, "is_new": false}, + {"term": "c", "confidence": 0.7, "is_new": false}, + {"term": "d", "confidence": 0.6, "is_new": false}, + {"term": "e", "confidence": 0.5, "is_new": false} + ]}'; + + $result = $method->invoke( $this->ability, $response, 'allow_new', array(), 'post_tag', 3 ); + + $this->assertIsArray( $result, 'Result should be an array' ); + $this->assertCount( 3, $result, 'Should be limited to 3 suggestions' ); + $this->assertEquals( 'a', $result[0]['term'], 'First suggestion should be highest confidence' ); + } + + /** + * Test that parse_suggestions() sorts by confidence descending. + * + * @since x.x.x + */ + public function test_parse_suggestions_sorts_by_confidence() { + $reflection = new \ReflectionClass( $this->ability ); + $method = $reflection->getMethod( 'parse_suggestions' ); + $method->setAccessible( true ); + + $response = '{"suggestions": [ + {"term": "low", "confidence": 0.3, "is_new": true}, + {"term": "high", "confidence": 0.95, "is_new": true}, + {"term": "mid", "confidence": 0.6, "is_new": true} + ]}'; + + $result = $method->invoke( $this->ability, $response, 'allow_new', array(), 'post_tag', 10 ); + + $this->assertEquals( 'high', $result[0]['term'], 'First should be highest confidence' ); + $this->assertEquals( 'mid', $result[1]['term'], 'Second should be mid confidence' ); + $this->assertEquals( 'low', $result[2]['term'], 'Third should be lowest confidence' ); + } + + /** + * Test that parse_suggestions() clamps confidence values to 0-1 range. + * + * @since x.x.x + */ + public function test_parse_suggestions_clamps_confidence() { + $reflection = new \ReflectionClass( $this->ability ); + $method = $reflection->getMethod( 'parse_suggestions' ); + $method->setAccessible( true ); + + $response = '{"suggestions": [ + {"term": "over", "confidence": 1.5, "is_new": true}, + {"term": "under", "confidence": -0.5, "is_new": true} + ]}'; + + $result = $method->invoke( $this->ability, $response, 'allow_new', array(), 'post_tag', 10 ); + + $this->assertEquals( 1.0, $result[0]['confidence'], 'Confidence above 1 should be clamped to 1.0' ); + $this->assertEquals( 0.0, $result[1]['confidence'], 'Confidence below 0 should be clamped to 0.0' ); + } + + /** + * Test that parse_suggestions() preserves parent field for hierarchical terms. + * + * @since x.x.x + */ + public function test_parse_suggestions_preserves_parent_field() { + $reflection = new \ReflectionClass( $this->ability ); + $method = $reflection->getMethod( 'parse_suggestions' ); + $method->setAccessible( true ); + + $response = '{"suggestions": [ + {"term": "Machine Learning", "confidence": 0.9, "parent": "Technology"}, + {"term": "Finance", "confidence": 0.8} + ]}'; + + // Use 'category' (hierarchical) so parent is preserved. + $result = $method->invoke( $this->ability, $response, 'allow_new', array(), 'category', 10 ); + + $this->assertArrayHasKey( 'parent', $result[0], 'First suggestion should have parent key' ); + $this->assertEquals( 'Technology', $result[0]['parent'], 'Parent should be Technology' ); + $this->assertArrayNotHasKey( 'parent', $result[1], 'Second suggestion should not have parent key' ); + } + + /** + * Test that parse_suggestions() strips parent for non-hierarchical taxonomies. + * + * @since x.x.x + */ + public function test_parse_suggestions_strips_parent_for_non_hierarchical_taxonomy() { + $reflection = new \ReflectionClass( $this->ability ); + $method = $reflection->getMethod( 'parse_suggestions' ); + $method->setAccessible( true ); + + $response = '{"suggestions": [ + {"term": "Machine Learning", "confidence": 0.9, "parent": "Technology"} + ]}'; + + $result = $method->invoke( $this->ability, $response, 'allow_new', array(), 'post_tag', 10 ); + + $this->assertArrayNotHasKey( 'parent', $result[0], 'Parent should be stripped for non-hierarchical taxonomy' ); + } + + /** + * Test that parse_suggestions() strips parent when it matches the taxonomy slug. + * + * @since x.x.x + */ + public function test_parse_suggestions_strips_taxonomy_slug_as_parent() { + $reflection = new \ReflectionClass( $this->ability ); + $method = $reflection->getMethod( 'parse_suggestions' ); + $method->setAccessible( true ); + + $response = '{"suggestions": [ + {"term": "Machine Learning", "confidence": 0.9, "parent": "category"} + ]}'; + + $result = $method->invoke( $this->ability, $response, 'allow_new', array(), 'category', 10 ); + + $this->assertArrayNotHasKey( 'parent', $result[0], 'Parent should be stripped when it matches the taxonomy slug' ); + } + + /** + * Test that parse_suggestions() skips items with empty or missing term. + * + * @since x.x.x + */ + public function test_parse_suggestions_skips_invalid_items() { + $reflection = new \ReflectionClass( $this->ability ); + $method = $reflection->getMethod( 'parse_suggestions' ); + $method->setAccessible( true ); + + $response = '{"suggestions": [ + {"term": "valid", "confidence": 0.9, "is_new": true}, + {"confidence": 0.8, "is_new": true}, + {"term": "", "confidence": 0.7, "is_new": true}, + "not an object", + {"term": "also valid", "confidence": 0.6, "is_new": true} + ]}'; + + $result = $method->invoke( $this->ability, $response, 'allow_new', array(), 'post_tag', 10 ); + + $this->assertCount( 2, $result, 'Should only have 2 valid suggestions' ); + $this->assertEquals( 'valid', $result[0]['term'] ); + $this->assertEquals( 'also valid', $result[1]['term'] ); + } + + /** + * Test that parse_suggestions() defaults confidence to 0.5 when missing. + * + * @since x.x.x + */ + public function test_parse_suggestions_defaults_missing_confidence() { + $reflection = new \ReflectionClass( $this->ability ); + $method = $reflection->getMethod( 'parse_suggestions' ); + $method->setAccessible( true ); + + $response = '{"suggestions": [{"term": "test", "is_new": true}]}'; + + $result = $method->invoke( $this->ability, $response, 'allow_new', array(), 'post_tag', 10 ); + + $this->assertEquals( 0.5, $result[0]['confidence'], 'Missing confidence should default to 0.5' ); + } + + /** + * Test that parse_suggestions() determines is_new based on existing terms, not AI response. + * + * @since x.x.x + */ + public function test_parse_suggestions_overrides_is_new_from_existing_terms() { + $reflection = new \ReflectionClass( $this->ability ); + $method = $reflection->getMethod( 'parse_suggestions' ); + $method->setAccessible( true ); + + // Create a real term so parse_suggestions can find it via get_existing_terms(). + $this->factory()->term->create( array( 'taxonomy' => 'post_tag', 'name' => 'Tech' ) ); + + // AI says "tech" is new, but it exists in the DB as "Tech". + $response = '{"suggestions": [{"term": "tech", "confidence": 0.9, "is_new": true}]}'; + + $result = $method->invoke( $this->ability, $response, 'existing_only', array(), 'post_tag', 10 ); + + $this->assertFalse( $result[0]['is_new'], 'Should be false because "Tech" exists (case-insensitive match)' ); + $this->assertEquals( 'Tech', $result[0]['term'], 'Should use the original capitalized term name from the existing terms list' ); + } + + /** + * Test that parse_suggestions() filters out assigned terms. + * + * @since x.x.x + */ + public function test_parse_suggestions_filters_assigned_terms() { + $reflection = new \ReflectionClass( $this->ability ); + $method = $reflection->getMethod( 'parse_suggestions' ); + $method->setAccessible( true ); + + $response = '{"suggestions": [ + {"term": "php", "confidence": 0.9, "is_new": true}, + {"term": "javascript", "confidence": 0.8, "is_new": true}, + {"term": "python", "confidence": 0.7, "is_new": true} + ]}'; + + $result = $method->invoke( $this->ability, $response, 'allow_new', array( 'PHP' ), 'post_tag', 10 ); + + $this->assertCount( 2, $result, 'Should exclude assigned term' ); + $this->assertEquals( 'javascript', $result[0]['term'] ); + $this->assertEquals( 'python', $result[1]['term'] ); + } + + /** + * Test that parse_suggestions() filters new terms for existing_only strategy. + * + * @since x.x.x + */ + public function test_parse_suggestions_existing_only_filters_new_terms() { + $reflection = new \ReflectionClass( $this->ability ); + $method = $reflection->getMethod( 'parse_suggestions' ); + $method->setAccessible( true ); + + // Create real terms so parse_suggestions can find them via get_existing_terms(). + $this->factory()->term->create( array( 'taxonomy' => 'post_tag', 'name' => 'php' ) ); + $this->factory()->term->create( array( 'taxonomy' => 'post_tag', 'name' => 'javascript' ) ); + + $response = '{"suggestions": [ + {"term": "php", "confidence": 0.9, "is_new": true}, + {"term": "brand new term", "confidence": 0.85, "is_new": true}, + {"term": "javascript", "confidence": 0.8, "is_new": true} + ]}'; + + $result = $method->invoke( $this->ability, $response, 'existing_only', array(), 'post_tag', 10 ); + + $this->assertCount( 2, $result, 'Should only include existing terms' ); + $this->assertEquals( 'php', $result[0]['term'] ); + $this->assertEquals( 'javascript', $result[1]['term'] ); + $this->assertFalse( $result[0]['is_new'] ); + $this->assertFalse( $result[1]['is_new'] ); + } + + /** + * Test that suggestions_schema() returns the expected structure. + * + * @since x.x.x + */ + public function test_suggestions_schema_returns_expected_structure() { + $reflection = new \ReflectionClass( $this->ability ); + $method = $reflection->getMethod( 'suggestions_schema' ); + $method->setAccessible( true ); + + $schema = $method->invoke( $this->ability ); + + $this->assertEquals( 'object', $schema['type'] ); + $this->assertArrayHasKey( 'suggestions', $schema['properties'] ); + $this->assertEquals( 'array', $schema['properties']['suggestions']['type'] ); + + $item_props = $schema['properties']['suggestions']['items']['properties']; + $this->assertArrayHasKey( 'term', $item_props ); + $this->assertArrayHasKey( 'confidence', $item_props ); + $this->assertArrayNotHasKey( 'is_new', $item_props, 'is_new is determined server-side, not by the LLM' ); + $this->assertArrayNotHasKey( 'parent', $item_props, 'parent is determined server-side from existing term hierarchy, not by the LLM' ); + + $required = $schema['properties']['suggestions']['items']['required']; + $this->assertContains( 'term', $required ); + $this->assertContains( 'confidence', $required ); + $this->assertNotContains( 'is_new', $required ); + $this->assertNotContains( 'parent', $required, 'parent should not be required so the LLM can omit it' ); + } + + /** + * Test that get_existing_terms() returns term names for a valid taxonomy. + * + * @since x.x.x + */ + public function test_get_existing_terms_returns_term_names() { + $reflection = new \ReflectionClass( $this->ability ); + $method = $reflection->getMethod( 'get_existing_terms' ); + $method->setAccessible( true ); + + // Create some terms. + $this->factory()->term->create( array( 'taxonomy' => 'post_tag', 'name' => 'PHP' ) ); + $this->factory()->term->create( array( 'taxonomy' => 'post_tag', 'name' => 'JavaScript' ) ); + + $result = $method->invoke( $this->ability, 'post_tag' ); + + $this->assertIsArray( $result ); + $this->assertContains( 'PHP', $result ); + $this->assertContains( 'JavaScript', $result ); + } + + /** + * Test that get_existing_terms() returns empty array for invalid taxonomy. + * + * @since x.x.x + */ + public function test_get_existing_terms_returns_empty_for_invalid_taxonomy() { + $reflection = new \ReflectionClass( $this->ability ); + $method = $reflection->getMethod( 'get_existing_terms' ); + $method->setAccessible( true ); + + $result = $method->invoke( $this->ability, 'nonexistent_taxonomy' ); + + $this->assertIsArray( $result ); + $this->assertEmpty( $result ); + } + + /** + * Test that get_top_terms() returns terms ordered by count. + * + * @since x.x.x + */ + public function test_get_top_terms_returns_terms() { + $reflection = new \ReflectionClass( $this->ability ); + $method = $reflection->getMethod( 'get_top_terms' ); + $method->setAccessible( true ); + + $this->factory()->term->create( array( 'taxonomy' => 'post_tag', 'name' => 'TopTerm' ) ); + $this->factory()->term->create( array( 'taxonomy' => 'post_tag', 'name' => 'AnotherTerm' ) ); + + $result = $method->invoke( $this->ability, 'post_tag' ); + + $this->assertIsArray( $result ); + $this->assertContains( 'TopTerm', $result ); + $this->assertContains( 'AnotherTerm', $result ); + } + + /** + * Test that get_top_terms() respects the limit parameter. + * + * @since x.x.x + */ + public function test_get_top_terms_respects_limit() { + $reflection = new \ReflectionClass( $this->ability ); + $method = $reflection->getMethod( 'get_top_terms' ); + $method->setAccessible( true ); + + $this->factory()->term->create( array( 'taxonomy' => 'post_tag', 'name' => 'Term1' ) ); + $this->factory()->term->create( array( 'taxonomy' => 'post_tag', 'name' => 'Term2' ) ); + $this->factory()->term->create( array( 'taxonomy' => 'post_tag', 'name' => 'Term3' ) ); + + $result = $method->invoke( $this->ability, 'post_tag', 2 ); + + $this->assertIsArray( $result ); + $this->assertCount( 2, $result, 'Should be limited to 2 terms' ); + } + + /** + * Test that get_top_terms() returns empty array for invalid taxonomy. + * + * @since x.x.x + */ + public function test_get_top_terms_returns_empty_for_invalid_taxonomy() { + $reflection = new \ReflectionClass( $this->ability ); + $method = $reflection->getMethod( 'get_top_terms' ); + $method->setAccessible( true ); + + $result = $method->invoke( $this->ability, 'nonexistent_taxonomy' ); + + $this->assertIsArray( $result ); + $this->assertEmpty( $result ); + } + + /** + * Test that permission_callback() returns true for user with edit_post capability on a specific post. + * + * @since x.x.x + */ + public function test_permission_callback_with_post_id_and_edit_capability() { + $reflection = new \ReflectionClass( $this->ability ); + $method = $reflection->getMethod( 'permission_callback' ); + $method->setAccessible( true ); + + $post_id = $this->factory->post->create( + array( + 'post_content' => 'Test content', + 'post_status' => 'publish', + ) + ); + + $user_id = $this->factory->user->create( array( 'role' => 'editor' ) ); + wp_set_current_user( $user_id ); + + $result = $method->invoke( $this->ability, array( 'post_id' => $post_id ) ); + + $this->assertTrue( $result, 'Permission should be granted for user with edit_post capability' ); + } + + /** + * Test that permission_callback() returns error for user without edit_post capability on a specific post. + * + * @since x.x.x + */ + public function test_permission_callback_with_post_id_without_edit_capability() { + $reflection = new \ReflectionClass( $this->ability ); + $method = $reflection->getMethod( 'permission_callback' ); + $method->setAccessible( true ); + + $post_id = $this->factory->post->create( + array( + 'post_content' => 'Test content', + 'post_status' => 'publish', + ) + ); + + $user_id = $this->factory->user->create( array( 'role' => 'subscriber' ) ); + wp_set_current_user( $user_id ); + + $result = $method->invoke( $this->ability, array( 'post_id' => $post_id ) ); + + $this->assertInstanceOf( WP_Error::class, $result, 'Result should be WP_Error' ); + $this->assertEquals( 'insufficient_capabilities', $result->get_error_code(), 'Error code should be insufficient_capabilities' ); + } + + /** + * Test that permission_callback() returns error for non-existent post. + * + * @since x.x.x + */ + public function test_permission_callback_with_nonexistent_post_id() { + $reflection = new \ReflectionClass( $this->ability ); + $method = $reflection->getMethod( 'permission_callback' ); + $method->setAccessible( true ); + + $user_id = $this->factory->user->create( array( 'role' => 'editor' ) ); + wp_set_current_user( $user_id ); + + $result = $method->invoke( $this->ability, array( 'post_id' => 99999 ) ); + + $this->assertInstanceOf( WP_Error::class, $result, 'Result should be WP_Error' ); + $this->assertEquals( 'post_not_found', $result->get_error_code(), 'Error code should be post_not_found' ); + } + + /** + * Test that permission_callback() returns true for user with edit_posts capability. + * + * @since x.x.x + */ + public function test_permission_callback_with_edit_posts_capability() { + $reflection = new \ReflectionClass( $this->ability ); + $method = $reflection->getMethod( 'permission_callback' ); + $method->setAccessible( true ); + + $user_id = $this->factory->user->create( array( 'role' => 'editor' ) ); + wp_set_current_user( $user_id ); + + $result = $method->invoke( $this->ability, array() ); + + $this->assertTrue( $result, 'Permission should be granted for user with edit_posts capability' ); + } + + /** + * Test that permission_callback() returns error for user without edit_posts capability. + * + * @since x.x.x + */ + public function test_permission_callback_without_edit_posts_capability() { + $reflection = new \ReflectionClass( $this->ability ); + $method = $reflection->getMethod( 'permission_callback' ); + $method->setAccessible( true ); + + $user_id = $this->factory->user->create( array( 'role' => 'subscriber' ) ); + wp_set_current_user( $user_id ); + + $result = $method->invoke( $this->ability, array() ); + + $this->assertInstanceOf( WP_Error::class, $result, 'Result should be WP_Error' ); + $this->assertEquals( 'insufficient_capabilities', $result->get_error_code(), 'Error code should be insufficient_capabilities' ); + } + + /** + * Test that permission_callback() returns error for logged out user. + * + * @since x.x.x + */ + public function test_permission_callback_for_logged_out_user() { + $reflection = new \ReflectionClass( $this->ability ); + $method = $reflection->getMethod( 'permission_callback' ); + $method->setAccessible( true ); + + wp_set_current_user( 0 ); + + $result = $method->invoke( $this->ability, array() ); + + $this->assertInstanceOf( WP_Error::class, $result, 'Result should be WP_Error' ); + $this->assertEquals( 'insufficient_capabilities', $result->get_error_code(), 'Error code should be insufficient_capabilities' ); + } + + /** + * Test that permission_callback() returns false for post type without show_in_rest. + * + * @since x.x.x + */ + public function test_permission_callback_with_post_type_without_show_in_rest() { + $reflection = new \ReflectionClass( $this->ability ); + $method = $reflection->getMethod( 'permission_callback' ); + $method->setAccessible( true ); + + register_post_type( + 'test_no_rest', + array( + 'public' => true, + 'show_in_rest' => false, + ) + ); + + $post_id = $this->factory->post->create( + array( + 'post_content' => 'Test content', + 'post_type' => 'test_no_rest', + 'post_status' => 'publish', + ) + ); + + $user_id = $this->factory->user->create( array( 'role' => 'editor' ) ); + wp_set_current_user( $user_id ); + + $result = $method->invoke( $this->ability, array( 'post_id' => $post_id ) ); + + $this->assertFalse( $result, 'Permission should be denied for post type without show_in_rest' ); + + unregister_post_type( 'test_no_rest' ); + } + + /** + * Test that meta() returns the expected meta structure. + * + * @since x.x.x + */ + public function test_meta_returns_expected_structure() { + $reflection = new \ReflectionClass( $this->ability ); + $method = $reflection->getMethod( 'meta' ); + $method->setAccessible( true ); + + $meta = $method->invoke( $this->ability ); + + $this->assertIsArray( $meta, 'Meta should be an array' ); + $this->assertArrayHasKey( 'show_in_rest', $meta, 'Meta should have show_in_rest' ); + $this->assertTrue( $meta['show_in_rest'], 'show_in_rest should be true' ); + } + + /** + * Test that get_prompt_builder() returns a WP_Error when no text generation model is available. + * + * @since x.x.x + */ + public function test_get_prompt_builder_returns_error_without_valid_model() { + $reflection = new \ReflectionClass( $this->ability ); + $method = $reflection->getMethod( 'get_prompt_builder' ); + $method->setAccessible( true ); + + $result = $method->invoke( $this->ability, 'Test prompt' ); + + // Without configured AI credentials, the builder should indicate no supported model. + if ( is_wp_error( $result ) ) { + $this->assertEquals( 'unsupported_model', $result->get_error_code(), 'Error code should be unsupported_model' ); + } else { + // If a model happens to be available in the test environment, verify it returns a builder. + $this->assertIsObject( $result, 'Should return a prompt builder object' ); + } + } + + /** + * Test that generate_suggestions() returns a WP_Error when no AI model is available. + * + * @since x.x.x + */ + public function test_generate_suggestions_returns_error_without_ai() { + $reflection = new \ReflectionClass( $this->ability ); + $method = $reflection->getMethod( 'generate_suggestions' ); + $method->setAccessible( true ); + + $result = $method->invoke( + $this->ability, + array( 'content' => 'Test content for suggestions.' ), + 'post_tag', + 'allow_new', + 5, + array() + ); + + // Without a configured AI provider, this should return a WP_Error. + $this->assertInstanceOf( WP_Error::class, $result, 'Should return WP_Error without AI provider' ); + } + + /** + * Test that generate_suggestions() builds prompt with assigned terms. + * + * Verifies the prompt filter receives the expected assigned terms. + * + * @since x.x.x + */ + public function test_generate_suggestions_passes_assigned_terms_to_prompt_filter() { + $captured_prompt = ''; + + add_filter( + 'wpai_content_classification_prompt', + static function ( $prompt ) use ( &$captured_prompt ) { + $captured_prompt = $prompt; + return $prompt; + } + ); + + $reflection = new \ReflectionClass( $this->ability ); + $method = $reflection->getMethod( 'generate_suggestions' ); + $method->setAccessible( true ); + + // This will fail at the AI client, but the filter fires before that. + $method->invoke( + $this->ability, + array( 'content' => 'Test content.' ), + 'post_tag', + 'allow_new', + 5, + array( 'existing-tag' ) + ); + + $this->assertStringContainsString( 'existing-tag', $captured_prompt, 'Prompt should contain assigned terms' ); + $this->assertStringContainsString( '', $captured_prompt, 'Prompt should contain content tags' ); + $this->assertStringContainsString( 'post_tag', $captured_prompt, 'Prompt should contain taxonomy' ); + } + + /** + * Test that generate_suggestions() includes available terms for existing_only strategy. + * + * @since x.x.x + */ + public function test_generate_suggestions_includes_available_terms_for_existing_only() { + // Create terms so they appear in the prompt. + $this->factory()->term->create( array( 'taxonomy' => 'post_tag', 'name' => 'AvailableTerm' ) ); + + $captured_prompt = ''; + + add_filter( + 'wpai_content_classification_prompt', + static function ( $prompt ) use ( &$captured_prompt ) { + $captured_prompt = $prompt; + return $prompt; + } + ); + + $reflection = new \ReflectionClass( $this->ability ); + $method = $reflection->getMethod( 'generate_suggestions' ); + $method->setAccessible( true ); + + $method->invoke( + $this->ability, + array( 'content' => 'Test content.' ), + 'post_tag', + 'existing_only', + 5, + array() + ); + + $this->assertStringContainsString( '', $captured_prompt, 'Prompt should contain available terms for existing_only strategy' ); + $this->assertStringContainsString( 'AvailableTerm', $captured_prompt, 'Prompt should include the created term' ); + } + + /** + * Test that generate_suggestions() omits available terms for allow_new strategy. + * + * @since x.x.x + */ + public function test_generate_suggestions_omits_available_terms_for_allow_new() { + $this->factory()->term->create( array( 'taxonomy' => 'post_tag', 'name' => 'SomeTerm' ) ); + + $captured_prompt = ''; + + add_filter( + 'wpai_content_classification_prompt', + static function ( $prompt ) use ( &$captured_prompt ) { + $captured_prompt = $prompt; + return $prompt; + } + ); + + $reflection = new \ReflectionClass( $this->ability ); + $method = $reflection->getMethod( 'generate_suggestions' ); + $method->setAccessible( true ); + + $method->invoke( + $this->ability, + array( 'content' => 'Test content.' ), + 'post_tag', + 'allow_new', + 5, + array() + ); + + $this->assertStringNotContainsString( '', $captured_prompt, 'Prompt should not contain available terms for allow_new strategy' ); + } +} diff --git a/tests/Integration/Includes/Experiments/Content_Classification/Content_ClassificationTest.php b/tests/Integration/Includes/Experiments/Content_Classification/Content_ClassificationTest.php new file mode 100644 index 000000000..fdeb3db46 --- /dev/null +++ b/tests/Integration/Includes/Experiments/Content_Classification/Content_ClassificationTest.php @@ -0,0 +1,261 @@ + 'test-api-key' ) ); + + // Mock has_valid_ai_credentials to return true for tests. + add_filter( 'wpai_pre_has_valid_credentials_check', '__return_true' ); + + // Enable experiments globally and individually. + update_option( 'wpai_features_enabled', true ); + update_option( 'wpai_feature_content-classification_enabled', true ); + + $registry = new Registry(); + $loader = new Loader( $registry ); + $loader->register_features(); + + $experiment = $registry->get_feature( 'content-classification' ); + $this->assertInstanceOf( Content_Classification::class, $experiment, 'Content classification experiment should be registered in the registry.' ); + } + + /** + * Tear down test case. + * + * @since x.x.x + */ + public function tearDown(): void { + wp_set_current_user( 0 ); + delete_option( 'wpai_features_enabled' ); + delete_option( 'wpai_feature_content-classification_enabled' ); + delete_option( 'wpai_feature_content-classification_field_strategy' ); + delete_option( 'wpai_feature_content-classification_field_max_suggestions' ); + delete_option( 'wp_ai_client_provider_credentials' ); + remove_filter( 'wpai_pre_has_valid_credentials_check', '__return_true' ); + remove_all_filters( 'wpai_content_classification_strategy' ); + remove_all_filters( 'wpai_content_classification_max_suggestions' ); + parent::tearDown(); + } + + /** + * Test that the experiment is registered correctly. + * + * @since x.x.x + */ + public function test_experiment_registration() { + $experiment = new Content_Classification(); + + $this->assertEquals( 'content-classification', $experiment->get_id() ); + $this->assertEquals( 'Content Classification', $experiment->get_label() ); + $this->assertEquals( Experiment_Category::EDITOR, $experiment->get_category() ); + $this->assertTrue( $experiment->is_enabled() ); + } + + /** + * Test that experiment settings are registered. + * + * @since x.x.x + */ + public function test_experiment_settings_registration() { + $experiment = new Content_Classification(); + $experiment->register_settings(); + + // Verify the settings are registered by checking they can be retrieved. + $strategy = get_option( 'wpai_feature_content-classification_field_strategy', 'existing_only' ); + $this->assertEquals( 'existing_only', $strategy ); + + $max_suggestions = get_option( 'wpai_feature_content-classification_field_max_suggestions', 5 ); + $this->assertEquals( 5, $max_suggestions ); + } + + /** + * Test that strategy sanitization works correctly. + * + * @since x.x.x + */ + public function test_sanitize_strategy() { + $experiment = new Content_Classification(); + + $this->assertEquals( 'existing_only', $experiment->sanitize_strategy( 'existing_only' ) ); + $this->assertEquals( 'allow_new', $experiment->sanitize_strategy( 'allow_new' ) ); + $this->assertEquals( 'existing_only', $experiment->sanitize_strategy( 'invalid_value' ) ); + $this->assertEquals( 'existing_only', $experiment->sanitize_strategy( '' ) ); + } + + /** + * Test that max suggestions sanitization works correctly. + * + * @since x.x.x + */ + public function test_sanitize_max_suggestions() { + $experiment = new Content_Classification(); + + $this->assertEquals( 5, $experiment->sanitize_max_suggestions( 5 ) ); + $this->assertEquals( 1, $experiment->sanitize_max_suggestions( 0 ) ); + $this->assertEquals( 1, $experiment->sanitize_max_suggestions( -1 ) ); + $this->assertEquals( 10, $experiment->sanitize_max_suggestions( 15 ) ); + $this->assertEquals( 7, $experiment->sanitize_max_suggestions( '7' ) ); + } + + /** + * Test that get_strategy() returns the default value. + * + * @since x.x.x + */ + public function test_get_strategy_returns_default() { + $experiment = new Content_Classification(); + + $this->assertEquals( 'existing_only', $experiment->get_strategy() ); + } + + /** + * Test that get_strategy() returns the saved option value. + * + * @since x.x.x + */ + public function test_get_strategy_returns_saved_option() { + update_option( 'wpai_feature_content-classification_field_strategy', 'allow_new' ); + $experiment = new Content_Classification(); + + $this->assertEquals( 'allow_new', $experiment->get_strategy() ); + } + + /** + * Test that get_strategy() is filterable. + * + * @since x.x.x + */ + public function test_get_strategy_is_filterable() { + $experiment = new Content_Classification(); + + add_filter( + 'wpai_content_classification_strategy', + static function () { + return 'allow_new'; + } + ); + + $this->assertEquals( 'allow_new', $experiment->get_strategy() ); + } + + /** + * Test that get_strategy() sanitizes filtered value. + * + * @since x.x.x + */ + public function test_get_strategy_sanitizes_filtered_value() { + $experiment = new Content_Classification(); + + add_filter( + 'wpai_content_classification_strategy', + static function () { + return 'malicious_value'; + } + ); + + $this->assertEquals( 'existing_only', $experiment->get_strategy(), 'Invalid filtered value should fall back to default' ); + } + + /** + * Test that get_max_suggestions() returns the default value. + * + * @since x.x.x + */ + public function test_get_max_suggestions_returns_default() { + $experiment = new Content_Classification(); + + $this->assertEquals( 5, $experiment->get_max_suggestions() ); + } + + /** + * Test that get_max_suggestions() returns the saved option value. + * + * @since x.x.x + */ + public function test_get_max_suggestions_returns_saved_option() { + update_option( 'wpai_feature_content-classification_field_max_suggestions', 8 ); + $experiment = new Content_Classification(); + + $this->assertEquals( 8, $experiment->get_max_suggestions() ); + } + + /** + * Test that get_max_suggestions() is filterable. + * + * @since x.x.x + */ + public function test_get_max_suggestions_is_filterable() { + $experiment = new Content_Classification(); + + add_filter( + 'wpai_content_classification_max_suggestions', + static function () { + return 3; + } + ); + + $this->assertEquals( 3, $experiment->get_max_suggestions() ); + } + + /** + * Test that get_max_suggestions() sanitizes filtered value that exceeds maximum. + * + * @since x.x.x + */ + public function test_get_max_suggestions_sanitizes_filtered_value_above_max() { + $experiment = new Content_Classification(); + + add_filter( + 'wpai_content_classification_max_suggestions', + static function () { + return 50; + } + ); + + $this->assertEquals( 10, $experiment->get_max_suggestions(), 'Filtered value above 10 should be clamped to 10' ); + } + + /** + * Test that get_max_suggestions() sanitizes filtered value below minimum. + * + * @since x.x.x + */ + public function test_get_max_suggestions_sanitizes_filtered_value_below_min() { + $experiment = new Content_Classification(); + + add_filter( + 'wpai_content_classification_max_suggestions', + static function () { + return 0; + } + ); + + $this->assertEquals( 1, $experiment->get_max_suggestions(), 'Filtered value of 0 should be clamped to 1' ); + } +} diff --git a/tests/e2e-request-mocking/e2e-request-mocking.php b/tests/e2e-request-mocking/e2e-request-mocking.php index f6009ab92..390a5d239 100644 --- a/tests/e2e-request-mocking/e2e-request-mocking.php +++ b/tests/e2e-request-mocking/e2e-request-mocking.php @@ -18,9 +18,9 @@ /** * Mock the HTTP requests and provide known responses. * - * @param mixed $preempt Whether to preempt an HTTP request's return value. - * @param array $parsed_args HTTP request arguments. - * @param string $url The request URL. + * @param mixed $preempt Whether to preempt an HTTP request's return value. + * @param array $parsed_args HTTP request arguments. + * @param string $url The request URL. * @return array|bool The response. */ function ai_e2e_test_request_mocking( $preempt, $parsed_args, $url ) { @@ -69,6 +69,9 @@ function ai_e2e_test_request_mocking( $preempt, $parsed_args, $url ) { // Route review-notes requests to their own fixture. if ( is_string( $body ) && str_contains( $body, 'Category guidance by block type' ) ) { $response = file_get_contents( __DIR__ . '/responses/OpenAI/review-notes-responses.json' ); + } elseif ( is_string( $body ) && str_contains( $body, 'content taxonomy assistant' ) ) { + // Route content-classification requests to their own fixture. + $response = file_get_contents( __DIR__ . '/responses/OpenAI/content-classification-responses.json' ); } else { $response = file_get_contents( __DIR__ . '/responses/OpenAI/responses.json' ); } @@ -81,6 +84,9 @@ function ai_e2e_test_request_mocking( $preempt, $parsed_args, $url ) { // Route review-notes requests to their own fixture. if ( is_string( $body ) && str_contains( $body, 'Category guidance by block type' ) ) { $response = file_get_contents( __DIR__ . '/responses/OpenAI/review-notes-completions.json' ); + } elseif ( is_string( $body ) && str_contains( $body, 'content taxonomy assistant' ) ) { + // Route content-classification requests to their own fixture. + $response = file_get_contents( __DIR__ . '/responses/OpenAI/content-classification-completions.json' ); } else { $response = file_get_contents( __DIR__ . '/responses/OpenAI/completions.json' ); } diff --git a/tests/e2e-request-mocking/responses/OpenAI/content-classification-completions.json b/tests/e2e-request-mocking/responses/OpenAI/content-classification-completions.json new file mode 100644 index 000000000..6328dd60b --- /dev/null +++ b/tests/e2e-request-mocking/responses/OpenAI/content-classification-completions.json @@ -0,0 +1,36 @@ +{ + "id": "chatcmpl-cc-e2e-test-mock-001", + "object": "chat.completion", + "created": 1766520705, + "model": "gpt-4o-mini-2024-07-18", + "choices": [ + { + "index": 0, + "message": { + "role": "assistant", + "content": "{\"suggestions\":[{\"term\":\"Technology\",\"confidence\":0.95},{\"term\":\"Innovation\",\"confidence\":0.88},{\"term\":\"Science\",\"confidence\":0.75}]}", + "refusal": null, + "annotations": [] + }, + "logprobs": null, + "finish_reason": "stop" + } + ], + "usage": { + "prompt_tokens": 490, + "completion_tokens": 50, + "total_tokens": 540, + "prompt_tokens_details": { + "cached_tokens": 0, + "audio_tokens": 0 + }, + "completion_tokens_details": { + "reasoning_tokens": 0, + "audio_tokens": 0, + "accepted_prediction_tokens": 0, + "rejected_prediction_tokens": 0 + } + }, + "service_tier": "default", + "system_fingerprint": "fp_cc_e2e_mock" +} diff --git a/tests/e2e-request-mocking/responses/OpenAI/content-classification-responses.json b/tests/e2e-request-mocking/responses/OpenAI/content-classification-responses.json new file mode 100644 index 000000000..30ce8f819 --- /dev/null +++ b/tests/e2e-request-mocking/responses/OpenAI/content-classification-responses.json @@ -0,0 +1,71 @@ +{ + "id": "resp_cc_e2e_test_mock_001", + "object": "response", + "created_at": 1771602524, + "status": "completed", + "background": false, + "billing": { + "payer": "developer" + }, + "completed_at": 1771602526, + "error": null, + "frequency_penalty": 0.0, + "incomplete_details": null, + "instructions": null, + "max_output_tokens": null, + "max_tool_calls": null, + "model": "gpt-4o-2024-08-06", + "output": [ + { + "id": "msg_cc_e2e_test_mock_001", + "type": "message", + "status": "completed", + "content": [ + { + "type": "output_text", + "annotations": [], + "logprobs": [], + "text": "{\"suggestions\":[{\"term\":\"Technology\",\"confidence\":0.95},{\"term\":\"Innovation\",\"confidence\":0.88},{\"term\":\"Science\",\"confidence\":0.75}]}" + } + ], + "role": "assistant" + } + ], + "parallel_tool_calls": true, + "presence_penalty": 0.0, + "previous_response_id": null, + "prompt_cache_key": null, + "prompt_cache_retention": null, + "reasoning": { + "effort": null, + "summary": null + }, + "safety_identifier": null, + "service_tier": "default", + "store": true, + "temperature": 0.5, + "text": { + "format": { + "type": "json_schema" + }, + "verbosity": "medium" + }, + "tool_choice": "auto", + "tools": [], + "top_logprobs": 0, + "top_p": 1.0, + "truncation": "disabled", + "usage": { + "input_tokens": 490, + "input_tokens_details": { + "cached_tokens": 0 + }, + "output_tokens": 50, + "output_tokens_details": { + "reasoning_tokens": 0 + }, + "total_tokens": 540 + }, + "user": null, + "metadata": {} +} diff --git a/tests/e2e/specs/experiments/content-classification.spec.js b/tests/e2e/specs/experiments/content-classification.spec.js new file mode 100644 index 000000000..a772ce77b --- /dev/null +++ b/tests/e2e/specs/experiments/content-classification.spec.js @@ -0,0 +1,351 @@ +/** + * WordPress dependencies + */ +import { test, expect } from '@wordpress/e2e-test-utils-playwright'; + +/** + * Internal dependencies + */ +import { + disableExperiment, + disableExperiments, + enableExperiments, + enableExperiment, +} from '../../utils/helpers'; + +const EXPERIMENT_ID = 'content-classification'; + +// Content Classification has a 150 word minimum requirement. +const LONG_CONTENT = + 'Artificial intelligence is transforming the technology landscape at an unprecedented pace. ' + + 'From machine learning algorithms that power recommendation engines to natural language processing ' + + 'systems that understand human speech, AI is reshaping how we interact with computers and the world ' + + 'around us. In the field of healthcare, AI-driven diagnostics are helping doctors detect diseases ' + + 'earlier and more accurately than ever before. In finance, algorithmic trading systems process vast ' + + 'amounts of data to make split-second decisions. The automotive industry is leveraging AI for ' + + 'self-driving vehicles that promise to revolutionize transportation. Education is being transformed ' + + 'through personalized learning platforms that adapt to each student. Meanwhile, creative industries ' + + 'are exploring how generative AI can assist with writing, music composition, and visual art. ' + + 'As these technologies continue to evolve, important questions about ethics, privacy, and the future ' + + 'of work demand careful consideration from policymakers, technologists, and society at large. ' + + 'The potential benefits are enormous, but so are the challenges we must navigate together.'; + +/** + * Opens the Post sidebar and expands a taxonomy panel by its header label. + * + * Taxonomy panels in the Gutenberg sidebar are collapsible. The content + * classification component only renders when the panel is expanded. + * + * @param {Object} editor The editor fixture. + * @param {Object} page The page object. + * @param {string} panelLabel The panel header label to expand (e.g. "Tags"). + */ +async function openTaxonomyPanel( editor, page, panelLabel ) { + await editor.openDocumentSettingsSidebar(); + + // Switch to the Post tab if the Block tab is active. + const postTab = page.getByRole( 'tab', { name: 'Post' } ); + if ( ( await postTab.count() ) > 0 ) { + await postTab.click(); + } + + // Expand the taxonomy panel if it is collapsed. + const panelToggle = page.locator( '.components-panel__body-toggle', { + hasText: panelLabel, + } ); + + if ( ( await panelToggle.count() ) > 0 ) { + // Check if the panel is collapsed by looking at the aria-expanded attribute. + const isExpanded = await panelToggle.getAttribute( 'aria-expanded' ); + if ( isExpanded === 'false' ) { + await panelToggle.click(); + } + } +} + +/** + * Sets the strategy option for the content classification experiment + * via the settings page. + * + * @param {Object} admin The admin fixture. + * @param {Object} page The page object. + * @param {string} strategy The strategy value ('existing_only' or 'allow_new'). + */ +async function setStrategy( admin, page, strategy ) { + await admin.visitAdminPage( 'options-general.php?page=ai' ); + await page + .locator( '#wpai_feature_content-classification_field_strategy' ) + .selectOption( strategy ); + await page.locator( '#submit' ).click(); + await expect( + page.locator( '.wrap .notice-success', { + hasText: 'Settings saved', + } ) + ).toHaveCount( 1 ); +} + +test.describe( 'Content Classification Experiment', () => { + test.beforeEach( async ( { admin, page } ) => { + // Globally turn on Experiments. + await enableExperiments( admin, page ); + + // Enable the Content Classification Experiment. + await enableExperiment( admin, page, EXPERIMENT_ID ); + } ); + + test( 'Shows the "Suggest Tags" button in the Tags panel', async ( { + admin, + editor, + page, + } ) => { + await admin.createNewPost( { title: 'Content Classification Test' } ); + + // Add enough content to enable suggestions. + await editor.insertBlock( { + name: 'core/paragraph', + attributes: { content: LONG_CONTENT }, + } ); + + // Open the Tags panel. + await openTaxonomyPanel( editor, page, 'Tags' ); + + // The suggest button should be visible within the Tags panel. + await expect( + page.locator( '.ai-content-classification button', { + hasText: 'Suggest Tags', + } ) + ).toBeVisible(); + } ); + + test( 'Shows the "Suggest Categories" button in the Categories panel', async ( { + admin, + editor, + page, + } ) => { + await admin.createNewPost( { + title: 'Content Classification Categories Test', + } ); + + // Add enough content to enable suggestions. + await editor.insertBlock( { + name: 'core/paragraph', + attributes: { content: LONG_CONTENT }, + } ); + + // Open the Categories panel. + await openTaxonomyPanel( editor, page, 'Categories' ); + + // The suggest button should be visible within the Categories panel. + await expect( + page.locator( '.ai-content-classification button', { + hasText: 'Suggest Categories', + } ) + ).toBeVisible(); + } ); + + test( 'Shows hint text when content is insufficient', async ( { + admin, + editor, + page, + } ) => { + await admin.createNewPost( { + title: 'Content Classification Hint Test', + } ); + + // Add a short paragraph (well under 150 words). + await editor.insertBlock( { + name: 'core/paragraph', + attributes: { content: 'This is a short paragraph.' }, + } ); + + // Open the Tags panel. + await openTaxonomyPanel( editor, page, 'Tags' ); + + // The hint should be visible. + await expect( + page.locator( '.ai-content-classification__hint', { + hasText: 'Add more content to enable AI suggestions', + } ) + ).toBeVisible(); + + // The suggest button should be disabled. + await expect( + page + .locator( '.ai-content-classification__generate-button' ) + .first() + ).toBeDisabled(); + } ); + + test( 'Generates and displays suggestion pills', async ( { + admin, + editor, + page, + } ) => { + // Set strategy to allow_new so mock suggestions (new terms) pass through. + await setStrategy( admin, page, 'allow_new' ); + + await admin.createNewPost( { + title: 'Content Classification Generate Test', + } ); + + // Add enough content. + await editor.insertBlock( { + name: 'core/paragraph', + attributes: { content: LONG_CONTENT }, + } ); + + await editor.saveDraft(); + await page.reload(); + + // Open the Tags panel. + await openTaxonomyPanel( editor, page, 'Tags' ); + + // Click the Suggest Tags button. + await page + .locator( '.ai-content-classification button', { + hasText: 'Suggest Tags', + } ) + .first() + .click(); + + // Wait for suggestions to appear. + await expect( + page.locator( '.ai-content-classification__suggestions' ).first() + ).toBeVisible(); + + // Verify suggestion pills are rendered. + await expect( + page.locator( '.ai-content-classification__pill' ).first() + ).toBeVisible(); + + // Verify the "Suggest again" and "Dismiss all" actions are visible. + await expect( + page + .locator( '.ai-content-classification__actions button', { + hasText: 'Suggest again', + } ) + .first() + ).toBeVisible(); + + await expect( + page + .locator( '.ai-content-classification__actions button', { + hasText: 'Dismiss all', + } ) + .first() + ).toBeVisible(); + } ); + + test( 'Dismiss all clears all suggestion pills', async ( { + admin, + editor, + page, + } ) => { + // Set strategy to allow_new so mock suggestions (new terms) pass through. + await setStrategy( admin, page, 'allow_new' ); + + await admin.createNewPost( { + title: 'Content Classification Dismiss All Test', + } ); + + // Add enough content. + await editor.insertBlock( { + name: 'core/paragraph', + attributes: { content: LONG_CONTENT }, + } ); + + await editor.saveDraft(); + await page.reload(); + + // Open the Tags panel. + await openTaxonomyPanel( editor, page, 'Tags' ); + + // Generate suggestions. + await page + .locator( '.ai-content-classification button', { + hasText: 'Suggest Tags', + } ) + .first() + .click(); + + // Wait for suggestions to appear. + await expect( + page.locator( '.ai-content-classification__suggestions' ).first() + ).toBeVisible(); + + // Click "Dismiss all". + await page + .locator( '.ai-content-classification__actions button', { + hasText: 'Dismiss all', + } ) + .first() + .click(); + + // Suggestions should be cleared and the generate button should reappear. + await expect( + page.locator( '.ai-content-classification__suggestions' ).first() + ).not.toBeVisible(); + + await expect( + page + .locator( '.ai-content-classification button', { + hasText: 'Suggest Tags', + } ) + .first() + ).toBeVisible(); + } ); + + test( 'UI is hidden when experiments are globally disabled', async ( { + admin, + editor, + page, + } ) => { + // Globally turn off Experiments. + await disableExperiments( admin, page ); + + // Create a new post with content. + await admin.createNewPost( { + title: 'Content Classification Globally Disabled Test', + } ); + + await editor.insertBlock( { + name: 'core/paragraph', + attributes: { content: LONG_CONTENT }, + } ); + + // Open the Tags panel. + await openTaxonomyPanel( editor, page, 'Tags' ); + + // The content classification UI should not be present. + await expect( + page.locator( '.ai-content-classification' ) + ).toHaveCount( 0 ); + } ); + + test( 'UI is hidden when experiment is individually disabled', async ( { + admin, + editor, + page, + } ) => { + // Disable the Content Classification Experiment. + await disableExperiment( admin, page, EXPERIMENT_ID ); + + // Create a new post with content. + await admin.createNewPost( { + title: 'Content Classification Disabled Test', + } ); + + await editor.insertBlock( { + name: 'core/paragraph', + attributes: { content: LONG_CONTENT }, + } ); + + // Open the Tags panel. + await openTaxonomyPanel( editor, page, 'Tags' ); + + // The content classification UI should not be present. + await expect( + page.locator( '.ai-content-classification' ) + ).toHaveCount( 0 ); + } ); +} ); diff --git a/webpack.config.js b/webpack.config.js index bb9292578..8743c3719 100644 --- a/webpack.config.js +++ b/webpack.config.js @@ -19,6 +19,11 @@ module.exports = { 'src/admin/settings', 'index.scss' ), + 'experiments/content-classification': path.resolve( + process.cwd(), + 'src/experiments/content-classification', + 'index.tsx' + ), 'experiments/abilities-explorer': path.resolve( process.cwd(), 'src/experiments/abilities-explorer',