Add SSL verification and chat model support for LiteLLM provider#130
Open
NiklasPhabian wants to merge 1 commit into
Open
Add SSL verification and chat model support for LiteLLM provider#130NiklasPhabian wants to merge 1 commit into
NiklasPhabian wants to merge 1 commit into
Conversation
- Add configurable verify_ssl property (defaults to true for security) - Set litellm.ssl_verify module setting based on user configuration - Fix inline completions to support chat-only models like Anthropic - Implement FIM-to-chat fallback mechanism for compatibility - Strip markdown code blocks from chat model responses Fixes plmbr#128 Fixes plmbr#129
mbektas
reviewed
Apr 8, 2026
| ) | ||
| return litellm_resp.choices[0].message.content | ||
| except Exception as e: | ||
| # Fall back to chat format for models that don't support FIM |
Collaborator
There was a problem hiding this comment.
@NiklasPhabian can you use the same logic / prompts as in #117?
mbektas
reviewed
Apr 8, 2026
| return litellm_resp.choices[0].message.content | ||
| except Exception as e: | ||
| # Fall back to chat format for models that don't support FIM | ||
| prompt_text = f"You are an autocomplete engine. Complete the {language} code at the cursor position. " |
Collaborator
There was a problem hiding this comment.
it would be better to move this logic to a method.
and keep track of models (based on model_id in a set) that failed, so that you can directly use the chat completions next time.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds two improvements to the LiteLLM compatible provider:
Fixes #128
Fixes #129
Changes
SSL Verification (Issue #128)
verify_sslproperty to bothLiteLLMCompatibleChatModelandLiteLLMCompatibleInlineCompletionModeltrue(secure by default)litellm.ssl_verifymodule setting before API calls"verify_ssl": "false"in their configChat Model Support for Inline Completions (Issue #129)
Testing
Tested with:
Backward Compatibility
verify_sslis optional and defaults totrue(secure)Usage Example
For users with self-signed certificates, add to your config:
{ "chat_model": { "properties": [ ... { "id": "verify_ssl", "name": "Verify SSL", "description": "Verify SSL certificates (disable for self-signed certificates)", "value": "false", "optional": true } ] } }