added tokenization tab to gradio app

[ParamThakkar123]

Changes

• Enhanced inference-ollama-gradio-chat/ with SSH tunneling support for remote Ollama instances
• Added tokenization preview interface to visualize how the model tokenizes text
• Updated gradio_chat.py to include SSH setup, tokenization function, and tabbed interface
• Modified task.yaml to include additional dependencies (sshtunnel, requests) and updated run commands

Features

• Chat interface for interacting with Ollama models
• SSH tunneling for accessing remote Ollama servers
• Tokenization preview to see how text is broken into tokens with logprobs
• Tabbed Gradio UI combining chat and tokenization features

Parameters

• OLLAMA_BASE_URL: Ollama server URL (default: http://127.0.0.1:11434)
• MODEL_NAME: Model name to use (default: llama2)
• SSH parameters: SSH_HOST, SSH_USER, SSH_KEY_PATH, SSH_PORT for remote access

How to Test

1. In TransformerLab, select the 'inference-ollama-gradio-chat' task
2. Configure parameters (especially SSH settings if using remote Ollama)
3. Run the task
4. Access the Gradio interface and test both chat and tokenization tabs

[mina-parham]

It seems there is a conflict we need to resolve. could you check that? I’m afraid I might break something if I try to fix it