diff --git a/docs/tutorials/springai-basic.md b/docs/tutorials/springai-basic.md index c29585c..9e8495a 100644 --- a/docs/tutorials/springai-basic.md +++ b/docs/tutorials/springai-basic.md @@ -6,61 +6,83 @@ myst: (springai-basic)= # Developing an LLM chat-client with Spring AI + This tutorial aims to provide a quick, hands-on introduction to Spring AI. Starting with [devpack-for-spring](https://www.youtube.com/watch?v=aHYwD8yTUZA), and using a locally installed large language model, the tutorial demonstrates how easy it is to create an LLM chat-client using Spring AI. + ## Introduction to Spring AI -The [Spring AI](https://docs.spring.io/spring-ai/reference/index.html) project aims to streamline development of applications that incorporate artificial intelligence functionality. It is an extension to the Spring and Spring Boot frameworks. It includes abstractions for various artificial intelligence interactions and protocols, that hide complexity and provide an easy programming interface for Java applications. Spring AI offers a consistent interface across AI providers and models, making your code portable. + +The [Spring AI](https://docs.spring.io/spring-ai/reference/index.html) project aims to streamline development of applications that incorporate artificial intelligence functionality. It is an extension to the Spring and Spring Boot frameworks. It includes abstractions for various artificial intelligence interactions and protocols that hide complexity and provide an easy programming interface for Java applications. Spring AI offers a consistent interface across AI providers and models, making your code portable. Spring AI's core abstractions include: + - [ChatClient API](https://docs.spring.io/spring-ai/reference/api/chatclient.html) - [Structured output converters](https://docs.spring.io/spring-ai/reference/api/structured-output-converter.html) - [Prompt templates](https://docs.spring.io/spring-ai/reference/api/prompt.html) - + The advanced features include support for: + - [Retrieval-Augmented Generation (RAG)](https://docs.spring.io/spring-ai/reference/api/chatclient.html#_retrieval_augmented_generation) - [Vector databases](https://docs.spring.io/spring-ai/reference/api/vectordbs.html) - [Tool/Function calling](https://docs.spring.io/spring-ai/reference/api/tools.html) - [Model Context Protocol (MCP)](https://docs.spring.io/spring-ai/reference/api/mcp/mcp-overview.html) - [Observability](https://docs.spring.io/spring-ai/reference/observability/index.html) - - The above feature-set makes LLM interactions effortless, and other complex interactions relatively easier. + + The above feature-set makes LLM interactions effortless and other complex interactions relatively easier. + ## An LLM chat-client using Spring AI -In this tutorial, Spring AI's ChatClient API is used to create a simple LLM chat-client. The application offers a REST API which may be used to send questions to a configured LLM, and relay the answers as responses. We will use Qwen 2.5 VL model installed locally using an inference snap. The `devpack-for-spring` CLI is used to initialize this application. + +In this tutorial, Spring AI's ChatClient API is used to create a simple LLM chat-client. The application offers a REST API to send questions to a configured LLM, and relay the answers as responses. We use Qwen 2.5 VL model installed locally using an inference snap. The {command}`devpack-for-spring` CLI is used to initialize this application. + ### Installing Qwen 2.5 VL locally -Install Qwen VL snap for the 2.5 track: -```shell + +Install {pkg}`qwen-vl` for the 2.5 track: + +```{terminal} + sudo snap install qwen-vl --channel 2.5/beta ``` -You may check the status using the `status` command. We will revisit this in a later section. -``` +To check the status, use the {command}`status` command. We revisit this in a later section. + +```{terminal} + qwen-vl status ``` -If the `status` command reports an openai endpoint, we have Qwen 2.5 VL installed and the server used to inference with it, up and running. To learn more, refer to the [documentation](https://documentation.ubuntu.com/inference-snaps/) on inference snaps. +When the {command}`status` command reports an openai endpoint, Qwen 2.5 VL is installed and the server used to perform inference with it is up and running. To learn more, refer to the [documentation](https://documentation.ubuntu.com/inference-snaps/) on inference snaps. + + +### Initializing the chat-client project + +Install {pkg}`devpack-for-spring` (skip this step when already installed): + +```{terminal} -### Initialzing the chat-client project -Install `devpack-for-spring` if you don't have it installed: -``` sudo snap install devpack-for-spring --classic ``` Install the content snap for Spring AI: -``` + +```{terminal} + devpack-for-spring snap install content-for-spring-ai-11 ``` + :::{important} -The `content-for-spring-ai-11` snap installs Spring AI 1.1.X, which is compatible with Spring Boot 3.5.X. +The {pkg}`content-for-spring-ai-11` snap installs Spring AI 1.1.X, which is compatible with Spring Boot 3.5.X. ::: -A Spring Boot and Spring AI project may be initialized using `devpack-for-spring`'s CLI wizard. +To initialize a Spring Boot and Spring AI project, use {command}`devpack-for-spring`'s CLI wizard. ![spring-ai-init](../images/spring-ai-init.gif) -Alternatively, you may initialize it using this command: -```shell +Alternatively, initialize it using this command: + +```{terminal} + devpack-for-spring boot start \ --path $PWD/chat-client \ --project gradle-project \ @@ -75,33 +97,44 @@ devpack-for-spring boot start \ --dependencies spring-ai-openai,web \ --packaging jar \ --java-version 21 - ``` - +``` + :::{important} -Please note the chosen dependencies. The spring-ai-openai dependency supports interaction with OpenAI-compatible models. The web dependency supports the creation of an HTTP endpoint. -::: +Note the chosen dependencies. The {pkg}`spring-ai-openai` dependency supports interaction with OpenAI-compatible models. The {pkg}`web` dependency supports the creation of an HTTP endpoint. +::: + ### Implementing the chat-client backend -After successful project initialization, you should find the file `src/main/java/demo/chatclient/ChatClientApplication.java`. The code in this file marks the primary entry point of your application. We now add code related to the LLM chat-client application. + +After successful project initialization, you find the file {file}`src/main/java/demo/chatclient/ChatClientApplication.java`. The code in this file marks the primary entry point of your application. We now add code related to the LLM chat-client application. Let us first define the basic abstractions. -File `src/main/java/demo/chatclient/Question.java` defines the Question type. -```java +File {file}`src/main/java/demo/chatclient/Question.java` defines the Question type. + +```{code-block} java +:caption: `src/main/java/demo/chatclient/Question.java` + package demo.chatclient; public record Question(String question) {} ``` -File `src/main/java/demo/chatclient/Answer.java` defines the Answer type. -```java +File {file}`src/main/java/demo/chatclient/Answer.java` defines the Answer type. + +```{code-block} java +:caption: `src/main/java/demo/chatclient/Answer.java` + package demo.chatclient; public record Answer(String answer) {} ``` -File `src/main/java/demo/chatclient/DemoChatClient.java` defines the basic chat-client interface. -```java +File {file}`src/main/java/demo/chatclient/DemoChatClient.java` defines the basic chat-client interface. + +```{code-block} java +:caption: `src/main/java/demo/chatclient/DemoChatClient.java` + package demo.chatclient; public interface DemoChatClient { @@ -109,10 +142,13 @@ public interface DemoChatClient { } ``` -We then add a service implementing the above interface, and a controller that defines the REST endpoint. +We then add a service implementing the above interface and a controller that defines the REST endpoint. + +File {file}`src/main/java/demo/chatclient/DemoChatService.java` implements the DemoChatClient interface. + +```{code-block} java +:caption: `src/main/java/demo/chatclient/DemoChatService.java` -File `src/main/java/demo/chatclient/DemoChatService.java` implements the DemoChatClient interface. -```java package demo.chatclient; import org.springframework.ai.chat.client.ChatClient; @@ -139,8 +175,11 @@ public class DemoChatService implements DemoChatClient { } ``` -File `src/main/java/demo/chatclient/DemoChatController.java` adds the RestController to create an endpoint named `/ask` -```java +File {file}`src/main/java/demo/chatclient/DemoChatController.java` adds the RestController to create an endpoint named `/ask` + +```{code-block} java +:caption: `src/main/java/demo/chatclient/DemoChatController.java` + package demo.chatclient; import org.springframework.web.bind.annotation.PostMapping; @@ -163,11 +202,14 @@ public class DemoChatController { } ``` + ### Adding a simple HTML frontend -Paste the following code into `src/main/resources/static/index.html` +Paste the following code into {file}`src/main/resources/static/index.html`: + +```{code-block} html +:caption: `src/main/resources/static/index.html` -```html @@ -240,54 +282,73 @@ Paste the following code into `src/main/resources/static/index.html` ``` + ### Configuring the LLM to be used -We installed the Qwen 2.5 VL model through the `qwen-vl` inference snap. To configure Spring AI to use this model, we need to find the model-id and its openai endpoint. +We installed the Qwen 2.5 VL model through the {pkg}`qwen-vl` inference snap. To configure Spring AI to use this model, we need to find the model-id and its openai endpoint. + +```{terminal} + +qwen-vl status -```shell -$ qwen-vl status engine: intel-gpu endpoints: openai: http://localhost:8326/v3 ``` + The openai endpoint is `http://localhost:8326/v3`. -```shell -$ curl http://localhost:8326/v3/models 2>/dev/null | jq | grep "id" +```{terminal} + +curl http://localhost:8326/v3/models 2>/dev/null | jq | grep "id" + "id": "Qwen2.5-VL-3B-Instruct-ov-int4", ``` + The model-id is `Qwen2.5-VL-3B-Instruct-ov-int4`. -Use the above values to define properties in `src/main/resources/application.properties`: -```properties +Use the above values to define properties in {file}`src/main/resources/application.properties`: + +```{code-block} properties +:caption: `src/main/resources/application.properties` + spring.application.name=chat-client spring.ai.openai.base-url=http://localhost:8326/v3 spring.ai.openai.api-key=ignored spring.ai.openai.chat.options.model=Qwen2.5-VL-3B-Instruct-ov-int4 ``` + :::{attention} Because we are using a local inference model, we do not need to provide an API key. The `spring.ai.openai.api-key` property is mandatory to access online models. ::: ### Launching and using the chat-client + We are now set to launch the chat-client application and run simple prompts. -Before launching the application, ensure you have `openjdk-21-jdk` installed. -```shell +Before launching the application, ensure you have {pkg}`openjdk-21-jdk` installed: + +```{terminal} + sudo apt install openjdk-21-jdk ``` Launch the application using this command: -```shell + +```{terminal} + ./gradlew bootRun ``` -If all goes well, you should see a log message indicating the application started. No exceptions must be logged. -``` +A successful launch produces a log message indicating the application started. No exceptions are logged. + +```{terminal} +:output-only: + 2026-06-18T21:17:05.427+05:30 INFO 977098 --- [chat-client] [ main] demo.chatclient.ChatClientApplication : Started ChatClientApplication in 1.364 seconds (process running for 1.584) ``` -You may now open `http://localhost:8080` in a browser to use the chat-client. A sample interaction is captured below: +Open `http://localhost:8080` in a browser to use the chat-client. A sample interaction is captured below: -![chat-client-interaction](../images/chat-client.gif) \ No newline at end of file +![chat-client-interaction](../images/chat-client.gif)