Skip to content

Review fixes: MyST markup, language, and grammar #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
rkratky wants to merge 1 commit into
base: springai-basic
Choose a base branch
from
Open

Review fixes: MyST markup, language, and grammar #1

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
161 changes: 111 additions & 50 deletions docs/tutorials/springai-basic.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,61 +6,83 @@ myst:

(springai-basic)=
# Developing an LLM chat-client with Spring AI

This tutorial aims to provide a quick, hands-on introduction to Spring AI. Starting with [devpack-for-spring](https://www.youtube.com/watch?v=aHYwD8yTUZA), and using a locally installed large language model, the tutorial demonstrates how easy it is to create an LLM chat-client using Spring AI.


## Introduction to Spring AI
The [Spring AI](https://docs.spring.io/spring-ai/reference/index.html) project aims to streamline development of applications that incorporate artificial intelligence functionality. It is an extension to the Spring and Spring Boot frameworks. It includes abstractions for various artificial intelligence interactions and protocols, that hide complexity and provide an easy programming interface for Java applications. Spring AI offers a consistent interface across AI providers and models, making your code portable.

The [Spring AI](https://docs.spring.io/spring-ai/reference/index.html) project aims to streamline development of applications that incorporate artificial intelligence functionality. It is an extension to the Spring and Spring Boot frameworks. It includes abstractions for various artificial intelligence interactions and protocols that hide complexity and provide an easy programming interface for Java applications. Spring AI offers a consistent interface across AI providers and models, making your code portable.

Spring AI's core abstractions include:

- [ChatClient API](https://docs.spring.io/spring-ai/reference/api/chatclient.html)
- [Structured output converters](https://docs.spring.io/spring-ai/reference/api/structured-output-converter.html)
- [Prompt templates](https://docs.spring.io/spring-ai/reference/api/prompt.html)

The advanced features include support for:

- [Retrieval-Augmented Generation (RAG)](https://docs.spring.io/spring-ai/reference/api/chatclient.html#_retrieval_augmented_generation)
- [Vector databases](https://docs.spring.io/spring-ai/reference/api/vectordbs.html)
- [Tool/Function calling](https://docs.spring.io/spring-ai/reference/api/tools.html)
- [Model Context Protocol (MCP)](https://docs.spring.io/spring-ai/reference/api/mcp/mcp-overview.html)
- [Observability](https://docs.spring.io/spring-ai/reference/observability/index.html)

The above feature-set makes LLM interactions effortless, and other complex interactions relatively easier.

The above feature-set makes LLM interactions effortless and other complex interactions relatively easier.


## An LLM chat-client using Spring AI
In this tutorial, Spring AI's ChatClient API is used to create a simple LLM chat-client. The application offers a REST API which may be used to send questions to a configured LLM, and relay the answers as responses. We will use Qwen 2.5 VL model installed locally using an inference snap. The `devpack-for-spring` CLI is used to initialize this application.

In this tutorial, Spring AI's ChatClient API is used to create a simple LLM chat-client. The application offers a REST API to send questions to a configured LLM, and relay the answers as responses. We use Qwen 2.5 VL model installed locally using an inference snap. The {command}`devpack-for-spring` CLI is used to initialize this application.


### Installing Qwen 2.5 VL locally
Install Qwen VL snap for the 2.5 track:
```shell

Install {pkg}`qwen-vl` for the 2.5 track:

```{terminal}

sudo snap install qwen-vl --channel 2.5/beta
```

You may check the status using the `status` command. We will revisit this in a later section.
```
To check the status, use the {command}`status` command. We revisit this in a later section.

```{terminal}

qwen-vl status
```

If the `status` command reports an openai endpoint, we have Qwen 2.5 VL installed and the server used to inference with it, up and running. To learn more, refer to the [documentation](https://documentation.ubuntu.com/inference-snaps/) on inference snaps.
When the {command}`status` command reports an openai endpoint, Qwen 2.5 VL is installed and the server used to perform inference with it is up and running. To learn more, refer to the [documentation](https://documentation.ubuntu.com/inference-snaps/) on inference snaps.


### Initializing the chat-client project

Install {pkg}`devpack-for-spring` (skip this step when already installed):

```{terminal}

### Initialzing the chat-client project
Install `devpack-for-spring` if you don't have it installed:
```
sudo snap install devpack-for-spring --classic
```

Install the content snap for Spring AI:
```

```{terminal}

devpack-for-spring snap install content-for-spring-ai-11
```

:::{important}
The `content-for-spring-ai-11` snap installs Spring AI 1.1.X, which is compatible with Spring Boot 3.5.X.
The {pkg}`content-for-spring-ai-11` snap installs Spring AI 1.1.X, which is compatible with Spring Boot 3.5.X.
:::

A Spring Boot and Spring AI project may be initialized using `devpack-for-spring`'s CLI wizard.
To initialize a Spring Boot and Spring AI project, use {command}`devpack-for-spring`'s CLI wizard.

![spring-ai-init](../images/spring-ai-init.gif)

Alternatively, you may initialize it using this command:
```shell
Alternatively, initialize it using this command:

```{terminal}

devpack-for-spring boot start \
--path $PWD/chat-client \
--project gradle-project \
Expand All @@ -75,44 +97,58 @@ devpack-for-spring boot start \
--dependencies spring-ai-openai,web \
--packaging jar \
--java-version 21
```
```

:::{important}
Please note the chosen dependencies. The spring-ai-openai dependency supports interaction with OpenAI-compatible models. The web dependency supports the creation of an HTTP endpoint.
:::
Note the chosen dependencies. The {pkg}`spring-ai-openai` dependency supports interaction with OpenAI-compatible models. The {pkg}`web` dependency supports the creation of an HTTP endpoint.
:::


### Implementing the chat-client backend
After successful project initialization, you should find the file `src/main/java/demo/chatclient/ChatClientApplication.java`. The code in this file marks the primary entry point of your application. We now add code related to the LLM chat-client application.

After successful project initialization, you find the file {file}`src/main/java/demo/chatclient/ChatClientApplication.java`. The code in this file marks the primary entry point of your application. We now add code related to the LLM chat-client application.

Let us first define the basic abstractions.

File `src/main/java/demo/chatclient/Question.java` defines the Question type.
```java
File {file}`src/main/java/demo/chatclient/Question.java` defines the Question type.

```{code-block} java
:caption: `src/main/java/demo/chatclient/Question.java`

package demo.chatclient;

public record Question(String question) {}
```

File `src/main/java/demo/chatclient/Answer.java` defines the Answer type.
```java
File {file}`src/main/java/demo/chatclient/Answer.java` defines the Answer type.

```{code-block} java
:caption: `src/main/java/demo/chatclient/Answer.java`

package demo.chatclient;

public record Answer(String answer) {}
```

File `src/main/java/demo/chatclient/DemoChatClient.java` defines the basic chat-client interface.
```java
File {file}`src/main/java/demo/chatclient/DemoChatClient.java` defines the basic chat-client interface.

```{code-block} java
:caption: `src/main/java/demo/chatclient/DemoChatClient.java`

package demo.chatclient;

public interface DemoChatClient {
Answer askQuestion(Question question);
}
```

We then add a service implementing the above interface, and a controller that defines the REST endpoint.
We then add a service implementing the above interface and a controller that defines the REST endpoint.

File {file}`src/main/java/demo/chatclient/DemoChatService.java` implements the DemoChatClient interface.

```{code-block} java
:caption: `src/main/java/demo/chatclient/DemoChatService.java`

File `src/main/java/demo/chatclient/DemoChatService.java` implements the DemoChatClient interface.
```java
package demo.chatclient;

import org.springframework.ai.chat.client.ChatClient;
Expand All @@ -139,8 +175,11 @@ public class DemoChatService implements DemoChatClient {
}
```

File `src/main/java/demo/chatclient/DemoChatController.java` adds the RestController to create an endpoint named `/ask`
```java
File {file}`src/main/java/demo/chatclient/DemoChatController.java` adds the RestController to create an endpoint named `/ask`

```{code-block} java
:caption: `src/main/java/demo/chatclient/DemoChatController.java`

package demo.chatclient;

import org.springframework.web.bind.annotation.PostMapping;
Expand All @@ -163,11 +202,14 @@ public class DemoChatController {
}
```


### Adding a simple HTML frontend

Paste the following code into `src/main/resources/static/index.html`
Paste the following code into {file}`src/main/resources/static/index.html`:

```{code-block} html
:caption: `src/main/resources/static/index.html`

```html
<!DOCTYPE html>
<html lang="en">
<head>
Expand Down Expand Up @@ -240,54 +282,73 @@ Paste the following code into `src/main/resources/static/index.html`
</html>
```


### Configuring the LLM to be used

We installed the Qwen 2.5 VL model through the `qwen-vl` inference snap. To configure Spring AI to use this model, we need to find the model-id and its openai endpoint.
We installed the Qwen 2.5 VL model through the {pkg}`qwen-vl` inference snap. To configure Spring AI to use this model, we need to find the model-id and its openai endpoint.

```{terminal}

qwen-vl status

```shell
$ qwen-vl status
engine: intel-gpu
endpoints:
openai: http://localhost:8326/v3
```

The openai endpoint is `http://localhost:8326/v3`.

```shell
$ curl http://localhost:8326/v3/models 2>/dev/null | jq | grep "id"
```{terminal}

curl http://localhost:8326/v3/models 2>/dev/null | jq | grep "id"

"id": "Qwen2.5-VL-3B-Instruct-ov-int4",
```

The model-id is `Qwen2.5-VL-3B-Instruct-ov-int4`.

Use the above values to define properties in `src/main/resources/application.properties`:
```properties
Use the above values to define properties in {file}`src/main/resources/application.properties`:

```{code-block} properties
:caption: `src/main/resources/application.properties`

spring.application.name=chat-client
spring.ai.openai.base-url=http://localhost:8326/v3
spring.ai.openai.api-key=ignored
spring.ai.openai.chat.options.model=Qwen2.5-VL-3B-Instruct-ov-int4
```

:::{attention}
Because we are using a local inference model, we do not need to provide an API key. The `spring.ai.openai.api-key` property is mandatory to access online models.
:::


### Launching and using the chat-client

We are now set to launch the chat-client application and run simple prompts.

Before launching the application, ensure you have `openjdk-21-jdk` installed.
```shell
Before launching the application, ensure you have {pkg}`openjdk-21-jdk` installed:

```{terminal}

sudo apt install openjdk-21-jdk
```

Launch the application using this command:
```shell

```{terminal}

./gradlew bootRun
```

If all goes well, you should see a log message indicating the application started. No exceptions must be logged.
```
A successful launch produces a log message indicating the application started. No exceptions are logged.

```{terminal}
:output-only:

2026-06-18T21:17:05.427+05:30 INFO 977098 --- [chat-client] [ main] demo.chatclient.ChatClientApplication : Started ChatClientApplication in 1.364 seconds (process running for 1.584)
```

You may now open `http://localhost:8080` in a browser to use the chat-client. A sample interaction is captured below:
Open `http://localhost:8080` in a browser to use the chat-client. A sample interaction is captured below:

![chat-client-interaction](../images/chat-client.gif)
![chat-client-interaction](../images/chat-client.gif)