OpenAI stream response in java

Question

The OpenAI chat/completions API supports a stream response by passing stream=true. This enables the response to come in chunks of data rather than one full response. It will be similar to how ChatGPT outputs answers.

I see similar question for different languages or tech stack (like here). But I can't find a full example of how to implement that in java (or kotlin) with OkHttp client.

How do get OpenAI's API response in form of stream? Thanks!

Answer 1

I modified the code, now ChatGPT can fully realize streaming conversation in the console via Java.

import com.theokanning.openai.completion.chat.*;
import com.theokanning.openai.service.OpenAiService;
import io.reactivex.Flowable;
import java.util.*;

public class Starter {
    // Create a new OpenAiService instance with the given API key
    public static OpenAiService service = new OpenAiService("Your API Token");

    public static void main(String[] args) {
        System.out.println("Streaming chat completion...");
        Scanner scanner = new Scanner(System.in);
        String userInput = scanner.nextLine();
        // Create a list of ChatMessage objects
        List<ChatMessage> message = new ArrayList<ChatMessage>();
        message.add(new ChatMessage(ChatMessageRole.USER.value(), userInput));
        // Create a ChatCompletionRequest object
        ChatCompletionRequest chatCompletionRequest;
        boolean running = true;
        // Run the loop until the user enters "exit"
        while (running) {
            chatCompletionRequest = ChatCompletionRequest
                    .builder()
                    .model("gpt-3.5-turbo")
                    .messages(message)
                    .n(1)
                    .maxTokens(500)
                    .logitBias(Collections.emptyMap())
                    .build();
            // Create a Flowable object to stream the chat completion
            Flowable<ChatCompletionChunk> flowableResult = service.streamChatCompletion(chatCompletionRequest);
            // Create a StringBuilder object to store the result
            StringBuilder buffer = new StringBuilder();
            // Subscribe to the Flowable object and print the result
            flowableResult.subscribe(chunk -> {
                chunk.getChoices().forEach(choice -> {
                    String result = choice.getMessage().getContent();
                    if (result != null) {
                        buffer.append(result);
                        System.out.print(choice.getMessage().getContent());
                    }
                });
            }, Throwable::printStackTrace, () -> System.out.println());
            // Get the user input
            userInput = scanner.nextLine();
            // Add the user input to the list of ChatMessage objects
            message.add(new ChatMessage(ChatMessageRole.SYSTEM.value(), buffer.toString()));
            message.add(new ChatMessage(ChatMessageRole.USER.value(), userInput));
            // Exit the loop if the user enters "exit"
            if (userInput.equals("exit")) {
                running = false;
            }
        }
        scanner.close();
        service.shutdownExecutor();
    }
}