@@ -40,22 +40,22 @@ In addition to OpenAI, this library supports many other LLM providers. For provi
40
40
| ----------| ------------------------| ---------------| -------------|
41
41
| [ OpenAI] ( https://platform.openai.com ) | Full | Standard + Responses API | Full API support |
42
42
| [ Azure OpenAI] ( https://azure.microsoft.com/en-us/products/ai-services/openai-service ) | Full | Standard + Responses API | OpenAI on Azure|
43
- | [ Azure AI] ( https://azure.microsoft.com/en-us/products/ai-studio ) | Varies | | Open-source models |
44
43
| [ Anthropic] ( https://www.anthropic.com/api ) | Implied | | Claude models |
45
- | [ Google Vertex AI] ( https://cloud.google.com/vertex-ai ) | Full | Yes | Gemini models |
46
- | [ Google Gemini] ( https://ai.google.dev/ ) (🔥 ** New** ) | Full | Yes | Google's models |
47
- | [ Groq] ( https://wow.groq.com/ ) | Only JSON object mode | Yes | Fast inference |
48
- | [ Grok] ( https://x.ai/ ) | Full | | x.AI models |
49
- | [ Fireworks AI] ( https://fireworks.ai/ ) | Only JSON object mode | | Cloud provider |
50
- | [ Octo AI] ( https://octo.ai/ ) | Only JSON object mode | | Cloud provider (obsolete) |
51
- | [ TogetherAI] ( https://www.together.ai/ ) | Only JSON object mode | | Cloud provider |
44
+ | [ Azure AI] ( https://azure.microsoft.com/en-us/products/ai-studio ) | Varies | | Open-source models |
52
45
| [ Cerebras] ( https://cerebras.ai/ ) | Only JSON object mode | | Fast inference |
53
- | [ Mistral] ( https://mistral.ai/ ) | Only JSON object mode | | Open-source leader |
54
46
| [ Deepseek] ( https://deepseek.com/ ) | Only JSON object mode | | Chinese provider |
55
- | [ Ollama] ( https://ollama.com/ ) | Varies | | Local LLMs |
56
47
| [ FastChat] ( https://github.com/lm-sys/FastChat ) | Varies | | Local LLMs |
48
+ | [ Fireworks AI] ( https://fireworks.ai/ ) | Only JSON object mode | | Cloud provider |
49
+ | [ Google Gemini] ( https://ai.google.dev/ ) (🔥 ** New** ) | Full | Yes | Google's models |
50
+ | [ Google Vertex AI] ( https://cloud.google.com/vertex-ai ) | Full | Yes | Gemini models |
51
+ | [ Grok] ( https://x.ai/ ) | Full | | x.AI models |
52
+ | [ Groq] ( https://wow.groq.com/ ) | Only JSON object mode | | Fast inference |
53
+ | [ Mistral] ( https://mistral.ai/ ) | Only JSON object mode | | Open-source leader |
57
54
| [ Novita] ( https://novita.ai/ ) (🔥 ** New** ) | Only JSON object mode | | Cloud provider |
55
+ | [ Octo AI] ( https://octo.ai/ ) | Only JSON object mode | | Cloud provider (obsolete) |
56
+ | [ Ollama] ( https://ollama.com/ ) | Varies | | Local LLMs |
58
57
| [ Perplexity Sonar] ( https://www.perplexity.ai/ ) (🔥 ** New** ) | Only implied | | Search-based AI |
58
+ | [ TogetherAI] ( https://www.together.ai/ ) | Only JSON object mode | | Cloud provider |
59
59
60
60
---
61
61
@@ -378,7 +378,7 @@ There is a new project [openai-scala-client-examples](./openai-examples/src/main
378
378
}
379
379
```
380
380
381
- - Create chat completion with json /structured output
381
+ - Create chat completion with ** JSON /structured output**
382
382
383
383
``` scala
384
384
val messages = Seq (
@@ -408,7 +408,7 @@ There is a new project [openai-scala-client-examples](./openai-examples/src/main
408
408
val jsonSchemaDef = JsonSchemaDef (
409
409
name = " capitals_response" ,
410
410
strict = true ,
411
- structure = schema
411
+ structure = capitalsSchema
412
412
)
413
413
414
414
service
@@ -427,7 +427,7 @@ There is a new project [openai-scala-client-examples](./openai-examples/src/main
427
427
}
428
428
```
429
429
430
- - Create chat completion with json /structured output using a handly implicit function (` createChatCompletionWithJSON[T] ` ) that handles JSON extraction with a potential repair, as well as deserialization to an object T.
430
+ - Create chat completion with ** JSON /structured output** using a handly implicit function (` createChatCompletionWithJSON[T] ` ) that handles JSON extraction with a potential repair, as well as deserialization to an object T.
431
431
432
432
``` scala
433
433
import io .cequence .openaiscala .service .OpenAIChatCompletionExtra ._
@@ -449,10 +449,12 @@ There is a new project [openai-scala-client-examples](./openai-examples/src/main
449
449
}
450
450
```
451
451
452
- - Failover
452
+ - ** Failover** to alternative models if the primary one fails
453
453
454
- ```
455
- private val messages = Seq(
454
+ ``` scala
455
+ import io .cequence .openaiscala .service .OpenAIChatCompletionExtra ._
456
+
457
+ val messages = Seq (
456
458
SystemMessage (" You are a helpful weather assistant." ),
457
459
UserMessage (" What is the weather like in Norway?" )
458
460
)
@@ -471,14 +473,235 @@ There is a new project [openai-scala-client-examples](./openai-examples/src/main
471
473
print(response.contentHead)
472
474
}
473
475
```
474
- - Failover with JSON/structred outuput
475
476
476
- - Responses API (basic)
477
+ - ** Failover ** with JSON/structured output
477
478
478
- - Responses API (tool use)
479
+ ``` scala
480
+ import io .cequence .openaiscala .service .OpenAIChatCompletionExtra ._
479
481
482
+ val capitalsSchema = JsonSchema .Object (
483
+ properties = Map (
484
+ " countries" -> JsonSchema .Array (
485
+ items = JsonSchema .Object (
486
+ properties = Map (
487
+ " country" -> JsonSchema .String (
488
+ description = Some (" The name of the country" )
489
+ ),
490
+ " capital" -> JsonSchema .String (
491
+ description = Some (" The capital city of the country" )
492
+ )
493
+ ),
494
+ required = Seq (" country" , " capital" )
495
+ )
496
+ )
497
+ ),
498
+ required = Seq (" countries" )
499
+ )
500
+
501
+ val jsonSchemaDef = JsonSchemaDef (
502
+ name = " capitals_response" ,
503
+ strict = true ,
504
+ structure = capitalsSchema
505
+ )
506
+
507
+ // Define the chat messages
508
+ val messages = Seq (
509
+ SystemMessage (" Give me the most populous capital cities in JSON format." ),
510
+ UserMessage (" List only african countries" )
511
+ )
480
512
513
+ // Call the service with failover support
514
+ service
515
+ .createChatCompletionWithJSON[JsObject ](
516
+ messages = messages,
517
+ settings = CreateChatCompletionSettings (
518
+ model = ModelId .o3_mini, // Primary model
519
+ max_tokens = Some (1000 ),
520
+ response_format_type = Some (ChatCompletionResponseFormatType .json_schema),
521
+ jsonSchema = Some (jsonSchemaDef)
522
+ ),
523
+ failoverModels = Seq (
524
+ ModelId .gpt_4_5_preview, // First fallback model
525
+ ModelId .gpt_4o // Second fallback model
526
+ ),
527
+ maxRetries = Some (3 ), // Maximum number of retries per model
528
+ retryOnAnyError = true , // Retry on any error, not just retryable ones
529
+ taskNameForLogging = Some (" capitals-query" ) // For better logging
530
+ )
531
+ .map { json =>
532
+ println(Json .prettyPrint(json))
533
+ }
534
+ ```
481
535
536
+ - ** Responses API** - basic usage with textual inputs / messages
537
+
538
+ ``` scala
539
+ import io .cequence .openaiscala .domain .responsesapi .Inputs
540
+
541
+ service
542
+ .createModelResponse(
543
+ Inputs .Text (" What is the capital of France?" )
544
+ )
545
+ .map { response =>
546
+ println(response.outputText.getOrElse(" N/A" ))
547
+ }
548
+ ```
549
+
550
+ ``` scala
551
+ import io .cequence .openaiscala .domain .responsesapi .Input
552
+
553
+ service
554
+ .createModelResponse(
555
+ Inputs .Items (
556
+ Input .ofInputSystemTextMessage(
557
+ " You are a helpful assistant. Be verbose and detailed and don't be afraid to use emojis."
558
+ ),
559
+ Input .ofInputUserTextMessage(" What is the capital of France?" )
560
+ )
561
+ )
562
+ .map { response =>
563
+ println(response.outputText.getOrElse(" N/A" ))
564
+ }
565
+ ```
566
+
567
+ - ** Responses API** - image input
568
+
569
+ ``` scala
570
+
571
+ import io .cequence .openaiscala .domain .responsesapi .{Inputs , Input }
572
+ import io .cequence .openaiscala .domain .responsesapi .InputMessageContent
573
+ import io .cequence .openaiscala .domain .ChatRole
574
+
575
+ service
576
+ .createModelResponse(
577
+ Inputs .Items (
578
+ Input .ofInputMessage(
579
+ Seq (
580
+ InputMessageContent .Text (" what is in this image?" ),
581
+ InputMessageContent .Image (
582
+ imageUrl = Some (
583
+ " https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg"
584
+ )
585
+ )
586
+ ),
587
+ role = ChatRole .User
588
+ )
589
+ )
590
+ )
591
+ .map { response =>
592
+ println(response.outputText.getOrElse(" N/A" ))
593
+ }
594
+ ```
595
+
596
+ - ** Responses API** - tool use (file search)
597
+
598
+ ``` scala
599
+
600
+ service
601
+ .createModelResponse(
602
+ Inputs .Text (" What are the attributes of an ancient brown dragon?" ),
603
+ settings = CreateModelResponseSettings (
604
+ model = ModelId .gpt_4o_2024_08_06,
605
+ tools = Seq (
606
+ FileSearchTool (
607
+ vectorStoreIds = Seq (" vs_1234567890" ),
608
+ maxNumResults = Some (20 ),
609
+ filters = None ,
610
+ rankingOptions = None
611
+ )
612
+ )
613
+ )
614
+ )
615
+ .map { response =>
616
+ println(response.outputText.getOrElse(" N/A" ))
617
+
618
+ // citations
619
+ val citations : Seq [Annotation .FileCitation ] = response.outputMessageContents.collect {
620
+ case e : OutputText =>
621
+ e.annotations.collect { case citation : Annotation .FileCitation => citation }
622
+ }.flatten
623
+
624
+ println(" Citations:" )
625
+ citations.foreach { citation =>
626
+ println(s " ${citation.fileId} - ${citation.filename}" )
627
+ }
628
+ }
629
+ ```
630
+
631
+ - ** Responses API** - tool use (web search)
632
+
633
+ ``` scala
634
+ service
635
+ .createModelResponse(
636
+ Inputs .Text (" What was a positive news story from today?" ),
637
+ settings = CreateModelResponseSettings (
638
+ model = ModelId .gpt_4o_2024_08_06,
639
+ tools = Seq (WebSearchTool ())
640
+ )
641
+ )
642
+ .map { response =>
643
+ println(response.outputText.getOrElse(" N/A" ))
644
+
645
+ // citations
646
+ val citations : Seq [Annotation .UrlCitation ] = response.outputMessageContents.collect {
647
+ case e : OutputText =>
648
+ e.annotations.collect { case citation : Annotation .UrlCitation => citation }
649
+ }.flatten
650
+
651
+ println(" Citations:" )
652
+ citations.foreach { citation =>
653
+ println(s " ${citation.title} - ${citation.url}" )
654
+ }
655
+ }
656
+ ```
657
+
658
+ - ** Responses API** - tool use (function call)
659
+
660
+ ``` scala
661
+ service
662
+ .createModelResponse(
663
+ Inputs .Text (" What is the weather like in Boston today?" ),
664
+ settings = CreateModelResponseSettings (
665
+ model = ModelId .gpt_4o_2024_08_06,
666
+ tools = Seq (
667
+ FunctionTool (
668
+ name = " get_current_weather" ,
669
+ parameters = JsonSchema .Object (
670
+ properties = Map (
671
+ " location" -> JsonSchema .String (
672
+ description = Some (" The city and state, e.g. San Francisco, CA" )
673
+ ),
674
+ " unit" -> JsonSchema .String (
675
+ `enum` = Seq (" celsius" , " fahrenheit" )
676
+ )
677
+ ),
678
+ required = Seq (" location" , " unit" )
679
+ ),
680
+ description = Some (" Get the current weather in a given location" ),
681
+ strict = true
682
+ )
683
+ ),
684
+ toolChoice = Some (ToolChoice .Mode .Auto )
685
+ )
686
+ )
687
+ .map { response =>
688
+ val functionCall = response.outputFunctionCalls.headOption
689
+ .getOrElse(throw new RuntimeException (" No function call output found" ))
690
+
691
+ println(
692
+ s """ Function Call Details:
693
+ |Name: ${functionCall.name}
694
+ |Arguments: ${functionCall.arguments}
695
+ |Call ID: ${functionCall.callId}
696
+ |ID: ${functionCall.id}
697
+ |Status: ${functionCall.status}""" .stripMargin
698
+ )
699
+
700
+ val toolsUsed = response.tools.map(_.typeString)
701
+
702
+ println(s " ${toolsUsed.size} tools used: ${toolsUsed.mkString(" , " )}" )
703
+ }
704
+ ```
482
705
483
706
- Count expected used tokens before calling ` createChatCompletions ` or ` createChatFunCompletions ` , this helps you select proper model and reduce costs. This is an experimental feature and it may not work for all models. Requires ` openai-scala-count-tokens ` lib.
484
707
@@ -598,7 +821,6 @@ Note that the adapters can be arbitrarily combined/stacked.
598
821
Some (println(_)) // simple logging
599
822
)
600
823
```
601
-
602
824
- ** Retry** on a specific function using [ RetryHelpers] ( ./openai-core/src/main/scala/io/cequence/openaiscala/RetryHelpers.scala ) directly
603
825
604
826
``` scala
@@ -710,4 +932,5 @@ This project is open-source and welcomes any contribution or feedback ([here](ht
710
932
711
933
Development of this library has been supported by [ <img src =" https://cequence.io/favicon-16x16.png " > - Cequence.io] ( https://cequence.io ) - ` The future of contracting `
712
934
713
- Created and maintained by [ Peter Banda] ( https://peterbanda.net ) .
935
+ Created and maintained by [ Peter Banda] ( https://peterbanda.net ) .
936
+
0 commit comments