README
¶
multimodal
Package for making calls to multimodal prompts in Google Cloud easier to deal with.
Example use
package main
import (
"fmt"
"log"
"github.com/xyproto/multimodal"
)
func main() {
// Select the model and temperature
mm := multimodal.New("gemini-1.0-pro-vision", 0.4)
// Build a prompt
mm.AddImage("frog.png")
mm.AddURI("gs://generativeai-downloads/images/scones.jpg")
mm.AddText("describe what is common for these two images")
// Use your location and project ID for a multimodal Vertex AI model in Google Cloud
const location = "us-central1"
const projectID = "123412341234"
// Submit the prompt and get a reponse
response, err := mm.Submit(projectID, location)
if err != nil {
log.Fatalln(err)
}
fmt.Println(response)
}
General info
- Version: 1.2.2
- License: Apache2
Documentation
¶
Overview ¶
Package multimodal abstracts the genai multimodal prompt building
Index ¶
- type MultiModal
- func (mm *MultiModal) AddData(mimeType string, data []byte)
- func (mm *MultiModal) AddImage(filename string) error
- func (mm *MultiModal) AddText(prompt string)
- func (mm *MultiModal) AddURI(URI string)
- func (mm *MultiModal) AddURL(URL string) error
- func (mm *MultiModal) MustAddImage(filename string)
- func (mm *MultiModal) SetTrim(trim bool)
- func (mm *MultiModal) SetVerbose(verbose bool)
- func (mm *MultiModal) Submit(projectID, location string) (string, error)
- func (mm *MultiModal) SubmitToClient(ctx context.Context, client *genai.Client) (string, error)
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type MultiModal ¶
type MultiModal struct {
// contains filtered or unexported fields
}
MultiModal represents multimodal prompt parts + configuration
func New ¶
func New(modelName string, temperature float32) *MultiModal
New creates a new MultiModal instance with a specified model name and temperature, initializing it with default values for parts, trim, and verbose settings.
func (*MultiModal) AddData ¶ added in v1.2.0
func (mm *MultiModal) AddData(mimeType string, data []byte)
AddData adds arbitrary data with a specified MIME type to the parts of the MultiModal instance.
func (*MultiModal) AddImage ¶
func (mm *MultiModal) AddImage(filename string) error
AddImage reads an image from a file, prepares it for processing, and adds it to the list of parts to be used by the model. It supports verbose logging of operations if enabled.
func (*MultiModal) AddText ¶
func (mm *MultiModal) AddText(prompt string)
AddText adds a textual part to the MultiModal instance.
func (*MultiModal) AddURI ¶
func (mm *MultiModal) AddURI(URI string)
AddURI adds a file part to the MultiModal instance from a Google Cloud URI, allowing for integration with cloud resources directly. Example URI: "gs://generativeai-downloads/images/scones.jpg"
func (*MultiModal) AddURL ¶ added in v1.1.0
func (mm *MultiModal) AddURL(URL string) error
AddURL downloads the file from the given URL, identifies the MIME type, and adds it as a genai.Part.
func (*MultiModal) MustAddImage ¶ added in v1.0.1
func (mm *MultiModal) MustAddImage(filename string)
MustAddImage is a convenience function that adds an image to the MultiModal instance, terminating the program if adding the image fails.
func (*MultiModal) SetTrim ¶
func (mm *MultiModal) SetTrim(trim bool)
SetTrim updates the trim flag of the MultiModal instance, controlling whether the output is trimmed for whitespace.
func (*MultiModal) SetVerbose ¶
func (mm *MultiModal) SetVerbose(verbose bool)
SetVerbose updates the verbose logging flag of the MultiModal instance, allowing for more detailed output during operations.
func (*MultiModal) Submit ¶
func (mm *MultiModal) Submit(projectID, location string) (string, error)
Submit sends all added parts to the specified Vertex AI model for processing, returning the model's response. It supports temperature configuration and response trimming. This function creates a temporary client, and is not meant to be used within Google Cloud (use SubmitToClient instead).
func (*MultiModal) SubmitToClient ¶ added in v1.2.1
SubmitToClient sends all added parts to the specified Vertex AI model for processing, returning the model's response. It supports temperature configuration and response trimming.
Source Files
Directories