Use Anthropic Claude 3 Models To Build Generative AI Applications With Go

Claude 3 models are multi-modal. In this article, let's explore how to use the Claude 3 models on Amazon Bedrock with Go.

Abhishek Gupta

CORE ·

May. 14, 24 · Tutorial

Like (1)

Save

846 Views

Anthropic's Claude 3 is a family of AI models with different capabilities and costs for a variety of tasks:

Claude 3 Haiku is a compact and fast model that provides near-instant responsiveness
Claude 3 Sonnet provides a balance between skills and speed
Claude 3 Opus is for highly complex tasks when you need high intelligence and performance

Claude 3 models are multi-modal. This means that they can accept both text and images as inputs (although they can only output text). Let's learn how to use the Claude 3 models on Amazon Bedrock with Go.

Basic Examples

Refer to the Before You Begin section in this blog post to complete the prerequisites for running the examples. This includes installing Go, configuring Amazon Bedrock access, and providing necessary IAM permissions.

Amazon Bedrock abstracts multiple models via a uniform set of APIs that exchange JSON payloads. The same applies to Claude 3.

Let's start with a simple example using AWS SDK for Go (v2).

You can run it as such:

    Go
   
   git clone https://github.com/abhirockzz/claude3-bedrock-go
cd claude3-bedrock-go

go run basic/main.go

The response may (or may not) be slightly different in your case:

    Go
   
 

   request payload:
 {"anthropic_version":"bedrock-2023-05-31","max_tokens":1024,"messages":[{"role":"user","content":[{"type":"text","text":"Hello, what's your name?"}]}]}
response payload:
 {"id":"msg_015e3SJ99WF6p1yhGTkc4HbK","type":"message","role":"assistant","content":[{"type":"text","text":"My name is Claude. It's nice to meet you!"}],"model":"claude-3-sonnet-28k-20240229","stop_reason":"end_turn","stop_sequence":null,"usage":{"input_tokens":14,"output_tokens":15}}
response string:
 My name is Claude. It's nice to meet you!
  

You can refer to the complete code here. I will break it down to make this simpler for you:

We start by creating the JSON payload — it's modeled as a struct (Claude3Request):

    Go
   
 

         msg := "Hello, what's your name?"

    payload := Claude3Request{
        AnthropicVersion: "bedrock-2023-05-31",
        MaxTokens:        1024,
        Messages: []Message{
            {
                Role: "user",
                Content: []Content{
                    {
                        Type: "text",
                        Text: msg,
                    },
                },
            },
        },
    }

  payloadBytes, err := json.Marshal(payload)
  

InvokeModel is used to call the model. The JSON response is converted to a struct (Claude3Response) and the text response is extracted from it.

    Go
   
   //....
  output, err := brc.InvokeModel(context.Background(), &bedrockruntime.InvokeModelInput{
      Body:        payloadBytes,
      ModelId:     aws.String(modelID),
      ContentType: aws.String("application/json"),
  })

  var resp Claude3Response
  err = json.Unmarshal(output.Body, &resp)

  log.Println("response string:\n", resp.ResponseContent[0].Text)

Chat With Streaming

Now moving on to a common example which involves a conversational exchange. We will also add a streaming element for a better experience — the client application does not need to wait for the complete response to be generated before it starts showing up in the conversation.

You can run it as such:

    Go
   
   go run chat-streaming/main.go

You can refer to the complete code here.

A streaming-based implementation is a bit more involved. First off, we use InvokeModelWithResponseStream (instead of Invoke).

    Go
   
 

       output, err := brc.InvokeModelWithResponseStream(context.Background(), &bedrockruntime.InvokeModelWithResponseStreamInput{
        Body:        payloadBytes,
        ModelId:     aws.String(modelID),
        ContentType: aws.String("application/json"),
    })

    resp, err := processStreamingOutput(output, func(ctx context.Context, part []byte) error {
        fmt.Print(string(part))
        return nil
    })
  

To process its output, we use the following:

    Go
   
 

       resp, err := processStreamingOutput(output, func(ctx context.Context, part []byte) error {
        fmt.Print(string(part))
        return nil
    })
//...
  

Here are a few bits from the processStreamingOutput function — you can check the code here. The important thing to understand is how the partial responses are collected together to produce the final output.

    Go
   
 

   func processStreamingOutput(output *bedrockruntime.InvokeModelWithResponseStreamOutput, handler StreamingOutputHandler) (Claude3Response, error) {

  //...
    for event := range output.GetStream().Events() {
        switch v := event.(type) {
        case *types.ResponseStreamMemberChunk:

            var pr PartialResponse
            err := json.NewDecoder(bytes.NewReader(v.Value.Bytes)).Decode(&pr)
            if err != nil {
                return resp, err
            }

            if pr.Type == partialResponseTypeContentBlockDelta {
                handler(context.Background(), []byte(pr.Delta.Text))
                combinedResult += pr.Delta.Text
            } else if pr.Type == partialResponseTypeMessageStart {
                resp.ID = pr.Message.ID
                resp.Usage.InputTokens = pr.Message.Usage.InputTokens
            } else if pr.Type == partialResponseTypeMessageDelta {
                resp.StopReason = pr.Delta.StopReason
                resp.Usage.OutputTokens = pr.Message.Usage.OutputTokens
            }
  //...
    }

  resp.ResponseContent[0].Text = combinedResult

    return resp, nil
}
  

Use Claude Haiku With Images

All the Claude3 models can accept images as inputs. Haiku is deemed to be good at OCR (optical), understanding images, etc. Let's make use of Claude Haiku the upcoming examples — no major changes are required, except for the model ID.

The example uses this image (actual StackOverflow question) along with this prompt Transcribe the code in the question. Only output the code.

You can run it as such:

    Go
   
   go run images/main.go

This is the response that Claude Haiku came up with. Impressive!

    Go
   
 

   func test() {
    cluster := gocql.NewCluster("hostIP")
    cluster.ProtoVersion = 4
    cluster.Authenticator = gocql.PasswordAuthenticator{
        Username: "<username>",
        Password: "<password>",
    }
    cluster.Keyspace = "myspace"
    cluster.Consistency = gocql.One
    session, err := cluster.CreateSession()
    if err != nil {
        panic(err)
    }
    defer session.Close()
}
  

I tried another one: "Can you suggest a solution to the question?"

Here is the response:

    Go
   
   Based on the error messages shown in the image, it seems that the issue is related to connecting to the Cassandra database using the gocql driver (Go: 1.8.3). The errors indicate that the application is unable to dial the control connection, create a session, or connect to the initial hosts within the timeout period.

A few things you could check to troubleshoot this issue:

1. Verify the connection details (host, port, username, password) are correct and match the Cassandra cluster configuration.

2. Ensure the Cassandra cluster is up and running and accessible from the application host.

3. Check the firewall settings to make sure the application host is able to connect to the Cassandra cluster on the required ports.

4. Inspect the Cassandra server logs for any errors or clues about the connection issue.

5. Try increasing the timeout values in the gocql configuration, as the current timeouts may be too short for the Cassandra cluster to respond.

6. Ensure the gocql version (1.8.3) is compatible with the Cassandra version (3.10.0) you are using.

7. Consider adding some error handling and retry logic in your code to gracefully handle connection failures and attempt to reconnect.

Without more context about your specific setup and environment, it's difficult to pinpoint the exact issue. However, the steps above should help you investigate the problem and find a solution.

You can refer to the complete code here.

Here is what the JSON payload for the request:

    Go
   
 

     payload := Claude3Request{
        AnthropicVersion: "bedrock-2023-05-31",
        MaxTokens:        1024,
        Messages: []Message{
            {
                Role: "user",
                Content: []Content{
                    {
                        Type: "image",
                        Source: &Source{
                            Type:      "base64",
                            MediaType: "image/jpeg",
                            Data:      imageContents,
                        },
                    },
                    {
                        Type:   "text",
                        Text:   msg,
                        Source: nil,
                    },
                },
            },
        },
    }
  

The imageContents in Data attribute is the base64 encoded version of the image which is calculated like this:

    Go
   
   func readImageAsBase64(filePath string) (string, error) {

    imageFile, err := os.ReadFile(filePath)

    if err != nil {
        return "", err
    }

    encodedString := base64.StdEncoding.EncodeToString(imageFile)
    return encodedString, nil
}

You can try a different image (for example this one), and check if it's able to calculate the cost of all menu items.

Combine Text and Image Conversations

Image and text data are not exclusive. The message structure (JSON payload) is flexible enough to support both.

You can refer to the complete code here.

Here is an example in which you can mix and match text and image-based messages. You can run it as such:

    Go
   
   go run multi-modal-chat-streaming/main.go

Here is the response (I used the same StackOverflow image):

Wrap Up

You don't have to depend on Python to build generative AI applications. Thanks to AWS SDK support, you can use the programming language of your choice (including Go!) to integrate with Amazon Bedrock.

AI applications Go (programming language) generative AI

Published at DZone with permission of Abhishek Gupta, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

Related

Trending

Use Anthropic Claude 3 Models To Build Generative AI Applications With Go

Claude 3 models are multi-modal. In this article, let's explore how to use the Claude 3 models on Amazon Bedrock with Go.

Basic Examples

Chat With Streaming

Use Claude Haiku With Images

Combine Text and Image Conversations

Wrap Up

Related

Partner Resources