Claude on Vertex

Google Cloud Build with AI Custom ML & MLOps
,
1

Hello,

Having the code bellow, which is trying to do a query on claude.

$PROJECT_ID = "project_id";
            $LOCATION = "us-central1";
            $MODEL = "claude-3-haiku@20240307";
            $formattedEndpoint = PredictionServiceClient::projectLocationPublisherModelName($PROJECT_ID, $LOCATION, 'google', $MODEL);
            Log::info($formattedEndpoint);
            $predictionServiceClient = new PredictionServiceClient([
                'apiEndpoint' => $LOCATION.'-aiplatform.googleapis.com',
            ]);

            $payload = [
                "anthropic_version" => "vertex-2023-10-16",
                "messages"          => [["role" => "user", "content" => $prompt]],
                "max_tokens"        => 4000,
                "stream"            => false,
                "temperature"       => floatval(env('CLAUDE_TEMPERATURE')),
            ];

            $httpBody = new HttpBody();
            $httpBody->setContentType('application/json;');
            $httpBody->setData(json_encode($payload));

            $request = RawPredictRequest::build($formattedEndpoint, $httpBody);

            return $predictionServiceClient->rawPredict($request);

No matter how I set that payload I always get back the same error:

{
    "message": "messages: Field required",
    "code": 400,
    "status": "UNRECOGNIZED_STATUS",
    "details": []
}

Any help is highly appreciated!

2

Hi @MariusC ,

Thank you for joining our community.

The error message “messages: Field required” indicates a missing field in your request. While your code includes the “messages” field, there might be an issue with how the message itself is constructed.

A common culprit is an empty $prompt variable within the “content” field. Ensure $prompt is defined and contains the data you want to send to Claude.

For reference on proper code structure, refer to this Anthropic - Vertex AI API article.

I hope I was able to provide you with useful insights.

3

Were you able to solve this eventually?

4 
<?php

namespace App\Services\ScannerEngine\Analyzers;

use Google\Client;
use Aws\ResultInterface;
use Google\Service\Aiplatform;
use Illuminate\Support\Facades\Log;
use Illuminate\Http\Client\Response;
use App\Services\ScannerEngine\AvailableLLM;
use Google\Cloud\AIPlatform\V1\Client\PredictionServiceClient;

class HaikuVertexApi extends AnalyzerAbstract
{
    public function analyze(string $message): Response|ResultInterface|bool
    {
        if (! getenv('GOOGLE_APPLICATION_CREDENTIALS')) {
            putenv('GOOGLE_APPLICATION_CREDENTIALS=' . base_path() . '/google-credentials.json');
        }

        $formattedEndpoint = PredictionServiceClient::projectLocationPublisherModelName(
            "GOOGLE_PROJECT_ID",
            "us-central1",
            'anthropic',
            $this->modelVersion ?? "claude-3-haiku@20240307"
        );

        $client = new Client();
        $client->useApplicationDefaultCredentials();
        $client->addScope(Aiplatform::CLOUD_PLATFORM);

        $httpClient = $client->authorize();
        $response = $httpClient->post('https://us-central1-aiplatform.googleapis.com/v1/' . $formattedEndpoint . ':rawPredict', [
            "headers" => [
                "Content-Type" => "application/json",
            ],
            "body" => json_encode([
                "anthropic_version" => "vertex-2023-10-16",
                "messages"          => [["role" => "user", "content" => $message]],
                "system"            => $this->systemPrompt ?? '',
                "max_tokens"        => 4000,
                "temperature"       => floatval($this->temperature ?? config('llm.' . AvailableLLM::CLAUDE_3_HAIKU_VERTEX->value . '.temperature')),
            ])
        ]);

        return new Response($response);
    }
}

This is the version that I ended up with.