Announcing support for the Inkling model family on Google Cloud

Google Cloud Community Articles
,
1

Google Cloud is thrilled to announce support for Thinking Machine Lab’s highly capable Inkling model family in Model Garden on Agent Platform via self-deployment. By bringing Inkling to our fully managed environment, developers and enterprises can now easily access and configure these newest multimodal models.

Alongside support for Inkling, we are excited to deliver day 0 support for Inkling-Small—a highly optimized, open-weights variant that accepts text, image, and audio inputs and generates text outputs, balancing flagship capabilities with outstanding operational efficiency.

Hosting the Inkling model family with Model Garden

Choosing the best model for a given job depends on task-relevant capabilities, modality, and cost. By deploying the Inkling models through Model Garden on Agent Platform, enterprise engineering teams can avoid the operational complexities of manual container setup and CUDA driver management alongside features such as:

  • Secure private connectivity: Deployed endpoints can be bound directly to internal Virtual Private Clouds using Private Service Connect (PSC), which routes real-time inference traffic securely without exposing endpoints to the public internet or risking IP address exhaustion.

  • Cost optimization: Scale to Zero automatically scales the active inference nodes (including spot or reserved GPUs and TPUs) down to zero replicas during periods of inactivity, dropping idle infrastructure costs to zero.

  • OpenAI API compatibility: The prebuilt Model Garden serving containers natively support the industry-standard OpenAI Chat Completions API protocol. This allows engineering teams to drop Inkling models directly into existing application codebases or developer toolchains by simply changing the endpoint base URL.

Option 1: Deploy via the Cloud Console UI

If you prefer a visual, code-less experience, you can deploy Inkling directly within the Google Cloud console:

1. Navigate to the Model Garden section of the Google Cloud Console.

2. Search for Inkling in the Model Garden search-box.

3. Click Deploy on your preferred variant (Inkling or Inkling-Small) and optionally configure your quantization, machine type, and region.

image
image1920×1147 127 KB

Option 2: Deploy programmatically via the Vertex AI SDK

For automated environments, integration pipelines, or developers who prefer code, you can easily initialize and deploy any Inkling model variant using the Vertex AI SDK. The sample below shows how to register and deploy Inkling-Small:

import vertexai
from vertexai import model_garden

PROJECT_ID = "YOUR_PROJECT_ID"
REGION = "us-central1"

vertexai.init(
    project=PROJECT_ID,
    location=REGION,
)

# Deploy Inkling-Small (or configure with 'inkling' for the larger model)
model_name = "inkling-small"
PUBLISHER_MODEL_NAME = f"publishers/thinkingmachines/models/inkling@{model_name}"

model = model_garden.OpenModel(PUBLISHER_MODEL_NAME)
endpoint = model.deploy(accept_eula=True)

print(f"Endpoint successfully deployed: {endpoint.resource_name}")

Once the endpoint is ready, you can perform standard predictions with customizable parameters:

# Perform inference with sampling parameters
instances = [
    {"text": "Explain the advantages of Mixture-of-Experts routing configurations."}
]

parameters = {
    "sampling_params": {
        "max_new_tokens": 128,
        "temperature": 1.0,
        "top_p": 0.95
    }
}

response = endpoint.predict(instances=instances, parameters=parameters)
print(response.predictions[0])

What’s next?

Ready to experiment with Inkling models on Google Cloud? Access the models via Model Garden or check out the direct Hugging Face links:

1. Access self-deploy options on Model Garden.
2. Explore the model card and access open weights directly on Hugging Face: Inkling, Inkling-Small

Acknowledgements

We want to thank the Thinking Machines Lab team for their collaboration on having the Inkling family in Model Garden on Agent Platform.

3 Likes