Create and manage templates

You use Model Armor templates to configure the screening of prompts and responses for your AI applications. The templates provide customized filters and thresholds for multiple safety and security categories. This document explains how to create and manage Model Armor templates. For more information, see Model Armor templates.

Use a regional or multi-regional endpoint

When working with a Model Armor template, you must use a regional or multi-regional endpoint (modelarmor.LOCATION.rep.googleapis.com) that matches the template's location.

The global endpoint (modelarmor.googleapis.com) doesn't support managing Model Armor templates or sanitizing prompts and responses.

Before you begin

Before you begin, complete the following tasks.

Obtain the required permissions

To get the permissions that you need to manage Model Armor templates, ask your administrator to grant you the Model Armor Admin (roles/modelarmor.admin) IAM role on Model Armor templates. For more information about granting roles, see Manage access to projects, folders, and organizations.

You might also be able to get the required permissions through custom roles or other predefined roles.

Enable APIs

You must enable the Model Armor API before you can use Model Armor.

Console

Enable the Model Armor API.
Roles required to enable APIs
To enable APIs, you need the serviceusage.services.enable permission. If you created the project, then you likely already have this permission through the Owner role (roles/owner). Otherwise, you can get this permission through the Service Usage Admin role (roles/serviceusage.serviceUsageAdmin). Learn how to grant roles.
Enable the API
Select the project where you want to activate Model Armor.

gcloud

Before you begin, follow these steps using the Google Cloud CLI with the Model Armor API:

In the Google Cloud console, activate Cloud Shell.

Activate Cloud Shell

At the bottom of the Google Cloud console, a Cloud Shell session starts and displays a command-line prompt. Cloud Shell is a shell environment with the Google Cloud CLI already installed and with values already set for your current project. It can take a few seconds for the session to initialize.
Set the API endpoint override using the gcloud CLI.

Set the API endpoint override using the gcloud CLI

This step is only required if you use the gcloud CLI with Model Armor and want to use a region or multi-region other than the default us multi-region. You must manually set the API endpoint override to ensure the gcloud CLI correctly routes requests to the Model Armor service.

Run the following command to set the API endpoint for the Model Armor service.

gcloud config set api_endpoint_overrides/modelarmor "https://modelarmor.LOCATION.rep.googleapis.com/"

Replace LOCATION with the region or multi-region where you want to use Model Armor.

Create a Model Armor template

Model Armor templates define the specific filters and thresholds that Model Armor uses to screen prompts and responses for safety and security risks. To create a Model Armor template, follow these steps:

Console

In the Google Cloud console, go to the Model Armor page.

Go to Model Armor
Verify that you are viewing the project that you activated Model Armor on.
On the Model Armor page, click Create Template. The Create Template page is displayed.
Specify the Template ID. The template ID can contain letters, digits, underscores, or hyphens. It cannot exceed 63 characters, contain spaces, or start with a hyphen.
In the Location type section, select a region or multi-region where the Model Armor operations will run. You cannot change the location later.
Optional: In the Data residency compliance section, choose whether to enable data residency enforcement. By default, any Model Armor feature that isn't locally hosted in your selected location is disabled to prevent your data from crossing to other jurisdictions. The disabled features are listed in this section.

You can enable the features that are restricted by data residency enforcement by disabling the Enforce data residency option. This action allows cross-jurisdictional routing of your data.

Caution: Cross-jurisdictional routing of your data can impact your data residency compliance.
Optional: In the Labels section, add any key-value pairs that you want to use to group related templates.
In the Filter version section, select either the filter version number or the filter version alias.
Select modality to specify whether you want to screen text, images, or both.

Note: Only us and eu multi-regions support image modality. Selecting any other region disables the Select modality field.
In the Detections section, configure the detection settings.
Optional: If you select Sensitive Data Protection detection, you need to configure the Sensitive Data Protection settings.
In the Responsible AI section, set the confidence level for each content filter.

Note: If you don't specify a confidence level, it is set to High by default.
Optional: Expand the Additional configurations (optional) section, and configure any of the following settings:
- In the Configure logging section, select the operations that you want to configure logging for.
- In the Multi-language support section, select Enable Multi-language support to use multi-language detection.
- In the Enforcement mode section, select an enforcement type.
Click Create.

gcloud

Run the following command:

 gcloud model-armor templates create TEMPLATE_ID --project=PROJECT_ID --location=LOCATION \
     --rai-settings-filters='[{ "filterType": "HATE_SPEECH", "confidenceLevel": "MEDIUM_AND_ABOVE" },{ "filterType": "HARASSMENT", "confidenceLevel": "MEDIUM_AND_ABOVE" },{ "filterType": "SEXUALLY_EXPLICIT", "confidenceLevel": "MEDIUM_AND_ABOVE" }]' \
     --basic-config-filter-enforcement=enabled  \
     --pi-and-jailbreak-filter-settings-enforcement=enabled \
     --pi-and-jailbreak-filter-settings-confidence-level=HIGH \
     --malicious-uri-filter-settings-enforcement=enabled \
     --template-metadata-custom-llm-response-safety-error-code=798 \
     --template-metadata-custom-llm-response-safety-error-message="test template llm response evaluation failed" \
     --template-metadata-custom-prompt-safety-error-code=799 \
     --template-metadata-custom-prompt-safety-error-message="test template prompt evaluation failed" \
     --template-metadata-ignore-partial-invocation-failures \
     --template-metadata-log-operations \
     --template-metadata-log-sanitize-operations

Replace the following:

LOCATION: the location of the template.
TEMPLATE_ID: the ID of the template.
PROJECT_ID: the ID of the project that the template belongs to.

REST

Use the following command to create a new Model Armor template.

  curl -X POST \
    -d "FILTER_CONFIG" \
    -H "Content-Type: application/json" \
    -H "Authorization: Bearer $(gcloud auth print-access-token)" \
        "https://modelarmor.LOCATION.rep.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/templates?template_id=TEMPLATE_ID"

Replace the following:

FILTER_CONFIG: the filter configuration for the template in JSON format.
PROJECT_ID: the ID of the project that the template belongs to.
TEMPLATE_ID: the ID of the template to be created.
LOCATION: the location of the template.

The following example shows the Model Armor filter configuration (FilterConfig), where you define what you want to detect. In this example, the Responsible AI filters are configured for hate speech, harassment, dangerous content, and sexually explicit content with varying confidence levels. The prompt injection and jailbreak detection filter is enabled with a HIGH confidence level, which means only content with a high likelihood will be flagged. The malicious URI filter is enabled.

  export FILTER_CONFIG='{
   "filterConfig": {
    "raiSettings": {
     "raiFilters": [{
       "filterType": "HATE_SPEECH",
       "confidenceLevel": "MEDIUM_AND_ABOVE"
      }, {
      "filterType": "HARASSMENT",
      "confidenceLevel": "HIGH"
    }, {
      "filterType": "DANGEROUS",
      "confidenceLevel": "MEDIUM_AND_ABOVE"
    },{
      "filterType": "SEXUALLY_EXPLICIT",
      "confidenceLevel": "MEDIUM_AND_ABOVE"
    }]
  },

  "piAndJailbreakFilterSettings": {
    "filterEnforcement": "ENABLED",
    "confidenceLevel": "HIGH"
  },
  "maliciousUriFilterSettings": {
    "filterEnforcement": "ENABLED"
  }
 }
}'

curl -X POST \
 -d "$FILTER_CONFIG" \
 -H "Content-Type: application/json" \
 -H "Authorization: Bearer $(gcloud auth print-access-token)" \
    "https://modelarmor.LOCATION.rep.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/templates?template_id=TEMPLATE_ID"

To define how Model Armor handles operations, errors, and logs, set the template metadata (TemplateMetadata) in your command. The template metadata includes settings for language detection, modalities, and data residency compliance.

The following example creates a Model Armor template configuration that has multi-language detection and modalities (image and text) enabled.

  export TEMPLATE_CONFIG='{
   "filterConfig": {
    "raiSettings": {
     "raiFilters": [{
       "filterType": "HATE_SPEECH",
       "confidenceLevel": "MEDIUM_AND_ABOVE"
      }, {
      "filterType": "HARASSMENT",
      "confidenceLevel": "HIGH"
    }, {
      "filterType": "DANGEROUS",
      "confidenceLevel": "MEDIUM_AND_ABOVE"
    },{
      "filterType": "SEXUALLY_EXPLICIT",
      "confidenceLevel": "MEDIUM_AND_ABOVE"
    }]
  },

  "piAndJailbreakFilterSettings": {
    "filterEnforcement": "ENABLED",
    "confidenceLevel": "HIGH"
  },
  "maliciousUriFilterSettings": {
    "filterEnforcement": "ENABLED"
  }
 },
 "templateMetadata": {
    "multiLanguageDetection": {
      "enableMultiLanguageDetection": true
    },
    "modalities": ["MODALITY_IMAGE", "MODALITY_TEXT"]
  }
}'

curl -X POST \
 -d "$TEMPLATE_CONFIG"  \
 -H "Content-Type: application/json" \
 -H "Authorization: Bearer $(gcloud auth print-access-token)" \
    "https://modelarmor.LOCATION.rep.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/templates?template_id=TEMPLATE_ID"

C#

To run this code, first set up a C# development environment and install the Model Armor C# SDK.

using Google.Api.Gax.ResourceNames;
using Google.Cloud.ModelArmor.V1;
using System.Collections.Generic;

public class CreateTemplateSample
{
    public Template CreateTemplate(
        string projectId = "my-project",
        string locationId = "us-central1",
        string templateId = "my-template"
    )
    {
        ModelArmorClient client = new ModelArmorClientBuilder
        {
            Endpoint = $"modelarmor.{locationId}.rep.googleapis.com",
        }.Build();

        // Build the Model Armor template with your preferred filters.
        // For more details on filters, please refer to the following doc:
        // https://cloud.google.com/security-command-center/docs/key-concepts-model-armor#ma-filters

        // Configure Responsible AI filter with multiple categories and their confidence
        // levels.
        RaiFilterSettings raiFilterSettings = new RaiFilterSettings();
        List<RaiFilterSettings.Types.RaiFilter> filters =
            new List<RaiFilterSettings.Types.RaiFilter>
            {
                new RaiFilterSettings.Types.RaiFilter
                {
                    FilterType = RaiFilterType.Dangerous,
                    ConfidenceLevel = DetectionConfidenceLevel.High,
                },
                new RaiFilterSettings.Types.RaiFilter
                {
                    FilterType = RaiFilterType.HateSpeech,
                    ConfidenceLevel = DetectionConfidenceLevel.High,
                },
                new RaiFilterSettings.Types.RaiFilter
                {
                    FilterType = RaiFilterType.SexuallyExplicit,
                    ConfidenceLevel = DetectionConfidenceLevel.LowAndAbove,
                },
                new RaiFilterSettings.Types.RaiFilter
                {
                    FilterType = RaiFilterType.Harassment,
                    ConfidenceLevel = DetectionConfidenceLevel.MediumAndAbove,
                },
            };

        raiFilterSettings.RaiFilters.Add(filters);

        Template template = new Template
        {
            FilterConfig = new FilterConfig { RaiSettings = raiFilterSettings },
        };

        CreateTemplateRequest request = new CreateTemplateRequest
        {
            ParentAsLocationName = LocationName.FromProjectLocation(projectId, locationId),
            TemplateId = templateId,
            Template = template,
        };

        Template createdTemplate = client.CreateTemplate(request);
        System.Console.WriteLine($"Created template: {createdTemplate.Name}");

        return createdTemplate;
    }
}

Go

To run this code, first set up a Go development environment and install the Model Armor Go SDK.


import (
	"context"
	"fmt"
	"io"

	modelarmor "cloud.google.com/go/modelarmor/apiv1"
	modelarmorpb "cloud.google.com/go/modelarmor/apiv1/modelarmorpb"
	"google.golang.org/api/option"
)

// createModelArmorTemplate method creates a new
// Model Armor template with the provided settings.
//
// w io.Writer: The writer to use for logging.
// projectID string: The ID of the Google Cloud project.
// locationID string: The ID of the Google Cloud location.
// templateID string: The ID of the template to create.
func createModelArmorTemplate(w io.Writer, projectID, locationID, templateID string) error {
	ctx := context.

Create and manage templates Stay organized with collections Save and categorize content based on your preferences.

Use a regional or multi-regional endpoint

Before you begin

Obtain the required permissions

Enable APIs

Console

gcloud

Set the API endpoint override using the gcloud CLI

Create a Model Armor template

Console

gcloud

REST

C#

Go

Create and manage templates