Create and manage deployments

Deployments let you schedule code to run—for example, pipelines—that's stored in remote Git repositories. You can choose the directory where your code is located and perform the following actions:

• Create and manage deployments connected to remote repositories.
• Configure triggers to pull code from remote repositories and run code on a regular basis.
• Run pipelines on a schedule or with other triggers.
• View logs of your executions directly within the Google Cloud console. For alerting, Dataform deployments integrate with Cloud Logging and Cloud Monitoring.

Limitations

• Deployments don't have built-in alerting. You must use Logging and Monitoring to configure alerts.
• Authentication with Developer Connect only supports connections and not account connectors.

Before you begin

Before you create a deployment, complete the following tasks:

• Opt in to using the Pipelines & Connections page in the Google Cloud console.

• Enable the BigQuery API, Dataform API, and Developer Connect API:

  Enable the APIs

• Connect a remote repository using Developer Connect.

Required roles

To get the permissions that you need to complete the tasks in this document, ask your administrator to grant you the following IAM roles:

• Dataform Admin (roles/dataform.admin) on the project
• BigQuery Job User (roles/bigquery.jobUser) on the project
• Developer Connect Viewer (roles/developerconnect.viewer) on the project
• Use a custom service account to run a deployment: Service Account User (roles/iam.serviceAccountUser) on the custom service account

For more information about granting roles, see Manage access to projects, folders, and organizations.

You might also be able to get the required permissions through custom roles or other predefined roles.

To use a custom service account to run a deployment, you must also grant the BigQuery Data Editor role (roles/bigquery.dataEditor) to the custom service account.

Create a deployment

To create a Dataform deployment, do the following:

1. In the Google Cloud console, go to the Pipelines & Connections page.

   Go to Pipelines & Connections

2. Click Create > Create deployment.

3. Select a repository authentication type, and then select one of the following options:

   Developer Connect

   1. Select a repository and branch.
   2. Enter the path for the directory where the code is located.
   3. Enter a name for the deployment.
   4. Select a region for the deployment.

   5. In the Schedule frequency section, do the following:

      1. In the Repeats menu, select the frequency of scheduled pipeline runs.
      2. In the At time field, enter the time for scheduled pipeline runs.
      3. In the Timezone menu, select the time zone for the schedule.

      Alternatively, select On-demand to run your deployment on demand.

   6. Optional: Enter the Google Cloud project ID.

   7. Optional: Enter a schema suffix.

   8. Optional: Enter a table prefix.

   9. Optional: Enter the default schema.