Analyze logs using Logs Explorer and Observability Analytics

This document describes how you query, view, and analyze log entries by using the Google Cloud console. There are two interfaces available to you, the Logs Explorer and Observability Analytics. You can query, view, and analyze logs with both interfaces; however, they use different query languages and they have different capabilities:

  • For troubleshooting and exploration of log data, use the Logs Explorer.

  • To join your log and trace data, or to generate insights and trends, use Observability Analytics.

You can query your logs and save your queries by issuing Logging API commands. You can also query your logs by using Google Cloud CLI.

Use Logs Explorer

The Logs Explorer is designed to help you troubleshoot and analyze the performance of your services and applications. For example, a histogram displays the rate of errors. If you see a spike in errors or something that is interesting, you can locate and view the corresponding log entries. When a log entry is associated with an error group, the log entry is annotated with a menu of options that let you access more information about the error group.

The same query language is supported by the Cloud Logging API, the Google Cloud CLI, and the Logs Explorer. To simplify query construction when you are using the Logs Explorer, you can build queries by using menus, by entering text, and, in some cases, by using options included with the display of an individual log entry.

The Logs Explorer doesn't support aggregate operations, like counting the number of log entries that contain a specific pattern. To perform aggregate operations, enable analytics on the log bucket and then use Observability Analytics.

For details about searching and viewing logs with the Logs Explorer, see View logs by using the Logs Explorer.

Explore Observability Analytics

Using Observability Analytics, you can generate insights by running queries that group and aggregate your log data. These insights can help reduce the time you spend troubleshooting issues. To view your query results, use a table, a chart, or both. Charts can help you identify patterns and trends in your log data. For example, the following screenshot shows a query result being displayed as a table and a chart:

User interface for Observability Analytics.

Observability Analytics supports the following:

  • Grouping and aggregating log data.

    For example, you can run a SQL query that groups log entries by the hour, and then computes for each group, the average latency for HTTP requests issued to a specific URL.

  • SQL queries that use pipe syntax.

  • Queries of log views and analytics views.

    Log views have a system-defined schema. You define the schema for analytics views.

  • Joins of log and trace data.

    For information about querying your trace data, see Query and analyze traces.

Cloud Logging also lets you query your log data from BigQuery, without exporting that data to BigQuery. After you upgrade your log bucket to use Observability Analytics, create a linked BigQuery dataset. You can query the linked BigQuery dataset by using BigQuery services.

Upgrading a log bucket doesn't affect your usage of the Logs Explorer. The Logs Explorer only requires that your log data be stored in a log bucket.

Restrictions

  • To upgrade an existing log bucket to use Observability Analytics, the following restrictions apply:

    • The log bucket was created at the Google Cloud project level.
    • The log bucket is unlocked unless it is the _Required bucket.
    • There aren't pending updates to the bucket.