Collect Apache web server metrics with the Ops Agent

Learn how to collect and monitor metrics from an Apache web server installed on a Compute Engine virtual machine (VM) instance by using the Ops Agent:

Create a Compute Engine VM instance and install the Ops Agent.
Install an Apache web server.
Configure the Ops Agent for the Apache web server.
Generate traffic to the Apache web server.
View metrics on the predefined Apache dashboard.
Create an alerting policy.
Test the alerting policy.
Clean up.

To follow step-by-step guidance for this task directly in the Google Cloud console, click Guide me:

Guide me

Before you begin

Security constraints defined by your organization might prevent you from completing the following steps. For troubleshooting information, see Develop applications in a constrained Google Cloud environment.

Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

Roles required to select or create a project

Select a project: Selecting a project doesn't require a specific IAM role—you can select any project that you've been granted a role on.
Create a project: To create a project, you need the Project Creator role (roles/resourcemanager.projectCreator), which contains the resourcemanager.projects.create permission. Learn how to grant roles.

Go to project selector

If you're using an existing project for this guide, verify that you have the permissions required to complete this guide. If you created a new project, then you already have the required permissions.

Verify that billing is enabled for your Google Cloud project.

Enable the Compute Engine, Cloud Monitoring, Cloud Logging, and OS Config APIs.

Roles required to enable APIs

To enable APIs, you need the serviceusage.services.enable permission. If you created the project, then you likely already have this permission through the Owner role (roles/owner). Otherwise, you can get this permission through the Service Usage Admin role (roles/serviceusage.serviceUsageAdmin). Learn how to grant roles.

Enable the APIs

In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

Roles required to select or create a project

Select a project: Selecting a project doesn't require a specific IAM role—you can select any project that you've been granted a role on.
Create a project: To create a project, you need the Project Creator role (roles/resourcemanager.projectCreator), which contains the resourcemanager.projects.create permission. Learn how to grant roles.

Go to project selector

If you're using an existing project for this guide, verify that you have the permissions required to complete this guide. If you created a new project, then you already have the required permissions.

Verify that billing is enabled for your Google Cloud project.

Enable the Compute Engine, Cloud Monitoring, Cloud Logging, and OS Config APIs.

Roles required to enable APIs

Enable the APIs

Required roles

To get the permissions that you need to create a Compute Engine instance, install Apache, and to view metric data, ask your administrator to grant you the following IAM roles on your project:

Compute Instance Admin (v1) (roles/compute.instanceAdmin.v1)
Monitoring Editor (roles/monitoring.editor)
OSPolicyAssignment Admin (roles/osconfig.osPolicyAssignmentAdmin)
Service Account User (roles/iam.serviceAccountUser)
Service Usage Admin (roles/serviceusage.serviceUsageAdmin)

For more information about granting roles, see Manage access to projects, folders, and organizations.

You might also be able to get the required permissions through custom roles or other predefined roles.

Create a VM instance

In the Google Cloud console, go to the VM instances page:
Go to VM instances

If you use the search bar to find this page, then select the result whose subheading is Compute Engine.
Create a VM by clicking Create instance. Configure your instance by using the options in the navigation menu.
In the Machine configuration option, do the following:
1. In the Name field, enter a descriptive name.
2. In the Machine type preset drop-down, select Shared-core > e2-small.
Verify that the OS and storage option displays Debian GNU/Linux. If not, click the OS and storage option and click Change. In the Boot disk dialog, set Version to Debian GNU/Linux.
In the Networking option, for Firewall, select both Allow HTTP traffic and Allow HTTPS traffic.
Verify that the Observability option displays Install Ops Agent. If not, click the Observability option and select Install Ops Agent for Monitoring and Logging.
Click Create.

Install an Apache web server

To install an Apache web server on your Compute Engine VM instance, do the following:

On the VM instances page, locate your new VM, go to the Connect column, and then click SSH.

Having trouble connecting? Refer to Troubleshooting SSH.
To update the package lists, copy the following command to your clipboard, paste the command into the SSH terminal, and then press enter:
```
sudo apt-get update
```
After you see the message "Reading package lists... Done", in the SSH terminal, run the following command to install an Apache2 web server:
```
sudo apt-get install apache2 php7.0
```
When asked to continue the installation, enter Y. If the install command fails, then use sudo apt-get install apache2 php.
When your command prompt returns, go to the VM instances page and copy the VM's external IP address into the following URL:
```
http://EXTERNAL_IP
```
To connect to your Apache web server, open a new browser tab, and then enter the URL from the previous step.

When the web server is successfully installed, the browser tab displays the Apache2 Debian default page.

Collect Apache web server logs and metrics

In these steps, you configure the Ops Agent to collect logs and metrics from your Apache web server:

Go to the SSH terminal for your VM instance. If you don't have a terminal open, then do the following:
1. In the Google Cloud console, go to the VM instances page:
  Go to VM instances
  
  If you use the search bar to find this page, then select the result whose subheading is Compute Engine.
2. Locate your new VM and then click SSH.

Copy the following command, then paste it into the terminal for your instance, and then press enter:

# Configures Ops Agent to collect telemetry from the app. You must restart the agent for the configuration to take effect.

set -e

# Check if the file exists
if [ ! -f /etc/google-cloud-ops-agent/config.yaml ]; then
  # Create the file if it doesn't exist.
  sudo mkdir -p /etc/google-cloud-ops-agent
  sudo touch /etc/google-cloud-ops-agent/config.yaml
fi

# Create a back up of the existing file so existing configurations are not lost.
sudo cp /etc/google-cloud-ops-agent/config.yaml /etc/google-cloud-ops-agent/config.yaml.bak

# Configure the Ops Agent.
sudo tee /etc/google-cloud-ops-agent/config.yaml > /dev/null << EOF
metrics:
  receivers:
    apache:
      type: apache
  service:
    pipelines