GCP

Export Cloud Billing data to BigQuery

Google Cloud Platform provides a service called "BigQuery" which enables to export detailed Google Cloud billing data to a Bigquery dataset.

a. Create Billing Account

1. Log into GCP console portal(https://console.cloud.google.com/) and it will navigate to OverView Tab.

Google Cloud Platform

2. At the top left corner, find an option to create a Project which helps to monitor multiple Billing Account. Select either an already existing project or create new as needed.

Create New Project

3. Give a friendly Project name and create the Project.

Google Project Creation

Project Dashboard

4. Once Project is created, navigate to "IAM & Actions" -> "Service Accounts" found at the top left corner navigation options.

Navigate to Billing Account Tab

5. On navigating to Service Account page, you will find a list of Billing Accounts monitored under the single Project.

Billing Account Tab

6. Create new Billing Account (Service Account).

7. Give a friendly name for the service account and create it.

Create Billing Account

8. To have access to a more comprehensive set of Google Cloud billing data for the analysis needs, it is recommend to enable Cloud Billing data export to BigQuery at the same time as Billing Account creation.

9. Navigate to "BigQuery" -> "Data Transfers" at the navigation bar.

Navigating to Data Transfer

10. Enable the Bigquery Data Transfer to access the bigquery SQL Workspace to get the cost export data.

Enable Big Query data Transfer

Creating a GCP Connection

To create a new connection, follow these steps:

1. Log in to Hybr Admin portal and navigate to Reporting → Data Source -> Connections page.

Reporting -> Data Sources -> Connections

2. List of connection created will be listed in Connection tab.

3. Create a GCP Connection using Create Connection button.

Create GCP Connection - Step 1

4. Select Connection category as "GCP".

5. Give a friendly connection name to the GCP connection.

Create GCP Connection - Step 2(a)

Create Gcp Connection - Step 2(b)

6. Provide necessary connection details asked in the second step for Gcp Connection Creation.

Parameter	Description
Connection Type	Specifies the type of the configuration file, which should be set to "service_account" for a GCP service account.
Access Key Id	Specifies the ID of the private key associated with the service account.
Access Key Secret	Specifies the private key contents associated with the service account, which is used for authentication.
Billing Account Id	Specifies the "Id" of the billing account created in the GCP project.
Billing Account Name	Specifies the "Name" of the billing account created in the GCP project
Project Id	Specifies the ID of the GCP project where the service account is created.
Dataset Id	Specifies to a unique identifier that is used to identify and manage datasets in various GCP services
Client Id	Specifies the client ID associated with the service account.
Client Email	Specifies the email address associated with the service account, which is used for authentication.
Token URL	Specifies the URL for the token endpoint used for exchanging authorization codes for access tokens.
Service Account Impersonation URL	Specifies the URL where the x509 public certificates can be retrieved for the service account, which is used for token validation.

6. Configure the Cost export start date, which ensures from when the cost export records should be collected for individual connection.

7. Save the changes.

Editing a GCP Connection

If you want to make changes to an existing connection, follow these steps.

1. Log in to Hybr Admin portal and navigate to Reporting → Data Sources -> Connections page.

2. Select the connection and click on the "Edit" button

Edit Connection

3. Make the desired changes to your connection.

4. Save your changes.