Log streaming: Grafana Cloud Logs
Last updated 2024-10-23
Fastly's Real-Time Log Streaming feature can send log files to Grafana Cloud Logs, a scalable, multi-tenant log aggregation system by Grafana Labs.
NOTE
Fastly does not provide direct support for third-party services. Read Fastly's Terms of Service for more information.
Prerequisites
Before adding Grafana Cloud Logs as a logging endpoint for Fastly services, you will need to:
- Register for a Grafana Cloud Account.
- Create a Grafana Cloud Loki instance in your Grafana stack. Check out the Grafana Cloud Logs documentation for more details.
- Create a Grafana Access Policy with
logs:write
access scoped to your Loki instance.
Grafana Streams
Grafana Cloud Logs to sends data to Grafana Cloud via a stream. Grafana uses labels, formatted as a JSON string (e.g., { "app": "my app", "env": "staging" }
), to define a particular log stream.
IMPORTANT
Dynamic labels are not supported with this logging endpoint. Be sure to follow Grafana's current best practices for using labels.
Adding Grafana Cloud Logs as a logging endpoint
Follow these instructions to add Grafana Cloud Logs as a logging endpoint.
- Deliver services
- Compute services
- Review the information in our guide to setting up remote log streaming.
- In the Grafana Cloud Logs area, click Create endpoint.
- Fill out the Create a Grafana Cloud Logs endpoint fields as follows:
- In the Name field, enter a human-readable name for the endpoint.
- In the Placement area, select where the logging call should be placed in the generated VCL. Valid values are Format Version Default, waf_debug (waf_debug_log), and None. Read our guide on changing log placement for more information.
- In the Log format field, enter the data to send to Grafana Cloud. Check out the example format section for details.
- In the User ID area, enter your Grafana User ID.
- In the URL area, enter the URL of your Grafana stack (e.g.,
https://myhost.grafana.net
). - In the Access Policy Token area, enter the Grafana Access Policy token with
logs:write
scope you created. - In the Stream Labels area, enter the JSON string used to identify this stream in Grafana (e.g.,
{ "app": "my app", "env": "staging" }
).
- Click Create to create the new logging endpoint.
- Click Activate to deploy your configuration changes.
Example format
Data sent to Grafana Cloud Logs can be sent as JSON or plain text. Here's an example of JSON message data sent to Grafana Cloud Logs:
123456789101112131415161718
{ "timestamp": "%{strftime(\{"%Y-%m-%dT%H:%M:%S"\}, time.start)}V", "client_ip": "%{req.http.Fastly-Client-IP}V", "geo_country": "%{client.geo.country_name}V", "geo_city": "%{client.geo.city}V", "host": "%{if(req.http.Fastly-Orig-Host, req.http.Fastly-Orig-Host, req.http.Host)}V", "url": "%{json.escape(req.url)}V", "request_method": "%{json.escape(req.method)}V", "request_protocol": "%{json.escape(req.proto)}V", "request_referer": "%{json.escape(req.http.referer)}V", "request_user_agent": "%{json.escape(req.http.User-Agent)}V", "response_state": "%{json.escape(fastly_info.state)}V", "response_status": %{resp.status}V, "response_reason": %{if(resp.response, "%22"+json.escape(resp.response)+"%22", "null")}V, "response_body_size": %{resp.body_bytes_written}V, "fastly_server": "%{json.escape(server.identity)}V", "fastly_is_edge": %{if(fastly.ff.visits_this_service == 0, "true", "false")}V}
Do not use this form to send sensitive information. If you need assistance, contact support. This form is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.