Compute@Edge log streaming: Coralogix

Fastly's Real-Time Log Streaming feature for Compute@Edge services can send log files to Coralogix. Coralogix provides an analytics platform that allows you to detect abnormal behavior via dynamic alerts, ratio alerts, flow anomaly detection, and threat discovery.

Prerequisites

If you don't already have a Coralogix account, you'll need to register for one by following the signup instructions on the Coralogix website. Once you've signed up, navigate to the Send Your Logs area in the Settings section of your Coralogix dashboard and make note of your unique private key. Coralogix uses this to associate data you send them with your account. You'll need it when you set up your endpoint with Fastly.

Adding Coralogix as a logging endpoint

Follow these instructions to add Coralogix as a logging endpoint:

  1. Review the information in our Setting Up Remote Log Streaming guide.
  2. Click the HTTPS Create endpoint button. The Create an HTTPS endpoint page appears.
  3. Fill out the Create an HTTPS endpoint fields as follows:
    • In the Name field, enter a human-readable name for the endpoint.
    • In the Placement area, select where the logging call should be placed in the generated VCL. Valid values are Format Version Default, waf_debug (waf_debug_log), and None. See our guide on changing log placement for more information.
    • In the URL field, enter the Coralogix regional FluentD API URL. See Coralogix documentation for more detail. For example, Coralogix accounts in the US would use https://api.coralogix.us/logs/rest/singles.
    • In the Maximum logs field, leave as 0 (the default).
    • In the Maximum bytes field, enter 2000000.
  4. Click the Advanced options link of the Create an HTTPS endpoint page. The Advanced options appear.
  5. Fill out the Advanced options of the Create an HTTPS endpoint page as follows:
    • In the Content type field, enter application/json.
    • In the Custom header name field, enter private_key.
    • In the Custom header value field, enter your Coralogix private key.
    • From the Method controls, select POST.
    • From the JSON log entry format controls, select Array of JSON.
    • Leave the Select a log line format controls set to the defaults.
    • Leave the TLS hostname, TLS CA certificate, TLS client certificate, and TLS client key fields blank.
  6. Click the Create button to create the new logging endpoint.
  7. Click the Activate button to deploy your configuration changes.

Data sent to Coralogix must be serialized in a way conforming to Coralogix's expectations. If your logs are not formatted properly, attempts at processing your logs by your Coralogix endpoint may fail. Here's an example format string for sending data to Coralogix:

1
2
3
4
5
6
7
8
9
10
11
12
{
    "timestamp": 1653088964764,
    "applicationName": "fastly",
    "subsystemName": "wasm",
    "severity": 3,
    "json": {
        "message": "Request happened",
        "response": {
          "status":200
       }
    }
}

You can follow the general JSON structure above regardless of the chosen language for your Compute@Edge service. The following fields are required:

  • timestamp: The format of this field is in milliseconds.
  • applicationName: Enter the name of the application.
  • subsystemName: Enter the name of the subsystem. This field is used to separate components. Use whatever subsystem name makes sense that helps you identify the subsystem.
  • severity: The severity of the log. You can specify the severity to all logs using the following choices: 1 (debug), 2 (verbose), 3 (info), 4 (warning), 5 (error), 6 (critical). This can be changed later using an extraction rule as described in the field below.
  • json (object): Used to specify additional log details as necessary. Nested JSON formats are supported.

    Specifying a nested response.status field is a useful way to identify the status for servicing the request. Using the Coralogix parsing rules, you can set a JSON Extract rule to use the status code value from the log to populate the severity field in the Coralogix interface. Specifically, you can automatically map an HTTP status code to a severity value. For example, status code 2xx will set the Coralogix severity as “INFO” and status code 4xx will set Coralogix severity as “ERROR”.

    In the Coralogix web interface, it will look like this:

    Creating a new Coralogix Rule

Configuring Coralogix dashboards and alerting

Coralogix provides tutorials for integrating their service with Fastly via dashboards and alerting. This includes examples of data dashboards created using Fastly data, including one for a general service overview, a visitor breakdown, and quality of service.

Their tutorials also describe how to set up user-defined alerts for situations like no logs being received from Fastly, outages at your origin, elevated error ratios and cache misses, unusual or suspicious requests of various types, as well as potential website defacement attempts.

Back to Top