search close

Data Flows

access_time Updated Sep 21, 2021

This document demonstrates various data flows between the Module and Agent. While MessagePack is the serialization protocol, the data is displayed here in JSON format for easy of reading.

Benign Post Request

Notice how in HeadersIn the Cookie value was redacted, and also that TLSProtocol and TLSCipher are filled in.

{
   "ModuleVersion": "sigsci-module-apache 0.214",
   "ServerVersion": "Apache/2.4.7 (Ubuntu) PHP/5.5.9-1ubuntu4.11 OpenSSL/1.0.1f",
   "ServerFlavor": "prefork",
   "ServerName": "soysauce.in",
   "Timestamp": 1438838135,
   "RemoteAddr": "198.51.100.209",
   "Method": "POST",
   "Scheme": "https",
   "URI": "/add-data"
   "Protocol": "HTTP/1.1",
   "TLSProtocol": "TLSv1.2",
   "TLSCipher": "ECDHE-RSA-AES128-SHA256",
   "HeadersIn": [
       [ "Host", "soysauce.in" ],
       [ "Accept", "*/*" ],
       [ "Connection", "keep-alive" ],
       [ "Cookie", "" ],
       [ "User-Agent", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_4) AppleWebKit/600.7.12 (KHTML, like Gecko) Version/8.0.7 Safari/600.7.12"],
       [ "Accept-Language", "en-us" ],
       [ "Referer", "https://soysauce.in/" ],
       [ "Accept-Encoding", "gzip, deflate" ],
   ],
   "PostData": "foo=bar&company=something"
}

This request was completely benign, so all that is returned is a 200 response (allow the request to proceed).

{
    "WAFResponse": 200
}

And that is end of the request.

Benign request (with 404 error)

$ curl -v '127.0.0.1:8085/junk'
*   Trying 127.0.0.1...
* Connected to 127.0.0.1 (127.0.0.1) port 8085 (#0)
> GET /junk HTTP/1.1
> User-Agent: curl/7.37.1
> Host: 127.0.0.1:8085
> Accept: */*
>
< HTTP/1.1 404 Not Found
< Content-Type: text/plain; charset=utf-8
< Date: Wed, 05 Aug 2015 18:38:24 GMT
< Content-Length: 19
<

would be converted into the following:

{
  "ModuleVersion": "sigsci-sdk-golang 1.0",
  "ServerVersion": "go1.4.2",
  "ServerFlavor": "",
  "ServerName": "127.0.0.1:8085",
  "Timestamp": 1438799904,
  "RemoteAddr": "127.0.0.1",
  "Method": "GET",
  "Scheme": "http",
  "URI": "/junk",
  "Protocol": "HTTP/1.1",
  "HeadersIn": [
      [ "User-Agent",  "curl/7.37.1" ],
      [ "Accept", "*/*" ],
  ],
}

Response is just 200 or allow the response to pass through.

{
    "WAFResponse": 200
}

The server proceeds normally. If at the end of the request, we find that a error condition occurred or that it had an exceptionally large output or took an exceptionally long time to process, we would followup with a PostRequest. Notice how ResponseCode, ResponseMillis, ResponseSize and filled out as well as HeadersOut

{
  "ModuleVersion": "sigsci-sdk-golang 1.0",
  "ServerVersion": "go1.4.2",
  "ServerFlavor": "",
  "ServerName": "127.0.0.1:8085",
  "Timestamp": 1438799904,
  "RemoteAddr": "127.0.0.1",
  "Method": "GET",
  "Scheme": "http",
  "URI": "/junk",
  "Protocol": "HTTP/1.1",
  "WAFResponse": 200,
  "ResponseCode": 404,
  "ResponseMillis": 1,
  "ResponseSize": 19,
  "HeadersIn": [
      [ "User-Agent", "curl/7.37.1" ],
      [ "Accept",  "*/*" ],
  ],
  "HeadersOut": [
      [ "Content-Type",  "text/plain; charset=utf-8" ]
  ]
}

Blocked Request with SQLI and 406

Here are the raw HTTP headers:

$ curl -v '127.0.0.1:8085/junk?id=1+UNION+ALL+SELECT+1'
* Connected to 127.0.0.1 (127.0.0.1) port 8085 (#0)
> GET /junk?id=1+UNION+ALL+SELECT+1 HTTP/1.1
> User-Agent: curl/7.37.1
> Host: 127.0.0.1:8085
> Accept: */*
>
< HTTP/1.1 406 Not Acceptable
< Content-Type: text/plain; charset=utf-8
< Date: Wed, 05 Aug 2015 17:59:46 GMT
< Content-Length: 19
<
406 not acceptable

This translates to the following flow.

Server/Module sends the following to the agent:

{
  "ModuleVersion": "sigsci-sdk-golang 1.0",
  "ServerVersion": "go1.4.2",
  "ServerFlavor": "",
  "ServerName": "127.0.0.1:8085",
  "Timestamp": 1438796694,
  "RemoteAddr": "127.0.0.1",
  "Method": "GET",
  "Scheme": "http",
  "URI": "/junk?id=1+UNION+ALL+SELECT+1",
  "Protocol": "HTTP/1.1",
  "HeadersIn": [
      [ "Accept",  "*/*" ],
      [ "User-Agent",  "curl/7.37.1" ],
  ],
}

The Agent replies with the following. Notice the RequestID is filled in, along with an X-SigSci-Tags header describing was found (SQLi in this case).

{
  "WAFResponse": 406,
  "RequestID": "55c24b96ca84c02201000001",
  "RequestHeaders": [
      [ "X-SigSci-Tags", "SQLI" ]
  ]
}

The request should be blocked, and at the end of the request, and UpdateRequest message.

{
  "RequestID": "55c24b96ca84c02201000001",
  "ResponseCode": 406,
  "ResponseMillis": 1,
  "ResponseSize": 19,
  "HeadersOut": [
      [ "Content-Type", "text/plain; charset=utf-8" ],
  ]
}