https://github.com/cosimo/vlogdump

Small tool to filter/convert/match raw varnishlog output
https://github.com/cosimo/vlogdump

awk logging varnish varnishlog

Last synced: about 1 month ago
JSON representation

Small tool to filter/convert/match raw varnishlog output

• Host: GitHub
• URL: https://github.com/cosimo/vlogdump
• Owner: cosimo
• Created: 2013-04-24T13:52:56.000Z (almost 12 years ago)
• Default Branch: master
• Last Pushed: 2018-09-05T09:38:32.000Z (over 6 years ago)
• Last Synced: 2024-11-05T20:50:14.083Z (3 months ago)
• Topics: awk, logging, varnish, varnishlog
• Language: Awk
• Homepage: http://www.streppone.it/cosimo/blog/2013/05/display-and-filter-traffic-at-the-varnish-level-vlogdump/
• Size: 16.6 KB
• Stars: 54
• Watchers: 10
• Forks: 7
• Open Issues: 0
• Metadata Files:
  • Readme: README.pod

Awesome Lists containing this project

README

        
=head1 NAME

vlogdump

=head1 DESCRIPTION

Tool to parse varnishlog output and keep track of transactions.

Match those that we want to debug and show detailed information

about them.

This tool is meant to help when setting up or troubleshooting

a varnish configuration.

It's confirmed working with Varnish 2.x, 3.x, 4.x and 5.x.

For some versions, it might be necessary to use a specific

development branch instead of master.

=head1 EXAMPLES

Here's some examples of usage.

=head2 Show all requests that are cache misses

  $ varnishlog | vlogdump -v only_misses=1

  172.22.0.15 => GET /assets/tn/m/mq/e85ed0a7b1b87120a0a2bfa025531c6733a48802 HTTP/1.0 MISS

              <= 200 OK                                                          28.432 ms

  172.22.0.18 => GET /assets/tn/m/mq/5a9e9440c5c85e8dc5d65e03e15c95e390901fa7 HTTP/1.0 MISS

              <= 200 OK                                                          36.905 ms

  172.22.0.18 => GET /assets//icons/categories/te/icon32x32-technology.png HTTP/1.0 MISS

              <= 304 Not Modified                                                 0.589 ms

  172.22.0.15 => GET /api/fetch/article-preview/?client=2&language=en-GB HTTP/1.1 MISS

              <= 301 MOVED PERMANENTLY                                            8.381 ms

  172.22.0.18 => GET /assets/tn/m/mq/c3830e95b717761005e26ce49ebab253e0ccb40b HTTP/1.0 MISS

              <= 200 OK                                                         291.354 ms

  172.22.0.18 => GET /api/1.0/category/list?client=2&language=en-GB HTTP/1.1 MISS

              <= 200 OK                                                          58.025 ms

  ...

=head2 Show only requests that are cache hits

Same as above, but C<< -v only_hits=1 >>.

=head2 Show only requests (not) for certain URLs

  $ varnishlog | vlogdump -v url_match=assets

To negate the pattern, prepend a C character before the

regular expression. You will get only requests for URLs that

B match C:

  $ varnishlog | vlogdump -v url_match=!assets

=head2 For each transaction, show the request headers

  $ varnishlog | vlogdump -v show_req_headers=1

  172.22.0.15 => GET /assets/3a626ed2f029fc5802c244673c6e168914568080 HTTP/1.0 MISS

              <= 200 OK                                                          51.483 ms

      req.http.Host = discovery.opera.com

      req.http.User-Agent = Amazon CloudFront

      req.http.X-Forwarded-For = 111.12.34.56

      req.http.X-Amz-Cf-Id = NS3KqU_AKnueKyQcJWQXbLyVAYR7jvPJ5QGl9raDg0AZZaPkt87avA==

      req.http.Connection = keep-alive

      ...

=head2 Only show transactions where the request headers match X

Where C can be a string or a regular expression:

  $ varnishlog | vlogdump -v show_req_headers=1 -v req_headers_match=X

=head2 ...

and there's more. Check the full list of options below.

=head1 OPTIONS

This being an C script, all options need to be supplied

in the form C<< -v = >>.

=over

=item client_match=

Shows only transactions from clients (IP addresses) matching

a given regular expression.

=item show_req_headers=0|1

Turn on or off the output of HTTP request headers

=item show_resp_headers=0|1

Turn on or off the output of HTTP response headers

=item req_headers_match=

Shows only transactions with request headers

(any header) matching the given regular expression

Prepend '!' to the pattern to negate the meaning,

to display only transactions that *do not* match the

pattern.

=item resp_headers_match=

Shows only transactions with response headers

(any header) matching the given regular expression

Prepend '!' to the pattern to negate the meaning,

to display only transactions that *do not* match the

pattern.

=item req_method=

Shows only transactions with the given HTTP request

method, or a method that matches the given regexp.

Prepend '!' to the pattern to negate the meaning,

to display only transactions that *do not* match the

pattern.

=item only_hits=0|1

Shows only transactions that were served from the cache

=item only_misses=0|1

Shows only transactions that were served by going to

the backend server.

=item only_slow=1

=item only_slow=

Shows only requests that are considered slow.

The time used is the backend response time.

If you use C, the default threshold is 1s.

You can also specify your own threshold, using

for example C. Value is in milliseconds.

=item url_match=

Show only the transactions for URLs matching the

given regular expression.

Prepend '!' to the pattern to negate the meaning,

to display only transactions where the URL *does not* match

the given regular expression.

Example:

    varnishlog | vlogdump -v url_match='!/assets'

=back

=head1 AUTHOR

Cosimo Streppone, C<< [email protected] >>