A couple days ago I released two widgets, and since then I've wanted to keep an eye on installations for bug detection and vanity purposes. Tailing the logs for this purpose was becoming cumbersome, so I whipped up this one-liner to tell me what is going on.
Enjoy!
Follow me on Twitter.
Or in Google reader.
Or by email.
Oh, and tell me what I should blog about: yegg@alum.mit.edu
grep [kp].js /var/log/nginx/nginx-access.log | awk '{print $10}' | perl -pe 's/^\"http:\/\/([^\/]+).*\"$/$1/' | sort | uniq -c | sort -n
I'm posting it here to remember it and because it might be useful to you. Here's what it does.
- Greps for the Web log lines desired. In my case, I'm looking for two JS files in a nginx log. In your case, you'll probably want to change everything but the grep.
- Awks out the referrer line. In my nginx log this is the 10th field. In your case, it might be a slightly different #.
- Perls out the domain. You could skip this step if you want to count each different referring URL differently. In my case, the widget is deployed on blogs, and so each post shows up as a different referring URL, and that creates noise, so I grouped them.
- Sorts the domains, so that 5 works.
- Uniqs the domains, i.e. groups & counts them (the -c).
- Sorts the grouped domains by the count, numerically (-n).
Enjoy!
