https://github.com/lucianmarin/logparser

Skip to content
 
Sign up

  * Why GitHub?
    Features -
      + Mobile -
      + Actions -
      + Codespaces -
      + Packages -
      + Security -
      + Code review -
      + Issues -
      + Integrations -
      + GitHub Sponsors -
      + Customer stories-
  * Team
  * Enterprise
  * Explore
      + Explore GitHub -

    Learn and contribute

      + Topics -
      + Collections -
      + Trending -
      + Learning Lab -
      + Open source guides -

    Connect with others

      + The ReadME Project -
      + Events -
      + Community forum -
      + GitHub Education -
      + GitHub Stars program -
  * Marketplace
  * Pricing
    Plans -
      + Compare plans -
      + Contact Sales -
      + Education -

[                    ] 

  *  
    #
    In this repository All GitHub |
    Jump to |

  * No suggested jump to results

  *  
    #
    In this repository All GitHub |
    Jump to |
  *  
    #
    In this user All GitHub |
    Jump to |
  *  
    #
    In this repository All GitHub |
    Jump to |

Sign in
Sign up
{{ message }}

lucianmarin / logparser Public

  * 
  * Notifications
  * Star 61
  * Fork 1

Command line parser for common log format.

View license
61 stars 1 fork
Star
Notifications

  * Code
  * Issues 0
  * Pull requests 0
  * Actions
  * Security
  * Insights

More

  * Code
  * Issues
  * Pull requests
  * Actions
  * Security
  * Insights

master
Switch branches/tags
[                    ]
Branches Tags
Could not load branches
Nothing to show
{{ refName }} default View all branches
Could not load tags
Nothing to show
{{ refName }} default
View all tags
1 branch 0 tags
Code

Latest commit

@lucianmarin
lucianmarin skip domains in referrers output
...
6b7accc Sep 23, 2021
skip domains in referrers output
6b7accc

Git stats

  * 37 commits

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
.github
Create FUNDING.yml
Dec 14, 2020
.gitignore
move tempalte to .py file
Sep 20, 2021
LICENSE
Update LICENSE
Jan 22, 2021
README.md
skip domains in referrers output
Sep 23, 2021
parse.py
skip domains in referrers output
Sep 23, 2021
requirements.txt
generate html using jinja2
Sep 20, 2021
template.py
removed devices
Sep 21, 2021
View code
Logparser Usage Speed Outputs

README.md

 Logparser

Command line parser for common log format (Nginx default).

 Usage

It counts most important data: referrers, operating systems, browsers
and daily unique visitors (IPs).

# Console output
python parse.py sitename.log.gz

# HTML output
python parse.py sitename.log.gz --html ~/sitename/logs.html

# Skip domains in referrers output
python parse.py sitename.log.gz --skip-ref "subreply.com,unfeeder.com"

Install and update PIP packages.

pip install -U -r requirements.txt

 Speed

  * logparser 24,249/s
  * GoAccess 6,234/s

 Outputs

  * HTML output is based on Jinja2 templates. It can be improved as
    you see fit.
  * Console output for Subreply for a fews days in Sep, 2021:

----- Days
 1402 2021-09-14 Tuesday
  893 2021-09-15 Wednesday
  797 2021-09-16 Thursday
  857 2021-09-17 Friday
  951 2021-09-18 Saturday
  945 2021-09-19 Sunday
  869 2021-09-20 Monday

----- Browsers
    2 Chrome Mobile iOS
    2 Bytespider
    2 HeadlessChrome
    3 Links
    4 Firefox iOS
    4 UC Browser
    6 IE
    7 Go-http-client
    7 Opera
    8 Other
   20 Chrome Mobile WebView
   20 Edge
   28 Firefox Mobile
   30 Safari
   42 Chrome Mobile
   56 Mobile Safari
  111 Firefox
 3468 Chrome

----- Operating Systems
   15 Other
   22 Ubuntu
   39 Linux
   63 iOS
   91 Android
  101 Mac OS X
 3474 Windows

----- Referrers
    2 twtxt.net
    2 l.instagram.com
    2 lobste.rs/s/bffayk/what_are_you_doing_this_weekend
    3 ro.linkedin.com
    3 199.247.2.88:80
    3 t.co/sv4adihlka
    3 lobste.rs/s/fe2eph/what_are_you_doing_this_weekend
    4 news.ycombinator.com
    4 1mb.club
    4 old.reddit.com/r/internetisbeautiful
    4 baidu.com
    4 t.co
    5 out.reddit.com/t3_pl9hiu
    5 sjmulder.nl/en/textonly.html
    8 nt
   10 199.247.2.88
   11 lobste.rs
   13 sjmulder.nl
   15 google.com
   16 lucianmarin.com
   49 reddit.com
   85 lobste.rs/s/8fzzmz/what_are_you_doing_this_week

About

Command line parser for common log format.

Topics

python nginx commonlogging

Resources

Readme

License

View license

Sponsor this project

  * ko_fi ko-fi.com/lucianmarin
  * paypal.me/lucianmarin

Languages

  * Python 100.0%

  * (c) 2021 GitHub, Inc.
  * Terms
  * Privacy
  * Security
  * Status
  * Docs

 

  * Contact GitHub
  * Pricing
  * API
  * Training
  * Blog
  * About

You can't perform that action at this time.
You signed in with another tab or window. Reload to refresh your
session. You signed out in another tab or window. Reload to refresh
your session.