Professional Documents
Culture Documents
Today, I had analyzed the Apache log files to view IP of the visitors to my website.
I used more & less command for this. After some time I got tired due to the size
of log file.
It’s not a easy task to read entire log when you want a specific information. I left
my work in between and I was thinking. Is there any other ways to read the log
files efficiently?
Initially I got few ideas to do that then I did the deep analyze and found so many
ways to do that.
I had decided to write about this an article so that others can get to know. What
are the ways to do?
Suggested Read : lnav – An Advanced Console Based Log File Viewer for Linux
It should be 01\/Feb\/2018:07:00:00.
Details:
or
203.99.204.141 - - [12/Feb/2018:08:06:19 -0700] "GET / HTTP/1.1" 301 2355 "-" "Mozilla/5.0 (Windows NT 6.1;
WOW64; Trident/7.0; rv:11.0) like Gecko"
203.99.204.141 - - [12/Feb/2018:08:06:19 -0700] "GET / HTTP/1.1" 200 6786 "-" "Mozilla/5.0 (Windows NT 6.1;
WOW64; Trident/7.0; rv:11.0) like Gecko"
The above output display one line with third day values. If you want to remove
that, use the following sed command.
203.99.204.141 - - [12/Feb/2018:08:06:19 -0700] "GET / HTTP/1.1" 301 2355 "-" "Mozilla/5.0 (Windows NT 6.1;
WOW64; Trident/7.0; rv:11.0) like Gecko"
203.99.204.141 - - [12/Feb/2018:08:06:19 -0700] "GET / HTTP/1.1" 200 6786 "-" "Mozilla/5.0 (Windows NT 6.1;
WOW64; Trident/7.0; rv:11.0) like Gecko"
Apart from Apache logs, most of the logs are logged on Linux in the following
format. Hence, adding an example for the same.
or
Feb 4 04:47:10 centos.2daygeek sshd[17502]: pam_unix(sshd:session): session closed for user magesh
Feb 4 04:49:45 centos.2daygeek sshd[19246]: Accepted password for magesh from 192.168.1.108 port 48336 ssh2
Feb 4 04:49:45 centos.2daygeek sshd[19246]: pam_unix(sshd:session): session opened for user magesh by (uid=0)
Feb 4 04:59:13 centos.2daygeek sshd[27670]: Accepted password for daygeek from 192.168.47.220 port 59739
ssh2
Feb 4 04:59:13 centos.2daygeek sshd[27670]: pam_unix(sshd:session): session opened for user daygeek by (uid=0)
Feb 5 23:00:52 centos.2daygeek sshd[2949]: pam_unix(sshd:session): session closed for user magesh
Feb 5 23:01:39 centos.2daygeek sshd[25377]: pam_unix(sshd:session): session closed for user magesh
Feb 5 23:04:44 centos.2daygeek sshd[7227]: Accepted password for magesh from 192.168.1.108 port 56142 ssh2
Feb 5 23:04:44 centos.2daygeek sshd[7227]: pam_unix(sshd:session): session opened for user magesh by (uid=0)
Feb 5 23:38:58 centos.2daygeek sshd[5486]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0
tty=ssh ruser= rhost=xxx.com
Feb 5 23:38:58 centos.2daygeek sshd[5486]: pam_succeed_if(sshd:auth): error retrieving information about user
thanu
Feb 6 00:13:52 centos.2daygeek sshd[5413]: Accepted password for sudha from 192.168.1.108 port 49273 ssh2
2) How to read log file between two timestamps with different Dates
Run the following commands to read the log file when you have the requirement
to read the files between two timestamps with in a day or different day. Make
sure you have to include date as well otherwise you can’t get the proper output.
This can be done using the following sed or awk command combination.
In this example, we are going to read Apache access log file from 12th Feb,
2018:14:51:17 to 13th Feb, 2018:10:18:30.
or
This can be done using the following sed or awk command combination.
In this example, we are going to read secure log file from 4th Feb, 2018 22:11:32
to 4th Feb, 2018 23:04:45.
or
Feb 4 22:11:32 centos.2daygeek sshd[28006]: pam_unix(sshd:session): session closed for user sudha
Feb 4 22:47:19 centos.2daygeek sshd[11080]: pam_unix(sshd:session): session closed for user magesh
Feb 4 22:49:45 centos.2daygeek sshd[1229]: Accepted password for magesh from 192.168.1.108 port 49058 ssh2
Feb 4 22:49:45 centos.2daygeek sshd[1229]: pam_unix(sshd:session): session opened for user magesh by (uid=0)
Feb 4 23:02:02 centos.2daygeek sshd[13323]: Accepted password for magesh from 192.168.1.108 port 51876 ssh2
Feb 4 23:02:02 centos.2daygeek sshd[13323]: pam_unix(sshd:session): session opened for user magesh by (uid=0)
Feb 4 23:02:49 centos.2daygeek sshd[1229]: pam_unix(sshd:session): session closed for user magesh
Feb 4 23:03:08 centos.2daygeek sshd[13323]: pam_unix(sshd:session): session closed for user magesh
Feb 4 23:04:45 centos.2daygeek sshd[16545]: Accepted password for magesh from 192.168.1.108 port 52486 ssh2
Feb 4 22:11:32 centos.2daygeek sshd[28006]: pam_unix(sshd:session): session closed for user sudha
Feb 4 22:47:19 centos.2daygeek sshd[11080]: pam_unix(sshd:session): session closed for user magesh
Feb 4 22:49:45 centos.2daygeek sshd[1229]: Accepted password for magesh from 192.168.1.108 port 49058 ssh2
Feb 4 22:49:45 centos.2daygeek sshd[1229]: pam_unix(sshd:session): session opened for user magesh by (uid=0)
Feb 4 23:02:02 centos.2daygeek sshd[13323]: Accepted password for magesh from 192.168.1.108 port 51876 ssh2
Feb 4 23:02:02 centos.2daygeek sshd[13323]: pam_unix(sshd:session): session opened for user magesh by (uid=0)
Feb 4 23:02:49 centos.2daygeek sshd[1229]: pam_unix(sshd:session): session closed for user magesh
Feb 4 23:03:08 centos.2daygeek sshd[13323]: pam_unix(sshd:session): session closed for user magesh
Feb 4 23:04:45 centos.2daygeek sshd[16545]: Accepted password for magesh from 192.168.1.108 port 52486 ssh2
Feb 4 23:04:45 centos.2daygeek sshd[16545]: pam_unix(sshd:session): session opened for user magesh by (uid=0)
Feb 5 02:18:37 centos.2daygeek sshd[6223]: Accepted password for daygeek from 192.168.1.108 port 51529 ssh2
Feb 5 02:18:37 centos.2daygeek sshd[6223]: pam_unix(sshd:session): session opened for user daygeek by (uid=0)
Feb 5 02:18:49 centos.2daygeek sshd[6392]: Accepted password for daygeek from 192.168.1.108 port 51531 ssh2
Feb 5 02:18:49 centos.2daygeek sshd[6392]: pam_unix(sshd:session): session opened for user daygeek by (uid=0)
Feb 5 02:43:37 centos.2daygeek sshd[30554]: Accepted password for daygeek from 192.168.1.108 port 51714 ssh2
Feb 4 11:04:45 centos.2daygeek sshd[3951]: Accepted password for magesh from 192.168.1.108 port 42864 ssh2
Feb 4 11:04:45 centos.2daygeek sshd[3951]: pam_unix(sshd:session): session opened for user magesh by (uid=0)
Feb 4 13:10:05 centos.2daygeek sshd[28006]: Accepted password for sudha from 192.168.1.108 port 46141 ssh2
Feb 4 13:10:05 centos.2daygeek sshd[28006]: pam_unix(sshd:session): session opened for user sudha by (uid=0)
Feb 4 13:10:05 centos.2daygeek sshd[28008]: subsystem request for ftp
Feb 4 13:11:16 centos.2daygeek sshd[29043]: Accepted password for sudha from 192.168.1.108 port 46162 ssh2
Feb 4 13:11:16 centos.2daygeek sshd[29043]: pam_unix(sshd:session): session opened for user sudha by (uid=0)
Feb 4 13:13:04 centos.2daygeek sshd[29043]: pam_unix(sshd:session): session closed for user sudha
Feb 4 16:46:59 centos.2daygeek sshd[3951]: pam_unix(sshd:session): session closed for user magesh
Feb 4 16:49:45 centos.2daygeek sshd[11080]: Accepted password for magesh from 192.168.1.108 port 59280 ssh2
Feb 4 16:49:45 centos.2daygeek sshd[11080]: pam_unix(sshd:session): session opened for user magesh by (uid=0)
Feb 4 21:49:04 centos.2daygeek sshd[5967]: pam_unix(sshd:session): session closed for user vinoth
Feb 4 22:11:32 centos.2daygeek sshd[28006]: pam_unix(sshd:session): session closed for user sudha
Feb 5 09:03:28 centos.2daygeek sshd[14950]: Accepted password for sudha from 192.168.1.108 port 37102 ssh2
Feb 5 09:03:28 centos.2daygeek sshd[14950]: pam_unix(sshd:session): session opened for user sudha by (uid=0)
Feb 5 09:04:32 centos.2daygeek sshd[14950]: pam_unix(sshd:session): session closed for user sudha
Feb 5 09:05:54 centos.2daygeek sshd[17960]: Accepted password for sudha from 192.168.1.108 port 37147 ssh2
Feb 5 09:05:54 centos.2daygeek sshd[17960]: pam_unix(sshd:session): session opened for user sudha by (uid=0)
.
- 192.168.1.103 www.2daygeek.com - - [15/Feb/2018:00:41:28 -0700] "GET / HTTP/1.1" 301 -
It’s for different format. The below command will print 3 days logs. Starting from
Feb 3rd, 2018 to Feb 6th, 2018.
Feb 4 04:47:10 centos.2daygeek.com sshd[17502]: pam_unix(sshd:session): session closed for user magesh
Feb 4 04:49:45 centos.2daygeek.com sshd[19246]: Accepted password for magesh from 192.168.1.105 port 48336
ssh2
Feb 4 04:49:45 centos.2daygeek.com sshd[19246]: pam_unix(sshd:session): session opened for user magesh by
(uid=0)
Feb 4 04:59:13 centos.2daygeek.com sshd[27670]: Accepted password for daygeek from 192.168.1.105 port 59739
ssh2
Feb 4 04:59:13 centos.2daygeek.com sshd[27670]: pam_unix(sshd:session): session opened for user daygeek by
(uid=0)
Feb 5 02:18:37 centos.2daygeek.com sshd[6223]: Accepted password for daygeek from 192.168.1.105 port 51529
ssh2
Feb 5 02:18:37 centos.2daygeek.com sshd[6223]: pam_unix(sshd:session): session opened for user daygeek by
(uid=0)
Feb 5 02:18:49 centos.2daygeek.com sshd[6392]: Accepted password for daygeek from 192.168.1.105 port 51531
ssh2
Feb 5 02:18:49 centos.2daygeek.com sshd[6392]: pam_unix(sshd:session): session opened for user daygeek by
(uid=0)
Feb 5 02:43:37 centos.2daygeek.com sshd[30554]: Accepted password for daygeek from 192.168.1.105 port 51714
ssh2
Feb 5 02:43:37 centos.2daygeek.com sshd[30554]: pam_unix(sshd:session): session opened for user daygeek by
(uid=0)
Feb 6 22:46:49 centos.2daygeek.com sshd[16959]: pam_unix(sshd:session): session closed for user magesh
Feb 6 22:49:44 centos.2daygeek.com sshd[29878]: Accepted password for magesh from 192.168.1.105 port 53248
ssh2
Feb 6 22:49:44 centos.2daygeek.com sshd[29878]: pam_unix(sshd:session): session opened for user magesh by
(uid=0)
Feb 6 23:02:46 centos.2daygeek.com sshd[10220]: Accepted password for magesh from 192.168.1.105 port 56332
ssh2
Feb 6 23:02:46 centos.2daygeek.com sshd[10220]: pam_unix(sshd:session): session opened for user magesh by
(uid=0)
Feb 6 23:02:51 centos.2daygeek.com sshd[10220]: pam_unix(sshd:session): session closed for user magesh
Feb 6 23:03:59 centos.2daygeek.com sshd[29878]: pam_unix(sshd:session): session closed for user magesh
Feb 6 23:04:44 centos.2daygeek.com sshd[12684]: Accepted password for magesh from 192.168.1.105 port 56730
ssh2
Feb 6 23:04:44 centos.2daygeek.com sshd[12684]: pam_unix(sshd:session): session opened for user magesh by
(uid=0)
/var/log/messages:Feb 14 11:26:42 Arch.2daygeek.com kernel: GPT: Use GNU Parted to correct GPT /errors.
/var/log/messages:Feb 14 11:26:55 Arch.2daygeek.com kernel: GPT: Use GNU Parted to correct GPT errors.
/var/log/messages:Feb 14 11:27:04 Arch.2daygeek.com kernel: GPT: Use GNU Parted to correct GPT errors.
/var/log/messages:Feb 14 11:27:13 Arch.2daygeek.com kernel: GPT: Use GNU Parted to correct GPT errors.
/var/log/messages:Feb 14 16:59:14 Arch.2daygeek.com kernel: GPT: Use GNU Parted to correct GPT errors.
/var/log/messages:Feb 14 16:59:14 Arch.2daygeek.com kernel: GPT: Use GNU Parted to correct GPT errors.
/var/log/messages:Feb 14 16:59:14 Arch.2daygeek.com kernel: GPT: Use GNU Parted to correct GPT errors.
/var/log/messages:Feb 14 16:59:14 Arch.2daygeek.com kernel: GPT: Use GNU Parted to correct GPT errors.
Feb 8 17:17:52 Arch.2daygeek.com kernel: GPT: Use GNU Parted to correct GPT errors.
Feb 8 17:17:52 Arch.2daygeek.com kernel: GPT: Use GNU Parted to correct GPT errors.
Feb 8 17:17:52 Arch.2daygeek.com kernel: GPT: Use GNU Parted to correct GPT errors.
Feb 8 17:17:52 Arch.2daygeek.com kernel: WARNING! power/level is deprecated; use power/control instead
or
/var/log/messages:Feb 8 17:17:52 Arch.2daygeek.com kernel: GPT: Use GNU Parted to correct GPT errors.
/var/log/messages:Feb 8 17:17:52 Arch.2daygeek.com kernel: GPT: Use GNU Parted to correct GPT errors.
/var/log/messages:Feb 8 17:17:52 Arch.2daygeek.com kernel: GPT: Use GNU Parted to correct GPT errors.
/var/log/messages:Feb 8 17:17:52 Arch.2daygeek.com kernel: GPT: Use GNU Parted to correct GPT errors.
/opt/magi.txt:Magi-Thanu
/opt/magi.txt:Magi-Thanu