How to search logs between two timestamps in a log file.

ram.venkat84 · 11-19-2009, 12:45 AM

The requirement was to write a shell script for a cron job set for every two hours for all days.
The Script has to scan log files (*.log) for the logs posted only for the last two hours.... and append them in a new file

the log file content appears as below

******************************************************

INFO 21 Oct 09 15:44:48 net.oit.utils.JmsReceiver - JMS Opened
DEBUG 21 Oct 09 15:44:48 net.oit.excite.mbeans.OutBound - Registered JMS listener
INFO 21 Oct 09 15:44:48 net.oit.mbeans.Base - Status now: Running
DEBUG 21 Oct 09 15:44:48 net.oit.excite.mbeans.OutBound - Started the JMS queue status = Running
DEBUG 21 Oct 09 15:44:48 net.oit.excite.mbeans.OutBound - Completed configureAndRun.
DEBUG 23 Oct 09 12:50:14 net.oit.excite.mbeans.OutBound - Started a JMS message......
DEBUG 23 Oct 09 12:50:14 net.oit.excite.mbeans.OutBound - JobId from message

********************************************************

I am clueless abt how to scan/compare based on time stamp seen in above logs.
Pls Help!!!

chrism01 · 11-19-2009, 12:59 AM

Well, you could use awk or cut to grab individual fields, but honestly this is perfect for Perl. It even understands dates in that format.

ghostdog74 · 11-19-2009, 01:04 AM

gawk

Code:

awk 'BEGIN{
    #get time in seconds the script is run
    now=systime()
    current = strftime("%Y:%m:%d:%H:%m:%S",now)
    print "Current date/time is: "current
    onehr = 3600 #seconds in 1 hr
    m=split("Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec",d,"|")
    # convert "Jan" to "01" etc
    for(o=1;o<=m;o++){
      date[d[o]]=sprintf("%02d",o)
    }     
}
{
    mth=date[$3] ; day=$2 ; year="20"$4    
    m=split($5,time,":")
    hr=time[1] ; min=time[2] ; sec=time[3]    
    # prepare to pass to mktime() function
    t=sprintf("%s %s %s %s %s %s" , year,mth,day,hr,min,sec)
    log_entry_time = mktime(t)
    if ( now - log_entry_time <= onehr ){
        # if less than one hour, print
        print $0
    }
}' file

see here for another similar example.