[SOLVED] syntax error while running a script

druuna · 12-19-2013, 04:11 AM

Quote:

Originally Posted by santosh0782

yes. today is a self written script

Code:

#!/bin/ksh

TT=$1
k=`date +%d%m%y`$TT
export k
echo "$k"

This seems to be over complicated...

The export k is pointless. today is the mother shell and k is only exported to child shells, it will not be accessible from the calling shell). Which leaves you with echo `date +%d%m%y`$TT. That can be used in the script and then there is no need for this external script at all.

Also: ksh today? If it is a script, call it that way: ./today (I've explained this to you in one of the previous posts).

santosh0782 · 12-19-2013, 05:01 AM

Quote:

Originally Posted by druuna

This seems to be over complicated...

The export k is pointless. today is the mother shell and k is only exported to child shells, it will not be accessible from the calling shell). Which leaves you with echo `date +%d%m%y`$TT. That can be used in the script and then there is no need for this external script at all.

Also: ksh today? If it is a script, call it that way: ./today (I've explained this to you in one of the previous posts).

ok.. we will use echo `date +%d%m%y`$TT this in script so there is no need for that external script.

also i am running the scripts ./scriptname(this way now)

now ..how would we make this existing script to accumulate data for every 30 mins instead of an hour?

druuna · 12-19-2013, 05:32 AM

Quote:

Originally Posted by santosh0782

now ..how would we make this existing script to accumulate data for every 30 mins instead of an hour?

You keep repeating the same question, which I already answered, twice....

I'll repeat for a third time:

Quote:

Originally Posted by druuna

In general: The whole script is build with a 1 hour time frame in mind which makes changing it into a 30 minute driven script rather hard (if not impossible). I would suggest having a good look at the data you have to start with (which seems to be the audit logs) and start fresh from there.

Without knowing what your exact task is and what the input data represents I'm not even capable of giving you a hint on how to start.

santosh0782 · 12-19-2013, 05:57 AM

Quote:

Originally Posted by druuna

You keep repeating the same question, which I already answered, twice....

I'll repeat for a third time:

Without knowing what your exact task is and what the input data represents I'm not even capable of giving you a hint on how to start.

Ans:

in the script:
part of code

Code:

#!/bin/ksh

cd /home/www/BCARD/data/dcp

rm -rf ./pay_dt.txt ./pay_dt1.txt ./DCP-hourly* ./CCCD-hourly* ./KIVR-hourly* ./AS_hourly_pay1*

user=biscsi
dt=`date '+%d%m%y'`

tody=`ksh today`

scp $user@picard.barclaycard.co.uk:/u/epayadm/log/audit.log ./
scp $user@picard.barclaycard.co.uk:/u/epayadm/log/audit$tody.log ./

cat ./audit.log ./audit$tody.log > ./audit1.tmp
sed 's/^/20/g' ./audit1.tmp > ./audit.tmp

for i in `cat ./channel | awk '{print $1}'`
do
fname=`egrep $i ./channel | awk '{print $2}'`
awk -F',' '{print substr($1,0,10),$14,$10,$11}' ./audit.tmp | awk '($2=='000')' | egrep "$i" | awk '{print $1}' | sort | uniq -c > ./$fname-hourly-pay1a.txt
awk -F',' '{print substr($1,0,10),$14,$10,$11}' ./audit.tmp | awk '($2!='000')' | egrep "$i" | awk '{print $1}' | sort | uniq -c > ./$fname-hourly-pay1d.txt
done

Sample of audit.log (First Column is date with HHMMSS):
131219000106,131219000110,100,033536,492181******6214,,,1609,MERCHANT-DECLINE,2000,2893907,826,826,106,8,00,CV2100,8,2,4,4,1,1,1,1
131219000116,131219000119,200,033460,492181******7018,,,1510,019405,15000,4451746,826,826,000,8,00,C V2200,8,2,2,2,1,1,1,1
131219000201,131219000204,100,033481,475130******4811,,,1603,703102,17000,2893907,826,826,000,8,00,C V2100,8,2,2,2,1,1,1,1
131219000204,131219000206,200,033528,475117******9618,,,1512,689211,21534,4451746,826,826,000,8,00,C V2200,8,2,2,2,1,1,1,1
131219000242,131219000244,100,033537,475130******3807,,,1503,637115,2745,2893907,826,826,000,8,00,CV 2100,8,2,2,2,1,1,1,1
131219000419,131219000423,100,033461,476367******6518,,,1406,MERCHANT-DECLINE,916,2893907,826,826,106,8,00,CV2100,8,2,4,4,1,1,1,1
131219000525,131219000528,100,033529,465902******1018,,,1610,052662,21165,2893907,826,826,000,8,00,C V2100,8,2,2,2,1,1,1,1
131219000600,131219000602,200,033622,454742******3999,,,1510,DECLINE,689135,4451746,826,826,002,4,00 ,CV2200,8,2,2,2,,,,
131219000645,131219000648,100,033632,465921******9208,,,1702,064741,22000,2893907,826,826,000,8,00,C V2100,8,2,2,2,1,1,1,1
131219000749,131219000752,100,033530,453978******7407,,,1702,075101,6000,2893907,826,826,000,8,00,CV 2100,8,2,2,2,1,1,1,1
131219000815,131219000818,100,033538,465942******7993,,,1502,003529,10000,2893907,826,826,000,8,00,C V2100,8,2,2,2,1,1,1,1
131219000827,131219000830,100,033539,446291******8627,,,1510,019809,11741,2893907,826,826,000,8,00,C V2100,8,2,2,2,1,1,1,1
131219000844,131219000847,100,033623,475130******6968,,,1608,058367,3000,2893907,826,826,000,8,00,CV 2100,8,2,2,2,1,1,1,1
131219000912,131219000916,200,033531,476367******6518,,,1406,MERCHANT-DECLINE,916,4451746,826,826,106,8,00,CV2200,8,2,4,4,1,1,1,1
131219001101,131219001104,100,033624,446291******8517,,,1601,019988,117215,2893907,826,826,000,8,00, CV2100,8,2,2,2,1,1,1,1
131219001113,131219001116,200,033692,476367******6518,,,1406,019879,916,4451746,826,826,000,8,00,CV2 200,8,2,4,4,1,0,0,0
131219001206,131219001209,100,033540,465942******6460,,,1609,002693,20000,2893907,826,826,000,8,00,C V2100,8,2,2,2,1,1,1,1
131219001215,131219001218,100,033625,492181******2146,,,1407,019639,4757,2893907,826,826,000,8,00,CV 2100,8,2,2,2,1,1,1,1
131219001234,131219001237,100,033633,475117******5299,,,1504,767330,26000,2893907,826,826,000,8,00,C V2100,8,2,2,2,1,1,1,1
131219001542,131219001544,100,033626,453979******5403,,,1611,054361,5000,2893907,826,826,000,8,00,CV 2100,8,2,2,2,1,1,1,1
131219001618,131219001621,100,033634,475117******0721,,,1512,495115,13936,2893907,826,826,000,8,00,C V2100,8,2,2,2,1,1,1,1
131219002109,131219002112,100,033627,475130******7734,,,1605,771357,20000,2893907,826,826,000,8,00,C V2100,8,2,2,2,1,1,1,1
131219002119,131219002122,100,033635,475129******8345,,,1507,300734,100000,2893907,826,826,000,8,00, CV2100,8,2,2,2,1,1,1,1
131219002158,131219002201,100,033693,454313******0425,,,1807,299696,19090,2893907,826,826,000,8,00,C V2100,8,2,2,2,1,1,1,1
131219002345,131219002349,100,033636,475714******6784,,,1411,427560,10000,2893907,826,826,000,8,00,C V2100,8,2,2,2,1,1,1,1
131219002618,131219002621,100,033628,475714******1535,,,1606,580350,5771,2893907,826,826,000,8,00,CV 2100,8,2,2,2,1,1,1,1
131219002634,131219002637,100,033637,453979******7323,,,1510,063628,16599,2893907,826,826,000,8,00,C V2100,8,2,2,2,1,1,1,1
131219002734,131219002737,100,033629,475127******4127,,,1601,765813,20676,2893907,826,826,000,8,00,C V2100,8,2,2,2,1,1,1,1
131219003212,131219003214,100,033541,465901******8002,,,1604,021378,37500,2893907,826,826,000,8,00,C V2100,8,2,2,2,1,1,1,1
131219003305,131219003308,100,033630,475130******9122,,,1402,730113,279679,2893907,826,826,000,8,00, CV2100,8,2,2,2,1,1,1,1
131219003358,131219003400,100,033638,475117******4818,,,1503,475470,15000,2893907,826,826,000,8,00,C V2100,8,2,2,2,1,1,1,1
131219003618,131219003621,100,033639,475130******9122,,,1402,778065,10000,2893907,826,826,000,8,00,C V2100,8,2,2,2,1,1,1,1
131219003723,131219003726,100,033640,454742******1310,,,1505,245030,139666,2893907,826,826,000,8,00, CV2100,8,2,2,2,1,1,1,1
131219003742,131219003745,100,033694,475129******7186,,,1312,212701,58356,2893907,826,826,000,8,00,C V2100,8,2,2,2,1,1,1,1
131219003806,131219003809,100,033912,465860******3103,,,1402,080788,17580,2893907,826,826,000,8,00,C V2100,8,2,2,2,1,1,1,1
131219003904,131219003907,100,033631,475128******4682,,,1605,025236,1927,2893907,826,826,000,8,00,CV 2100,8,2,2,2,1,1,1,1
131219004613,131219004616,100,033641,465859******2101,,,1502,061473,10000,2893907,826,826,000,8,00,C V2100,8,2,2,2,1,1,1,1
131219004628,131219004631,100,034012,675922******7662,,0412,1501,479800,4397,2893907,826,826,000,8,0 0,CV2100,8,2,2,2,1,1,1,1
131219004704,131219004707,100,034022,475116******1794,,,1407,413400,10175,2893907,826,826,000,8,00,C V2100,8,2,2,4,1,1,1,1
131219004730,131219004733,100,033695,465901******9016,,,1502,MERCHANT-DECLINE,60000,2893907,826,826,106,8,00,CV2100,8,2,4,4,1,1,1,1
131219005010,131219005013,200,034023,446272******4036,,,1510,019556,4397,4451746,826,826,000,8,00,CV 2200,8,2,2,2,1,1,1,1
131219005102,131219005105,100,034013,492182******9664,,,1505,019836,268847,2893907,826,826,000,8,00, CV2100,8,2,2,2,1,1,1,1
131219005745,131219005747,100,033913,475132******2962,,,1502,581696,10156,2893907,826,826,000,8,00,C V2100,8,2,2,2,1,1,1,1

in the present script this data is redirecting it to audit.tmp
now the data available in audit.tmp(First column is date with HHMMSS) is:
20131219000106,131219000110,100,033536,492181******6214,,,1609,MERCHANT-DECLINE,2000,2893907,826,826,106,8,00,CV2100,8,2,4,4,1,1,1,1
20131219000116,131219000119,200,033460,492181******7018,,,1510,019405,15000,4451746,826,826,000,8,00 ,CV2200,8,2,2,2,1,1,1,1
20131219000201,131219000204,100,033481,475130******4811,,,1603,703102,17000,2893907,826,826,000,8,00 ,CV2100,8,2,2,2,1,1,1,1
20131219000204,131219000206,200,033528,475117******9618,,,1512,689211,21534,4451746,826,826,000,8,00 ,CV2200,8,2,2,2,1,1,1,1
20131219000242,131219000244,100,033537,475130******3807,,,1503,637115,2745,2893907,826,826,000,8,00, CV2100,8,2,2,2,1,1,1,1
20131219000419,131219000423,100,033461,476367******6518,,,1406,MERCHANT-DECLINE,916,2893907,826,826,106,8,00,CV2100,8,2,4,4,1,1,1,1
20131219000525,131219000528,100,033529,465902******1018,,,1610,052662,21165,2893907,826,826,000,8,00 ,CV2100,8,2,2,2,1,1,1,1
20131219000600,131219000602,200,033622,454742******3999,,,1510,DECLINE,689135,4451746,826,826,002,4, 00,CV2200,8,2,2,2,,,,
20131219000645,131219000648,100,033632,465921******9208,,,1702,064741,22000,2893907,826,826,000,8,00 ,CV2100,8,2,2,2,1,1,1,1
20131219000749,131219000752,100,033530,453978******7407,,,1702,075101,6000,2893907,826,826,000,8,00, CV2100,8,2,2,2,1,1,1,1
20131219000815,131219000818,100,033538,465942******7993,,,1502,003529,10000,2893907,826,826,000,8,00 ,CV2100,8,2,2,2,1,1,1,1
20131219000827,131219000830,100,033539,446291******8627,,,1510,019809,11741,2893907,826,826,000,8,00 ,CV2100,8,2,2,2,1,1,1,1
20131219000844,131219000847,100,033623,475130******6968,,,1608,058367,3000,2893907,826,826,000,8,00, CV2100,8,2,2,2,1,1,1,1
20131219000912,131219000916,200,033531,476367******6518,,,1406,MERCHANT-DECLINE,916,4451746,826,826,106,8,00,CV2200,8,2,4,4,1,1,1,1
20131219001101,131219001104,100,033624,446291******8517,,,1601,019988,117215,2893907,826,826,000,8,0 0,CV2100,8,2,2,2,1,1,1,1
20131219001113,131219001116,200,033692,476367******6518,,,1406,019879,916,4451746,826,826,000,8,00,C V2200,8,2,4,4,1,0,0,0
20131219001206,131219001209,100,033540,465942******6460,,,1609,002693,20000,2893907,826,826,000,8,00 ,CV2100,8,2,2,2,1,1,1,1
20131219001215,131219001218,100,033625,492181******2146,,,1407,019639,4757,2893907,826,826,000,8,00, CV2100,8,2,2,2,1,1,1,1
20131219001234,131219001237,100,033633,475117******5299,,,1504,767330,26000,2893907,826,826,000,8,00 ,CV2100,8,2,2,2,1,1,1,1
20131219001542,131219001544,100,033626,453979******5403,,,1611,054361,5000,2893907,826,826,000,8,00, CV2100,8,2,2,2,1,1,1,1
20131219001618,131219001621,100,033634,475117******0721,,,1512,495115,13936,2893907,826,826,000,8,00 ,CV2100,8,2,2,2,1,1,1,1
20131219002109,131219002112,100,033627,475130******7734,,,1605,771357,20000,2893907,826,826,000,8,00 ,CV2100,8,2,2,2,1,1,1,1
20131219002119,131219002122,100,033635,475129******8345,,,1507,300734,100000,2893907,826,826,000,8,0 0,CV2100,8,2,2,2,1,1,1,1
20131219002158,131219002201,100,033693,454313******0425,,,1807,299696,19090,2893907,826,826,000,8,00 ,CV2100,8,2,2,2,1,1,1,1
20131219002345,131219002349,100,033636,475714******6784,,,1411,427560,10000,2893907,826,826,000,8,00 ,CV2100,8,2,2,2,1,1,1,1
20131219002618,131219002621,100,033628,475714******1535,,,1606,580350,5771,2893907,826,826,000,8,00, CV2100,8,2,2,2,1,1,1,1
20131219002634,131219002637,100,033637,453979******7323,,,1510,063628,16599,2893907,826,826,000,8,00 ,CV2100,8,2,2,2,1,1,1,1
20131219002734,131219002737,100,033629,475127******4127,,,1601,765813,20676,2893907,826,826,000,8,00 ,CV2100,8,2,2,2,1,1,1,1
20131219003212,131219003214,100,033541,465901******8002,,,1604,021378,37500,2893907,826,826,000,8,00 ,CV2100,8,2,2,2,1,1,1,1
20131219003305,131219003308,100,033630,475130******9122,,,1402,730113,279679,2893907,826,826,000,8,0 0,CV2100,8,2,2,2,1,1,1,1
20131219003358,131219003400,100,033638,475117******4818,,,1503,475470,15000,2893907,826,826,000,8,00 ,CV2100,8,2,2,2,1,1,1,1
20131219003618,131219003621,100,033639,475130******9122,,,1402,778065,10000,2893907,826,826,000,8,00 ,CV2100,8,2,2,2,1,1,1,1
20131219003723,131219003726,100,033640,454742******1310,,,1505,245030,139666,2893907,826,826,000,8,0 0,CV2100,8,2,2,2,1,1,1,1
20131219003742,131219003745,100,033694,475129******7186,,,1312,212701,58356,2893907,826,826,000,8,00 ,CV2100,8,2,2,2,1,1,1,1
20131219003806,131219003809,100,033912,465860******3103,,,1402,080788,17580,2893907,826,826,000,8,00 ,CV2100,8,2,2,2,1,1,1,1
20131219003904,131219003907,100,033631,475128******4682,,,1605,025236,1927,2893907,826,826,000,8,00, CV2100,8,2,2,2,1,1,1,1
20131219004613,131219004616,100,033641,465859******2101,,,1502,061473,10000,2893907,826,826,000,8,00 ,CV2100,8,2,2,2,1,1,1,1
20131219004628,131219004631,100,034012,675922******7662,,0412,1501,479800,4397,2893907,826,826,000,8 ,00,CV2100,8,2,2,2,1,1,1,1
20131219004704,131219004707,100,034022,475116******1794,,,1407,413400,10175,2893907,826,826,000,8,00 ,CV2100,8,2,2,4,1,1,1,1
20131219004730,131219004733,100,033695,465901******9016,,,1502,MERCHANT-DECLINE,60000,2893907,826,826,106,8,00,CV2100,8,2,4,4,1,1,1,1
20131219005010,131219005013,200,034023,446272******4036,,,1510,019556,4397,4451746,826,826,000,8,00, CV2200,8,2,2,2,1,1,1,1
20131219005102,131219005105,100,034013,492182******9664,,,1505,019836,268847,2893907,826,826,000,8,0 0,CV2100,8,2,2,2,1,1,1,1
20131219005745,131219005747,100,033913,475132******2962,,,1502,581696,10156,2893907,826,826,000,8,00 ,CV2100,8,2,2,2,1,1,1,1

==============

now this file audit.tmp is used every where in the script.

druuna · 12-19-2013, 06:42 AM

You don't understand what I'm trying to tell you.

The current script cannot be re-written to handle 30 minute intervals. You need a different approach.

You have to get a clear picture of what needs to be done:
- What is the actual task you are given (what is this script supposed to do),
- What input data do you have (audit.log and channel, anything else),
- what do the fields in these data files represent and how can they be used,
- Do you see anything you might run into that is not described in the task given to you,

If you have that information sorted out you can start thinking of ways to implement it into a script. You might even come to the conclusion that a script isn't the way to go and a (small) database solution is better.

santosh0782 · 01-02-2014, 05:57 AM

Hi

i have a file test1 with data :
15|201401021105
19|201401021106
17|201401021107
24|201401021108
18|201401021109
9|201401021110
21|201401021111
16|201401021112
15|201401021113
16|201401021114
13|201401021115
10|201401021116
18|201401021117
11|201401021118
17|201401021119
21|201401021120
22|201401021121
10|201401021122
20|201401021123
19|201401021124
13|201401021125
14|201401021126
25|201401021127
18|201401021128
19|201401021129
21|201401021130
8|201401021131
16|201401021132
3|201401021133
9|201401021134
21|201401021135
16|201401021136
21|201401021137
19|201401021138
16|201401021139
14|201401021140
21|201401021141
16|201401021142
18|201401021143
12|201401021144
17|201401021145
16|201401021146
19|201401021147

i want to add first column (only for first 15 records), how would i do it?

i tried this:
cat test1|awk '{ sum+=$1 } END { print $sum }'

but its not working...

druuna · 01-02-2014, 06:19 AM

The delimiter in the test1 file is a | and awk uses a space/tab as default, so you need to tell awk to use | instead. There are 2 ways of doing this:

Code:

awk -F"|" '....

or

Code:

awk 'BEGIN { FS="|" } ...

You also mention that you only want the first 15 records and not the rest. Awk has some build in variables you can use (8 Powerful Awk Built-in Variables – FS, OFS, RS, ORS, NR, NF, FILENAME, FNR)

In this case you can use NR:

Code:

awk -F"|" 'NR <=15 { sum+=$1 } END { print sum }' test1

santosh0782 · 01-04-2014, 11:30 PM

ok..Thanks a Lot :-)

I'll go through the link provided by you.

santosh0782 · 01-06-2014, 05:32 AM

Hi,

I have a system hour(current hour) in this format 2014010611(YmdH).
I want to create a file which will have last 24 hours time in the above format, starting from current time with window of 1 hour.

could you please help me to get this ?

druuna · 01-06-2014, 06:50 AM

Quote:

Originally Posted by santosh0782

I have a system hour(current hour) in this format 2014010611(YmdH).
I want to create a file which will have last 24 hours time in the above format, starting from current time with window of 1 hour.

I'm not sure what it is you want/need.

- Do you want a file to be filled with 2014010611 -> 2014010710, each entry on one line, or 2014010512 -> 2014010611 or ???
- What decides the start point? Current date or manual input or ???

Please provide an input/start and output example.

santosh0782 · 01-06-2014, 10:17 PM

Quote:

Originally Posted by druuna

I'm not sure what it is you want/need.

- Do you want a file to be filled with 2014010611 -> 2014010710, each entry on one line, or 2014010512 -> 2014010611 or ???
- What decides the start point? Current date or manual input or ???

Please provide an input/start and output example.

ok..let me put the proper requirement.. i'll update here in few mins

Sample Input File audit.tmp which keeps on loading data into it with every few seconds(First column is the count for that particular date and second column is the date(format is YmdHM):
2 201401060002
1 201401060003
1 201401060004
2 201401060005
2 201401060006
1 201401060007
3 201401060008
1 201401060009
2 201401060010
1 201401060012
1 201401060014
2 201401060015
1 201401060016
1 201401060018
2 201401060020
1 201401060021
1 201401060022
1 201401060023
1 201401060025
1 201401060026
1 201401060027
2 201401060028
2 201401060029
1 201401060032
2 201401060034
1 201401060035
1 201401060036
2 201401060037
1 201401060039
1 201401060044
4 201401060049
1 201401060051
2 201401060054
1 201401060057
1 201401060059
1 201401060102
3 201401060105
1 201401060111
1 201401060113
1 201401060117
2 201401060118
1 201401060119
1 201401060120
1 201401060124
2 201401060125
1 201401060129
1 201401060133
1 201401060136
1 201401060138
1 201401060142
1 201401060144
1 201401060147
1 201401060158
1 201401060258
1 201401060308
1 201401060317
1 201401060318
1 201401060333
1 201401060340
1 201401060345
1 201401060357
1 201401060358
1 201401060407
1 201401060410
1 201401060412
1 201401060416
1 201401060422
1 201401060426
1 201401060428
1 201401060433
1 201401060442
1 201401060446
1 201401060448
1 201401060453
1 201401060458
1 201401060503
1 201401060505
1 201401060507
3 201401060511
1 201401060516
1 201401060521
1 201401060522
1 201401060528
1 201401060529
2 201401060534
1 201401060536
1 201401060537
2 201401060538
1 201401060539
1 201401060540
2 201401060541
2 201401060547
1 201401060555
1 201401060557
1 201401060559
2 201401060601
2 201401060602
3 201401060605
1 201401060606
1 201401060609
1 201401060610
1 201401060611
1 201401060614
2 201401060615
4 201401060616
2 201401060617
2 201401060618
2 201401060619
1 201401060620
2 201401060621
1 201401060622
1 201401060623
1 201401060624
1 201401060626
2 201401060627
1 201401060628
4 201401060629
2 201401060631
1 201401060632
3 201401060634
2 201401060636
1 201401060637
3 201401060638
1 201401060639
1 201401060641
1 201401060642
3 201401060643
3 201401060645
4 201401060646
3 201401060647
2 201401060648
1 201401060649
1 201401060650
1 201401060651
1 201401060652
2 201401060653
1 201401060654
2 201401060655
4 201401060656
1 201401060657
2 201401060659
2 201401060700
3 201401060701
2 201401060703
1 201401060704
6 201401060705
5 201401060706
1 201401060707
5 201401060708
2 201401060709
1 201401060710
3 201401060712
2 201401060713
2 201401060714
3 201401060715
2 201401060716
2 201401060719
3 201401060720
4 201401060721
5 201401060722
5 201401060723
2 201401060724
4 201401060725
6 201401060726
4 201401060727
7 201401060728
5 201401060729
2 201401060730
5 201401060731
4 201401060732
1 201401060733
1 201401060734
1 201401060735
5 201401060736
5 201401060737
4 201401060738
2 201401060739
2 201401060740
5 201401060741
1 201401060742
6 201401060743
3 201401060744
4 201401060745
8 201401060746
8 201401060747
6 201401060748
1 201401060749
3 201401060750
8 201401060751
8 201401060752
5 201401060753
4 201401060754
3 201401060755
7 201401060756
6 201401060757
5 201401060758
11 201401060759
7 201401060800
5 201401060801
6 201401060802
5 201401060803
6 201401060804
4 201401060805
4 201401060806
8 201401060807
12 201401060808
2 201401070007
1 201401070009
1 201401070010
1 201401070011
1 201401070012
1 201401070013
1 201401070014
2 201401070015
1 201401070016
1 201401070018
1 201401070021
2 201401070024
1 201401070026
2 201401070028
1 201401070033
4 201401070034
1 201401070039
1 201401070040
1 201401070041
2 201401070045
1 201401070046
1 201401070049
1 201401070054
1 201401070055
1 201401070056
1 201401070058
1 201401070116
3 201401070120
1 201401070123
1 201401070131
1 201401070133
1 201401070137
1 201401070157
1 201401070208
1 201401070213
1 201401070232
1 201401070234
1 201401070255
1 201401070305
1 201401070318
1 201401070336
1 201401070344
1 201401070345
1 201401070411
1 201401070432
1 201401070454
2 201401070459
1 201401070517
1 201401070521
1 201401070523
1 201401070527
1 201401070531
1 201401070533
1 201401070535
2 201401070537
1 201401070538
1 201401070540
2 201401070542
2 201401070543
1 201401070546
2 201401070551
1 201401070553
1 201401070556
2 201401070557
1 201401070559
1 201401070600
1 201401070601
1 201401070604
2 201401070607
1 201401070608
1 201401070609
1 201401070610
1 201401070611
1 201401070613
2 201401070615
1 201401070616
2 201401070617
1 201401070619
1 201401070620
1 201401070621
2 201401070622
2 201401070623
1 201401070626
1 201401070627
2 201401070628

now the requirement is that we need to add every 15 mins count and display it.
let's say current date is 2014010607 now i want to add every 15 mis count for that particular hour and similarly for last 24 hours
for current hour, first 15 mins it will add count from 201401060700 to 201401060715 and will display it.
next from 201401060716 to 201401060730 it will add the count and will display it
next from 201401060731 to 201401060745 it will add the count and will display it
next from 201401060746 to 201401060800 it will add the count and will display it.

similar way it will do for last 24 hours.

start point would be the current hours calculation(with every 15 mins)
.
.
.
.
.
End point would be the last 24 hours calculation(with every 15 mins)

out put would be like this:

Current datehour| addition of 00mins-15 mins count.
Current datehour| addition of 16 mins-30 mins count.
current datehour|addition of 31mins-45mins count.
current datehour|addition of 46mins-59mins count

previous datehour|addition of 00mins-15 mins count.
previous datehour|addition of 16mins-30 mins count.
previous datehour|addition of 31mins-45 mins count.
previous datehour|addition of 46mins-59 mins count.

previous to previous datehour|addition of 00mins-15 mins count.
previous to previous datehour|addition of 16mins-30 mins count.
previous to previous datehour|addition of 31mins-45 mins count.
previous to previous datehour|addition of 46mins-59 mins count.

.
.
.
.
.
.
.
Similar way last 24 hours data we required.

druuna · 01-07-2014, 04:14 AM

I have an idea what it is you want, but I do have another question about the following:

Quote:

Originally Posted by santosh0782

Sample Input File audit.tmp which keeps on loading data into it with every few seconds

Do the audit file(s) you use as input contain one day of input (201401060000 -> 201401062359) or are the start and end points arbitrary (201401060123 -> 201401081847)?

If the second example is true than this will become a lot harder to do. Consider this: The date is 20160228; Is the next day 20160301 or 20160229 (the last, its a leap year) and you also need to take into account the days of normal month's (30 or 31).

santosh0782 · 01-07-2014, 04:59 AM

Quote:

Originally Posted by druuna

I have an idea what it is you want, but I do have another question about the following

o the audit file(s) you use as input contain one day of input (201401060000 -> 201401062359) or are the start and end points arbitrary (201401060123 -> 201401081847)?

If the second example is true than this will become a lot harder to do. Consider this: The date is 20160228; Is the next day 20160301 or 20160229 (the last, its a leap year) and you also need to take into account the days of normal month's (30 or 31).

audit.tmp contains data(yesterdays and today's till current time)

druuna · 01-07-2014, 05:55 AM

Maybe this is something you can use:

Code:

#!/bin/bash

# ------------------------------------------------------------------ #
# start main function
function perDay() {
  startDate="$1"
  i=0

  # fill interval array
  for HOUR in $( seq -f '%02g' 0 23 )
  do
    for MINS in $( seq -f '%02g' 0 15 59 )
    do
      intCheck[i]="${startDate}${HOUR}${MINS}"
      (( i++ ))
    done
  done
  
  # parse input file
  unset cntDelta
  while read VALUE TSTAMP
  do
    for D1 in $( seq 0 95 )
    do
      D2=$(( intCheck[D1] + 14 ))
      if [[ "$TSTAMP" -ge "${intCheck[$D1]}" ]] && [[ "$TSTAMP" -le "$D2" ]]
      then
        cntDelta[$D1]=$(( cntDelta[$D1] + VALUE ))
      fi
    done
  done < audit.tmp
  
  # print found values
  echo "count per 15 minute interval for $startDate :"
  for X in $( seq 0 95 )
  do
    [[ "${cntDelta[$X]}" == "" ]] && cntDelta[$X]="0"
    echo "- ${intCheck[$X]} : ${cntDelta[$X]}"
  done
}

# ------------------------------------------------------------------ #
# Main
todayDate=$( date '+%Y%m%d' )
yesterdayDate=$( date --date="1 day ago" '+%Y%m%d' )

for thisDate in $yesterdayDate $todayDate
do
  perDay "$thisDate" > out.$thisDate
done

# ------------------------------------------------------------------ #
exit 0

The above creates 2 output files, one for today's date and one for yesterdays date, thecontent looks like this:

Code:

$ head  out.20140106
count per 15 minute interval for 20140106 :
- 201401060000 : 17
- 201401060015 : 16
- 201401060030 : 9
- 201401060045 : 9
- 201401060100 : 6
- 201401060115 : 9
- 201401060130 : 5
- 201401060145 : 2
- 201401060200 : 0

$  head out.20140107
count per 15 minute interval for 20140107 :
- 201401070000 : 8
- 201401070015 : 10
- 201401070030 : 8
- 201401070045 : 8
- 201401070100 : 0
- 201401070115 : 5
- 201401070130 : 3
- 201401070145 : 1
- 201401070200 : 2

santosh0782 · 01-08-2014, 12:35 AM

Thanks a Lot :-) let me try this, will let you know the status.