[SOLVED] How to delete large no of files with grep

deva420patra · 09-14-2016, 02:54 AM

Dear Team

I need to delete lots files in a folder (wc is around 2392934). The folder contains files of year 2015 & year 2016.
I want to delete files for year 2015 only and for that I have created a script below:-

echo 'going to gived path'
cd /usr/local/nagios/var/spool/xidpe/
pwd
b=$(ls -lrt | grep '2015' | awk '{print $9}')
echo 'listen tail outcome'
echo 'going to del'
rm -rf $b
exit

but it gives error :- ./test.sh: line 10: /bin/rm: Argument list too long.

Is there any changes need to be done in my script or any other ways to delete these files(only for year 2015).

pan64 · 09-14-2016, 03:01 AM

just try xargs

TenTenths · 09-14-2016, 03:26 AM

Did you try:

Code:

find /usr/local/nagios/var/spool/xidpe/ -type f -name "*2015*" -delete

Try it first without the final -delete to ensure it's going to give the correct results.

TenTenths · 09-14-2016, 03:32 AM

If you really, really, really want to use grep then you could try (untested):

Code:

echo 'going to gived path'
cd /usr/local/nagios/var/spool/xidpe/
pwd
ls -lrt | grep '2015' | awk '{print $9}' | while read -r FILE ; do
  echo 'listen tail outcome'
  echo 'going to del'
  rm -f "${FILE}"
done
exit

Also, in your initial script you use rm -rf which would be dangerous if you've any folders with 2015 in the folder name as it would delete everything inside and under that folder. (Which may be your intended result, but I'm just pointing it out.

syg00 · 09-14-2016, 03:43 AM

Scary.
Better to use find - then you can decide between -delete or -exec. Much more controllable.

deva420patra · 09-14-2016, 05:06 AM

Thanx a lot, The script really worked. Resolved my issue.

Quote:

Originally Posted by TenTenths

If you really, really, really want to use grep then you could try (untested):

Code:

echo 'going to gived path'
cd /usr/local/nagios/var/spool/xidpe/
pwd
ls -lrt | grep '2015' | awk '{print $9}' | while read -r FILE ; do
  echo 'listen tail outcome'
  echo 'going to del'
  rm -f "${FILE}"
done
exit

Also, in your initial script you use rm -rf which would be dangerous if you've any folders with 2015 in the folder name as it would delete everything inside and under that folder. (Which may be your intended result, but I'm just pointing it out.

Turbocapitalist · 09-14-2016, 06:38 AM

Quote:

Originally Posted by TenTenths

Did you try:

Code:

find /usr/local/nagios/var/spool/xidpe/ -type f -name "*2015*" -delete

Try it first without the final -delete to ensure it's going to give the correct results.

The utility "find" also has some simple regex capabilities using the -regex option. But it's missing built-in PCRE but that can be worked around:

Since "find" does a logical AND between tests, you could use an extra -exec to run "perl" and get full perl regular expressions.

Code:

find . -name '*' -exec perl -e 'if ( q({}) =~ /2015/ ) { exit 0 } else { exit 1 }' \; -exec echo '-delete "{}"' \; -print

Habitual · 09-14-2016, 07:14 PM

Quote:

Originally Posted by deva420patra

I want to delete files for year 2015 only and for that I have created a script below:-

Code:

echo 'going to gived path'
cd /usr/local/nagios/var/spool/xidpe/
pwd
b=$(ls -lrt | grep '2015' | awk '{print $9}')
echo 'listen tail outcome'
echo 'going to del'
rm -rf $b
exit

Sorry, but no.

Code:

b=$(ls -lrt | grep '2015' | awk '{print $9}')

is a train-wreck.
See http://mywiki.wooledge.org/BashPitfalls
for better example of how to ls stuff and process it.

Observations:

Code:

ls -lrt

output may vary on different systems as "ls" is aliased all over the place these days.
To illustrate run

Code:

bash --norc

then

Code:

ls -ltr

Try find with -mtime or -ctime? See also http://www.cyberciti.biz/faq/howto-f...files-by-date/

Let us know!

Sefyir · 09-14-2016, 07:50 PM

Quote:

See http://mywiki.wooledge.org/BashPitfalls

Nice link, going to read that

Quote:

Thanx a lot, The script really worked. Resolved my issue.

Why did you use this when there are better, safer and simpler methods? It was provided as a untested pseudo example.
Use proper methods and tools. It will save you from data loss / headache in the future.

Habitual · 09-14-2016, 10:14 PM

Quote:

Originally Posted by Sefyir

Nice link, going to read that

Why did you use this when there are better, safer and simpler methods? It was provided as a untested pseudo example.
Use proper methods and tools. It will save you from data loss / headache in the future.

Well, they did say

Quote:

Originally Posted by deva420patra

any other ways to delete these files(only for year 2015).

and TeamLQ responded in spades.
Perhaps they just need some encouragement/links/examples...

balvinders · 09-15-2016, 01:15 AM

i m also serching for the same. plz help me. thanks in advance

pan64 · 09-15-2016, 01:43 AM

what kind of help do you need? Where did you stuck?