sed/awk: Three consecutive blank lines in a file, how to delete two of them?

recomboDNA · 06-16-2010, 12:16 PM

I've spent a stupidly long amount of time trying to figure this out. I think at this point I'm losing braincells instead of gaining knowledge.

Like the subject says, I have a file with three consecutive blank lines. I want to delete two and keep one.

Also, if anyone could direct me towards a guide on regular expressions particularly as they apply to sed, I would be grateful. I am having a hell of a time figuring out the syntax.

druuna · 06-16-2010, 12:24 PM

Hi,

Sed one-liners

That link also has an answer to your question:

Quote:

# delete all CONSECUTIVE blank lines from file except the first; also
# deletes all blank lines from top and end of file (emulates "cat -s")
sed '/./,/^$/!d' # method 1, allows 0 blanks at top, 1 at EOF
sed '/^$/N;/\n$/D' # method 2, allows 1 blank at top, 0 at EOF

Sed - Regular expressions
Feed your search engine with Sed and Regular Expression, you get a ton of hits.

If a book is your thing then I would suggest O'Reilly's Sed & AWK.

Hope this helps.

grail · 06-17-2010, 05:04 AM

If you have awk pointing at gawk, the following can work:

Code:

awk 'gsub(/\n\n\n/,"\n")' RS="\0" file

MTK358 · 06-17-2010, 07:35 AM

Quote:

Originally Posted by grail

If you have awk pointing at gawk, the following can work:

Code:

awk 'gsub(/\n\n\n/,"\n")' RS="\0" file

The part highlighted in bold won't work.

Code:

awk 'BEGIN { RS="\0" } { gsub(/\n\n\n/,"\n")}' file

grail · 06-17-2010, 08:18 AM

Quote:

The part highlighted in bold won't work.

I am guessing that is dependent on versions ... works just fine for me

You can also do other variables like FS after as well

MTK358 · 06-17-2010, 08:19 AM

I don't know, maybe it would work. Haven't tried it.

grail · 06-17-2010, 09:04 AM

Quote:

I don't know, maybe it would work. Haven't tried it.

Are you kidding me!! Please have your facts straight or at least try something before you say it doesn't work or is incorrect.

MTK358 · 06-17-2010, 09:11 AM

Code:

$ awk -v myvar='it works!' 'BEGIN {print myvar}' test.sh
it works!
$ awk myvar='it works!' 'BEGIN {print myvar}' test.sh
awk: cmd. line:1: myvar=it works!
awk: cmd. line:1:                ^ unexpected newline or end of string
$ awk 'BEGIN {print myvar}' myvar='it works!' test.sh

$ awk 'BEGIN {print myvar}' -v myvar='it works!' test.sh

$

grail · 06-17-2010, 09:50 AM

hmmm ... so your pointing out that you didn't read my post!!

In either post do I talk about setting user defined variables? No

If you care to review the awk manuals easily found on the net you will see that the awk internal variables, such as FS or RS, can be set post the
action and pattern string, eg:

Code:

awk '{print NF} FS=":" file

This has the effect of setting the field separator which can be done pre action / pattern statement with -F switch or included inside the statement
using the BEGIN{} setting.

Hopefully this will help you learn a little more about awk.