match newline, space, newline

vincix · 03-07-2017, 08:52 AM

Given the following text:

Code:

Consult Section 3.1 in the Owner and Operator

 Guide for a description of the tape drives
available on your system.

I'm trying to substitute "Owner and Operator" with "Installation" and to bring "Guide" on the same line, so as to read "Installation Guide", and then start a newline (the last thing is easy). But to do that I want to match "Owner and Operator*newline*newline*space*. Even more, I would like it to match any combination of newline and spaces (perhaps including tabs), regardless of the order. For example, newline-space-newline or doublespace-newline-tab-newline and so on, and so forth.

If I remember correctly, you could do it with square brackets, but I'm not sure how I can do that using \n, \s or \t or regex classes.

This is the original idea:

Code:

/Owner/{
N
s/Owner *\n*and *\n*Operator *\n*Guide/Installation Guide\
/
}

which matches "Owner Operator Guide", even if it's broken across multiple lines, but it doesn't take into account tabs or spaces that occur after the newline,or tabs before the line or some other random order.

vincix · 03-07-2017, 12:30 PM

I guess one of the first problems I need to solve is to somehow 'catch' the second newline. The first newline is caught with N (if I understand correctly). So how can I match several newlines?

rtmistler · 03-07-2017, 01:22 PM

Hi vincix,

I've moved this thread to the Programming forum. I know that persons regularly on the Programming forum will have some thoughts. Myself, I've not played much with newline and regex, instead I've always tackled that internally within an editor a'la copy or cut and then paste that into the search-replace buffer.

danielbmartin · 03-07-2017, 01:52 PM

This is a brute-force solution.

With this InFile ...

Code:

Consult Section 3.1 in the Owner and Operator

 Guide for a description of the tape drives
available on your system.
 
This is another paragraph.
 
This is yet another paragraph.

... this code ...

Code:

tr "\n" "~" <$InFile  \
|sed 's/Owner and Operator~~ Guide /Installation Guide~/g'  \
|tr "~" "\n"  \
>$OutFile

... produced this OutFile ...

Code:

Consult Section 3.1 in the Installation Guide
for a description of the tape drives
available on your system.
 
This is another paragraph.
 
This is yet another paragraph.

Explanation:
tr replaces all newlines with tilde.
sed implements the desired transformation.
tr replaces all tildes with newlines.

Daniel B. Martin

vincix · 03-07-2017, 02:11 PM

I find it a little bit hard to digest it, but at the same time the logic seems simple enough. So you're using tildes just for transition. That's interesting. Thanks. I did expect it to solve it only with sed, just to get to know the tool better, but I guess it's unecessarily intricate.

danielbmartin · 03-07-2017, 02:20 PM

Quote:

Originally Posted by vincix

I find it a little bit hard to digest it, but at the same time the logic seems simple enough. So you're using tildes just for transition. That's interesting. Thanks. I did expect it to solve it only with sed, just to get to know the tool better, but I guess it's unnecessarily intricate.

The first tr changes the entire file into one long line, with tildes as "ghosts" of former newlines. The sed becomes less complicated because it isn't trying to "peek" one or two lines ahead. After sed does its work, another tr replaces any remaining "ghosts" with newlines.

Daniel B. Martin

astrogeek · 03-07-2017, 04:58 PM

Here is one that works with sed only:

Code:

$ sed -r '/Owner/,/Guide/{:a;N;$!ba;s/[ \t\n]+/ /g;s/Owner and Operator/Installation/;s/Guide /Guide\n/}' file.txt
Consult Section 3.1 in the Installation Guide
for a description of the tape drives available on your system.

I have put it on a single line, but you might want to break it up for readability.

UPDATE: End address had no effect because of inner loop as pointed out by danielbmartin - but I had not noticed because with simpler infile it did not matter in quick testing. Moving end address to inner loop test (where it should have been) fixes it for longer infiles, see below.

danielbmartin · 03-07-2017, 06:12 PM

Quote:

Originally Posted by astrogeek

Here is one that works with sed only:

Code:

$ sed -r '/Owner/,/Guide/{:a;N;$!ba;s/[ \t\n]+/ /g;s/Owner and Operator/Installation/;s/Guide /Guide\n/}' file.txt

Using the InFile shown in post #4 this sed produced this OutFile ...

Code:

Consult Section 3.1 in the Installation Guide
for a description of the tape drives available on your system. This is another paragraph. This is yet another paragraph.

It blew away the newlines which separate paragraphs.

Daniel B. Martin

astrogeek · 03-07-2017, 06:27 PM

AH! So it did! I was working only with the original file snippet.

Try this then...

Code:

$ sed -r '/Owner/{:a;N;/Guide/!ba;s/[ \t\n]+/ /g;s/Owner and Operator/Installation/;s/Guide */Guide\n/}' file.txt
Consult Section 3.1 in the Installation Guide
for a description of the tape drives
available on your system.

This is another paragraph.

This is yet another paragraph.

I see a few other changes that could be made, but I think this meets the originally stated conditions. The main point being, of course, to remove multi-line whitespace between the start and end condition and make the given replacements, adjust to suite actual requirements!

MadeInGermany · 03-09-2017, 01:29 PM

Code:

/Owner and Operator/{
# substitute what has matched
s//Installation/
:L
# append the following line with a \n
N
# delete it if blank
s/\n *$//
# repeat if successful
tL
# join the following line
s/\n//
}

MadeInGermany · 03-09-2017, 02:10 PM

A multi-line replacement can be done with perl:

Code:

perl -pe 'BEGIN {undef $/} s/Owner\s+and\s+Operator\s+Guide/Installation Guide\n/'

danielbmartin · 03-09-2017, 02:27 PM

Quote:

Originally Posted by MadeInGermany

A multi-line replacement can be done with perl:

Code:

perl -pe 'BEGIN {undef $/} s/Owner\s+and\s+Operator\s+Guide/Installation Guide\n/'

With this InFile ...

Code:

Consult Section 3.1 in the Owner and Operator

 Guide for a description of the tape drives
available on your system.
 
This is another paragraph.
 
This is yet another paragraph.

Consult Section 3.2 in the Owner and Operator

 Guide for a description of the disk drives
available on your system.
 
This is another paragraph.
 
This is yet another paragraph.

... the perl solution treats only the first Owner and Operator text string. Previous solutions in this thread handle all instances. Is there an easy fix?

Daniel B. Martin

MadeInGermany · 03-09-2017, 02:31 PM

Sure, with the /g modifier.
The whole file is treated as one line (and must fit into memory).

Code:

perl -pe 'BEGIN {undef $/} s/Owner\s+and\s+Operator\s+Guide/Installation Guide\n/g'

suicidaleggroll · 03-09-2017, 03:14 PM

You can do it in vi with:

Code:

:%s/Owner and Operator\n\n Guide /Installation Guide^M/g

Note that to get that "^M" you would hit Ctrl+v Enter