LinuxQuestions.org

There is more to sed(1) than s///

As a followup, bradvan had asked about using sed to delete a block if it contains specific text.

The following deletes all blocks of text that begin with 'aa' and end with 'cc' if they also happend to contain 'bb' somewhere in between:

Code:

Begin='aa';
End='cc';

sed -e "/^$Begin/,/^$End/{H;\$!d;}; x;/bb/d;\${p;x;}" somefile.txt;

This makes use of the hold buffer.

Posted 07-16-2018 at 01:56 PM by Turbocapitalist Turbocapitalist is offline

	Sharing Write Access to a Web Directory for Multiple Users Such much better than the 777 nonsense you see posted around various places. People just don't seem to see the security caveats of such open permissions... personally I think 777 is taking a gamble.
	Posted 02-15-2017 at 05:54 AM by r3sistance Updated 02-15-2017 at 05:56 AM by r3sistance

There is more to sed(1) than s///

syg00 posted a good example of conditional replacement spanning multiple lines. The idea is to remove any line breaks between a string at the start of the line and the next occurring semicolon, whether it is on the same line or one of the next. It also leaves other lines untouched.

Code:

sed '/^Test/ {:a;/;/!{N;$!ba};s/\\n/ /g;n};' test.file

So on lines starting with 'Test', sed(1) loops until either a semicolon is found or the end of the file is encountered. During the loop each line is appended to the pattern space. Once the loop exits, new lines are removed from the whole pattern space.

So with the following input:

Code:

Test_Macro(abc, def, "\n string1 string2 \n test string",
            "test string2 \n");
printf("Welcome to C world..\n");
// Some code or text
     
Test_Macro(asdsadas, abc.str(), "test String1\n");
Another_Macro("Done with this file...\n\n");
// Some code...

The following output would be produced:

Code:

Test_Macro(abc, def, " string1 string2  test string",
            "test string2 ");
printf("Welcome to C world..\n");
// Some code or text
     
Test_Macro(asdsadas, abc.str(), "test String1");
Another_Macro("Done with this file...\n\n");
// Some code...

Edit:

And grail came up with a method using the record separator and output record separator instead:

Code:

awk '/Test/{gsub(/\\n/,"")}1' RS=";" ORS=";" file

Posted 10-28-2016 at 01:52 AM by Turbocapitalist Turbocapitalist is offline

Updated 10-28-2016 at 02:18 AM by Turbocapitalist

	There is more to sed(1) than s/// I see now I said 'line number' where I really should have said 'range'. At the time I could not think of the correct term. You caught on to my idea, however. And for fun, you can combine ranges and patterns - replace foo with bar on lines 3 to 5 inclusive, only if they contain baz: sed '3,5{/baz/s/foo/bar/}'
	Posted 10-20-2016 at 08:47 AM by goumba

	Misuse of sudo(8) and sudoers(5) For those who prefer talks given at meetings there is also an interesting video of Michael W. Lucas: http://blather.michaelwlucas.com/archives/2266
	Posted 10-19-2016 at 11:45 AM by JZL240I-U

There is more to sed(1) than s///

Quote:

Originally Posted by goumba

One other (overlooked) thing about sed, and I frequently see head piped to sed is that sed can operate on lines by line number as well.

You're right. I thought about it when planning the post but could not come up with good use-cases. When just looking for something, I just use grep(1). If I need a little context, I add -A or -B. If it's something I need to change, such as a configuration file or a script, then I fire up the appropriate editor, especially if it's because of an error message.

I have the feeling that maybe line numbers and thus operating on a file by line number was a lot more relevant in the paper teletype days. One couldn't just scroll back and forth interactively.

About the line numbers, I was thinking about how sed(1) can do ranges as well.

sed '3,5s/foo/bar/' will do replacements on lines 3 to 5, inclusive.

The criteria can also be patterns.

sed '/c/,/e/s/foo/bar/' will do replacements on lines between one that has a c to on that has an e, inclusive. Though if the first pattern is not found, it will never start trying the replacements. And if the second pattern is not found, it will keep replacing on through to the end of the file.

What's a good use-case for line numbers?

Posted 10-17-2016 at 04:45 AM by Turbocapitalist Turbocapitalist is offline

There is more to sed(1) than s///

One other (overlooked) thing about sed, and I frequently see head piped to sed is that sed can operate on lines by line number as well. So, one can do things like:

Code:

sed -i '1,10s#^#command="/usr/bin/uptime -p" #' *.pub

Which will only perform your example on the first ten lines of the file(s).

One other point, something people fail to remember (or even read at all), what makes sed less than ideal for X/HTML as you mention is that it only has an idea of a single line, and in these files a block can span multiple.

Posted 10-17-2016 at 01:58 AM by goumba goumba is offline

Updated 10-17-2016 at 02:00 AM by goumba

	Misuse of sudo(8) and sudoers(5) Quote: Or whatever is the graphical equivalent for "apt-get". I'll have to look into that. I guess the closest would be synaptic(8).
	Posted 10-08-2016 at 06:51 AM by goumba

Misuse of sudo(8) and sudoers(5)

Thanks for the comments!

It took me longer to write than I'd care to admit. When to allow or disallow su(1) isn't in there because I ignored it on purpose because the Lucas book covers that pretty well. And I left out speculation on the directions some of the distros risk heading if sudo(8) becomes perceived as just a longer way of writing su(1), but that is what I want to head off.

I agree, mostly, about the *buntus (and Linux Mints and whatevers), and it is actually their defaults that finally motivated me to write the post. It is a big enough topic that it'd be worth it's own post, though I don't think I can cover enough use-cases myself.

About remote root logins, allow me to disagree at least partially. I'd rather not have to worry about the validity of remote root logins especially if there are any roadwarriors to deal with. So let all direct attempts on root be invalid, but have the intermediate step of a relatively unprivileged user login to allow quick identification for accountability. When something is changed as root, there are usually is usually important information that needs to be communicated. In cases where full root access it accepted, it is essential to know who logged in as root and why. Also as one scales up in speed, quantity, or pressure, it becomes increasingly important to limit the possibility for chaos.

Back to sudo(8), to illustrate what is appropriate for the default user, the following is for discussion.

Code:

%sudo ALL=(root:root) /usr/sbin/visudo "",
        /usr/bin/apt-get

Or whatever is the graphical equivalent for "apt-get". I'll have to look into that.

Posted 10-08-2016 at 01:23 AM by Turbocapitalist Turbocapitalist is offline

	Misuse of sudo(8) and sudoers(5) Excellent blogpost. Well done! I echo frankbell's grumpyness on this topic. Misuse of 'sudo' is rife and I mostly blame Canonical/Ubuntu for setting people a bad example.
	Posted 10-07-2016 at 03:57 PM by GazL

	Misuse of sudo(8) and sudoers(5) Excellent. Aside: One of my pet peeves is the creepy sudo fetish of the *buntus, which seems to have persuaded many that using su or (gasp!) logging in as root so as to do administrative tasks are somehow inherently insecure. But I'm just a grumpy old man.
	Posted 10-07-2016 at 02:48 PM by frankbell

	Misuse of sudo(8) and sudoers(5) I don't have much to say other than this was a nicely written piece on something many of us use, and likely not very effectively at that.
	Posted 10-07-2016 at 07:44 AM by goumba

Sharing Write Access to a Web Directory for Multiple Users

You so rock!

Posted 05-29-2016 at 11:29 AM by Habitual