LinuxQuestions.org - [SOLVED] how to escape $ (dollar sign) when echoing environment variable

- Linux - General (https://www.linuxquestions.org/questions/linux-general-1/)

- - how to escape $ (dollar sign) when echoing environment variable (https://www.linuxquestions.org/questions/linux-general-1/how-to-escape-%24-dollar-sign-when-echoing-environment-variable-4175736499/)

how to escape $ (dollar sign) when echoing environment variable

I am looping through a list of files:

Code:

#!/bin/bash

cat fileList | while read

do

    ls "$REPLY"

done

The problem is that some of the files have '$' signs, e.g. "dir/$this file". echo, and printf attempt to expand $this as an environment variable giving "dir/ file" which of course the ls command doesn't find.

Is there a way to preserve the '$'? I probably need to change that to "dir/\$this file". How can I do that?

Oh my.
There are a few things wrong with this, but let me ask before we start
IS this homework? Because HOW I answer depends upon WHY you are asking.

And while I wait for that information:
Do you understand how to use "read"? Because that is not how you use "read" in a loop".
And the related...
Where does the environment variable REPLY get set? Because it is never set in that script.

Where does the fileList come from? Has it structure, and what does it look like?

WHY in the world have you files containing shell special characters in the names???

Why call 'cat' in this script?

Is there some restriction that you must use BASH for this, because it may not be the optimal tool for the job.

It shouldn't. I tested your script and it prints dollars in filenames just fine on my machine. Weird. Maybe some shopt? Or maybe you simplified the bug out of it?

later: or is it bash at all? Could be busybox, if it's a router or something. What does 'ls -l /bin/bash' say?

Quote:

Originally Posted by wpeckham (Post 6498644)

Do you understand how to use "read"? Because that is not how you use "read" in a loop".
And the related...
Where does the environment variable REPLY get set? Because it is never set in that script.

Ooooohhh.... this is the worst way to make a fool of yourself - a didactic rant, and you are wrong. This will smart :)

The read command without a name variable will automatically save input to $REPLY.

Code:

read

echo $REPLY

The loop as written is technically correct syntax but not best practices. I agree that there must be something else going on because it should work. It might depend on the exact contents of your filelist.

Code:

filelist.txt

/dir1/$file1

/dir2/$file2



while read -r line

do

    echo "$line"

done < filelist.txt

Thank you both for that correction. I had always used explicit variables and NEVER used the REPLY variable!

I would still like to see the answers to some of my other questions.

OF note, we both want to see the nature of that file. Without understand the input the processing and output are not determined.

Quote:

Originally Posted by lvm_ (Post 6498648)

later: or is it bash at all? Could be busybox, if it's a router or something. What does 'ls -l /bin/bash' say?

Note the first line (the #! line) of the script. To my knowledge, that would not work under busybox. (Although I have NOT tested that.)

OK, yes, I simplified the script too much. The problem is with echo. To clarify, fileList is ... a list of files! an example of an entry in this list containing a "$":

Code:

/mnt/public/Pension Files/1 Unpaid Leave-Military Leave/Military Leave reports/~$2023.02.25 Military Leave.xlsx

if I assign that to a variable:

Code:

$ x=$(echo "/mnt/public/Pension Files/1 Unpaid Leave-Military Leave/Military Leave reports/~$2023.02.25 Military Leave.xlsx")

$ echo $x

/mnt/public/Pension Files/1 Unpaid Leave-Military Leave/Military Leave reports/~023.02.25 Military Leave.xlsx

You can see that the "$2023" becomes "023". Why I am doing this in a while loop shouldn't matter to the question, and yes, I solved the problem another way, but short answer is that I have a list of files in a directory and subdirectories collected from another machine and I want to do something with these filepaths. The way I solved it for my purposes was to edit fileList and replace all $ with \$.

So, the real question is how to preserve '$' in an echo and not have echo attempt to replace with an env. variable.

To the other questions and comments:

Quote:

Originally Posted by wpeckham (Post 6498644)

Oh my.
There are a few things wrong with this, but let me ask before we start
Do you understand how to use "read"? Because that is not how you use "read" in a loop".

Well, I've been using read in such loops for 20+ years. If you read input lines into a bash script some other way, I'd be interested to see that.

Quote:

And the related...
Where does the environment variable REPLY get set? Because it is never set in that script.

I believe this was answered by michaelk. In addition, using REPLY preserves leading whitespace in the line read whereas 'while read var' does not, even if you quote it inside the loop: echo "$var".

Quote:

Where does the fileList come from? Has it structure, and what does it look like?

I gave an acutal example above.

Quote:

WHY in the world have you files containing shell special characters in the names???

Not my choice. These are Windows filenames.

Quote:

Why call 'cat' in this script?

Why not? I could redirect stdin as shown in michaelk's example, but I think this is a matter of preference.

Quote:

Is there some restriction that you must use BASH for this, because it may not be the optimal tool for the job.

Sure, but I'd like to know if there is a way to deal with this in bash.

Quote:

Originally Posted by michaelk (Post 6498672)

The read command without a name variable will automatically save input to $REPLY.
The loop as written is technically correct syntax but not best practices. ...

Hmmmm - what's not "best practice" about it?

Quote:

Originally Posted by wpeckham (Post 6498725)

Note the first line (the #! line) of the script. To my knowledge, that would not work under busybox. (Although I have NOT tested that.)

This is pretty standard https://www.baeldung.com/linux/shebang:

Quote:

If we try to run a text file, execve expects the first two characters of the file to be “#!” (read “shebang” or “hashbang”) followed by a path to the interpreter that will be used to interpret the rest of the script.

Pretty much all my distro supplied bash scripts have "#!/bin/bash" or "#!/bin/sh" as the first line. And it can be used for scripts besides bash: "#!/usr/bin/perl" for .pl scripts.

So, if anyone has a solution to the echo "$var" problem where var contains '$', I'm still interested.

x=$(echo "something") is just wrong. I mean it is valid, probably you wanted to do that, but in real life it is just x="something" (usually).
It is not the echo which removes the $, but the syntax you use means you want to evaluate variables within " ". That's why it is removed/evaluated. Actually there was no $2 (it is empty or undefined), therefore you got the result what you posted.

But anyway, we cannot show you any solution without knowing your script. At least the relevant part.

This cat | while is the typical UUoC (useless use of cat), you should avoid that. Post #4 offers a much better way.

It's not a problem where var contains '$', it's a quoting problem. To quote bash man page "Enclosing characters in double quotes preserves the literal value of all characters within the quotes, with the exception of $, ...". Use single quotes or xargs.

Quote:

Hmmmm - what's not "best practice" about it?

It falls under the useless use of cat.

I guess you need to sed to add an escape character...

The use of cat generates an extra I/O call to load cat to do something for which it is not needed.
You can live without it and make your script faster and less vulnerable.

Allowing bash to interpret the string causes it to apply regex expansions, causing the problem. Echo, used as you did, is called as a bash internal. It is not needed and adds interpretation.

Escaping the character, as you did, is ONE way to avoid that problem. There are others. Not allowing the bash command line handling at your file name data would be optimal. Using a different tool than bash would be another.

Quote:

using REPLY preserves leading whitespace in the line read whereas 'while read var' does not

Indeed! (I was surprised.)

I wouldn't bet this is true in other shells (Posix sh, ksh, zsh).
Portable is 'while IFS= read var'
The empty IFS environment variable makes read preserve leading whitespace. It is a temporary setting just for the read command (technically it happens between fork() and exec()).

Quote:

Originally Posted by pan64 (Post 6498867)

Tough to show my actual script since I don't have it any more as I solved the problem another way. This is the best of my recollection:

Code:

cat fileList | while read

do

    x=$(echo "$REPLY" | sed 's#/mnt##')

    if [ -e "$x" ]; then echo yup; else echo nope; fi

done

I am trying remove or substitute characters in the $REPLY varliable. $x ends up with the "~$2023..." changed to "~023". Single quotes as lvm_ suggested can't be used because the $REPLY would not be substituted with its value.

And yes, I know I could use other tools. I'm interested specifically in a bash solution. If there isn't one, that's the answer.

For reading file names use IFS= and -r
Use the built-in string operators (parameter modifiers).

Code:

while IFS= read -r fn

do

  x=${fn/\/mnt/} # deletes(substitutes) the first occurrence of /mnt (like sed 's#/mnt##')

#  x=${fn/#\/mnt/} # deletes(substitutes) a /mnt at the beginning (like sed 's#^/mnt##')

#  x=${fn#/mnt} # deletes a /mnt at the beginning (like sed 's#^/mnt##')

    if [ -e "$x" ]; then echo yup; else echo nope; fi

done < fileList

If you must use sed then this is almost safe:

Code:

x=$(echo "$fn" | sed 's#/mnt##')

Exception: $fn is a echo option like -n or -e
100% safe is printf with a format:

Code:

x=$(printf "%s\n" "$fn" | sed 's#/mnt##')

$-expressions within " " (like "$fn") are evaluated/substituted; no further substitution or expansion happens.
An assignment (like x=val) does not need quotes (x="val"). But a command does:

Code:

printf "%s\n" "$(printf "%s\n" "$fn" | sed 's#/mnt##')"

$( ) is a sub shell; the quotes in it don't interfere with the main shell.
For demonstration, you could write it as

Code:

printf "%s\n" "$(

# sub shell starts

printf "%s\n" "$fn" | sed 's#/mnt##'

# sub shell ends

)"

Quote:

Originally Posted by mfoley (Post 6499502)

I am trying remove or substitute characters in the $REPLY varliable. $x ends up with the "~@2023..." changed to "~023". Single quotes as lvm_ suggested can't be used because the $REPLY would not be substituted with its value.

I still think you brought this trouble on yourself. I mean it is all wrong. You did something wrong and are now trying to add some workarounds to make it work. The real solution would be to fix the initial issue, not to find a way to add patches.
But we would need to know how this variable was generated and if there was any way to make that differently.

Quote:

Originally Posted by mfoley (Post 6499502)

And yes, I know I could use other tools. I'm interested specifically in a bash solution. If there isn't one, that's the answer.

In most cases it can be solved in bash, it has a huge amount of features. It is explained in the previous post. Sometimes it is not that efficient, and yes, this language is ugly, but there are many ways to make it work and also many ways to fail.