[SOLVED] wanted to make file content in required format.

azheruddin · 01-31-2013, 08:42 PM

Hi,

I do have one file which contains some data in format as....

pqrs
abcd
efgh
xyz
....

so i wanted to make that everything should be in a single line as below format like ..

'pqrs','abcd','efgh','xyz',.....

means each word should be in a single line with seprator as , and should be quoated with single quoate.

i was trying with sed but single quoate creating a problem..i guess single quote doesnt work properly with sed utility..

so any other way to make in this format...
any other way to make this...

grail · 01-31-2013, 09:13 PM

Show us your attempt so we may assist in a solution?

azheruddin · 01-31-2013, 09:23 PM

so the file contents are..

123.gzip
234.gzp
356.gzip

and iam trying to do like

'123.gzip','234.gzip','356.gzip'

by command
sed -e 's/[0-9]*$/'&'/g' filename

but not working..

sangfroid · 01-31-2013, 09:47 PM

hmm...

try this one..
awk '{printf "\x27%s\x27,", $1}' data

where data is the file that contains your list..

was an interesting one..

danielbmartin · 01-31-2013, 09:47 PM

Have ...

Code:

123.gzip
234.gzp
356.gzip

Want ...

Code:

'123.gzip','234.gzip','356.gzip'

Try this ...

Code:

sed "s/^/'/; s/$/'/" $InFile |paste -s -d","

Daniel B. Martin

danielbmartin · 01-31-2013, 09:57 PM

Quote:

Originally Posted by sangfroid

hmm...

try this one..
awk '{printf "\x27%s\x27,", $1}' data

where data is the file that contains your list..

was an interesting one..

OP asked for this...

Code:

'123.gzip','234.gzp','356.gzip'

Your awk delivered this ...

Code:

'123.gzip','234.gzp','356.gzip',

... which has an unwanted trailing comma.

Daniel B. Martin

azheruddin · 01-31-2013, 10:05 PM

Thanks! Sangfriod its worked out

can you please elaborate awk '{printf "\x27%s\x27,", $1}' .How it works out...
or redirect please redirect me to the reference page

Thanks Again!

sangfroid · 01-31-2013, 11:18 PM

x27 is the hex code for '. Basically, it is going through each of the line of the file and printing 'line_content (variable $1) and a comma (,) and again apostrophe.... it is doing that thing for each and every line.

grail · 02-01-2013, 02:32 AM

Here is an all sed:

Code:

sed -r "s/.*/'&'/;:a N;s/(.*)\n(.*)/\1,'\2'/;ta" file

Probably can be done a little better as my sedfu is not great.

David the H. · 02-01-2013, 06:37 AM

The 's'ubstitute command in sed cannot, by default, operate on newlines due to the way it processes input. sed takes each line in turn into its pattern buffer minus the newline, and applies the given expressions to it, then empties the buffer before grabbing the next line. There are never any newline characters in the buffer for it to operate on.

Multi-line editing requires use of 'N' and/or the various hold buffer commands. These commands can put multiple lines into the pattern buffer at the same time, with newlines between them, allowing you to target them.

See the relevant sections of the grymoire and the sedfaq:

Here are a few useful sed references:
http://www.grymoire.com/Unix/Sed.html
http://sed.sourceforge.net/sedfaq.html
http://sed.sourceforge.net/grabbag/
http://sed.sourceforge.net/sed1line.txt
http://www.catonmat.net/series/sed-one-liners-explained

BTW, here's an ed solution for you too. Since it's a full-fledged text editor, it's able to do many multi-line operations more easily than sed.

Code:

printf '%s\n' "%s/.*/'&'/" '%-1s/$/,/' '%j' 'w newfile.txt' | ed -s oldfile.txt

The first 's' expression surrounds every line with quotemarks. The second 's' adds a comma to the end of every line except the last. The third 'j'oins all lines together. Finally it 'w'rites the modified buffer to a file, which can be the same as the input file (just leave the filename off).

You can insert a '%p' (print all) command to view the entire file at any step.

How to use ed:
http://wiki.bash-hackers.org/howto/edit-ed
http://snap.nlc.dcccd.edu/learn/nlc/ed.html
(also read the info page)

danielbmartin · 02-01-2013, 07:57 AM

These tests ...

Code:

echo; echo "Method of LQ Newbie sangfroid"
awk '{printf "\x27%s\x27,", $1}' $InFile > $OutFile
echo "OutFile ..."; cat $OutFile; echo "END"

echo; echo "Method of LQ Member danielbmartin"
sed "s/^/'/; s/$/'/" $InFile |paste -s -d"," > $OutFile
echo "OutFile ..."; cat $OutFile; echo "END"

echo; echo "Method of LQ Guru grail"
sed -r "s/.*/'&'/;:a N;s/(.*)\n(.*)/\1,'\2'/;ta" $InFile > $OutFile
echo "OutFile ..."; cat $OutFile; echo "END"

echo; echo "Method of LQ Guru David the H."
printf '%s\n' "%s/.*/'&'/" '%-1s/$/,/' '%j' 'w newfile.txt' | ed -s $InFile > $OutFile
echo "OutFile ..."; cat $OutFile; echo "END"

Produced these results ...

Code:

Method of LQ Newbie sangfroid
OutFile ...
'123.gzip','234.gzp','356.gzip',END

Method of LQ Member danielbmartin
OutFile ...
'123.gzip','234.gzp','356.gzip'
END

Method of LQ Guru grail
OutFile ...
'123.gzip','234.gzp','356.gzip'
END

Method of LQ Guru David the H.
OutFile ...
END

Question 1) Why did sangfroid's solution omit a linefeed? Is this a bug?
Question 2) Why did David the H.'s solution fail?

Daniel B. Martin

David the H. · 02-01-2013, 08:32 AM

Hmm. What does the input look like, exactly?

I tried it on multiple inputs, and they all did just fine, except for one file that had dos-format newlines (I'd forgotten I'd set it as such for a previous test). It worked just fine too, after I converted it back.

Let me play with it.

Edit: Ah, my ed command writes directly to an output file, which I named "newfile.txt". You're trying to redirect the non-existant stdout to a different file.

Change the final command to "w $OutFile" and drop the redirection, or alternately to "%p" to have it print the buffer to stdout instead.

colucix · 02-01-2013, 08:43 AM

To put a multiline input into a single line you can use echo. In this way you are a step ahead in order to apply a simple sed solution, e.g.

Code:

echo $(<file) | sed -r "s/^|$/'/g; s/ /','/g"

However this fails if the original lines contain blank spaces, since they will be changed to ',' as well.

Regarding the awk solution you can easily avoid the trailing comma if you add the separator before the string, except for the first iteration. This is easily done by means of a C-like conditional expression:

Code:

awk 'BEGIN{ q = "\x27" } { s ? s = s q "," q $0 : s = q $0 } END { print s q }' file

This takes care of inner blank spaces, since no substitution is made and the characters or strings are added sequentially.

David the H. · 02-01-2013, 08:53 AM

BTW, as for Q1, the output doesn't have any newlines because sangfroid's printf format string doesn't have any. It's a byproduct of the concatenation process. You can tack on an 'END{ print }' to have it insert a final one.

danielbmartin · 02-01-2013, 12:57 PM

I "polished" my code and now have this ...

Code:

sed "s/.*/'&'/" $InFile |paste -sd, > $OutFile

This is the most concise of all solutions offered in this thread.
It's also the most readable, though readability is subjective.

I really like paste because, in one swoop, it ...
- inserts commas where they are wanted
- removes linefeeds where they are not wanted.

Daniel B. Martin