Modify rows to columns in bash script??

Drigo · 07-30-2015, 06:50 PM

I have this file.txt:
0 0 0 1 2 3
0 0 0 4 5 6
0 0 0 7 8 9

I want it to look like file2.txt:
0 0 0
0 0 0
0 0 0
1 4 7
2 5 8
3 6 9

What is the easiest way to do it? Prob. a single command line?

astrogeek · 07-30-2015, 07:24 PM

I don't have a simple command, but awk would be the tool. Shouldn't be too difficult.

mralk3 · 07-30-2015, 08:15 PM

You could read each line into a loop and make each line an array. Then format the second file accordingly.

grail · 07-31-2015, 01:52 AM

It would be nice to see what you have tried first, otherwise it is all academic and you can use any number of tools for the job.

ondoho · 07-31-2015, 08:26 AM

i recently tried something similar with text.
it requires 2 nested for-loops, if you get them right it's a really short script.

allend · 07-31-2015, 10:05 AM

Using bash looks like homework. In real life I would read into a spreadsheet, then copy and paste with the transpose option.

Drigo · 07-31-2015, 11:09 AM

Not homework, but I'll give it a try. I wasn't looking for a for loop but seems that might do the trick...

Drigo · 07-31-2015, 02:22 PM

Not a homework, but here's the solution:
#!/bin/bash
#Read in the bvecs file
BVECFILE=$1
BVECNEWFILE="bvecs_new"
#Store it as an array
readarray BVEC < $BVECFILE
BVECNUM=${#BVEC[@]}
#Check the number of directions
GRADNUM=$(echo $BVEC | wc -w )
#Indexing at i=1 so we get rid of the first vol0 indexed as 0
if [ -f $BVECNEWFILE ] ; then
echo "$BVECNEWFILE already exists. Removing and replacing..."
rm $BVECNEWFILE
fi
echo "New bvecs are generated in "$BVECNEWFILE" "
#echo "GRADX is $GRADX"
for ((i=2; i<=${GRADNUM}; i++ ));
do
gx=$(echo ${BVEC[0]} | awk -v x=$i '{print $x}')
gy=$(echo ${BVEC[1]} | awk -v x=$i '{print $x}')
gz=$(echo ${BVEC[2]} | awk -v x=$i '{print $x}')
echo "$gx $gy $gz" >> $BVECNEWFILE
done

HMW · 07-31-2015, 03:22 PM

^ Nice! I never seem to have the time to learn awk properly. I found this to be an entertaining problem, so I did my own variant in Python. Looks like this:

Code:

#!/usr/bin/env python3

# Declare our matrix 3 chars wide, 6 lines
matrix = [[ 0 for i in range(3)] for j in range(6) ]

x = 0
y = 0
count = 0

# Open the original file
with open("f1.txt") as f1:
    txt = f1.read()

# Read the file, char by char.
# Insert chars at correct place in matrix.
for char in txt:
    if count > 5:
        count = 0
        x = 0
        y += 1
    if char != "\n":
        matrix[x][y] = char
        x += 1
        count+=1


# Now let's open a file to write the result to, and create it if not 
# already existing
outfile = open("f2.txt", "w")

x = 0
y = 0

while True:
    outfile.write(matrix[x][y])
    y += 1
    if y > 2:
        outfile.write("\n")
        y = 0
        x += 1
    if x > 5:
        break

The original file:

Code:

$ cat f1.txt
000123
000456
000789

The output (outfile) from my program:

Code:

$ cat f2.txt 
000
000
000
147
258
369

Thanks for a good exercise!

HMW

astrogeek · 07-31-2015, 05:02 PM

OK, I couldn't NOT come up with a minimalist awk example since no one else did (actually I had my son do it).

Because we are turning the corner, let's call it corner.awk, with the following lines:

Code:

{
        for(f=1;f<=NF;++f) arr[NR ":" f]=$f;
}
END{
        for(f=1;f<=NF;++f){
                for(n=1;n<=NR;++n)
                        printf("%s ",arr[n ":" f]);
                printf("\n");
        }
}

Assuming your data file is file.txt, you would invoke it like this...

Code:

# awk -f corner.awk file.txt
0 0 0
0 0 0
0 0 0
1 4 7
2 5 8
3 6 9

OR save to file...

# awk -f corner.awk file.txt >file2.txt

Also works for any width of field, alpha or numeric, and any number of columns and lines so long as all rows have same number of space separated columns!

Gotta love awk!

And thanks for the exercise as well!

syg00 · 07-31-2015, 09:51 PM

Not minimalist, but mine was similarly done (in awk) but using an array of arrays - I find them easy to "walk" as you can use "for i in array" followed by "for j in array[i]" ...
No need to know the bounds in advance.

Edit: -

not relevant here as the prints needed to be ordered - I did similar to @astrogeek for the printing.

Just a nit - hanging whitespace on lines makes regex using the "$" anchor more complex than necessary.

astrogeek · 07-31-2015, 10:26 PM

Quote:

Originally Posted by syg00

Not minimalist, but mine was similarly done (in awk) but using an array of arrays - I find them easy to "walk" as you can use "for i in array" followed by "for j in array[i]" ...
No need to know the bounds in advance.

Edit: -

not relevant here as the prints needed to be ordered - I did similar to @astrogeek for the printing.

Just a nit - hanging whitespace on lines makes regex using the "$" anchor more complex than necessary.

HAHA! Yes I was aware of the trailing space but was out of time - hoped no one would notice!

Here is one without trailing whitespace and two lines shorter!

Code:

{
        for(f=1;f<=NF;++f) arr[NR ":" f]=$f;
}
END{
        for(f=1;f<=NF;++f)
                for(n=1;n<=NR;++n)
                        printf("%s%s",arr[n ":" f],(n<NR ? " " : "\n"));
}

Thanks! These are fun when you have the time and the right mindset, fortunate superposition today!

EDIT ****
Although I use a linear array the associative keys are effectively compound to produce the same logic as would an array of arrays, allowing same use of (i,j), or (n,f) in my own case.

syg00 · 07-31-2015, 11:17 PM

Cute - I did it as follows, but needs an extra print statement for the newline. I might steal that idea ...

Code:

printf(n==NR?"%d" : "%d ",_[n][m])

astrogeek · 07-31-2015, 11:43 PM

Quote:

Originally Posted by syg00

Cute - I did it as follows, but needs an extra print statement for the newline. I might steal that idea ...

Code:

printf(n==NR?"d" : "%d ",_[n][m])

Thanks for the warm fuzzy feeling of satisfaction!

When my son produced his first cut, he had a proper BEGIN block and used print(...) statements.

He also gave us the linear array idea which is good! (My own first attempt used multiple arrays as well.)

In order to get the newlines right with print he had to define ORS="" to override the default (\n), among other things.

I wanted to rely on defaults without a BEGIN block to keep it minimal so I changed to use of printf(...), and hastily introduced the trailing space because I was somewhat rushed, but knew a ternary would fix it - just didn't do it.

No need to "steal" the idea, you would have thought of it yourself in another minute or two!

Anyway, sharing and immitation are both good things - and I have FREELY shared the thought! I like to never miss an opportunity to make that point, so let's make it "official" with a universal statement of copy rights...

Code:

# Acknowledgment of right to use, copy, modify and distribute:
#
# You already have the right to use, modify and distribute this
# or any other thought or idea, and need no license or other
# permission from anyone to do so!
#
# Exercise it freely and never concede it to anyone!

HMW · 08-01-2015, 01:36 AM

^ Awesome, all of the above!

I toyed with only using loops for a while, but found it easier to, eh, "visualize" using a matrix. As I keep saying, one of these days, I might find the time and energy to learn awk properly.