LinuxQuestions.org - cat command - for every combination of 6 files, merge into a single image file.

- Linux - Newbie (https://www.linuxquestions.org/questions/linux-newbie-8/)

- - cat command - for every combination of 6 files, merge into a single image file. (https://www.linuxquestions.org/questions/linux-newbie-8/cat-command-for-every-combination-of-6-files-merge-into-a-single-image-file-4175632562/)

Numptius

06-24-2018 06:03 AM

cat command - for every combination of 6 files, merge into a single image file.

Hi this is my first question - apologies, I searched but couldn't find anything similar. (Though I guess it is covered already - I just don't know how to ask correctly).

I have 6 image files part.img, part1.img.. .. part5.img - each of which is 2GB in size.

To combine all of these into a single image file is possible with the following:

Code:

cat part*.img >> Result.img

This has failed to generate an image that can be mounted..

I'm just curious is all and wanted to try to merge each segment file in every possible combination and save that as a Result[n].img file..

..then try to mount the individual results (i doubt it will work any way.. give an image that can be mounted). I just wanted to know if this is possible from the command line (to automate) or should I really be doing this in python or similar?)

Thanks.

(edit: I guess the question was, "Can you put a FOR loop into the command line" and the answer looks to be yes.. probably in a bash script?

Code:

for i in {1..5}; do COMMAND-HERE; done

syg00

06-24-2018 06:14 AM

Hmmm. It's certainly possible with fairly simple loop constructs, but I'd be inclined to enforce collating sequence by renaming part.img to part0.img (that's a zero). Then try that cat.
Looks definitely suspect that naming ...

Numptius

06-24-2018 07:37 AM

Hi Syg00 - this is part of an effort to restore data from an Android mobile (Cubot) - They generate 6 userdata backup files during recovery, each named userdata_ddmmyy_hh_ss.backup and then userdata_ddmmyy_hh_ss.backup1 and so on..

I've converted to .img and skipped the first 512 bytes, which when merged, should give a mountable image file (that's reported to work for other phones) but not Cubots.. I just thought, what if the .backup + .backup1 + .backup3 was intentionally misleading..

BW-userx

06-24-2018 02:13 PM

make copies of them first, then keep the originals in a safe, different place, then go into experimentation mode with your copies, if it messes up, make another set of copies until you get it figured out. command line or script both will work.

if you use a part1 part2 ... etc.
you can use a part of this to get the next file by its number.

Code:

userx@manjaro:~/test



$ for i in 1 2 3 4 ; do touch file"$((c++))" ; done





$ ls

file0  file1  file2  file3

AwesomeMachine

06-24-2018 07:16 PM

If you use the wildcard, you need '>>', so each file doesn't write over the previous one. Then it should work.

Numptius

06-25-2018 08:11 AM

Quote:

Originally Posted by BW-userx (Post 5871431)

Code:

userx@manjaro:~/test



$ for i in 1 2 3 4 ; do touch file"$((c++))" ; done





$ ls

file0  file1  file2  file3

Thanks, that's a nice example. Much appreciated. Cheers.

Numptius

06-25-2018 08:27 AM

Quote:

Originally Posted by AwesomeMachine (Post 5871544)

If you use the wildcard, you need '>>', so each file doesn't write over the previous one. Then it should work.

I thought this related to the example by BW? (but I can't wait for 6:30pm to find out now :) cheers.)

BW-userx

06-25-2018 08:35 AM

this should help you in figuring out how to split, and put them back together again.
https://unix.stackexchange.com/quest...splitting-them

I've only joined zip files back together, and that was a long while ago using cat.

Numptius

06-25-2018 12:54 PM

Quote:

Originally Posted by AwesomeMachine (Post 5871544)

If you use the wildcard, you need '>>', so each file doesn't write over the previous one. Then it should work.

Cheers, I was hoping it would work (it is definitely the correct syntax and I'll update the original post).

I read on the XDA forum that there's something unique about the headers for the Cubot '.backup[n]' files.. It's from my own phone, so it's no big deal.. it was just a learning exercise (mainly because every other Android device I own, either doesn't list 'Backup Userdata' as a recovery option or the phone is seemingly beyond (without jtag) repair.. Riff box looks likely in the near future..

trying to get started on the bash script but woman and online banking are blocking :)

Numptius

06-25-2018 01:55 PM

Quote:

Originally Posted by syg00 (Post 5871279)

I think you and BW's link are my next efforts.. reading on the aforementioned link.. 'split' (which i've not had cause to use) has a note about the correct naming of files... if split hasn't done the naming, then there is no guarantee that cat should give a viable output? (i hope the answer is yes)

Numptius

06-26-2018 01:15 PM

Code:

#!/bin/bash



# Shell script to generate all combinations of image files part 1 to part 6



echo "input the desired output filename:"



read ofilename



clear



for i in {6..1..1} 



        do

                echo $i



                        for j in {6..1..1} 



                                do

                                        echo $j

                                

                                                for k in {6..1..1} 



                                                        do

                                                                echo $k



                                                                        for l in {6..1..1} 



                                                                                do



                                                                                        echo $l



                                                                                                for m in {6..1..1} 



                                                                                                        do

                                                                                                                echo $m

                                                                        

                                                                                                                        for n in {6..1..1} 



                                                                                                                                do 



                                                                                                                                        echo $n

cat >> $ofilename <<EOF

part$i.img

part$j.img

part$k.img

part$l.img

part$m.img

part$n.img

EOF



echo "$ofilename $i$j$k$l$m$n"



done



done



done



done



done



done

I've just run that, was going to append the $i$j$k$l$m$n to the filename to create the individual files.

Running it with very small test files.. it did work (i think) and planned to add a bit more at the end to mount and check each file.. delete if no good.. perhaps should finish it anyway but it's glaringly obvious that if the final result was a 12GB img file.. errr.. that's a long time to do on even my reasonable/modest desktop.. it can do one 12GB cat combination in around 4 minutes but the vast qty of combinations.. or am i being soft..

I got three months of runtime in my head.. and that's not including the mounting/checking time??? mmm thanks for all your help.. i don't know who should get the solved points.. BW was very helpful but i'd still be over catting files (with out Awesome Machine's eagle eye.. toss of a coin.. Tails.. AwesomeMachine... (hold on.. how do you even mark a thread as solved..)

Cheers for your help.

Numptius

06-27-2018 01:14 PM

I'm an idiot, hence the name.. it's not 3 months (blaming it on the heat or the lack of sleep).. it's 3 months for every combination (reusing the digits 1 2 3 4 5 6) it's a permutation problem? 6! = 720 = approx 2.5 days.. less than a week (for 12GB output file) which seems much better.. or at least worth solving...

keefaz

06-27-2018 03:18 PM

Are you sure the mounting error does not come from the 512 first bytes you skipped (from what you said on post #3)?

It seems unlikely the image files are named with random number

Numptius

06-28-2018 02:40 AM

Hi Keefaz, this was my starting point.. it describes the requirement to remove the first 512 bytes:

https://forum.xda-developers.com/sho...2528969&page=7

NB: I'm not really as bothered about mounting a viable image now, not as much as generating the permutations and checking.. just a learning exercise.. I've done a little programming but relatively new to bash..

All times are GMT -5. The time now is 10:40 AM.