Interlacing the lines of two text files

khdubya · 02-02-2007, 01:41 PM

I am trying to merge two large text files. I need to write twenty lines from one file, and then one line from another. I dont know if I should be using sed or awk or something else. Can anyone provide a tip?

Dox Systems - Brian · 02-02-2007, 01:45 PM

head and/or tail, combined with the appropriate pipes might get you what you need.

khdubya · 02-02-2007, 03:15 PM

Ok i put some heads and tails in a loop but it is not working out so well. The files are 400 megabytes so it goes way too slow after the first few thousand. Is there a simple way to do it that wont require me to load up the files everytime I loop?

makyo · 02-03-2007, 05:27 PM

Hi, khdubya.

If you are still interested in doing this, I have been working on a rudimentary perl code that may work.

Consider the two files:

Code:

File: data1

  1 one
  2 two
  3 three
  4 four
  5 five
  6 six
  7 seven
  8 eight
  9 nine
 10 ten

File: data2

  1 alpha
  2 beta
  3 gamma

The perl code would be called with a series of arguments like: filename:line-count (default:1) and produces output like so:

Code:

% ./shuffle data1:3 data2
one
two
three
alpha
four
five
six
beta
seven
eight
nine
gamma
ten

I timed it on sample files for your ratio l1:20 l2 of (line count) size:

Code:

 14547 l1
   766 l2

to get:

Code:

real    0m0.044s
user    0m0.031s
sys     0m0.006s

The drawback is that while the code seems to work, I have not finished it, there could be bugs in it, and so I provide no warranty.

Let me know if you'd like me to post it; about 80 lines ... cheers, makyo