LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 09-25-2017, 10:19 AM   #1
Santhoo87545
LQ Newbie
 
Registered: Sep 2017
Posts: 1

Rep: Reputation: Disabled
Merging log files in the created order


Hi...

I want a shell script to merge all files into a single file which are exist in a directory , But the merging should be done in the order of files created.

ex: there are 5 files in a folder i.e
a.txt -- > Created at 7.00 pm
b.txt -- > Created at 7.10 pm
c.txt -- > Created at 7.20 pm
d.txt -- > Created at 7.30 pm
e.txt -- > Created at 7.40 pm

Note:- The files are dynamically comes form the system and we do not know how many files will be there

Now all these files should be merge into a single file in the same order.

Can any one help me on this.
 
Old 09-25-2017, 11:57 AM   #2
MensaWater
LQ Guru
 
Registered: May 2005
Location: Atlanta Georgia USA
Distribution: Redhat (RHEL), CentOS, Fedora, CoreOS, Debian, FreeBSD, HP-UX, Solaris, SCO
Posts: 7,831
Blog Entries: 15

Rep: Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669
cat a.txt >new.txt
cat b.txt >>new.txt
cat c.txt >>new.txt

The first line creates (or overwrites if it already exists) new.txt with the single redirect.
The following lines with double redirect append to the file new.txt. If run in order they will append in order.

If they are fairly large you might want to put sleep statements in between.

If these files grow large you may wish to delete the originals after doing the append. You DO want to make sure the filesystem into which your putting your new.txt has enough space to handle this operation.

If a cron job is creating each of the txt files you could modify it so it does the append after creating the next file.
 
Old 09-25-2017, 05:08 PM   #3
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 21,159

Rep: Reputation: 4125Reputation: 4125Reputation: 4125Reputation: 4125Reputation: 4125Reputation: 4125Reputation: 4125Reputation: 4125Reputation: 4125Reputation: 4125Reputation: 4125
If they are in collating sequence, a simple for "i in *" loop should suffice. If not "ls" has options for time ordering.
 
Old 09-26-2017, 07:48 AM   #4
rtmistler
Moderator
 
Registered: Mar 2011
Location: USA
Distribution: MINT Debian, Angstrom, SUSE, Ubuntu, Debian
Posts: 9,885
Blog Entries: 13

Rep: Reputation: 4931Reputation: 4931Reputation: 4931Reputation: 4931Reputation: 4931Reputation: 4931Reputation: 4931Reputation: 4931Reputation: 4931Reputation: 4931Reputation: 4931
Hi Santhoo87545 and welcome to LQ.

There are some excellent ideas offered. What you should do at this point is start writing your script.

Please note that LQ members are volunteers and we are here to help, however the customary way of this forum is to enable you to complete these tasks on your own so that you learn from them. Therefore no one here should be just writing a script for you, however we're happy to review what you have and offer suggestions as to what directions to take with it.

For starters, do you have a script language in mind? Such as bash, or other?

And do you have skills and knowledge to write a script?

I think this problem breaks down into two parts: (1) identifying and sorting the input files by time/date (2) concatenating them to a single file (which incidentally you don't end up adding that new file to your concatenation ...)

Please update us with some more information about your script language choice and any attempts you have put forth to trying to solve this problem.
 
Old 09-30-2017, 05:05 AM   #5
MadeInGermany
Senior Member
 
Registered: Dec 2011
Location: Simplicity
Posts: 2,842

Rep: Reputation: 1221Reputation: 1221Reputation: 1221Reputation: 1221Reputation: 1221Reputation: 1221Reputation: 1221Reputation: 1221Reputation: 1221
Perhaps just one command
Code:
cat *.txt > ../new.txt
?
I have given another directory for new.txt, so it will not match if the command is repeated.
 
Old 09-30-2017, 11:54 AM   #6
MensaWater
LQ Guru
 
Registered: May 2005
Location: Atlanta Georgia USA
Distribution: Redhat (RHEL), CentOS, Fedora, CoreOS, Debian, FreeBSD, HP-UX, Solaris, SCO
Posts: 7,831
Blog Entries: 15

Rep: Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669
Quote:
Originally Posted by MadeInGermany View Post
Perhaps just one command
Code:
cat *.txt > ../new.txt
There is no guarantee *.txt would list the files in order as the OP requested.
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
log file created when c++ or java files compiled anitha1972 Linux - Newbie 2 10-02-2009 10:01 PM
mp3's created with kaudiocreator play in wrong order MartyJay Linux - Software 14 10-31-2008 06:08 AM
compress log files created yesterday? MacSob Linux - Enterprise 1 09-16-2005 03:55 PM
logrotate log files - keeping in order when using grep or cat and bash dmellem Linux - Software 6 02-11-2005 01:05 PM
merging log files help please digitalgravy Linux - Newbie 3 12-10-2003 02:26 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 07:26 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration