[SOLVED] Does a command, or a combination of commands exist that would move a bunch of files into a new directory and once that directory is full ...
Linux - SoftwareThis forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
Does a command, or a combination of commands exist that would move a bunch of files into a new directory and once that directory is full ...
Hey LQ,
So basically I've got a directory with like 30k .gz files in it. These are back up files.
I want to burn these to DISC media. What I normally do is browse the directory, keep selecting archive files until the file manager reports to me that I am nearing 4.5GB, then I CUT them, create a directory and name the directory with two dates, the first date is the date of the oldest archive I currently have CUT, the second date is the date of the newest archive I have CUT.
Naming of the directories is simple for me to do, I don't mind doing that manually, but is there a command that would do this cutting and creation of directories for me once I specify how big I want the directories to be?
I presume this is more in the realm of shell scripting rather than piping and chaining up commands together. I am prepared to do this as well if need be, but I recently learned about zgrep and was surprised someone had built a utility just for that purpose, so I figured I'd inquire.
Any tips in the right direction greatly appreciated as usual.
I don't think datapacker can be helpful here - it doesn't have any date functionality, and appears to consider file size not disk usage, which matters when you're aggregating files to burn to DVD.
Wouldn't surprise me if a proper tool existed, but also wouldn't surprise me if most people just use a script.
Possibly start with something like:
Code:
find . -type f -print0 | du --block-size=2K --time --files0-from=- | sort -k2
This outputs a list of filenames (no directories) with their disk usage and last modified time (which the sort orders by), and this can then be looped through in Awk or Bash, summing and compiling the filenames until the threshold, logging the end/start dates, then repeating until all files are batched, which I'd then review before performing the operation.
When dealing with cases like this I often don't do file modification directly - instead I'll generate a script that does it, so I can manually review it before running - in case any oddities stick out. (An example in the above would be if there are any filenames with newlines in; the first half will deal with them, but the second half needs additional params if they are a possibility; such a situation can/should be guarded against in the script, but if that were overlooked, may manifest itself in an command that looks odd.)
On a separate note, title of this thread is not a good one: titles should be a concise description of what a thread is about.
The first half of this thread's title does not contribute anything to that description, but readers don't know that until they've got a dozen words in - some will have given up by then.
Like a ladder, it is best when they are long enough to get from one place to another, but not so long that it takes a prolonged crawl to get there.
On a separate note, title of this thread is not a good one: titles should be a concise description of what a thread is about.
The first half of this thread's title does not contribute anything to that description, but readers don't know that until they've got a dozen words in - some will have given up by then.
Like a ladder, it is best when they are long enough to get from one place to another, but not so long that it takes a prolonged crawl to get there.
That's as many characters as the title field would let me put in there. Now that you've read the post, what would you have titled it as?
That's as many characters as the title field would let me put in there.
That's ... not surprising. You should have taken that as a hint!
Again, titles should be concise, direct, succinct - not verbose.
Quote:
Now that you've read the post, what would you have titled it as?
As I already said, the entire first half of it is redundant - everything upto and including the word "would" could be removed and it would improve the title, but even that second part could use half the words and still convey the same detail: "move files into directories until full".
An even better one would communicate the overall problem, e.g. "batching files into DVD-sized groups".
Thanks for all the responses. I ended up writing a PHP script to do it all for me (I know PHP, don't know Bash at all haha).
Cheers.
Perfect opportunity for you to begin learning bash.
Bash is after all the commonly used shell and everything you do on the command line (including launching php & probably writing your php script) begins with using bash.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.