LinuxQuestions.org
Review your favorite Linux distribution.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices


Reply
  Search this Thread
Old 11-12-2022, 08:35 PM   #1
Skaperen
Senior Member
 
Registered: May 2009
Location: center of singularity
Distribution: Xubuntu, Ubuntu, Slackware, Amazon Linux, OpenBSD, LFS (on Sparc_32 and i386)
Posts: 2,688
Blog Entries: 31

Rep: Reputation: 176Reputation: 176
find growing files


it would be easy to write a script to list out several largest files in a given tree. this can even be done in a one line command by those well experienced with their shell. BTDT.

i am looking for a script that, given an optional time duration and the tree path(s), will list out the fastest growing files in those trees, in slow to fast order, with the size change (over the duration) and current size on the same line as the file. i'm too lazy in my retirement to write one myself.

most preferred are scripts implemented in bash, pike, python, or rust. giving the total growth rate at the end of the list would be a nice plus.
 
Old 11-13-2022, 02:30 AM   #2
pan64
LQ Addict
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 22,039

Rep: Reputation: 7347Reputation: 7347Reputation: 7347Reputation: 7347Reputation: 7347Reputation: 7347Reputation: 7347Reputation: 7347Reputation: 7347Reputation: 7347Reputation: 7347
there is no such thing, so if you need that you need to implement it yourself.
From the other hand "fastest growing file" is not really well defined, so hard to handle. Usually files are overwritten, so the size will change in steps (but in such cases sometimes the inodes change too). Sometimes they are renamed. Sometimes they are kept open and continuously grow, but probably you won't recognize it as long as they are not really saved.
Regarding the growth of scripts and similar things usually we use a version control system and that will make it easier - for the managed files.
 
Old 11-13-2022, 03:01 AM   #3
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 21,152

Rep: Reputation: 4125Reputation: 4125Reputation: 4125Reputation: 4125Reputation: 4125Reputation: 4125Reputation: 4125Reputation: 4125Reputation: 4125Reputation: 4125Reputation: 4125
Quote:
Originally Posted by Skaperen View Post
i'm too lazy in my retirement to write one myself.
Says it all really. Why should anyone else waste their time, if you're too lazy to even try ?.
 
Old 11-13-2022, 12:58 PM   #4
Jan K.
Member
 
Registered: Apr 2019
Location: Esbjerg
Distribution: Windows 7...
Posts: 773

Rep: Reputation: 489Reputation: 489Reputation: 489Reputation: 489Reputation: 489
He pays very well?

But why not let a basic script write name, size, date to csv-files? Could be a cron job.
Then import these csv-files into a spreadsheet, sort the data and all kinds of diagrams, charts and what not is at your disposal...

You're welcome!
 
Old 11-13-2022, 01:06 PM   #5
Turbocapitalist
LQ Guru
 
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 7,357
Blog Entries: 3

Rep: Reputation: 3767Reputation: 3767Reputation: 3767Reputation: 3767Reputation: 3767Reputation: 3767Reputation: 3767Reputation: 3767Reputation: 3767Reputation: 3767Reputation: 3767
A key-value pair database would probably more appropriate than CSV. YottaDB would be one end of the extreme a tied hash in Perl on the other.
 
Old 11-13-2022, 01:47 PM   #6
MadeInGermany
Senior Member
 
Registered: Dec 2011
Location: Simplicity
Posts: 2,832

Rep: Reputation: 1219Reputation: 1219Reputation: 1219Reputation: 1219Reputation: 1219Reputation: 1219Reputation: 1219Reputation: 1219Reputation: 1219
A growing file has a recent mtime.
List all files that were modified in the last hour:
Code:
find . -type f -size +0 -mmin -60 -ls
These are the candidates.
Redirect it to a file.
Rerun it after 10 minutes, get the differences of the sizes. And get the maximum.
 
1 members found this post helpful.
Old 11-13-2022, 02:15 PM   #7
Skaperen
Senior Member
 
Registered: May 2009
Location: center of singularity
Distribution: Xubuntu, Ubuntu, Slackware, Amazon Linux, OpenBSD, LFS (on Sparc_32 and i386)
Posts: 2,688

Original Poster
Blog Entries: 31

Rep: Reputation: 176Reputation: 176
Quote:
Originally Posted by syg00 View Post
Says it all really. Why should anyone else waste their time, if you're too lazy to even try ?.
maybe (and pan64 seems to be quite sure not so) someone else (perhaps not retired) might have considered such a tool to be of great value.

just because i am that lazy, does not mean everyone else also is. we'll see.
 
Old 11-13-2022, 02:20 PM   #8
Skaperen
Senior Member
 
Registered: May 2009
Location: center of singularity
Distribution: Xubuntu, Ubuntu, Slackware, Amazon Linux, OpenBSD, LFS (on Sparc_32 and i386)
Posts: 2,688

Original Poster
Blog Entries: 31

Rep: Reputation: 176Reputation: 176
Quote:
Originally Posted by MadeInGermany View Post
A growing file has a recent mtime.
List all files that were modified in the last hour:
Code:
find . -type f -size +0 -mmin -60 -ls
These are the candidates.
Redirect it to a file.
Rerun it after 10 minutes, get the differences of the sizes. And get the maximum.
sounds good.
 
Old 11-13-2022, 02:34 PM   #9
boughtonp
Senior Member
 
Registered: Feb 2007
Location: UK
Distribution: Debian
Posts: 3,627

Rep: Reputation: 2556Reputation: 2556Reputation: 2556Reputation: 2556Reputation: 2556Reputation: 2556Reputation: 2556Reputation: 2556Reputation: 2556Reputation: 2556Reputation: 2556
Quote:
Originally Posted by Skaperen View Post
just because i am that lazy, does not mean everyone else also is. we'll see.
LQ is about helping people who have put in effort and are stuck, not satisfying the demands of lazy people.

https://www.linuxquestions.org/questions/faq.php?faq=welcome

 
Old 11-13-2022, 07:28 PM   #10
Skaperen
Senior Member
 
Registered: May 2009
Location: center of singularity
Distribution: Xubuntu, Ubuntu, Slackware, Amazon Linux, OpenBSD, LFS (on Sparc_32 and i386)
Posts: 2,688

Original Poster
Blog Entries: 31

Rep: Reputation: 176Reputation: 176
Quote:
Originally Posted by boughtonp View Post
LQ is about helping people who have put in effort and are stuck, not satisfying the demands of lazy people.

https://www.linuxquestions.org/questions/faq.php?faq=welcome

and it also seems to be about people who don't comprehend what they read.

i no longer care to carry out a development project (i'm retired). many times when i did (cared and carried out) i discovered that someone else already did it and such software was already available, often free and open source. now days, the wise move is to first look for what might already exist. and i still find the Google and Duckduckgo fail to find what can be found (or at least search hints provided) by some of the more friendly people on LQ.
 
Old 11-14-2022, 12:01 AM   #11
pan64
LQ Addict
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 22,039

Rep: Reputation: 7347Reputation: 7347Reputation: 7347Reputation: 7347Reputation: 7347Reputation: 7347Reputation: 7347Reputation: 7347Reputation: 7347Reputation: 7347Reputation: 7347
Quote:
Originally Posted by Skaperen View Post
i no longer care to carry out a development project (i'm retired). many times when i did (cared and carried out) i discovered that someone else already did it and such software was already available, often free and open source.
That part is completely acceptable. Why do you (or we) want to implement anything if it was already done?
Quote:
Originally Posted by Skaperen View Post
now days, the wise move is to first look for what might already exist. and i still find the Google and Duckduckgo fail to find what can be found (or at least search hints provided) by some of the more friendly people on LQ.
In this special case it looks like the tool does not exist and also it looks like it is only you who need it. So (from my side) you need to start to implement it and this LQ community will help you, but do not expect anybody will do it for you.
From the other hand it looks like the usual xy problem for me. Why do you need it, what do you want to achieve at all?
 
2 members found this post helpful.
Old 11-14-2022, 07:04 AM   #12
boughtonp
Senior Member
 
Registered: Feb 2007
Location: UK
Distribution: Debian
Posts: 3,627

Rep: Reputation: 2556Reputation: 2556Reputation: 2556Reputation: 2556Reputation: 2556Reputation: 2556Reputation: 2556Reputation: 2556Reputation: 2556Reputation: 2556Reputation: 2556
Quote:
Originally Posted by Skaperen View Post
now days, the wise move is to first look for what might already exist. and i still find the Google and Duckduckgo fail to find what can be found (or at least search hints provided) by some of the more friendly people on LQ.
Searching first has always been what should be done - and if you don't find and decide to start a thread, explicitly include the search terms you used in your first post.

This both demonstrates that you've made some effort and it helps the next person to use those terms.

 
Old 11-14-2022, 08:28 AM   #13
dugan
LQ Guru
 
Registered: Nov 2003
Location: Canada
Distribution: distro hopper
Posts: 11,249

Rep: Reputation: 5323Reputation: 5323Reputation: 5323Reputation: 5323Reputation: 5323Reputation: 5323Reputation: 5323Reputation: 5323Reputation: 5323Reputation: 5323Reputation: 5323
https://aarvik.dk/how-to-find-growin...nux/index.html

https://serverfault.com/questions/33...a-linux-system

These are two of the more interesting hits my search got.
 
Old 11-14-2022, 01:14 PM   #14
Skaperen
Senior Member
 
Registered: May 2009
Location: center of singularity
Distribution: Xubuntu, Ubuntu, Slackware, Amazon Linux, OpenBSD, LFS (on Sparc_32 and i386)
Posts: 2,688

Original Poster
Blog Entries: 31

Rep: Reputation: 176Reputation: 176
Quote:
Originally Posted by pan64 View Post
That part is completely acceptable. Why do you (or we) want to implement anything if it was already done?
i would not want to if the existing one met my needs. if it was close, i might adapt it to meet my needs. i suspect you and others might do similar.

i don't know of anything along this line that is done. for other reasons, i'd rather not make the effort to implement such a thing (although it is not entirely ruled out). these reasons apply to pretty much any project.

Quote:
Originally Posted by pan64 View Post
In this special case it looks like the tool does not exist and also it looks like it is only you who need it. So (from my side) you need to start to implement it and this LQ community will help you, but do not expect anybody will do it for you.
From the other hand it looks like the usual xy problem for me. Why do you need it, what do you want to achieve at all?
as to the xy problem, i am not expecting anyone to do it for me. i am merely looking to see if it has already been done. and i agree it looks like it does not exist. i ask as a final check because my search skills are not 100%. there has been more than one case of software i was looking for found by asking that searching did not find, often because of different terms likely due to different perspectives on the problem, or other search failure modes.
 
Old 11-14-2022, 01:32 PM   #15
Skaperen
Senior Member
 
Registered: May 2009
Location: center of singularity
Distribution: Xubuntu, Ubuntu, Slackware, Amazon Linux, OpenBSD, LFS (on Sparc_32 and i386)
Posts: 2,688

Original Poster
Blog Entries: 31

Rep: Reputation: 176Reputation: 176
Quote:
Originally Posted by dugan View Post
these are good links. thanks! i remember seeing that first one or one like it but passed it over as just useful for implementing a script. if i do implement something, it can be a good source of info, though i would not follow it exactly (i would not want the script to write any file). filtering by time of last writing (a file last written a much earlier is of less interest) makes sense as does file size (small files are likely of lesser concern compared to large files).
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
LXer: Linux Keeps Growing and Growing and Growing LXer Syndicated Linux News 1 06-05-2020 04:38 AM
LXer: Growing Pains, a platformer where you're constantly growing is now on Linux LXer Syndicated Linux News 0 09-10-2019 11:33 PM
find continuously growing files in the file system dnaqvi Linux - General 7 04-26-2010 02:12 PM
MySQL ibdata1 growing and growing and growing and... jnojr Linux - Software 1 07-20-2009 07:11 PM
X utilaziion is growing and growing amirb Linux - Desktop 2 05-28-2007 03:55 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - General

All times are GMT -5. The time now is 01:54 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration