RMLINT -- move files instead of deleting them

ericlindellnyc · 12-05-2023, 07:18 PM

I'm using RMLINT as a substitute for DUPEGURU, which is CPU- and RAM-intensive. However, RMLINT doesn't give me the option of moving duplicate files to a new folder instead of deleting them -- a feature that DUPEGURU has and which I can't find on any command-line script.
This is important, especially when I'm new to RMLINT, to avoid accidental erasures by keeping duplicates for a while before deleting.

I see a feature request from 2017, though I don't know if it's been implemented.
It's hard to find a command-line script that does all I need, including
searching for duplicate files or folders
only matches with the same name
using a reference folder
selecting file types to delete by extension
dry runs
moving duplicate hierarchy to separate folder

syg00 · 12-05-2023, 07:29 PM

Never used it (there are a plethora of CLI tools out there), but their own doco states:

Quote:

rmlint itself WILL NOT DELETE ANY FILES. It does however produce executable output (for example a shell script) to help you delete the files if you want to.

Most of the (useful) tools do likewise. It's just a script, hack it to your needs.

Edit: Missed the update as I was off looking for the doco. Get a tool that does most of what you want, and then manage it in the script. All the heavy lifting should be done for you.

ericlindellnyc · 12-05-2023, 09:31 PM

Quote:

Originally Posted by syg00

Never used it (there are a plethora of CLI tools out there), but their own doco states:Most of the (useful) tools do likewise. It's just a script, hack it to your needs.

Edit: Missed the update as I was off looking for the doco. Get a tool that does most of what you want, and then manage it in the script. All the heavy lifting should be done for you.

That's a good idea. The rm command in the output file can be replaced with mv. But it helps to move the entire directory structure of duplicate files -- as dupeguru does. Dupeguru does everything I need, except it's so RAM/CPU-hungry that I can use it only on limited numbers of files.

I don't trust any script, app, or myself for that matter -- to handle this without error, which is why I insist on moving the files instead of deleting -- at least for a while.

I believe dupeguru does byte-for-byte comparison, which could account for its inefficiency. I can't find a switch to use only the hashing algorithm.

Thanks for your reply.

ericlindellnyc · 05-17-2024, 08:43 PM

Update. I am trying to use rmlint to remove duplicates. I dont want to delete them -- just move them to a different drive. I would like the original directory structure preserved.

My original command was executed from:

Code:

/Volumes/rmlintTest/testData15may24/

The original command was

Code:

rmlint -gvbex -T df /Volumes/rmlintTest/testData15may24/

The destination tmp was at

Code:

/Volumes/rmlintTest/testData15may24/tmp

which produces a ".sh" output file containing the original rm_command code. I replaced that code with the following -- taken from rmlint documentation.

Code:

remove_cmd() {
    echo 'Deleting:' "$1"
    if original_check "$1" "$2"; then
        if [ -z "$DO_DRY_RUN" ]; then
            # was: rm -rf "$1"
            mv -p "$1" "/tmp$1"
        fi
    fi
}

Here, I tried moving "$1" up a level from "." level -- namely a folder called tmp.
However, the folder tmp is misconstrued as being below Volumes, like this -- in MacOS.
cp: tmp/Volumes/rmlintTest/testData15may24/dir2/11-testLint.txt: No such file or directory.

I've tried cp instead of mv -- by cd'ing into the target folder before cp'ing. That does move the files -- but doesn't preserve the folder structure.

I've been on this for a while and am totally stumped. Assistance appreciated.

MadeInGermany · 05-18-2024, 09:24 AM

Maybe mkdir the path first?

Code:

    newfn="/tmp/${1#/}"
    mkdir -p "${newfn%/*}" &&
    mv -v "$1" "$newfn"