[SOLVED] How to compare a list of files in two directories: compare content and print size

Batistuta_g_2000 · 03-23-2013, 10:18 AM

Hi

I have a script which unzips a list of files from a tar to a temp directory.

I now need to compare/diff on each unzipped file with the one in its original source directory.

Not sure where to start even, can find the files withe same file name in the source directory but not sure what to do next.

for i in `ls`;do find $IMPORTANT_FILES_DIRECTORY -name $i; done

Can i do an && and something like below? How would I do it with two lists of files in two directories?

for i in `ls`;
do
[[ find $IMPORTANT_FILES_DIRECTORY -name $i ]] | if diff $1 file2 >/dev/null ; then
echo Same
else
echo Different
fi;

done

Code:

#!/bin/bash

BASEDIR=$(cd `dirname $0` && pwd)
CONFILE=$BASEDIR/confile.cfg
BACKUP_FILES=$HOME/assignment1/backups
RESTORE_COMP_DIRECTORY=$HOME/assignment1/temp
FULL_LOGGING=false
LOGFLAG=false
LOGS=$HOME/assignment1/logs
LOGERR=$LOGS/log.err
LOGTMP=$LOGS/log.tmp
LOGOUT=$LOGS/log.out
TIMESTAMP=$(date +%Y%m%d-%H%M%S)
ACTION=backup

cd "$BASEDIR"
source "$CONFILE"

 rm -R $RESTORE_COMP_DIRECTORY
if [[ ! -d "$RESTORE_COMP_DIRECTORY" ]]; then
           mkdir -p "$RESTORE_COMP_DIRECTORY"
    fi

# if dir is not empty exit

if [[ "$(ls -A $RESTORE_COMP_DIRECTORY)" ]]; then
    echo "Dir "$RESTORE_COMP_DIRECTORY" is not empty exiting"               
    exit   
fi

# take filenames into array
cd $BACKUP_FILES_DIRECTORY

RESTORES="${@}"
        for f in $RESTORES
        do
        f=`echo "$f" | sed 's/^[ \t]*//;s/[ \t]*$//'` # trailing whitespaces
        if [[ -f ${f}.tar.gz ]]
        then
        tar -xzvf $BACKUP_FILES_DIRECTORY/$f.tar.gz -C $RESTORE_COMP_DIRECTORY
        echo "SUCCESS"
        sleep 3s
        else
        echo -e "File not found for "$f" please make sure is present. ...."
         sleep 3s
         exit
         fi
        done

     cd $RESTORE_COMP_DIRECTORY
     for i in `ls`;do find $IMPORTANT_FILES_DIRECTORY -name $i; done

Batistuta_g_2000 · 03-23-2013, 10:57 AM

Am pretty close with this but am getting permission denied on the files (am root)
Also how would I echo the "files are the same" or files are different from something like this?

for file in $(ls "$RESTORE_COMP_DIRECTORY"); do echo -e "\n$file";
find "$IMPORTANT_FILES_DIRECTORY" -name "$($RESTORE_FILES_DIRECTORY/$file)" -print -exec diff "$file" {} \; ; done

sleep 2s

Batistuta_g_2000 · 03-23-2013, 11:23 AM

This one shows the actual difference in the files and which are identical:

for file in $RESTORE_COMP_DIRECTORY/*; do diff -s "$file" "$IMPORTANT_FILES_DIRECTORY/${file##*/}"; done

but how do I get more information on both files - display both paths/names/file sizes

So run my restore - untar - check filenames - do diff - if diff show info if same echo same

unSpawn · 03-23-2013, 11:59 AM

Quote:

Originally Posted by Batistuta_g_2000

I now need to compare/diff on each unzipped file with the one in its original source directory.

Wouldn't it be easier to compare hashes as in 'md5sum -c /file/containing/hashes'?

Batistuta_g_2000 · 03-23-2013, 01:31 PM

Quote:

Originally Posted by unSpawn

Wouldn't it be easier to compare hashes as in 'md5sum -c /file/containing/hashes'?

hey unspawn, thanks but my mind is at meltdown as it is trying to figure out why basic things will not work for me....such as:

cd $RESTORE_COMP_DIRECTORY
for i in `ls`;do find $IMPORTANT_FILES_DIRECTORY -name $i;
if [ "$?" = "0" ]; then
echo -e "Found file "$i" in source directory"
else
echo -e "Did not find file "$i" source directory"
fi
done

giving all success even if file is not there??!!! will check out md5sum though thanks.

unSpawn · 03-23-2013, 02:14 PM

Code:

# This may be an easier way to find and unpack tar balls:
find "${BACKUP_FILES_DIRECTORY}" -type f -iname \*.tar.gz | while read ITEM; do
 tar -C "${RESTORE_COMP_DIRECTORY}" -xf "${ITEM}" || { echo "Unpack failed on \"${ITEM}\"."; break; }
done

Code:

# When testing existence this asserts the relative directory structure in
# RESTORE_COMP_DIRECTORY is similarly rooted in IMPORTANT_FILES_DIRECTORY:
cd "${RESTORE_COMP_DIRECTORY}"; find . -type f | while read ITEM; do
 echo -en ""${ITEM:2}" in source directory: "
 if [ -f "${IMPORTANT_FILES_DIRECTORY}/${ITEM:2}" ]; then
  echo -en "yes "
  cmp -s "${ITEM}" "${IMPORTANT_FILES_DIRECTORY}/${ITEM:2}"
  case $? in
   0) echo "...and they are similar.";;
   1) echo "...and they differ.";;
   *) echo "...but you better call Houston right now ;-p";;
  esac
 else
  echo "no"
 fi
done

jpollard · 03-23-2013, 08:10 PM

Quote:

Originally Posted by Batistuta_g_2000

hey unspawn, thanks but my mind is at meltdown as it is trying to figure out why basic things will not work for me....such as:

cd $RESTORE_COMP_DIRECTORY
for i in `ls`;do find $IMPORTANT_FILES_DIRECTORY -name $i;
if [ "$?" = "0" ]; then
echo -e "Found file "$i" in source directory"
else
echo -e "Did not find file "$i" source directory"
fi
done

giving all success even if file is not there??!!! will check out md5sum though thanks.

Because the exit status of find does not indicate anything about whether the file is there or not - it only indicates a command error. If there is no command error then it always exits with a success return.

Batistuta_g_2000 · 03-24-2013, 06:31 AM

Quote:

Originally Posted by unSpawn

Code:

# This may be an easier way to find and unpack tar balls:
find "${BACKUP_FILES_DIRECTORY}" -type f -iname \*.tar.gz | while read ITEM; do
 tar -C "${RESTORE_COMP_DIRECTORY}" -xf "${ITEM}" || { echo "Unpack failed on \"${ITEM}\"."; break; }
done

Code:

# When testing existence this asserts the relative directory structure in
# RESTORE_COMP_DIRECTORY is similarly rooted in IMPORTANT_FILES_DIRECTORY:
cd "${RESTORE_COMP_DIRECTORY}"; find . -type f | while read ITEM; do
 echo -en ""${ITEM:2}" in source directory: "
 if [ -f "${IMPORTANT_FILES_DIRECTORY}/${ITEM:2}" ]; then
  echo -en "yes "
  cmp -s "${ITEM}" "${IMPORTANT_FILES_DIRECTORY}/${ITEM:2}"
  case $? in
   0) echo "...and they are similar.";;
   1) echo "...and they differ.";;
   *) echo "...but you better call Houston right now ;-p";;
  esac
 else
  echo "no"
 fi
done

This is incredibly helpful - learning so much now, thanks a lot, some people on here go out of their way to help it's really appreciated.

Batistuta_g_2000 · 03-24-2013, 06:33 AM

Quote:

Originally Posted by jpollard

Because the exit status of find does not indicate anything about whether the file is there or not - it only indicates a command error. If there is no command error then it always exits with a success return.

Thanks jpollard, just spotted after the post, now testing for:

for i in `ls`;do find $IMPORTANT_FILES_DIRECTORY -name $i;
if [[ -f $IMPORTANT_FILES_DIRECTORY/$i ]]; then
### changed the above from bad::::if [ "$?" = "1" ]; then
echo -e "Found file "$i" in source directory"

unSpawn · 03-24-2013, 07:05 AM

Quote:

Originally Posted by Batistuta_g_2000

This is incredibly helpful - learning so much now, thanks a lot, some people on here go out of their way to help it's really appreciated.

It's just basic scripting. Here's some BASH scripting guides if you didn't know them already:

http://www.tldp.org/HOWTO/Bash-Prog-Intro-HOWTO.html
http://www.tldp.org/LDP/Bash-Beginne...tml/index.html
http://www.gnu.org/software/bash/man...ode/index.html
http://www.grymoire.com/Unix/Sh.html
http://www.tldp.org/LDP/abs/html/
http://mywiki.wooledge.org/BashFAQ
http://mywiki.wooledge.org/BashPitfalls

Just don't try to grok them all at once ;-p