Hi , i am building a script that will check if from a list of http links if they are online .
since i am using a list , then i am using a loop for this , every time the script does a loop it reads the next line from a file (containing the urls) , then get that url , executes wget to get that file from url , then after 5 seconds of downloading kills wget process .
After this point it goes check the incomplete downloaded file for its size , if size is near 0 then it says that link is down , else it says that link is ok .
during all this process the script removes from a temp folder the pid file that it got when wget was starting and it removes the incompleted downloaded file so it can start all over again to the next line of the url file list .
The script is very simple , but somehow during the process i get an error saying that the pid from wget was already terminated !!! , i have no idea why i get this message in the end of the loop , because that pid was killed a few lines before and i an not killing it again .
It looks complicated but it is not , however i am unable to write here a simple practical example but just some part of the code i am using , and a screenshot from terminal .
I will explain line by line what the script does , so you have an idea how it is working .
Code:
# this line gets from a specific file all the urls starting in "http"
# and removes from the captured urls the ones that have png of jpg
glnk=$(grep -F "http" < "$path/temp/1" | sed '/jpg/d' | sed '/png/d')
# this line writes all the desired urls to a new file
echo "$glnk" | tr " " "\n" > "$path/temp/2"
# this line counts all the urls to be checked so i can setup a loop
srvnmb=$(wc -l "$path/temp/2" | awk '{print$1}')
# these both lines remove any temporary temp files
rm -rf "$path/temp/stream" >/dev/null 2>&1
rm -rf "$path/temp/pid.tmp" >/dev/null 2>&1
# loop start from 1 to the number from variable srvnmb
for i in $(seq "$srvnmb")
do
#this line reads the line number in i
chkf=$(sed -n "${i}p" < "$path/temp/2")
#this line reads the first 4 letters from the previous line
chkurl=$(echo "$chkf" | head -c 4)
# here it checks if it is an url http or a rtmp
case "$chkurl" in
http|rtmp|HTTP)
rm -rf "$path/temp/stream" >/dev/null 2>&1
rm -rf "$path/temp/pid.tmp" >/dev/null 2>&1
#here starts wget , that will get the file from that url and
# at same time the script will create a pid file with wget pid to #kill later
wget "$chkf" -O "$path/temp/stream" >/dev/null 2>&1 & echo $! > "$path/temp/pid.tmp"
# here i read the current pid from wget to a variable
pid=$(sed -n 1p < "$path/temp/pid.tmp")
# this timer is to wait for wget to retrieve some data from url
sleep 5
# on this line i check if wget pid is still active
rpid=$(ps -p "$pid" -o pid= | awk '{print$1}')
# in case the pid is active then kill it
if [[ "$rpid" == "$pid" ]]
then
kill "$pid" > /dev/null 2>&1
fi
# this if cause i check if the downloaded file exists
if [[ ! -f "$path/temp/stream" ]]
then
echo "Error reading captured file"
else
# on this line i check the size in bytes from the downloaded file
stsz=$(wc -c "$path/temp/stream" | awk '{print$1}')
# if the size of the file is less that 100 bytes then url is down
if [[ "$stsz" -le "100" ]]
then
echo "$chkf is down"
else
echo "$chkf is ok"
fi
fi
;;
# this next case option is in case the url does not starts with http
#this way i can bypass any error in wget
*)
;;
esac
done
after this point the loop starts all over again
The next image is from my terminal running the script , it says line 21 but line 21 in script is where the "case" sentence starts , and have nothing to do with this error .
https://s14.postimg.org/fg1c2bq4x/iptv-check_031.jpg
I remember to had this error in some scripts before , but i never knew the reason for this to happen .
Does anyone have a slight idea ?
Last note : It is interesting that if the link is down then the error does not appear , despite the fact that a 0 bytes file is created , but if link is ok then these errors start to appear .
The difference between them is that one file have size , and other not , but both exists .
But the error is not even on checking the output file , but with the killing process that i called to check somehow !??? , and then i get those wget calls without the variables previous readed .
I even try to put some helpers on the code to identify the error , but everything is ok .
By example i placed an echo to print the variable that wget will check after this code :
case "$chkurl" in
http|rtmp|HTTP)
echo "$chkf"
and everything is fine .
However , i believe that one of the causes of this issue is that wget may be not outside of execution memory when next call is given to it , but i am not sure .
If until tomorrow no one had a clue why this is happening because it may have some difficulty to test the code , then i will try to build a simple test script based on this one for all of you test there .
