[SOLVED] Search string on pattern of special characters Bash

rbees · 03-17-2015, 01:59 PM

Ladies & Gents

Thanks for all the great help and guidance you offer.

I need to test 550 unique strings for the presence of special characters in them and preform actions based on which pattern is detected.

Those five patterns are;
#- as in the sample string et0106.htm#9-22 (400 of these ?)
#-# as in the sample string et0101.htm#1-et0102.htm#3 (100 of these?)
#-#-# as in the sample string et0119.htm#21-et0120.htm#-et0121.htm#4
; as in the sample string et1027.htm#6-et1028.htm#13;et1029.htm#22-23
and a fall through et0534.htm# (there is only this one)

This does not work

Code:

for ALIYAH in "${arr[@]:1:11}";do
      pat=[#-]
      if [[ "$ALIYAH" == "$pat" ]]; then
	SHIR1="$(echo "$ALIYAH" |awk -F \# '{print $1}')"
	START1="$(echo "$ALIYAH" |awk -F \# '{print $2}'|awk -F - '{print $1}')"
	END1="$(echo "$ALIYAH" |awk -F - '{print $2}')"
      pat2=[#-#]
      elif [[ "$ALIYAH" == "$pat2" ]]; then
	# some actions
      pat3=[#-#-#]
      elif [[ "$ALIYAH" == "$pat3" ]]; then
	# some action
      pat4=[;]
      elif [[ "$ALIYAH" == "$pat4" ]]; then
	# some actions
      else
	SHIR10="$(echo "$ALIYAH" |awk -F \# '{print $1}')"
      fi

It falls right out the bottom to the fall through "else". Some of the debug

Code:

+ for ALIYAH in '"${arr[@]:2:11}"'
+ pat='[#-]'
+ [[ some.htm#11-38 == \[\#\-\] ]]
++ echo some.htm#11-38
++ awk -F '#' '{print $1}'
+ SHIR10=some.htm

The pattern of the special characters is the only constant in the strings but the number of [a-z][0-9] between them changes as can be seen from the samples above. So clearly something more complex than what I have is required to make it ignore the regular characters between the special characters.

To break the most complex one down

et1027.htm # 6 - et1028.htm # 13 ; et1029.htm # 22 - 23
filename # regex in file - filename # regex in file ; filename # regex in file - regex in file

The file names can be one character longer but never shorter
The regex can be as many as 3 numbers long (123)

Trying to change the special characters to something else is somewhat problematic because of the difference in the strings. I already had to generate them by hand, I don't want to go through and change them by hand too.

Thanks Again

rbees · 03-17-2015, 04:14 PM

So after some more looking for the issue I found a possible solution

Code:

if [[ "$ALIYAH" =~ [#-] ]]; then

But as I feared it also tests positive for pattern 2 #-#-# and I really don't want to change the order of the testing to test for the least likely matches first.

rbees · 03-17-2015, 04:39 PM

I do have a different problem too that is not related to the conditional tests directly. In the processing resulting from the conditional test I have

Code:

	w3m -dump -T text/html $STORDIR/JPS/$SHIR1 | sed '1,5d' | sed -e :a -e '$d;N;2,6ba' -e 'P;D' | sed -n "/$START1/,/$END1/p" >  $TMPDIR/$PARSHA$arr[]

But it is not being processed in the correct order by bash for some reason which can be seen in the debug code and the resulting file.

Code:

+ w3m -dump -T text/html /home/kingbee/bin/shabbat/data/JPS/et0306.htm
+ sed 1,5d
+ sed -n /1/,/12/p
+ sed -e :a -e '$d;N;2,6ba' -e 'P;D'

Can I combine some of the sed's somehow?

Thanks again

grail · 03-17-2015, 07:32 PM

Yes you can combine the seds, try looking here

Quote:

I really don't want to change the order of the testing to test for the least likely matches first.

In this case you may then find it impossible to solve. You cannot force the system just because you do not wish to change something.
You either wish to find the most regular occurrence first so as the others are only run every now and then or you need to run through process of elimination.
Your examples of #-, #-# and #-#-# obviously grow from the first to the last so if you test for #- first then you can expect to never get to the other solutions.

rbees · 03-17-2015, 07:55 PM

Thanks grail

I thought that may be the case. In my search looking for an answer I came across a simple way to use the numerical value of the strings to do the test instead. So I changed them to

Code:

if [[ ${#ALIYAH} -gt 13 ]] && [[ ${#ALIYAH} -lt 18 ]]

if [[ ${#ALIYAH} -gt 23 ]] && [[ ${#ALIYAH} -lt 30 ]]

if [[ ${#ALIYAH} -gt 36 ]] && [[ "$ALIYAH" -ne "$pat4" ]]

and it seams to work fairly well.

I will look into the sed link you gave me and see what I can do about the other thing.

Thanks again

grail · 03-17-2015, 11:33 PM

Personally I would use (()) over [[]] so you can use arithmetic tests. You can also place the && inside the brackets so only one set is needed.

I would expect the last test to never work as pat4 = ';' and ALIYAH contains ';' but is never equal to a single ';'

rbees · 03-18-2015, 05:37 AM

Thanks grail

I am having an issue with this line even parsing correctly. Shellcheck reports

Code:

pat4="[;]"
elif (( ${#ALIYAH} > 36 )) && [[ "$ALIYAH" ~= "$pat4" ]]; then
            ^SC1009 The mentioned parser error was in this elif clause.
                                          ^SC1073 Couldn't parse this test expression.
                                                         ^SC1072  Fix any mentioned problems and try again.

I have tried both (()) and [[]] and with && inside and outside, still no joy.

I can't just use a simple string length comparison because the strings that contain the ; and the ones that don't have overlap in length, bummer.

AH, found it

Code:

[[ ${#ALIYAH} -gt 36 ]] -a [["$ALIYAH" ~= [\;] ]]

Thanks again

rbees · 03-18-2015, 05:54 AM

Guess I spoke too soon.

Shellcheck reports it good but bash be sayin me no think so

Code:

./ReadingTest: line 81: syntax error near unexpected token `-a'                                                                                                                           
./ReadingTest: line 81: `      elif [[ ${#ALIYAH} -gt 36 ]] -a [["$ALIYAH" ~= [\;] ]]; then'

and if I change the -a to && bash says

Code:

$ ./ReadingTest                                                                                                                                  
./ReadingTest: line 94: conditional binary operator expected                                                                                                                              
./ReadingTest: line 94: syntax error near `~='                                                                                                                                            
./ReadingTest: line 94: `      elif [[ "$ALIYAH" ~= [\;] ]]; then'

Which is the same error I had that made me pat4="[;]" thinking that would be parsed correctly but then you tell me no it won't. So now me be even more confused than before.

I can change the ; to something else in the file name, but it would have to be a letter not [abehmt] as those letters and all numbers are part of the file names. Will have to give that some consideration as to what may be the best.

rbees · 03-18-2015, 06:11 AM

Needed a space and changing the ; to a Z did the trick.

Now back to getting it to process the sed's correctly, and naming the output file to something that makes sense.

grail · 03-18-2015, 06:49 AM

I am guessing all the places you have '~=' is a typo and you have placed these correctly in the working version?

rbees · 03-23-2015, 04:59 PM

Thanks again for all the guidance & help