Writing multiple awk programs in a single file

Faki · 02-15-2023, 03:49 PM

Have been writing some awk scripts. For each type of file I have a different implementation for what needs to do. Would it be possible to have everything in a single file, and if so, can I see an exampled of how such a thing can be done?

Currently I am using different awk files.

Turbocapitalist · 02-15-2023, 08:20 PM

It depends on the specific AWK scripts which you have and how you decide to merge them but in principle it can be done.

I'd write some example but LQ is blocking it with some javascript:

Code:

 Icon for www.linuxquestions.org www.linuxquestions.org
Checking if the site connection is secure
Enable JavaScript and cookies to continue
www.linuxquestions.org needs to review the security of your connection before proceeding.

The example would have been above and started with #!/usr/bin/awk -f as the shebang.

Client-side processing is inappropriate on so many levels.

teckk · 02-15-2023, 09:34 PM

I can't post this at all with a Qt5/Qt6webengine browser. I can post this using webkit2gtk.

Example1.awk

Code:

#!/usr/bin/awk -f 

BEGIN { printf "%s\n","This is an awk script test." }

Example2.awk

Code:

#!/usr/bin/awk -f 
 
BEGIN {
    x=0

    do {
        print x;
        x+=1;
    }
    while(x<=10)
}

Just something that I noticed.

pan64 · 02-16-2023, 01:01 AM

Using more than one file is ok. I mean putting everything into one file is not necessary, but if you want you can do that.

Code:

CMD1='BEGIN {
    x=0

    do {
        print x;
        x+=1;
    }
    while(x<=10)
}'
CMD2='BEGIN { printf "%s\n","This is an awk script test." }'
CMD3='.....'

awk "${CMD1}" filename
awk "${CMD2}" oth_file
....

MadeInGermany · 02-16-2023, 02:22 AM

There is an awk variable FILENAME

Code:

awk '
# FNR == Line number per file
FNR == 1 { print "Processing line #1 of", FILENAME }
' filea fileb

You can branch on a FILENAME .ext like this:

Code:

awk '
FILENAME != pfn {
  ext=pfn=FILENAME
  sub(/.*[.]/, "", ext)
}
{
  if (ext == "e1") {
    print "Do e1 action on", FILENAME, "line", FNR
  } else if (ext == "e2") {
    print "Do e2 action on", FILENAME, "line", FNR
  } else {
    print "Do other action on", FILENAME, "line", FNR
  }
}
' filea.e1 fileb.e2 filec.e3

But it's not efficient, because it must test/branch for each input line.

MadeInGermany · 02-16-2023, 02:31 AM

More efficient: keep your different awk scripts and test/branch in the calling shell script.

Code:

#!/bin/sh
for fn in filea.e1 fileb.e2 filec.e3
do
  case ${fn##*.} in
  ( e1 ) echo "run awk.e1 on $fn"
  ;;
  ( e2 ) echo "run awk.e2 on $fn"
  ;;
  ( * ) echo "do other action on $fn"
  esac
done

Faki · 02-16-2023, 06:59 AM

@MadeInGermany The more efficient version is what I currently have (separate files for different purposes).
I could not improve on the aforementioned strategy. It is a shame that awk and sed suffer such limitations.
In shell scripts one can conveniently use functions for different tasks. Although with awk one can specify
function, they are not identical to bash functions.

MadeInGermany · 02-16-2023, 07:52 AM

I do not see your problem.

You can bundle common awk functions in a file, like this:

Code:

awk -f functions.awk -f e1.awk filea.e1
awk -f functions.awk -f e2.awk fileb.e2

Faki · 02-16-2023, 08:34 AM

Can one put BEGIN clauses in functions and whatever? The -f option reads the awk program source from source-file, rather than using a specific function inside an awk file. In a bash script for instance, one can call a specific function, which itself can take positional arguments. And also be able to call a different bash funcion doing completely different things in the same bash file. I cannot see that I can do the same thing for awk files.

One still ends up for multiple files, each of which is quite short. Right? Then it is much easier to just call awk commands from bash without the need to put the awk code in separate awk files.

MadeInGermany · 02-16-2023, 09:16 AM

Sure awk has got functions.
Example:

Code:

function abs(num) {
  return (num < 0 ? -num : num)
}

{
  print abs($1), abs($2)
}

Other examples:
https://www.gnu.org/software/gawk/ma...n-Example.html

Faki · 02-16-2023, 10:23 AM

@MadeInGermany Sure, one can use make awk functions and call them within an awk scripts. But can one access specific awk functions from bash? Can one include different BEGIN and END clauses in different awk functions. And can one write patterns and actions in awk functions?

pan64 · 02-16-2023, 11:06 AM

Quote:

Originally Posted by Faki

@MadeInGermany Suce, one can use make awk functions and call them within an awk scripts. But can one access specific awk functions from bash? Can one include different BEGIN and END clauses in different awk functions. And can one write patterns and actions in awk functions?

Actually I don't really understand it. awk and bash are two different languages. You can start any number of awk scripts from bash and also you can start any number of scripts from awk too. But you cannot invoke a function written in awk directly in bash.
In such cases you can put your awk scripts into different files (or variables) and run (execute) the one you need. But I think you need to use only one language to implement a functionality, mixing two (or more) will make things definitely harder.
bash is good enough in a lot of cases, but cannot do calculations very well.

MadeInGermany · 02-16-2023, 11:34 AM

awk and shell are different languages. They don't share anything (e.g. variables and functions are separate).
The shell has good interfaces to the processes that it runs.

BEGIN and END can run functions.
A function cannot contain BEGIN or END, why should it??

Do you mean
pattern {action}
?
You can always have an if clause in an action block:
{if (pattern) {further action}}
regardless if the action block is in the main (input loop) section or in a BEGIN or END or function.

Faki · 02-16-2023, 12:05 PM

I have been scrutinising the possibility of writing various awk functionalities without actually requiring a separate awk file for each task. Bash scripts seem to be much more useful.

Thank you for your statement regarding the non-permissibility of having BEGIN or END clauses in awk functions. My reason for my question was the possibility of having multiple BEGIN clauses in the same file with an option to do a particular one. This to avoid having a separate awk file. My awk files for the different things I do are very small, few tens of lines and was looking forward to the task of putting them together in a single file.

From the discussion, I would be better served using small separate files as I am currently doing. Would using positional argument options possibly handle different BEGIN clauses in an awk file?

Faki · 02-16-2023, 12:37 PM

The problem with

Code:

pattern {action}

is that the pattern is always tested. Suppose I do not want the pattern to be tested in certain situations. Could I avoid testing for pattern? Could I just branch so it does nothing?