Learning AWK Programming: A fast, and simple cutting-edge utility for text-processing on the Unix-like environment

Different flavors of GNU/Linux distribution have different package-management utilities. If you are using a Debian-based GNU/Linux distribution, such as Ubuntu, Mint, or Debian, then you can install it using the Advance Package Tool (APT) package manager, as follows:

[ shiwang@linux ~ ] $ sudo apt-get update -y
[ shiwang@linux ~ ] $ sudo apt-get install gawk -y

Similarly, to install AWK on an RPM-based GNU/Linux distribution, such as Fedora, CentOS, or RHEL, you can use the Yellowdog Updator Modified (YUM) package manager, as follows:

[ root@linux ~ ] # yum update -y 
[ root@linux ~ ] # yum install gawk -y

For installation of AWK on openSUSE, you can use the zypper (zypper command line) package-management utility, as follows:

[ root@linux ~ ] # zypper update -y
[ root@linux ~ ] # zypper install gawk -y

Once installation is finished, make sure AWK is accessible through the command line. We can check that using the which command, which will return the absolute path of AWK on our system:

[ root@linux ~ ] # which awk
/usr/bin/awk

You can also use awk --versionto find the AWK version on our system:

 [ root@linux ~ ] # awk --version 

Like every other open source utility, the GNU AWK source code is freely available for download as part of the GNU project. Previously, you saw how to install AWK using the package manager; now, you will see how to install AWK by compiling from its source code on the GNU/Linux distribution. The following steps are applicable to most of the GNU/Linux software for installation:

[ shiwang@linux ~ ] $ wget http://ftp.gnu.org/gnu/gawk/gawk-4.1.3.tar.xz

[ shiwang@linux ~ ] $ tar xvf gawk-4.1.3.tar.xz

[ shiwang@linux ~ ] $ cd gawk-4.1.3  &&  ./configure

[ shiwang@linux ~ ] $ make

[ shiwang@linux ~ ] $ sudo make install

[ root@linux ~ ] # which awk
/usr/bin/awk

Now you have a working AWK/GAWK installation and we are ready to begin AWK programming, but before that, our next section describes the workflow of the AWK interpreter.

Before proceeding, let's create a empinfo.txt file for practice. Each line in the file contains the name of the employee, their phone number, email address, job profile, salary in USD, and working days in a week:

Jack 9857532312 [email protected] hr 2000 5
Jane 9837432312 [email protected] hr 1800 5
Eva 8827232115 [email protected] lgs 2100 6
amit 9911887766 [email protected] lgs 2350 6
Julie 8826234556 [email protected] hr 2500 5

The syntax of the awk command with a pattern only is as follows:

awk   '/ pattern /'    inputfilename

In the given example, all lines of the empinfo.txt file are processed, and those that contain the Jane pattern are printed:

$ awk '/Jane/' empinfo.txt
Jane 9837432312 [email protected] hr 1800 5

The syntax of the awk command with an action only is as follows:

awk '{ action statements / awk-commands }' inputfilenames

If you omit the pattern and give the action statement (AWK commands), then the given action is performed on all input lines, for example:

$ awk '{ print $1 }' empinfo.txt 
Jack
Jane
Eva
amit
Julie

In the given example, all employee names are printed on the screen as $1, representing the first field of each input line.

We can print each input record of the input file in multiple ways, as shown in the following example. All the given examples will produce the same result by printing all the input records of the input file.

In our first example, we specify the empty pattern without any action statement to print each input record of the input file, as follows:

$ awk '//' empinfo.txt

In this example, we specify the print action statement only, without giving any pattern for matching, and print each input record of the input file, as follows:

$ awk '{ print }' empinfo.txt

In this example, we specify the $0 default variable, along with the print action statement, to print each input record of the input file, as follows:

$ awk '{ print $0 }' empinfo.txt

In this example, we specify the empty expression along with the print action statement to print each input record of the input file, as follows:

$ awk '//{ print }' empinfo.txt

All of the given examples perform the basic printing operation using AWK. On execution of any of the preceding examples, we will get the following output:

Jack 9857532312 [email protected] hr 2000 5
Jane 9837432312 [email protected] hr 1800 5
Eva 8827232115 [email protected] lgs 2100 6
amit 9911887766 [email protected] lgs 2350 6
Julie 8826234556 [email protected] hr 2500 5

AWK contains two special keywords, BEGIN and END, for patterns where an action is required. Both of them are optional and are used without slashes. They don't match any input lines.

The BEGIN block is executed in the beginning, before the first input line of the first input file is read. This block is executed once only when the AWK program is started. It is frequently used to initialize the variables or to change the value of the AWK built-in variables, such as FS. The syntax of the BEGIN block is as follows:

BEGIN { action / awk-commands }

It is the same pattern-action block that we discussed at the beginning of the chapter. The syntax of the body block is as follows:

/ search pattern / { action / awk-commands }

In the body block, AWK commands are applied by default on each input line, however, we can restrict this behavior with the help of patterns. There are no keywords for the body block.

The END block is executed after the last input line of the last input file is read. This block is executed at the end and is generally used for rendering totals, averages, and for processing data and figures that were read in the various input records. The syntax of the END block is as follows:

END { action / awk-commands }

Here is an example of the usage of the BEGIN and END blocks:

$ awk  'BEGIN { print "==Employee Info==" }              # begin block
{ print }                                                # body block
END { print "==ends here==" }'  empinfo.txt              # end block

On executing the code, we get the following result:

==Employee Info==
Jack 9857532312 [email protected] hr 2000 5
Jane 9837432312 [email protected] hr 1800 5
Eva 8827232115 [email protected] lgs 2100 6
amit 9911887766 [email protected] lgs 2350 6
Julie 8826234556 [email protected] hr 2500 5
===ends here==

This is the most-used method of running AWK programs. In this method program, AWK commands are given in single quotes as the first argument of the AWK command line, as follows:

$ awk  'program' input file1 file2 file3 .......fileN

Here, program refers to the sequence of pattern-action statements discussed earlier. In this format, the AWK interpreter is invoked from the shell or Terminal to process the input line of files. The quotes around program instruct the shell not to interpret the AWK character as a special shell character and treat the entire argument as singular, for the AWK program not for the shell. It also enables the program to continue on more than one line.

The format used to call the AWK program from inside of a shell script is the same one we used on the Unix/Linux command line. For example:

$ awk  '{ print }' empinfo.txt /etc/passwd 

The preceding command will print every line of the empinfo.txt file, followed by the lines of the /etc/passwd file on your system, as follows:

Jack 9857532312 [email protected] hr 2000 5
Jane 9837432312 [email protected] hr 1800 5
Eva 8827232115 [email protected] lgs 2100 6
amit 9911887766 [email protected] lgs 2350 6
Julie 8826234556 [email protected] hr 2500 5
at:x:25:25:Batch jobs daemon:/var/spool/atjobs:/bin/bash
avahi:x:481:480:User for Avahi:/run/avahi-daemon:/bin/false
avahi-autoipd:x:493:493:User for Avahi IPv4LL:/var/lib/avahi-autoipd:/bin/false
bin:x:1:1:bin:/bin:/bin/bash
daemon:x:2:2:Daemon:/sbin:/bin/bash
games:x:12:100:Games account:/var/games:/bin/bash
man:x:13:62:Manual pages viewer:/var/cache/man:/bin/bash
messagebus:x:499:499:User for D-Bus:/run/dbus:/bin/false
......................
...................... till last line in /etc/passwd

Filter commands can take their input from stdin instead of reading it from the file. We can omit giving input filenames at the command line while executing the awk program, and simply call it from the Terminal as:

$ awk  'program'

In the previous example, AWK applies the program to whatever you type on the standard input, that is, the Terminal, until you type end-of-file by pressing Ctrl + D, for example:

$ awk  '$2==50{ print }'
apple 50
apple 50
banana 60
litchi 50
litchi 50
mango 55
grapes 40
pineapple 60
........

The line that contains 50 in the second field is printed, hence it's repeated twice on the Terminal. This functionality of AWK can be used to experiment with AWK; all you need is to type your AWK commands first, then type data, and see what happens next. The only thing you have to take care of here is to enclose your AWK commands in single quotes on the command line. This prevents the shell expansion of special characters, such as $, and also allows your program to be longer than one line.

Here is one more example in which we take input from the pipe and process it with the AWK command:

$ echo -e "jack \nsam \ntarly \njerry" | awk '/sam/{ print }' 

On executing this code, you get the following result:

sam

We will be using examples of executing the AWK command line on the Terminal throughout the book for explaining various topics. This type of operation is performed when the program (AWK commands) is short (up to a few lines).

When AWK programs are long, it is more convenient to put them in a separate file. Putting AWK programs in a file reduces errors and retyping. Its syntax is as follows:

$ awk -f source_file inputfile1 inputfile2 ............inputfileN

The -f option tells the AWK utility to read the AWK commands from source_file. Any filename can be used in place of source_file. For example, we can create a cmd.awk text file containing the AWK commands, as follows:

$ vi cmd.awk
BEGIN { print "***Emp Info***" }
{ print }

Now, we instruct AWK to read the commands from the cmd.awk file and perform the given actions:

$ awk -f cmd.awk empinfo.txt

On executing the preceding command, we get the following result:

***Emp Info***
Jack 9857532312 [email protected] hr 2000 5
Jane 9837432312 [email protected] hr 1800 5
Eva 8827232115 [email protected] lgs 2100 6
amit 9911887766 [email protected] lgs 2350 6
Julie 8826234556 [email protected] hr 2500 5

It does the same thing as this:

$ awk 'BEGIN { print "***Emp Info***" } { print }' empinfo.txt

We can write self-contained AWK scripts to execute AWK commands, like we have with shell scripts to execute shell commands. We create the AWK script by using  #!, followed by the absolute path of the AWK interpreter and the -f optional argument. The line beginning with #! tells the operating system to run the immediately-followed interpreter with the given argument and the full argument list of the executed program. For example, we can update the cmd.awk file to emp.awk, as follows:

$ vi emp.awk
#!/usr/bin/awk -f
BEGIN { print "***Emp Info***" }
{ print }

Give this file executable permissions (with the chmod utility), then simply run ./emp.awk empinfo.txt at the shell and the system will run AWK as if you had typed awk -f cmd.awk empinfo.txt:

$ chmod +x emp.awk
$./emp.awk empinfo.txt 
***Emp Info***
Jack 9857532312 [email protected] hr 2000 5
Jane 9837432312 [email protected] hr 1800 5
Eva 8827232115 [email protected] lgs 2100 6
amit 9911887766 [email protected] lgs 2350 6
Julie 8826234556 [email protected] hr 2500 5

Self-contained executable AWK scripts are useful when you want to write AWK programs that users can invoke without having to know it was written in AWK.

For short AWK programs, it is most convenient to execute them on the command line. This is done by enclosing AWK commands in single quotes. Yet at times, the AWK commands that you want to execute on the command line are longer than one line. In these situations, you can extend the AWK commands to multiple lines using \ as the last element on each line. It is also mandatory at that time to enclose AWK commands in single quotes. For example:

$ awk  'BEGIN { print "***Emp Info***" } \
> { print } \
> END { print "***Ends Here***" } '    empinfo.txt

It is the same as if we have executed the AWK command on a single line, as follows:

$ awk 'BEGIN { print "***Emp Info***" } { print } END{ print "***Ends Here***" }' empinfo.txt

The output of the previous executed AWK command is:

***Emp Info***
Jack 9857532312 [email protected] hr 2000 5
Jane 9837432312 [email protected] hr 1800 5
Eva 8827232115 [email protected] lgs 2100 6
amit 9911887766 [email protected] lgs 2350 6
Julie 8826234556 [email protected] hr 2500 5
***Ends Here***

A comment is some text that is included in a program for documentation or human information. It is not an executable part of the program. It explains what a program does and how it does it. Almost every programming language has comments, as they make the program construct understandable.

In the AWK programming language, a comment begins with the hash symbol (#) and continues till the end of the line. It is not mandatory to have # as the first character on the line to mark it as a comment. Anything written after # is ignored by the AWK commands. For example, we can put the following in emp.awk and update it as emp_comment.awk:

$ vi emp_comment.awk
#!/usr/bin/awk -f

# Info        : This program displays the employees information 
# Date        : 09 Sept 2017
# Version     : 1.0 
# Author      : Shiwang 

# Header part is defined in BEGIN block to display Company information 

BEGIN { print "****Employee Information of HMT Corp.****" } 
# Body Block comment
{ print }           
# End Block comment                                                                                                    
END { print "***Information Database ends here****" }   

Now, give this program executable permission (using chmod) and execute it as follows:

$ ./emp_comment.awk empinfo.txt

Here is the output:

****Employee Information of HMT Corp.****
Jack 9857532312 [email protected] hr 2000 5
Jane 9837432312 [email protected] hr 1800 5
Eva 8827232115 [email protected] lgs 2100 6
amit 9911887766 [email protected] lgs 2350 6
Julie 8826234556 [email protected] hr 2500 5
***Information Database ends here****

As you have seen, we will be using the command line for most of our short AWK programs. The best way to use it is by enclosing the entire program in single quotes, as follows:

$ awk '/ search pattern / { awk commands }' inputfile1 inputfile2

When you are working on a shell, it is good to have a basic understanding of shell quoting rules. The following rules apply only to the POSIX-compliant, GNU Bourne Again Shell:

$ echo "Welcome to " Learning "awk"
>>>
Welcome to Learning awk

$ echo  "Apple are \$10 a dozen"
>>>
Apple are $10 a dozen

$ echo 'Apple are $10 a dozen'
>>>
Apple are $10 a dozen

$ echo  "Hi, \" Jack \" "
>>>
Hi, "Jack"

$ awk  'BEGIN { print "Hello world" }' 

It can be performed as follows:

$ awk  "BEGIN { print \"Hello world \" }"

Hello world

Sometimes, dealing with single quotes or double quotes becomes confusing. In these instances, you can use octal escape sequences. For example:

$ awk "BEGIN { print \"single quote' \" }" 

$ awk 'BEGIN { print "single quote'\'' " }'

$ awk 'BEGIN { print "single quote\47" }' 

$ awk -v q="'" 'BEGIN { print "single quote"q }'  

single quote'

AWK can have multiple pattern-action statements. They are executed in the order in which they appear in the AWK program. If one pattern-action rule matches the same linethat was matched with the previous rule, then it is printed twice. This continues until the program reaches the end of the file. In the next example, we have an AWK program with two rules:

$ awk '/maruti/ { print NR, $0 } 
/2007/ { print NR, $0 }' cars.dat

The output on execution of this code is as follows:

1 maruti          swift       2007        50000       5
1 maruti          swift       2007        50000       5
3 maruti          dezire      2009        3100        6
8 maruti          swift       2009        4100        5
9 maruti          esteem      1997        98000       1
12 fiat            punto       2007        45000       3

The record number 1 is printed twice because it matches both rule1 and rule2.

Sometimes, we may need to read input from standard input and from the pipe. The way to name the standard input, with all versions of AWK, is by using a single minus or dash sign,  -.  For example:

$ cat cars.dat | awk '{ print }' -

This can also be performed as follows:

$ cat cars.dat | awk  '{ print }'  /dev/stdin ( used with gawk only )

The output on execution of this code is as follows:

maruti          swift       2007        50000       5
honda           city        2005        60000       3
maruti          dezire      2009        3100        6
chevy           beat        2005        33000       2
honda           city        2010        33000       6
chevy           tavera      1999        10000       4
toyota          corolla     1995        95000       2
maruti          swift       2009        4100        5
maruti          esteem      1997        98000       1
ford            ikon        1995        80000       1
honda           accord      2000        60000       2
fiat            punto       2007        45000       3

We can also first read the input from one file, then read the standard input coming from the pipe, and then read another file again. In that case, the first file's data, the data from the pipe, and the other file's data, all become a single input. All of that data is read consecutively. In the following example, the input from cars.dat is read first, then the echo statement is taken as input, followed by the emp.dat file. Any pattern you apply in this AWK program will be applied on the whole input and not each file, as follows:

$ echo "======================================================" | \
awk '{ print NR , $0 }' cars.dat - emp.dat

The output on execution of this code is as follows:

1 maruti          swift       2007        50000       5
2 honda           city        2005        60000       3
3 maruti          dezire      2009        3100        6
4 chevy           beat        2005        33000       2
5 honda           city        2010        33000       6
6 chevy           tavera      1999        10000       4
7 toyota          corolla     1995        95000       2
8 maruti          swift       2009        4100        5
9 maruti          esteem      1997        98000       1
10 ford            ikon        1995        80000       1
11 honda           accord      2000        60000       2
12 fiat            punto       2007        45000       3
13 ======================================================
14 Jack    Singh   9857532312  [email protected]      M   hr      2000
15 Jane    Kaur    9837432312  [email protected]      F   hr      1800
16 Eva     Chabra  8827232115  [email protected]       F   lgs     2100
17 Amit    Sharma  9911887766  [email protected]      M   lgs     2350
18 Julie   Kapur   8826234556  [email protected]     F   Ops     2500
19 Ana     Khanna  9856422312  [email protected]    F   Ops     2700
20 Hari    Singh   8827255666  [email protected]      M   Ops     2350
21 Victor  Sharma  8826567898  [email protected]    M   Ops     2500
22 John    Kapur   9911556789  [email protected]      M   hr      2200
23 Billy   Chabra  9911664321  [email protected]      M   lgs     1900
24 Sam     khanna  8856345512  [email protected]     F   lgs     2300
25 Ginny   Singh   9857123466  [email protected]     F   hr      2250
26 Emily   Kaur    8826175812  [email protected]     F   Ops     2100
27 Amy     Sharma  9857536898  [email protected]    F   Ops     2500
28 Vina    Singh   8811776612  [email protected]      F   lgs     2300

Using command-line arguments: The AWK command line can have different forms, as follows:

awk 'program' file1 file2, file3 ………….

     awk -f source_file file1 file2, file3 ………….

         awk -Fsep 'program' file1 file2, file3 ………….

               awk -Fsep -f source_file file1 file2, file3 ………….

In the given command lines, file1, file2, file3, and so on are command-line arguments that generally represent filenames.The command-line arguments are accessed in the AWK program with a built-in array called ARGV. The number of arguments in the AWK program is stored in the ARGC built-in variable, its value is one more than the actual number of arguments in the command line. For example:

$ awk -f source_file a b c

Here, ARGV is AWKs' built-in array variable that stores the value of command-line arguments. We access the value stored in the ARGV array by suffixing it with an array index in square brackets, as follows:

ARGC has the value of four, ARGC is one more than the number of arguments because in AWK the name of the command is counted as argument zero, similar to C programs.

For example, the following program displays the number of arguments given to the AWK command and displays their value:

$ vi displayargs.awk 
# echo - print command-line arguments
BEGIN { 
     printf  "No. of command line args is : %d\n", ARGC-1;
     for  ( i = 1; i < ARGC; i++)
     printf  "ARG [ %d ] is : %s \n", i, ARGV[ i ]
}

Now, we call this AWK program with the hello how are you command line argument. Here, hello is the first command line argument, how is the second, are is the third, and you is the fourth:

$ awk -f displayargs.awk hello how are you

The output on execution of the preceding code is as follows:

No. of command line args is : 4
ARG[1] is : hello
ARG[2] is : how
ARG[3] is : are
ARG[4] is : you

The AWK commands, source filename, or other options, such as -f or -F followed by field separator, are not treated as arguments. Let's try another useful example of a command-line argument.In this program, we use command-line arguments to generate sequences of integers, as follows:

$ vi seq.awk 

# Program to print sequences of integers
BEGIN {

# If only one argument is given start from number 1
if  ( ARGC == 2 )
    for ( i = 1; i <= ARGV[1]; i++ )
    print i

# If 2 arguments are given start from first number upto second number 
else if (  ARGC == 3  )
    for ( i = ARGV[1]; i <= ARGV[2]; i++ )
    print i

# If 3 arguments are given start from first number through second with a stepping of third number
else if (  ARGC == 4  )
    for ( i = ARGV[1]; i <= ARGV[2]; i += ARGV[3] )
    print i
}

Now, let's execute the preceding script with three different parameters:

$ awk -f seq.awk 10 
$ awk -f seq.awk 1 10
$ awk -f seq.awk 1 10 1

All the given commands will generate the integers one through ten. Without the second argument, it begins printing the numbers from 1 to the first argument. If two arguments are given, then it prints the number starting from the first argument to the second argument. In the third case, if you specify three arguments, then it prints the numbers between the first and second argument, leaving out the third argument. The output on execution of any of these commands is as follows:

1
2
3
4
5
6
7
8
9
10

By default, fields in the input record are separated by any number of spaces or tabs. This behavior can be altered by using the -F option or the FS variable. The value of the field separator can beeither a single character or a regular expression.Itcontrols the way AWK splits an input record into fields:

-Ffs

--field-separator

In the next example, we illustrate the use of the -F option. Here we used the -F to print the list of usernames that has been assigned to the bash shell from the /etc/passwd file. This file contains userdetails separated by a colon (:):

$ awk -F: '/bash/ { print $1 }' /etc/passwd 

This can also be performed as follows:

$ awk --field-separator=: '/bash/ { print $1 }' /etc/passwd

The output on execution of this code is as follows:

at
bin
daemon
ftp
games
lp
man
news
nobody
...................
...................

This option is used to read the program source from the source file instead of in the first non-option argument. If this option is given multiple times, the program consists of the concatenation of the contents of each specified source file:

-f source-file

     --file=source-file

First, we will create 2 programs to print line number 2 and line number 4, respectively. Then, we will use the -f option to source those files for execution with the interpreter, as follows:

$ vi f1.awk
NR==2 { print NR, $0 }  
$ vi f2.
NR==4 { print NR, $0 }

Now, first use only f1. for sourcing:

$ awk -f f1.awk  cars.dat 

This can also be performed as follows:

awk --file=f1.awk cars.dat

The output on execution of this code is as follows:

2 honda           city        2005        60000       3

Now, we will source both the files together. AWK will concatenate the contents of the two sourced files and execute them on the cars.dat filename, as follows:

$ awk -f f1.awk -f f2.awk cars.dat

This can also be performed as follows:

$ awk --file=f1.awk --file=f2.awk  cars.dat

The output on execution of this code is as follows:

2 honda           city        2005        60000       3
4 chevy           beat        2005        33000       2

This option assigns a value to a variable before the program executes. Such variable values are available inside the BEGIN block. The -v option can only set one variable at a time, but it can be used more than once, setting another variable each time:

-v var=val

--assign var=val

The following example describes the usage of the -v option:

$ awk -v name=Jagmohon 'BEGIN{ printf "Name = %s\n", name }'

This can also be performed as follows:

$ awk --assign=Jagmohan 'BEGIN{ printf "Name = %s\n", name }'

The output on execution of this code is as follows:

Name = Jagmohan

Here is a multiple-value assignment example:

$ awk -v name=Jagmohon -v age=42 'BEGIN{ printf "Name = %s\nAge = %s\n", name, age }'

The output on execution of this code is:

Name = Jagmohan
Age = 42

This option is used to print a sorted list of global variables, their types, and final values to file. By default, it prints this list to a file named awkvars.out in the current directory. It is good to have a list of all global variables to avoid errors that are created by using the same name function in your programs. The following is the command to print the list in the default file:

-d[file]--dump-variables[=file]

$ awk --dump-variables  ' ' 

This can also be performed as follows:

 $ awk   -d   ' '

 On execution of this command, we will have a file with the name awkvars.out in our current working directory, which has the following contents:

$ cat awkvars.out
ARGC: 1
ARGIND: 0
ARGV: array, 1 elements
BINMODE: 0
CONVFMT: "%.6g"
ENVIRON: array, 99 elements
ERRNO: ""
FIELDWIDTHS: ""
FILENAME: ""
FNR: 0
FPAT: "[^[:space:]]+"
FS: " "
IGNORECASE: 0
LINT: 0
NF: 0
NR: 0
OFMT: "%.6g"
OFS: " "
ORS: "\n"
PREC: 53
PROCINFO: array, 15 elements
RLENGTH: 0
ROUNDMODE: "N"
RS: "\n"
RSTART: 0
RT: ""
SUBSEP: "\034"
TEXTDOMAIN: "messages"

This option enables the profiling of AWK programs, that is, it generates a pretty-printed version of the program in a file. By default, the profile is created in a file named awkprof.out. The optional file argument allows you to specify a different filename for the profile file. No space is allowed between -p and the filename, if a filename is supplied:

-p[file]--profile[=file]

The profile file created contains execution counts for each statement in the program in the left margin, and function call counts for each function. In the next example, we will create a file with a name sample and redirect the output of the AWK command to /dev/null:

$ awk --profile=sample \
'BEGIN { print "**header**" }
{ print } 
END{ print "**footer**" }' cars.dat > /dev/null

This same action can also be performed as follows:

$ awk -psample \
'BEGIN { print "**header**" } 
{ print }
END{ print "**footer**" }' cars.dat > /dev/null

To view the content of profile, we execute the cat command, as follows:

$ cat sample
        # gawk profile, created Thu Sep 14 17:20:27 2017

        # BEGIN rule(s)

        BEGIN {
     1          print "**header**"
        }

        # Rule(s)

    12  {
    12          print $0
        }

        # END rule(s)

        END {
     1          print "**footer**"
        }

The –pretty-print option: It is the same profiling option discussed in the preceding section:

-o[file]--pretty-print[=file]

 This option disables the execution of the system() function, which can execute shell commands supplied as an expression to AWK. It also disables the input redirections with getline, output redirections with print and printf, and dynamic extensions. This is very useful when you want to run AWK scripts from questionable/untrusted sources and need to make sure the scripts can't access your system (other than the specified input data file):

-S

--sandbox

In the following example, we first execute the echo command within the system function without the --sandbox  option, and then again with the --sandbox option to see the difference:

$ awk 'BEGIN { system("echo hello") }'

The preceding AWK command executes the echo hello command using the system function and returns a 0 value to the system upon successful execution. The output on execution of the preceding command is:

hello

Now, we use the --sandbox option with the AWK command to disable the execution of the echo hello shell command using the system function of AWK. In the next example, the system function will not execute as we have used the --sandbox option while executing it:

$ awk --sandbox 'BEGIN{ system("echo hello")}'

The output on execution of the preceding command is:

awk: cmd. line:1: fatal: 'system' function not allowed in sandbox mode

This option is equivalent to the @include directive, which is used to source a file in the current AWK program. However, it is different from the -f option in two aspects. First, when we use the -i option, the program sourced is not loaded if it has been previously loaded, whereas -f always loads a file. The second difference is this after processing an -i argument, GAWK still expects to find the main source code via the -f option or on the command line:

-i source-file--include source-file

In the next example, we will use the f1.awk and f2.awk files we created earlier to describe how the -i option works:

$ awk  -i  f1.awk  'NR==5 { print NR,  $0 }'  cars.dat

The output on execution of the given code is:

2 honda           city        2005        60000       3
5 honda           city        2010        33000       6

Now, we are using the -i option to include the f1.awk file inside the -f option to execute f2.awk, as follows:

$ awk  -i  f1.awk   -f   f2.awk    cars.dat

The output on execution of the preceding code is:

2 honda           city        2005        60000       3
4 chevy           beat        2005        33000       2

The next example shows it is mandatory to specify the AWK command or main source file using the -f option for executing a program with the -i option:

$ awk  -i  f1.awk cars.dat

The output on execution of the code is:

awk: cmd. line:1: cars.dat
awk: cmd. line:1:     ^ syntax error

This is a feature that is specific to GAWK. The @include keyword can be used to read external AWK source files and load in your running program before execution. Using this feature, we can split large AWK source files into smaller programs and also reuse common AWK code from various AWK scripts. It is useful for grouping together various AWK functions in a file and creating function libraries, using the @include keyword and including it in your program. It is similar to the -i option, which we discussed earlier.

The following example illustrates this. We'll create two AWK scripts, inc1 and inc2. Here is the inc1 script:

$ vi inc1
BEGIN { print "This is inc1." }

And now we create inc2, which includes inc1, using the @include keyword:

$ vi inc2
@include "inc1"
BEGIN { print "This is inc2." }

$ gawk -f  inc2

On executing GAWK with inc2, we get the following output:

This is inc1.
This is inc2.

So, to include external AWK source files, we just use @include followed by the name of the file to be included, enclosed in double quotes. The files to be included may be nested. For example, create a third script, inc3, that will include inc2:

$ vi inc3
@include "inc2"
BEGIN {  print "This is inc3." }

$ gawk -f inc3

On execution of GAWK with the inc3 script, we get the following results:

This is inc1.
This is inc2.
This is inc3.

The filename can be a relative path or an absolute path. For example:

@include "/home/usr/scripts/inc1"

This can also be performed as follows:

@include "../inc1"

Since AWK has the ability to specify multiple -f options on the command line, the @include mechanism is not strictly necessary. However, the @include keyword can help you in constructing self-contained GAWK programs, and eliminates the need to write complex command lines repetitively.

This option displays the version information for a running copy of GAWK as well as license info. This allows you to determine whether your copy of GAWK is up to date with respect to whatever the Free Software Foundation (FSF) is currently distributing. This can be done as shown in the following code block:

$ awk -V

It can also be performed as follows:

$ awk --version

The output on execution of the preceding code is: