Index

A

• Advance Package Tool (APT) / Using the package manager
• anchors
  • using / Anchors
  • used, for matching at beginning of string / Matching at the beginning of a string
  • used, for matching at end of string / Matching at the end of a string
• arithmetic expressions
  • building, binary operators used / Arithmetic expressions using binary operators
• arithmetic functions
  • sin (expr) function / The sin (expr) function
  • cos (expr) function / The cos (expr) function
  • atan2 (x, y) function / The atan2 (x, y) function 
  • int (expr) function / The int (expr) function
  • exp (expr) function / The exp (expr) function
  • log (expr) function / The log (expr) function
  • sqrt (expr) function / The sqrt (expr) function
  • rand() function / The rand() function
  • srand ([expr]) function / The srand ([expr]) function 
  • about / Summary table of built-in arithmetic functions
• arrays
  • assignment / Assignment in arrays
  • elements, accessing / Accessing elements in arrays
  • members, referring / Referring to members in arrays
  • processing, loops used / Processing arrays using loops
  • creating, split() function used / Using the split() function to create arrays
  • delete operation / Delete operation in arrays
  • sorting / Array sorting
  • sorting, by values with asort() / Sort array by values using asort( )
  • array indexes, asorti() used / Sort array indexes using asorti()
• assignment expressions
  • about / Assignment expressions
  • shorthand addition (+=) / Assignment expressions
  • shorthand subtraction (-=) / Assignment expressions
  • shorthand Multiplication (*=) / Assignment expressions
  • shorthand division (/=) / Assignment expressions
  • shorthand modulo (%=) / Assignment expressions
  • shorthand exponential (^=) / Assignment expressions
  • shorthand exponential (**=) / Assignment expressions
• AWKC++
  • reference / Types of AWK
• AWK programming language
  • overview / AWK programming language overview
  • about / What is AWK?
  • types / Types of AWK
  • using / When and where to use AWK, Getting started with AWK
  • installation, on Linux / Installation on Linux
  • workflow / Workflow of AWK
  • READ OPERATION / Workflow of AWK
  • EXECUTE OPERATION / Workflow of AWK
  • REPEAT OPERATION / Workflow of AWK
  • action / Action and pattern structure of AWK
  • pattern structure / Action and pattern structure of AWK
  • example data file /
  • pattern-only statements / Pattern-only statements
  • action-only statements / Action-only statements
  • input line/record, printing / Printing each input line/record
  • BEGIN and END blocks construct, using / Using the BEGIN and END blocks construct
  • BEGIN block / The BEGIN block
  • body block / The body block
  • END block / The END block
  • standard options / AWK standard options
  • variable / AWK variables and constants
  • constants / AWK variables and constants
• AWK programs
  • executing / Running AWK programs
  • using, as Unix command line / AWK as a Unix command line
  • using, as filter / AWK as a filter (reading input from the Terminal)
  • executing, from source file / Running AWK programs from the source file
  • as executable script files / AWK programs as executable script files
  • command line, executing on multiple lines / Extending the AWK command line on multiple lines
  • comments / Comments in AWK
  • shell quotes / Shell quotes with AWK
  • data files / Data files used as examples in this book
  • examples / Some simple examples with default usage
  • multiple rules / Multiple rules with AWK
  • standard input names, using with names / Using standard input with names in AWK
• AWK standard options
  • standard command-line options / Standard command-line options
  • GAWK-only options / GAWK-only options
• AWS, installation on Linux
  • package manager, using / Using the package manager
  • compiling, from source code /

B

• backreference / Backreferencing in regular expressions – sed and grep
• backreferencing of sub-expression / Backreferencing in regular expressions – sed and grep
• binary operators
  • used, for building arithmetic expressions / Arithmetic expressions using binary operators
• bit-manipulating function
  • and (num1, num2) function / The and (num1, num2) function
  • or (num1, num2) function / The or (num1, num2) function
  • xor (num1, num2) function / The xor (num1, num2) function
  • lshift (val, count) function / The lshift (val, count) function
  • rshift (val, count) function / The rshift (val, count) function
  • compl (num) function / The compl (num) function
• Boolean expressions / Logical or Boolean expressions
  • about / Logical or Boolean expressions
  • Logical OR (| |) / Logical or Boolean expressions
  • Logical NOT (!) / Logical or Boolean expressions
• bracket expression
  • about / Brackets expressions
  • character classes / Character classes
  • named character classes (POSIX standard) / Named character classes (POSIX standard)
• built-in command line debugger, GAWK
  • about / GAWK's built-in command line debugger
  • debugging / What is debugging?
  • breakpoint / Debugger concepts
  • watchpoint / Debugger concepts
  • stackframe / Debugger concepts
• built-in functions
  • about / Built-in functions
  • arithmetic functions / Arithmetic functions
  • string functions / String functions
  • input/output (I/O) functions / Input/output functions
  • time function / The time function
  • bit-manipulating functions / Bit-manipulating functions
• built-in variables
  • about / Built-in variables in AWK
  • field separator (FS) / Field separator 
  • output field separator / Output field separator
  • record separator (RS) / Record separator
  • record separator (RS), outputting / Outputting the record separator
  • NR / NR and NF
  • NF / NR and NF
  • FILENAME / FILENAME

C

• call stack / Debugger concepts
• case-sensitive matching / Case-sensitive matching
• complemented bracket expressions
  • about / Complemented bracket expressions
  • complemented character classes / Complemented character classes
  • complemented named character classes / Complemented named character classes
• compound statements / Conditional statements
• conditional expression / Ternary expressions
• conditional or comparison operators / Relational expressions
• conditional statements
  • about / Conditional statements
  • if statement / The if statement
  • switch statement / The switch statement (a GAWK-specific feature)

D

• decrement expressions / Increment and decrement expressions
• do...while loop statement / do...while loop statement

E

• elements
  • accessing, in arrays / Accessing elements in arrays
• END block
  • about / The END block
  • patterns / Patterns
  • actions / Actions
• entry-controlled loop / do...while loop statement
• environment variables, AWK
  • about / Environment variables in AWK
  • ARGV / ARGC and ARGV
  • ARGC / ARGC and ARGV
  • CONVFMT / CONVFMT and OFMT
  • OFMT / CONVFMT and OFMT
  • RLENGTH / RLENGTH and RSTART
  • RSTART / RLENGTH and RSTART
  • FNR / FNR
  • ENVIRON / ENVIRON and SUBSET
  • SUBSET / ENVIRON and SUBSET
  • FIELD (POSITIONAL) VARIABLE ($0 and $n) / FIELD (POSITIONAL) VARIABLE ($0 and $n)
• environment variables, GAWK
  • about / Environment variables in GAWK
  • ARGIND / ARGIND
  • ERRNO / ERRNO
  • FIELDWIDTHS / FIELDWIDTHS
  • IGNORECASE / IGNORECASE
  • PROCINFO / PROCINFO
• escape sequences / Basic regular expression construct, Escape sequences
• exponential expressions
  • about / Exponential expressions
  • exponential format 1 (^) / Exponential expressions
  • exponential format 2 (**) / Exponential expressions

F

• field separator (FS)
  • single character, using as value of FS / Using a single character or simple string as a value of the FS
  • regular expressions, using as value of FS / Using regular expressions as values of the FS
  • each character, using as separate field / Using each character as a separate field
  • command line, used for setting FS as -F / Using the command line to set the FS as -F
  • split (str, arr, regex) function / The split (str, arr, regex) function
• flow control
  • altering, with statements / Statements affecting flow control
• for each loop statement / For each loop statement
• for loop statement / The for loop statement
• format specification modifiers
  • about / Format specification modifiers
  • printing, with fixed column width / Printing with fixed column width 
  • minus modifier (-), using / Using the minus modifier (-) for left justification
  • fixed width, printing with / Printing with fixed width – right justified
  • hash modifier (#), using / Using hash modifier (#)
  • plus modifier (+), used for prefixing with sign/symbol / Using plus modifier (+) for prefixing with sign/symbol
  • prefix sign/symbol, used for printing / Printing with prefix sign/symbol
  • dot precision, using as modifier / Dot precision as modifier
  • positional modifier, using integer constant / Positional modifier using integer constant followed by $ (N$):
• format specifier
  • format control characters / Different format control characters in the format specifier
• format string / Pretty printing with the printf statement
• Free Software Foundation (FSF) / The -V option

G

• GAWK
  • about / GAWK-specific regular expression operators, Things you don't know about GAWK
  • matching whitespaces / Matching whitespaces
  • matching not whitespaces / Matching not whitespaces
  • matching words (\w) / Matching words (\w)
  • matching non-words / Matching non-words
  • word boundaries, matching / Matching word boundaries
  • matching, not as sub-string / Matching not as a sub-string using
  • string, matching as sub-string / Matching a string as sub-string only using
  • non-decimal input, reading / Reading non-decimal input
  • built-in command line debugger / GAWK's built-in command line debugger
  • array sorting / Array sorting
  • two-way inter-process communication / Two-way inter-process communication
  • used, for network programming / Using GAWK for network programming
  • profiling / Profiling
• GAWK, using as debugger
  • about / Using GAWK as a debugger
  • starting step / Starting the debugger
  • set breakpoint / Set breakpoint
  • breakpoint, removing / Removing the breakpoint
  • program, executing / Running the program
  • program, exploring / Looking inside the program
  • variables and data, displaying / Displaying some variables and data
  • unwatch, setting / Setting watch and unwatch
  • watch, setting / Setting watch and unwatch
  • execution, controlling / Controlling the execution
  • environment information, viewing / Viewing environment information
  • commands, saving in file / Saving the commands in file
  • exiting step / Exiting the debugger
• GAWK-only options
  • about / GAWK-only options
  • --dump-variables option (AWK global variables) / The --dump-variables option (AWK global variables)
  • --profile option (profiling) / The --profile option (profiling)
  • --sandbox option / The --sandbox option
  • -i option / The -i option (including other files in your program)
  • @include option / Include other files in the GAWK program (using @include)
  • -V option / The -V option
• getline command
  • simple way / Simple getline
  • fetching, into variable / Getline into a variable
  • used, for reading in file / Getline from a file
  • used, for obtaining variable from file / Using getline to get a variable from a file
  • used, for outputting into pipe / Using getline to output into a pipe
  • used, for changing output into variable from pipe / Using getline to change the output into a variable from a pipe
  • used, for changing output into variable from coprocess / Using getline to change the output into a variable from a coprocess

H

• Hypertext Transfer Protocol (HTTP ) / Reading a web page using HttpService

I

• if statement
  • about / The if statement
  • if / if
  • If...else / If...else
  • if...else...if statement / The if...else...if statement
• increment expression / Increment and decrement expressions
• input/output (I/O) functions
  • close (filename [to/from]) function / The close (filename [to/from]) function
  • fflush ([ filename ]) function / The fflush ([ filename ]) function
  • system (command) function / The system (command) function
  • getline command / The getline command
  • nextfile() function / The nextfile() function

L

• literals / Basic regular expression construct
• logical expressions / Logical or Boolean expressions
• looping statement
  • about / Looping statement
  • while loop / The while loop
  • do...while loop statement / do...while loop statement
  • for loop statement / The for loop statement
  • for each loop statement / For each loop statement
• loops
  • used, for processing arrays / Processing arrays using loops

M

• members
  • referring, to arrays / Referring to members in arrays
• metacharacters, regular expression
  • about / Understanding regular expression metacharacters
  • quoted metacharacter / Quoted metacharacter
  • anchors, using / Anchors
  • dot / Dot
  • bracket expression / Brackets expressions
  • complemented bracket expressions / Complemented bracket expressions
  • alternation operator / Alternation operator
  • unary operator for repetition / Unary operator for repetition
  • repetition ranges, with interval expressions / Repetition ranges with interval expressions
  • grouping, parentheses used / Grouping using parentheses
• multidimensional arrays / Multidimensional arrays

N

• network programming, with GAWK / TCP client and server (/inet/tcp)
  • about / Using GAWK for network programming
  • UDP client and server ( /inet/udp ) / UDP client and server ( /inet/udp )
  • HttpService, used for reading web page / Reading a web page using HttpService
• numbers
  • conversion / Conversion between strings and numbers
• numeric constants / Numeric constants

O

• one-dimensional arrays / One-dimensional arrays
• one-liners
  • used, for text processing and pattern matching / Working with one-liners for text processing and pattern matching with AWK
  • used, for printing of lines / Selective printing of lines with AWK
  • used, for modifying line spacing / Modifying line spacing in a file with AWK
  • used, for performing calculations / Numbering and calculations with AWK
  • used, for numbering of lines / Numbering and calculations with AWK
  • used, for selective deletion of lines / Selective deletion of certain lines in a file with AWK
  • used, for performing string operation on selected line / String operation on selected lines with AWK
  • used, for creating arrays / Array creation with AWK one-liner
  • working, in text conversion and substitution in files / Text conversion and substitution in files with AWK
  • for system administrators / One-liners for system administrators
• operators' precedence / Operators' Precedence
• output
  • redirecting, to file / Redirecting output to file, Redirecting output to a file (>)
  • appending, to file / Appending output to a file (>>)
  • sending, pipe (|) used / Sending output on other commands using pipe (|)
  • redirecting, with special file / Special file for redirecting output (/dev/null, stderr)
  • files, closing / Closing files and pipes
  • pipes, closing / Closing files and pipes
• Output Field Separator (OFS) / The print statement
• Output Record Separator (ORS) / The print statement
• output separator
  • in print statement / Role of output separator in print statement

P

• parentheses
  • used, for grouping / Grouping using parentheses
  • alternation operator, used for concatenation / Concatenation using alternation operator within parentheses
• pattern matching
  • use case examples / Use case examples of pattern matching using AWK
• printf statement
  • used, for printing / Pretty printing with the printf statement
• print statement
  • about / The print statement
  • output separator / Role of output separator in print statement

R

• record separator (RS)
  • about / Record separator
  • outputting / Outputting the record separator
• regular expression operators
  • about / Regular expression operators
  • match operator (~) / Regular expression operators
  • not match operator (!~) / Regular expression operators
• regular expressions
  • about / What is a regular expression?
  • using / Why use regular expressions?
  • using, with AWK / Using regular expressions with AWK
  • using, as string-matching patterns with AWK / Regular expressions as string-matching patterns with AWK
  • basic construct / Basic regular expression construct
  • backreferencing / Backreferencing in regular expressions – sed and grep
  • precedence / Precedence in regular expressions
• relational expression
  • about / Relational expressions
  • equal to (==) / Relational expressions
  • not equal to (!=) / Relational expressions
  • less than (<) / Relational expressions
  • less than or equal to (< =) / Relational expressions
• repetition ranges, with interval expressions
  • single number in bracket / A single number in brackets
  • single number, followed by comma in bracket / A single number followed by a comma in brackets
  • two numbers, in brackets / Two numbers in brackets

S

• special characters
  • construct / Basic regular expression construct
  • printing, with escape sequences / Escape sequences for special character printing
  • horizontal tab / Escape sequences for special character printing
  • new line / Escape sequences for special character printing
  • vertical tab / Escape sequences for special character printing
  • backspace / Escape sequences for special character printing
  • carriage return / Escape sequences for special character printing
• split() function
  • used, for creating arrays / Using the split() function to create arrays
• stack frame / Debugger concepts
• standard command-line options
  • -F option / The -F option – field separator
  • -f option (read source file) / The -f option (read source file)
  • -v option (assigning variables) / The -v option (assigning variables)
• statements, used for altering flow control
  • break statement, using / Break usage
  • continue statement, using / Usage of continue
  • exit statement, using / Exit usage
  • next command, using / Next usage
• string concatenation / String concatenation
• string constants / String constants
• string functions
  • about / String functions, Summary table of built-in string functions
  • index (str, sub) function / The index (str, sub) function
  • length ( string ) function / The length ( string ) function
  • ssplit (str, arr, regex) function / The split (str, arr, regex) function
  • substr (str, start, [ length ]) function / The substr (str, start, [ length ]) function
  • sub (regex, replacement, string) function / The sub (regex, replacement, string) function
  • gsub (regex, replacement, string) function / The gsub (regex, replacement, string) function
  • gensub (regex, replacement, occurrence, [ string ]) function / The gensub (regex, replacement, occurrence, [ string ]) function
  • match (string, regex) function / The match (string, regex) function
  • tolower (string) function / The tolower (string) function
  • toupper (string) function / The toupper (string) function
  • sprintf (format, expression) function / The sprintf (format, expression) function
  • strtonum (string) function / The strtonum (string) function
• strings
  • conversion / Conversion between strings and numbers
• sub-expression / Backreferencing in regular expressions – sed and grep
• switch statement / The switch statement (a GAWK-specific feature)

T

• ternary expressions / Ternary expressions
• text processing and pattern matching
  • one-liners, working with / Working with one-liners for text processing and pattern matching with AWK
• time function
  • about / The time function
  • systime() function / The systime() function
  • mktime (datespec) function / The mktime (datespec) function
  • strftime (format, timestamp) function / The strftime (format, timestamp) function
• types, AWK
  • about / Types of AWK
  • NAWK / Types of AWK
  • GAWK / Types of AWK

U

• unary expressions
  • about / Unary expressions
  • unary plus / Unary expressions
  • unary minus / Unary expressions
• unary operators
  • for repetition / Unary operator for repetition
  • closures / Closure
  • positive closure / Positive closure
  • zero or one operator (?) / Zero or one
• use case examples, for pattern matching
  • web server (Apache/Nginx) log files, parsing / Parsing web server (Apache/Nginx) log files
  • top 10 request IP addresses, printing with GeoIP information / Printing the top 10 request IP addresses with their GeoIP information
  • website unique visits, printing / Counting and printing unique visits to a website
  • website unique visits, counting / Counting and printing unique visits to a website
  • Real-Time IP address lookup, for requests / Real-time IP address lookup for requests
  • text, converting to HTML table / Converting text to HTML table
  • decimal, converting to binary / Converting decimal to binary
  • files, renaming in directory / Renaming files in a directory with AWK
  • generated sequence of numbers, printing / Printing a generated sequence of numbers in a specified columnate format
  • matrix, transposing / Transposing a matrix
  • multiple files, processing / Processing multiple files using AWK
• user-defined functions
  • about / User-defined functions
  • function definition and syntax / Function definition and syntax
  • user-defined functions, calling / Calling user-defined functions
  • variable scope, controlling / Controlling variable scope
  • return statement / Return statement
  • indirect function calls, making / Making indirect function calls

V

• variables / AWK variables and constants

W

• web server (Apache/Nginx) log files
  • Apache combined log format / Understanding the Apache combined log format
  • different log fields , processing with AWK / Using AWK for processing different log fields 
  • running website issues, identifying / Identifying problems with the running website
• while loop / The while loop
• word boundaries
  • matching, at beginning / Matching at the beginning of a word 
  • matching, at end of word / Matching at the end of a word 

Y

• Yellowdog Updator Modified (YUM) / Using the package manager