Preface
Shell scripts are an essential part of any modern operating system, such as Unix, Linux, or Windows. The scripting language and its syntax may vary from OS to OS, but the fundamental principles remain the same. I first encountered Linux shell scripts during the development of embedded Linux product development. Shell scripts initialized the complete product, from basic booting procedure to the user logging, to the complete operating system being initialized. Another situation was automation of regular activities such as build and release management of the source codes of very complex products, where more than 10,000 files were part of the single project. Similarly, another very common requirement is, automatic routine administration activities.
Initially, I learned scripts to solve practical problems and customize pre-existing products. This book is a summary of what I have learned over the years about Linux shell scripting through project development work, consultancy, and corporate training and Q&A sessions.
In this book, we will learn the very basics of shell scripting to real-world complex, customized automation. By the end of the book, the reader will be able to confidently use their own shell scripts for the real-world problems out there. The idea is to be as practical as possible and give the reader the look and feel of what real-world scripting looks like.
This book covers the GNU Bourne Again Shell (BASH) scripting. You can use the knowledge gained by reading this book for any shell of any of the UNIX flavors or Linux distributions. You may need to take care of few syntax changes if you are working in other shells, such as Korn or similar. You should be able to read this book cover to cover, or just pick it up and read anything you find interesting. But, perhaps most importantly, if you have a question about how to solve a particular problem or you need a hint, you will find it easy to find the right solution—or something close enough—to save your time and energy.
Who this book is for
This book is for readers who are proficient at working with Linux and who want to learn about shell scripting to improve their efficiency and practical skills. The following are few examples where we can use the skills we learned in this book:
- Shell scripting for automating tasks such as taking periodic backup
- System administration
- Database maintenance and backup
- Test processing and report generation
- Customization of system initialization
- Embedded Linux product developments
What this book covers
Chapter 1, Getting Started and Working with Shell Scripting, introduces different ways to write and run shell scripts. We'll also learn ways to handle files and directories, as well as working with permissions.
Chapter 2, Drilling Deep into Process Management, Job Control, and Automation, speaks about basic process management. We will learn about command ps and also about job management using commands such as jobs, fg, bg, kill, and pkill. Later on, we will learn about process monitoring tools top, iostat, vmstat, and sar.
Chapter 3, Using Text Processing and Filters in Your Scripts, speaks about using more, less, head, and tail commands. We will also learn about text processing tools such as cut, paste, comm, and uniq. We will learn what is a standard input, output, and standard error. Later on, we will learn about meta-characters and pattern matching using VI and grep.
Chapter 4, Working with Commands, explains how shell interprets any command entered on the command line. We will also learn about command substitution, separators, and pipes in detail.
Chapter 5, Exploring Expressions and Variables, speaks about variables in general and environment variables in particular. This includes how to export environment variables, set, shift, read-only variables, command-line arguments, and create and handle arrays.
Chapter 6, Neat Tricks with Shell Scripting, talks about debugging, here operator, and interactive shell scripts for taking input from keyboard and file handling.
Chapter 7, Performing Arithmetic Operations in Shell Scripts, covers performing arithmetic operations in various ways such as using declare, let, expr, and arithmetic expressions. We will also learn about representing numbers in different bases such as hex, octal, and binary. The chapter also covers using bc utility for performing floating point or fractional arithmetic.
Chapter 8, Automating Decision Making in Scripts, talks about using decision making in scripts by working with Test, if-else, and switching case. We will also learn about how to use select with for loop along with the menu.
Chapter 9, Automating Repetitive Tasks, speaks about repeating tasks such as doing routine administration activities using the for loop, while loop, and do while loop. We will also learn how control loops using break statement and continue statement.
Chapter 10, Working with Functions, speaks about functions in shell scripts. We will learn how to define and display functions, and further how to remove the function from the shell. We will also learn about passing arguments to functions, sharing data between functions, declaring local variables in a function, returning result from a function, and running functions in the background. We will finally learn about using source and .(dot) commands. We will use these commands for using the library of functions.
Chapter 11, Using Advanced Functionality in Scripts, covers using traps and signals. We will also learn about creating menus with the help of dialog utility.
Chapter 12, System Start-up and Customizing a Linux System, speaks about the Linux system start-up, from power on until user login and how to customize a Linux system environment.
Chapter 13, Pattern Matching and Regular Expressions with sed and awk, talks about regular expressions and using sed (stream editor) and awk for text processing. We will learn how to use various commands and options along with a lot of examples for using sed and awk.
Chapter 14, Taking Backup and Embedding Other Languages in Shell Scripts, speaks about taking backup locally as well as across the network. We will also learn about automating it using crontab. We will learn about embedding other languages in bash scripts such as Python, Ruby, and Pearl.
Chapter 15, Database Administration Using Shell Scripts, talks about how to write and execute MySQL commands in a shell script as well as how to write and execute Oracle commands in a shell script. By using learnings from this chapter, we will be able to automate frequently required database administration tasks.
To get the most out of this book
Any computer that has Linux OS installed in it will be sufficient for learning all the topics discussed in this book. For the first edition, we used Ubuntu Linux distribution. For this second edition, we have used CentOS Linux distribution. I have personally tested all the commands and scripts in Ubuntu 16.04, as well as in the CentOS 7.0 distribution.
During the course, if you find that any particular utility is not installed in Ubuntu or any Debian-based distribution, then enter the following command to install that utility:
$ sudo apt-get update $ sudo apt-get install package-name
A good internet connection should be available for the preceding commands to run.
In CentOS or any other rpm-based distribution, enter the following commands:
$ sudo yum update $ sudo yum install package-name
If the internet is connected, then using these commands you can install any command or utility that is not already installed.
Download the example code files
You can download the example code files for this book from your account at www.packtpub.com. If you purchased this book elsewhere, you can visit www.packtpub.com/support and register to have the files emailed directly to you.
You can download the code files by following these steps:
- Log in or register at www.packtpub.com.
- Select the
SUPPORTtab. - Click on
Code Downloads & Errata. - Enter the name of the book in the
Searchbox and follow the onscreen instructions.
Once the file is downloaded, please make sure that you unzip or extract the folder using the latest version of:
- WinRAR/7-Zip for Windows
- Zipeg/iZip/UnRarX for Mac
- 7-Zip/PeaZip for Linux
The code bundle for the book is also hosted on GitHub at https://github.com/PacktPublishing/Learning-Linux-Shell-Scripting-Second-Edition. In case there's an update to the code, it will be updated on the existing GitHub repository.
We also have other code bundles from our rich catalogue of books and videos available at https://github.com/PacktPublishing/. Check them out!
Conventions used
There are a number of text conventions used throughout this book.
CodeInText: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: "Mount the downloaded WebStorm-10*.dmg disk image file as another disk in your system."
A block of code is set as follows:
#!/bin/bash # This is comment line echo "Hello World" ls date
When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold:
#!/bin/bash
# This is comment line
echo "Hello World"
ls
date Any command-line input or output is written as follows:
$ bash hello.shBold: Indicates a new term, an important word, or words that you see onscreen. For example, words in menus or dialog boxes appear in the text like this. Here is an example: "Select System info from the Administration panel."
Note
Warnings or important notes appear like this.
Note
Tips and tricks appear like this.
Get in touch
Feedback from our readers is always welcome.
General feedback: Email [email protected] and mention the book title in the subject of your message. If you have questions about any aspect of this book, please email us at [email protected].
Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packtpub.com/submit-errata, selecting your book, clicking on the Errata Submission Form link, and entering the details.
Piracy: If you come across any illegal copies of our works in any form on the Internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [email protected] with a link to the material.
If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com.
Reviews
Please leave a review. Once you have read and used this book, why not leave a review on the site that you purchased it from? Potential readers can then see and use your unbiased opinion to make purchase decisions, we at Packt can understand what you think about our products, and our authors can see your feedback on their book. Thank you!
For more information about Packt, please visit packtpub.com.
