Advanced Bash-Scripting Guide

1 Advanced Bash-Scripting GuideAn in-depth exploration of the art of shell scriptingMendel Mar 2014 Revision HistoryRevision Apr 2012 Revised by: mc'TUNGSTENBERRY' releaseRevision Nov 2012 Revised by: mc'YTTERBIUMBERRY' releaseRevision 1010 Mar 2014 Revised by: mc'PUBLICDOMAIN' releaseThis tutorial assumes no previous knowledge of scripting or programming , yet progresses rapidly toward anintermediate/ Advanced level of instruction .. all the while sneaking in little nuggets of unix wisdom andlore. It serves as a textbook, a manual for self-study, and as a reference and source of knowledge on shellscripting techniques. The exercises and heavily-commented examples invite active reader participation, underthe premise that the only way to really learn scripting is to write book is suitable for classroom use as a general introduction to programming document is herewith granted to the Public Domain. No copyright!

2 DedicationFor Anita, the source of all the magicTable of ContentsChapter 1. Shell programming !..1 Chapter 2. Starting Off With a Invoking the Preliminary 2. 3. Special 4. Introduction to Variables and Variable Variable Bash Variables Are Special Variable 5. Quoting 6. Exit and Exit 7. Test File test Other Comparison Nestedif/then Condition Testing Your Knowledge of 8. Operations and Related Numerical The Double-Parentheses Operator 3. Beyond the 9. Another Look at Internal Typing variables: declare or Another use for $RANDOM: generate random 10. Manipulating Manipulating Manipulating strings using Further Parameter Bash-Scripting GuideiTable of ContentsChapter 11. Loops and Nested Loop Testing and 12. Command 13. Arithmetic 14. Recess 4. 15. Internal Commands and Job Control 16. External Filters, Programs and Basic Complex Time / Date Text Processing File and Archiving Communications Terminal Control Math Miscellaneous 17.

3 System and Administrative Analyzing a System 5. Advanced 18. Regular A Brief Introduction to Regular 19. Here Here 20. I/O Using Redirecting Code 21. Bash-Scripting GuideiiTable of ContentsChapter 22. Restricted 23. Process 24. Complex Functions and Function Local Local variables and Recursion Without Local 25. 26. List 27. 28. Indirect 30. Network 31. Of Zeros and 32. 33. 34. 35. Scripting With Unofficial Shell Scripting 36. Interactive and non-interactive shells and Shell Tests and Comparisons: Recursion: a script calling "Colorizing" Assorted Ideas for more powerful Security Infected Shell Hiding Shell Script Bash-Scripting GuideiiiTable of ContentsChapter 36. Writing Secure Shell Portability A Test Shell Scripting Under 37. Bash, versions 2, 3, and Bash, version Bash, version Bash, version Bash, version Bash, version Bash, version Bash, version 38.

4 Author's About the Where to Go For Tools Used to Produce This Software and A. Contributed B. Reference C. A Sed and Awk D. Parsing and Managing E. Exit Codes With Special F. A Detailed Introduction to I/O and I/O G. Command-Line Standard Command-Line Bash Command-Line H. Important Bash-Scripting GuideivTable of ContentsAppendix I. Important System J. An Introduction to Programmable K. L. History M. N. Converting DOS Batch Files to Shell O. Analyzing Writing P. Revision Q. Download and Mirror R. To Do S. T. ASCII Bash-Scripting GuidevChapter 1. Shell programming !No programming language is perfect. There isnot even a single best language; there are onlylanguages well suited or perhaps poorly suitedfor particular MayerA working knowledge of shell scripting is essential to anyone wishing to become reasonably proficient atsystem administration, even if they do not anticipate ever having to actually write a script.

5 Consider that as aLinux machine boots up, it executes the shell scripts in / to restore the system configuration andset up services. A detailed understanding of these startup scripts is important for analyzing the behavior of asystem, and possibly modifying craft of scripting is not hard to master, since scripts can be built in bite-sized sections and there is only afairly small set of shell-specific operators and options [1] to learn. The syntax is simple -- even austere --similar to that of invoking and chaining together utilities at the command line, and there are only a few "rules"governing their use. Most short scripts work right the first time, and debugging even the longer ones isstraightforward. In the early days of personal computing, the BASIC language enabled anyone reasonably computer proficient to write programs on an early generation of microcomputers. Decades later, the Bash scripting language enables anyone with a rudimentary knowledge of Linux or unix to do the same on modern machines.

6 We now have miniaturized single-board computers with amazing capabilities, such as the Raspberry Pi. Bash scripting provides a way to explore the capabilities of these fascinating shell script is a quick-and-dirty method of prototyping a complex application. Getting even a limited subsetof the functionality to work in a script is often a useful first stage in project development. In this way, thestructure of the application can be tested and tinkered with, and the major pitfalls found before proceeding tothe final coding in C, C++, Java, Perl, or scripting hearkens back to the classic unix philosophy of breaking complex projects into simplersubtasks, of chaining together components and utilities. Many consider this a better, or at least moreesthetically pleasing approach to problem solving than using one of the new generation of high-poweredall-in-one languages, such as Perl, which attempt to be all things to all people, but at the cost of forcing you toalter your thinking processes to fit the to Herbert Mayer, "a useful language needs arrays, pointers, and a generic mechanism for buildingdata structures.

7 " By these criteria, shell scripting falls somewhat short of being "useful." Or, perhaps not..When not to use shell scriptsChapter 1. Shell programming !1 Resource-intensive tasks, especially where speed is a factor (sorting, hashing, recursion [2] ..) Procedures involving heavy-duty math operations, especially floating point arithmetic, arbitraryprecision calculations, or complex numbers (use C++ or FORTRAN instead) Cross-platform portability required (use C or Java instead) Complex applications, where structured programming is a necessity (type-checking of variables,function prototypes, etc.) Mission-critical applications upon which you are betting the future of the company Situations where security is important, where you need to guarantee the integrity of your system andprotect against intrusion, cracking, and vandalism Project consists of subcomponents with interlocking dependencies Extensive file operations required (Bash is limited to serial file access, and that only in aparticularly clumsy and inefficient line-by-line fashion.)

8 Need native support for multi-dimensional arrays Need data structures, such as linked lists or trees Need to generate / manipulate graphics or GUIs Need direct access to system hardware or external peripherals Need port or socket I/O Need to use libraries or interface with legacy code Proprietary, closed-source applications (Shell scripts put the source code right out in the open for allthe world to see.) If any of the above applies, consider a more powerful scripting language -- perhaps Perl, Tcl, Python, Ruby-- or possibly a compiled language such as C, C++, or Java. Even then, prototyping the application as ashell script might still be a useful development will be using Bash, an acronym [3] for "Bourne-Again shell" and a pun on Stephen Bourne's now classicBourne shell. Bash has become a de facto standard for shell scripting on most flavors of unix . Most of theprinciples this book covers apply equally well to scripting with other shells, such as the Korn Shell, fromwhich Bash derives some of its features, [4] and the C Shell and its variants.

9 (Note that C Shell programmingis not recommended due to certain inherent problems, as pointed out in an October, 1993 Usenet post by TomChristiansen.)What follows is a tutorial on shell scripting. It relies heavily on examples to illustrate various features of theshell. The example scripts work -- they've been tested, insofar as possible -- and some of them are even usefulin real life. The reader can play with the actual working code of the examples in the source archive( or ), [5] give them execute permission (chmod u+rxscriptname), then run them to see what happens. Should the source archive not be available, thencut-and-paste from the HTML or pdf rendered versions. Be aware that some of the scripts presented hereintroduce features before they are explained, and this may require the reader to temporarily skip ahead otherwise noted, the author of this book wrote the example scripts that countenance was bold and bashed SpenserAdvanced Bash-Scripting GuideChapter 1.

10 Shell programming !2 Chapter 2. Starting Off With a Sha-BangShell programming is a 1950s juke box ..--Larry WallIn the simplest case, a script is nothing more than a list of system commands stored in a file. At the very least,this saves the effort of retyping that particular sequence of commands each time it is 2-1. cleanup: A script to clean up log files in /var/log# Cleanup# Run as root, of /var/logcat /dev/null > messagescat /dev/null > wtmpecho "Log files cleaned up."There is nothing unusual here, only a set of commands that could just as easily have been invoked one by onefrom the command-line on the console or in a terminal window. The advantages of placing the commands in ascript go far beyond not having to retype them time and again. The script becomes a program -- a tool -- and itcan easily be modified or customized for a particular 2-2. cleanup: An improved clean-up script#!/bin/bash# Proper header for a Bash script.