by Ed Hartnett
Prompt History Lesson
Back in the '70s, everyone was delighted with the invention of the first glass teletypes. They replaced the paper teletype—an IBM Selectric typewriter, modified to print out the computer's response underneath whatever you typed on the typewriter. The paper teletype used boxes and boxes of computer-fold paper, and sounded like a machine gun firing. (It sounds primitive now, but compared to punch cards, it was very high-tech.)
The glass teletype, as it was called, was the first cathode-ray tube monitor. The CRT monitor was a technology that had a long run, but is now almost completely obsolete, due to the LCD screen. At the height of its development (i.e., just as it's going obsolete), the CRT delivers super-sharp pictures in amazing color.
The first glass teletypes also had color: green. They displayed everything in the same ugly green text on the screen. But it wasn't long before the so-called "smart" terminal was developed, one that could let the user experience things like colored text, blinking text, etc. This was another technology that was approaching perfection and obsolescence, as the graphical user interface was introduced and the command-line interface became something that appears in just one window on a screen full of fancy gadgets and applications.
These days, no one makes a "smart" text terminal; they just emulate them on your general-purpose computer. Similarly, few rely fully on the command-line interface, but that doesn't mean you can't take advantage of the perfection of this technology to get your screen to look more fun.
Combining UNIX Commands
"Do one thing, and do it well." This is a statement that inspired early UNIX programmers. UNIX consists of a wide variety of tools, each extremely focused. But when you combine them, you can produce sophisticated results. The fundamental difference between Windows and UNIX is that Window is designed to be easy to use, while UNIX is designed for pure power.
But these powerful results wouldn't be possible without some way for these separate programs to share and communicate data.
Three Streams of Communication
Just about every UNIX tool can handle three streams of communication. One, stdin, is a way for you to provide information to the program. Another, stdout, is where the program sends its normal output. Finally, stderr is a way for the tool to send error information.
Redirection
Suppose you want to save the output of a command to a file. This is useful if you want to send a list of files through email. The greater-than sign (>) handles this in UNIX. Take a look at the following command:
ls -l > listing.txt
This will create a file called "listing.txt." The file will contain everything you would see on screen if you typed "ls -l". The greater-than sign (>) is called the redirection operator in geek-speak. It tells UNIX to send the output of a command to a file. In other words, whatever the ls program sends to stdout will be written into a file. If the file already exists, it will be overwritten. That is, the original contents will be destroyed. The prevent this, use two greater-than signs, which will append output to the file:
lp -l >> listing.txt
The redirection operator works in both directions:
lpr < listing.txt
This tells the computer to send listing.txt into stdin of the lpr program. It's not used as often, because programs that take input (like lpr) usually let you specify the filename on the command line. So the file can also be printed like this:
lpr listing.txt.
Piping
Sometimes you need to send the output of one command into another. You could redirect the output to a file, and then use that file as the input for the next command.
For examle, to get a sorted list of files, try:
ls -1 > listing.txt
sort < listing.txt
There is a way to do this in one step: the pipe operator. It does for data what a pipe does for water. It's the vertical line (sometimes found with a break in the middle) found above the "" key on most keyboards.
For your sorted list of files, try:
ls | sort
This will give you a sorted list of files in the current directory. The pipe takes the stdout from one
command and sends it to the stdin of another.
Piping with Grep
The grep command searches through one or more files for a text regular expression. For example:
grep "howdy" *.txt
This command will search through all .txt files in the current directory for the string "howdy."
Often, I use two grep commands connected with a pipe. This allows me to refine my search. If the grep command above returned hundreds of hits, I might refine the search:
grep "howdy" *.txt | grep -v "partner"
The first grep will generate a list of lines containing the word "howdy." The second grep will act on that list and find all the lines that contain "partner."
The grep command can be told to show lines that don't match the criteria. To show all the lines of a file that don't contain "howdy," try:
grep -v "howdy" *.txt
To find the lines that contain "howdy" but not "partner":
grep "howdy" *.txt | grep -v "partner"
The Find Command
The find command goes beyond merely finding files by name. It can also find files larger (or smaller) than a certain size.
The real power of the find command is to provide input to the grep command (using xargs command along the way). This is regularly done by programmers. Take a look at:
find /usr/home/ed -name "*.cpp"|xargs grep "variable"
This command will find all C++ source files under my home directory, and all subdirectories, and search them all for the string "variable."
More or Less
In the grand old days before the GUI, there were no slide bars on the sides of windows. Indeed, there weren't even windows! How then did early computer users examine long documents? They used something called "more." The more command will display any document one page at a time. For example:
more my_long_file.txt
As the years went by, the more command was improved in various ways, and released as a new tool called less. The less command works just the same as the more command:
less my_long_file.txt
More or less are used with commands that have a long output. A directory might have hundreds of files, in which case the ls -1 command will scroll right by, without giving you a chance to read it. Try using a pipe to more:
ls -1 | more
Now the output will be presented one page at a time.
More Text Manipulation
UNIX has a tremendous selection of tools to work with text- far too many to mention here. Any modern Web site developer or programmer can benefit from these tools, and the combinations that are possible with pipes. Text tools commonly used with pipes include the sort command; the uniq command, which can sort and eliminate duplicates in a list; and cat, which spits out the contents of a text file.
The tr command allows lots of fun manipulation of text, including changing case from upper to lower, change one character for another, or removing every instance of a character in the output.
Some Combinations
Once you get the hand of using pipes, it's amazing how much you can do with them. Three or more pipes can be used in a command. One common application is to search the output of grep with another grep. For example:
find .-name ".cpp|xargs grep "somestring"|grep "otherstring"
The above command will search all C++ files in and under the current directory for the string "somestring." It will then search the output of that for the string "otherstring."
To find the ten most popular words in a document, try:
cat file.txt|tr -c 'a-zA-Z' 'n'
filter: grayscale(100%); " class="sharethis-inline-share-buttons">share