Difference between revision 5 and current revision
Summary: Leading and trailing whitespace
No diff available.The special variable FS is a field separator that is used to determine how awk will split each record into fields.
The default value for the field separator is a single space character. In data files, fields are often separated by multiple whitespace characters, rather than a single space. The awk interpreter allows a single space separator as matching any stretch of whitespace. This prevents two consectutive spaces from being interpreted as separators delimiting an empty field. Note that only a single space is treated as a special case in this manner. Any other single characters would be treated as delimiters for an empty field:
#!/bin/sh # This gives only three fields, because double spaces do not delimit an empty field echo "Rhubarb Custard Bananas" | awk 'BEGIN { FS = " " } { for (l=1;l<=NF;l++) print l ":" $l }' # This gives five fields, because groups of other delimiters are treated as separating empty fields echo "Rhubarb##Custard##Bananas" | awk 'BEGIN { FS = "#" } { for (l=1;l<=NF;l++) print l ":" $l }'
If you really want to use a single space delimiter, allowing groups of spaces to be treated as delimiting an empty field, then it is possible to use a regular expression containing a space in a box enclosures to represent a single space:
# This gives five fields echo "Rhubarb Custard Bananas" | awk 'BEGIN { FS = "[ ]" } { for (l=1;l<=NF;l++) print l ":" $l }'
When the default single space field separator is being used, the awk interpreter strips leading and and trailing whitespace is stripped from the record, before it is split into fields. However, with other field separators the whitespace is not stripped and is treated as part of the record:
#!/bin/sh echo ' Rhubarb Custard Bananas '| awk 'BEGIN { FS = " " } { print $2 }' # This gives Custard echo ' Rhubarb Custard Bananas '| awk 'BEGIN { FS = "[ ]" } { print $2 }' # This gives Rhubarb
Note that the second awk command gives a different result, because the leading whitespace is not stripped and acts as a delimiter from the previous empty record.
The field separator can be changed by assignment like any other variable. Note that the field separator can be a single character or a regular expression:
# Change the field separator to a colon FS = ":"
In the following example, we use a regular expression '''l.[^l]''', which represents a double letter l not followed by a third l:
#!/bin/sh echo Llanfairpwllgwyngyllgogerychwyrndrobwllllantysiliogogogoch| awk 'BEGIN { FS = "l.[^l]" } { for (l=1;l<=NF;l++) print l ":" $l }'
The -F command line switch enables the default field separator to be changed. This variable can be set on the command line.
The /etc/passwd has its data fields separated by colons. Here we change the field separator to a colon, in order to extract field information from the /etc/passwd file:
>awk -F: '{print $1,$3 }' /etc/passwd
The behaviour of the awk interpreter when the field separator is set to an [[empty_string?]] is undefined. Some implementation of awk will treat the record as having only one field, others will put each character of the record in a separate field see AwkFeatureComparison
Which awk has this beahviour? all the awk I have access to seem to treat t as just a normal letter
If the -F command line switch is given a single letter t as a parameter, the awk interpreter assumes that the fields should be separated by tab characters, rather than by a letter t. This behaviour is implemented to avoid complications with shell quoting where backslash escape characters can become lost. If the letter t should really be used as a field separator, then it is necessary to use a regular expression containing the letter t in a box enclosures to represent the letter t:
>awk -F\[t\]