Unix / Linux - Regular Expressions with SED

Hello there, future command-line wizards! Today, we're diving into the magical world of regular expressions (regex) and the powerful SED command in Unix/Linux. Buckle up, because we're about to embark on a thrilling adventure that will transform the way you manipulate text!

Invoking sed

Let's start with the basics. SED, which stands for "Stream Editor," is a powerful text processing tool. To use it, we simply type "sed" followed by our commands. It's like summoning a text-manipulating genie!

sed 'command' filename

For example:

sed 's/hello/bonjour/' greetings.txt

This command replaces "hello" with "bonjour" in the file greetings.txt. Simple, right?

The sed General Syntax

The general syntax for sed is like a magic spell:

sed OPTIONS... [SCRIPT] [INPUTFILE...]

Don't worry if this looks intimidating. We'll break it down piece by piece, like solving a fun puzzle!

Deleting All Lines with sed

Want to make all the text in a file disappear? Sed can do that! Here's how:

sed 'd' filename

This deletes all lines in the file. It's like using an eraser on your digital notebook!

The sed Addresses

Addresses in sed are like GPS coordinates for your text. They tell sed where to perform its magic. Here are some examples:

sed '2d' file.txt  # Deletes the 2nd line
sed '/pattern/d' file.txt  # Deletes lines containing 'pattern'

The sed Address Ranges

Sometimes, we want to cast our spell on a range of lines. Here's how:

sed '2,5d' file.txt  # Deletes lines 2 through 5
sed '2,$d' file.txt  # Deletes from line 2 to the end of the file

The Substitution Command

The substitution command is sed's bread and butter. It's like "find and replace" on steroids!

sed 's/old/new/' file.txt

This replaces the first occurrence of "old" with "new" on each line.

Substitution Flags

Flags are like power-ups for your substitution commands. Here's a table of common flags:

Flag	Description
g	Replace all occurrences, not just the first
i	Ignore case
p	Print the changed line
w	Write the result to a file

Example:

sed 's/cat/dog/g' pets.txt

This replaces all occurrences of "cat" with "dog".

Using an Alternative String Separator

Sometimes, your text contains a lot of slashes. No worries! We can use different separators:

sed 's#/usr/local/bin#/common/bin#' paths.txt

Here, we're using '#' as our separator instead of '/'.

Replacing with Empty Space

Want to make text disappear without a trace? Here's how:

sed 's/unwanted//g' file.txt

This removes all occurrences of "unwanted" from the file.

Address Substitution

We can combine addresses with substitution for precision text surgery:

sed '3,6s/foo/bar/g' file.txt

This replaces all "foo" with "bar", but only on lines 3 through 6.

The Matching Command

The matching command is like a spotlight, illuminating the lines we're interested in:

sed -n '/pattern/p' file.txt

This prints only the lines containing "pattern".

Using Regular Expression

Now we're getting to the real magic! Regular expressions are like wildcards on steroids. Here's a simple example:

sed -n '/^The/p' story.txt

This prints all lines starting with "The".

Matching Characters

Regular expressions have special characters that act like shapeshifters:

Character	Meaning
.	Any single character
*	Zero or more of the previous character
^	Start of line
$	End of line

Example:

sed -n '/c.t/p' animals.txt

This matches "cat", "cot", "cut", etc.

Character Class Keywords

Character classes are like teams of characters. Here are some MVP players:

Class	Matches
[:alpha:]	Alphabetic characters
[:digit:]	Numeric characters
[:alnum:]	Alphanumeric characters

Example:

sed -n '/[[:digit:]]/p' data.txt

This prints lines containing any digit.

Ampersand Referencing

The ampersand (&) is like a magic mirror, reflecting what was matched:

sed 's/[0-9]/(&)/' numbers.txt

This puts parentheses around each number.

Using Multiple sed Commands

Want to cast multiple spells at once? Use the -e option:

sed -e 's/foo/bar/g' -e 's/baz/qux/g' file.txt

This performs two substitutions in one go!

Back References

Back references are like time machines, letting us reuse parts of the match:

sed 's/\(.*\):\(.*\)/\2:\1/' names.txt

This swaps the text before and after a colon.

And there you have it, my dear students! We've journeyed through the land of sed and regular expressions. Remember, practice makes perfect. So go forth and manipulate text like the command-line sorcerers you are becoming! Happy coding!

Credits: Image by storyset