Regular expressions: grep -E and sed

Why regex

In the lesson on pipes, we filtered logs through grep by a plain string. But you often need to find not an exact word but a pattern: all IP addresses, all lines with a 5xx response code, all emails. That's what regular expressions (regex) are for. A regular expression describes a text pattern. The syntax is almost identical in grep, sed, awk, and most programming languages, with differences in the details.

The pattern in the commands below is always in single quotes. That way the shell doesn't touch $, \, and * inside it, and passes the pattern to the command as-is.

Anchors and character classes

The basic building blocks:

• ^ start of line, $ end of line.
• . any single character.
• [abc] one of the characters in brackets, [a-z] a range, [^abc] anything except the ones listed.
• [0-9] a digit. The same thing is written as [[:digit:]], which works in any grep. There's also \w (a letter, digit, or underscore) and \s (a whitespace character); both are GNU extensions.

grep -E '^ERROR' app.log        # lines that start with ERROR
grep -E '[0-9]{3}$' app.log     # lines that end with three digits

The short notation \d for a digit doesn't exist in grep -E, it's PCRE syntax (Perl, Python, JavaScript). There won't be an error though. GNU grep 3.11 silently reads \d as the letter d:

printf 'code 503\nid 7d\nabc\n' | grep -E '\d'

code 503
id 7d

Lines with the letter d were found, not with a digit. PCRE in GNU grep is turned on with the -P flag, but macOS doesn't have it. It's more reliable to write [0-9].

ERE and BRE: why the -E flag

grep and sed have two dialects. Without the flag, the basic one (BRE) works; with -E, the extended one (ERE). In ERE, the characters +, ?, |, (), and {} work as special characters. In BRE, the same characters without \ mean themselves:

echo aaa | grep 'a+'        # looks for "a" followed by the character "+": won't match
echo aaa | grep 'a\+'       # GNU BRE: one or more "a", matches
echo aaa | grep -E 'a+'     # ERE: the same thing without backslashes

That's why grep -E and sed -E are used by default: the pattern is shorter and reads the same way in both commands.

Quantifiers

Quantifiers say how many times the preceding element repeats:

• * zero or more, + one or more, ? zero or one.
• {n} exactly n times, {n,m} from n to m.

grep -E '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' access.log

This is a rough IP address pattern: four groups of 1-3 digits separated by dots. The dot is escaped as \., because without escaping, . means "any character". Without \, the pattern would also match the string 10x0y0z1.

The pattern really is rough. It will let 999.999.999.999 through and will cut 1.2.3.4 out of the version number 1.2.3.4.5:

echo '10.0.0.1 999.999.999.999 1.2.3.4.5' | grep -oE '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}'

10.0.0.1
999.999.999.999
1.2.3.4

The -o flag prints not the whole line but just the matched piece, one per output line. For logs, this level of precision is usually enough. Checking that an address is actually valid is better done with something other than a regex.

Groups and alternation

() groups, | means "or":

grep -E '(GET|POST) /api' access.log    # lines with GET or POST to /api

The group (GET|POST) matches either option. Without the parentheses, GET|POST /api reads as "GET or POST /api" and would match any GET, even to /home. Groups are also used in replacement, to refer back to what was matched.

A couple of grep flags that often go along with a pattern: -i case-insensitive, -v lines that don't match the pattern, -c the count of matching lines.

Replacing with sed

grep finds, sed replaces by pattern. The syntax is s/pattern/replacement/:

echo 'user=alice' | sed -E 's/user=(.+)/login: \1/'

login: alice

The group (.+) captures the match, and \1 substitutes it into the replacement. There can be several groups; they're numbered left to right by their opening parenthesis:

echo '2026-10-09' | sed -E 's/([0-9]{4})-([0-9]{2})-([0-9]{2})/\3.\2.\1/'

09.10.2026

Without -E, parentheses in sed mean themselves, and \1 has nothing to refer to. GNU sed will say exactly that: invalid reference \1 on 's' command's RHS.

The g flag at the end (s/.../.../g) replaces all occurrences in the line, not just the first one: echo a-b-c | sed 's/-/+/' gives a+b-c, with g you get a+b+c. sed -i can edit a file in place; it was covered in the lesson on sed.

Greediness

* and + grab as much text as they can. If the line has two quoted values, "(.*)" will capture everything from the first quote to the last:

echo 'key="a" other="b"' | sed -E 's/"(.*)"/<\1>/'

key=<a" other="b>

Instead of "any character" you need "any character except a quote":

echo 'key="a" other="b"' | sed -E 's/"([^"]*)"/<\1>/g'

key=<a> other=<b>

Common mistakes

• \d instead of [0-9]. In grep -E and sed -E this is the letter d. There's no error, just the wrong matches.
• An unescaped dot. 10.0.0.1 as a pattern will also match 10x0y0z1. A literal dot character is written \..
• A pattern wider than it needs to be. grep 500 on a web server log will match the response code, the size 15003, and the path /orders/500. Anchor the pattern to its surroundings: spaces, quotes, ^, $.
• Alternation without parentheses. GET|POST /api is not the same as (GET|POST) /api.
• A greedy .* eats more than you expect. A class like [^"]* or [^ ]* helps.
• Forgot -E. In grep, a pattern with + or | silently matches nothing; in sed, the \1 reference will error out.
• The pattern in double quotes. The shell will expand $var and eat some of the \. Keep the regex in single quotes.
• macOS. There, grep and sed are the BSD versions: -E and \1 work, but grep -P doesn't, and sed -i has different syntax. In this lesson's tasks everything runs in a container with the GNU versions.

What you'll need for the tasks

• Write patterns with anchors, classes [0-9] and [^ ], quantifiers {n} and +, groups (a|b); escape the dot.
• Anchor the pattern to its surroundings in the line so it doesn't catch extra matches.
• Search with grep -E, count with grep -c, cut out a piece with grep -oE.
• Pull part of a line out with sed -E 's/...(group).../\1/'.