Strings in R and Python

In R a string is a single element of a character vector, and string functions are vectorized over those elements.

In Python a str is an immutable sequence of characters that can be indexed and sliced, and its methods act on one string at a time.

x = c("apple", "banana", "kiwi")

nchar(x)

[1] 5 6 4

substr(x, 1, 3)

[1] "app" "ban" "kiw"

toupper(x)

[1] "APPLE"  "BANANA" "KIWI"  

x = "banana"

len(x)

6

x[0:3]

'ban'

x.upper()

'BANANA'

[
  s.upper()
  for s in ["apple", "banana", "kiwi"]
]

['APPLE', 'BANANA', 'KIWI']

stringr

stringr is the tidyverse’s string package, built on stringi and the ICU C library. Base R has equivalents for most of its functions (paste(), substr(), nchar(), gsub(), …), but their names and argument orders are inconsistent.

• Most string-manipulation functions start with str_.

• Most take the string as the first argument, making them convenient to use with pipes.

• Functions are vectorized over the string inputs. Some, such as str_detect(), also accept a vector of patterns.

• Missing strings generally produce missing results.

String operations

Most everyday string tasks have a direct equivalent in each language.

Strings in data frames

stringr functions are vectorized, so they work directly inside mutate().

Python’s string methods are not, so pandas and polars expose vectorized versions through the .str accessor.

d = tibble(
  name = c("Bob Smith", "Alice Jones")
)

import polars as pl
d = pl.DataFrame({
  "name": ["Bob Smith", "Alice Jones"]
})

d |>
  mutate(
    upper = str_to_upper(name),
    n = str_length(name)
  )

# A tibble: 2 × 3
  name        upper           n
  <chr>       <chr>       <int>
1 Bob Smith   BOB SMITH       9
2 Alice Jones ALICE JONES    11

d.with_columns(
  upper = pl.col("name").str.to_uppercase(),
  n = pl.col("name").str.len_chars()
)

Regular expressions

A regular expression (regex) is a sequence of characters that describes a pattern to be matched in text. Regexes are used to detect, extract, replace, and split strings, and to validate input.

• The idea dates to the 1950s, when Stephen Kleene formalized regular languages.

• They came into common use in the late 1960s and early 1970s through Unix text processing tools like ed, grep, and sed.

• Most modern languages use a syntax descended from Perl’s. The core is shared, but each engine differs in its details.

• In R, stringr uses the ICU engine and base R uses TRE by default or PCRE when perl = TRUE.

• We will work through the syntax using stringr, and then see how it translates to Python’s re module.

stringr regular expression functions

Many of these functions have variants with an _all suffix (e.g. str_replace_all) which will match more than one occurrence of the pattern in a string.

Simple pattern detection

text = c("The quick brown", "fox jumps over", "the lazy dog")

str_detect(text, "quick")

[1]  TRUE FALSE FALSE

str_subset(text, "o")

[1] "The quick brown" "fox jumps over"  "the lazy dog"   

str_detect(text, "the")

[1] FALSE FALSE  TRUE

str_detect(text, regex("the", ignore_case = TRUE))

[1]  TRUE FALSE  TRUE

Escape characters

An escape character changes the interpretation of the character(s) that follow it. In R string literals the escape character is \.

Escapes in practice

print() shows a string as it would be typed, escapes included. cat() shows the actual characters.

print("a\tb")

[1] "a\tb"

cat("a\tb")

a   b

nchar("a\tb")

[1] 3

print("a\\b")

[1] "a\\b"

cat("a\\b")

a\b

nchar("a\\b")

[1] 3

print("a\nb")

[1] "a\nb"

cat("a\nb")

a
b

nchar("a\nb")

[1] 3

print("a\"b")

[1] "a\"b"

cat("a\"b")

a"b

nchar("a\"b")

[1] 3

Raw strings

A raw string turns off escape processing, so a backslash is just a backslash. As of R 4.0 these are written as r"(...)".

cat("\\int_0^\\infty 1/e^x")

\int_0^\infty 1/e^x

cat(r"(\int_0^\infty 1/e^x)")

\int_0^\infty 1/e^x

r"(\d+\.\d+)"

[1] "\\d+\\.\\d+"

"\\d+\\.\\d+" == r"(\d+\.\d+)"

[1] TRUE

Regex metacharacters

The power of regular expressions comes from metacharacters, which modify how the pattern is matched.

. ^ $ * + ? { } [ ] \ | ( )

Matching metacharacters

str_detect("abc[def", "\[")

## Error: '\[' is an unrecognized escape

str_detect("abc[def", "\\[")

[1] TRUE

cat("abc\\def")

abc\def

str_detect("abc\\def", "\\\\")

[1] TRUE

str_detect("abc\\def", r"(\\)")

[1] TRUE

str_detect("abc[def", fixed("["))

[1] TRUE

XKCD’s take

Anchors

Sometimes we want a pattern to match only at a particular location in a string. Anchors match a position rather than a character.

text = "the brown owl ate the egg"

str_replace(text, "^the", "---")

[1] "--- brown owl ate the egg"

str_replace(text, "the$", "---")

[1] "the brown owl ate the egg"

str_replace(text, "egg$", "---")

[1] "the brown owl ate the ---"

str_replace_all(text, "e", "*")

[1] "th* brown owl at* th* *gg"

str_replace_all(text, "e\\b", "*")

[1] "th* brown owl at* th* egg"

str_replace_all(text, "e\\B", "*")

[1] "the brown owl ate the *gg"

Alternation

If there is more than one pattern we would like to match we can use the or (|) metacharacter. It has the lowest precedence of any operator, so use parentheses to limit its scope.

str_replace_all(text, "the|egg", "---")

[1] "--- brown owl ate --- ---"

str_replace_all(text, "a|e|i|o|u", "*")

[1] "th* br*wn *wl *t* th* *gg"

str_replace_all(text, "\\ba|e|i|o|u", "*")

[1] "th* br*wn *wl *t* th* *gg"

str_replace_all(text, "\\b(a|e|i|o|u)", "*")

[1] "the brown *wl *te the *gg"

Classes and ranges

A character class matches any one character from a set, which is written inside square brackets. Ranges simplify classes made up of contiguous letters or digits.

text = c("apple", "(219) 733-8965")

str_replace_all(text, "[aeiou]", "*")

[1] "*ppl*"          "(219) 733-8965"

str_replace_all(text, "[13579]", "*")

[1] "apple"          "(2**) ***-8*6*"

str_replace_all(text, "[1-5a-f]", "*")

[1] "*ppl*"          "(**9) 7**-896*"

str_replace_all(
  text, "[^1-5a-f]", "*"
)

[1] "a***e"          "*21****33****5"

Character classes

Some classes are common enough that they have built-in shorthands.

Unicode character classes

stringr’s character classes follow Unicode categories, so they include characters beyond ASCII. Here ٣ is the Arabic-Indic digit three, _ is both a word character and punctuation, and + is a symbol rather than punctuation.

tibble(
  char = c("A", "é", "٣", "_", "+"),
  digit = str_detect(char, r"(\d)"),
  word = str_detect(char, r"(\w)"),
  punctuation = str_detect(char, "[[:punct:]]")
)

# A tibble: 5 × 4
  char  digit word  punctuation
  <chr> <lgl> <lgl> <lgl>      
1 A     FALSE TRUE  FALSE      
2 é     FALSE TRUE  FALSE      
3 ٣     TRUE  TRUE  FALSE      
4 _     FALSE TRUE  TRUE       
5 +     FALSE FALSE FALSE      

Example

How would we write a regular expression to match a telephone number with the form (###) ###-####?

text = c("apple", "(219) 733-8965", "(329) 293-8753")

str_detect(text, "(\\d\\d\\d) \\d\\d\\d-\\d\\d\\d\\d")

[1] FALSE FALSE FALSE

str_detect(text, "\\(\\d\\d\\d\\) \\d\\d\\d-\\d\\d\\d\\d")

[1] FALSE  TRUE  TRUE

Quantifiers

Attached to literals, character classes, or groups, these allow a match to repeat some number of times.

Example

How would we improve our previous regular expression for matching a telephone number with the form (###) ###-####?

text = c("apple", "(219) 733-8965", "(329) 293-8753")

str_detect(text, "\\(\\d\\d\\d\\) \\d\\d\\d-\\d\\d\\d\\d")

[1] FALSE  TRUE  TRUE

str_detect(text, "\\(\\d{3}\\) \\d{3}-\\d{4}")

[1] FALSE  TRUE  TRUE

str_extract(text, "\\(\\d{3}\\) \\d{3}-\\d{4}")

[1] NA               "(219) 733-8965" "(329) 293-8753"

Exercise 1

For the following vector of randomly generated names, write a regular expression that,

• detects if the person’s first name starts with a vowel (a,e,i,o,u)

• detects if the person’s last name starts with a vowel

• detects if either the person’s first or last name starts with a vowel

• detects if neither the person’s first nor last name starts with a vowel

x = c("Jeremy Cruz", "Nathaniel Le", "Jasmine Chu", "Bradley Calderon Raygoza",
      "Quinten Weller", "Katelien Kanamu-Hauanio", "Zuhriyaa al-Amen",
      "Travale York", "Alexis Ahmed", "David Alcocer", "Jairo Martinez",
      "Dwone Gallegos", "Amanda Sherwood", "Hadiyya el-Eid", "Shaimaaa al-Can",
      "Sarah Love", "Shelby Villano", "Sundus al-Hashmi", "Dyani Loving",
      "Shanelle Douglas")

Extracting matches

str_extract() returns the first match in each string, or NA when there is none. str_extract_all() returns every match, as a list with one element for each string.

x = c(
  "Midterms on 2026-10-14 and 2026-11-18",
  "Final on 2026-12-12",
  "No project"
)
date = "\\d{4}-\\d{2}-\\d{2}"

str_extract(x, date)

[1] "2026-10-14" "2026-12-12" NA          

str_extract_all(x, date)

[[1]]
[1] "2026-10-14" "2026-11-18"

[[2]]
[1] "2026-12-12"

[[3]]
character(0)

Greedy vs non-greedy matching

At a given starting position, a greedy quantifier tries to consume as much text as possible while allowing the rest of the pattern to match. Adding ? makes it non-greedy, so it consumes as little as possible while still allowing the rest to match.

text = "<div class='main'> <div> <a href='here.pdf'>Here!</a> </div> </div>"

str_extract(text, "<div>.*</div>")

[1] "<div> <a href='here.pdf'>Here!</a> </div> </div>"

str_extract(text, "<div>.*?</div>")

[1] "<div> <a href='here.pdf'>Here!</a> </div>"

Groups

Groups allow you to connect pieces of a regular expression for modification or capture.

Groups in stringr

str_extract() returns the full match by default. Its group argument can select a capturing group. str_match() returns a matrix with the full match in the first column, followed by one column per capturing group.

text = c("Bob Smith", "Alice Smith", "Apple")

str_extract(text, "^(\\w+) \\w+")

[1] "Bob Smith"   "Alice Smith" NA           

str_extract(text, "^(\\w+) \\w+", group = 1)

[1] "Bob"   "Alice" NA     

str_match(text, "^(\\w+) \\w+")

     [,1]          [,2]   
[1,] "Bob Smith"   "Bob"  
[2,] "Alice Smith" "Alice"
[3,] NA            NA     

str_extract(text, "^(\\w+) (\\w+)")

[1] "Bob Smith"   "Alice Smith" NA           

str_match(text, "^(\\w+) (\\w+)")

     [,1]          [,2]    [,3]   
[1,] "Bob Smith"   "Bob"   "Smith"
[2,] "Alice Smith" "Alice" "Smith"
[3,] NA            NA      NA     

Multiple matches and groups

str_match_all() returns a list with one matrix per input string. Each row is a match. The first column contains the full match, followed by one column per capturing group.

str_match_all("A12 B34", "([A-Z])([0-9]+)")

[[1]]
     [,1]  [,2] [,3]
[1,] "A12" "A"  "12"
[2,] "B34" "B"  "34"

Both matches are retained, with the letter and digits captured separately. An input with no matches gets a matrix with zero rows.

Backreferences

Backreferences refer to previously captured groups using \1, \2, etc. They can be used in a replacement string, or within the pattern itself to match repeated text.

text = c("Bob Smith", "Alice Smith", "Apple")

str_replace(text, "^(\\w+) (\\w+)", "\\2, \\1")

[1] "Smith, Bob"   "Smith, Alice" "Apple"       

str_detect(c("abab", "cdcd", "abcd"), "^(..)\\1$")

[1]  TRUE  TRUE FALSE

str_extract(c("hello", "banana", "apple"), "(.)\\1")

[1] "ll" NA   "pp"

Flags

Flags change how a pattern is interpreted. In stringr they are set by wrapping the pattern in regex().

str_count("the cat\nthe hat", "^the")

[1] 1

str_count("the cat\nthe hat", regex("^the", multiline = TRUE))

[1] 2

Regex in data frames

stringr’s functions are vectorized, so they can be used directly on the columns of a data frame.

d = tibble(x = c("Bob Smith", "Alice Jones"))

d |>
  mutate(
    first = str_extract(x, "^\\w+"),
    s = str_detect(x, "S")
  )

# A tibble: 2 × 3
  x           first s    
  <chr>       <chr> <lgl>
1 Bob Smith   Bob   TRUE 
2 Alice Jones Alice FALSE

d |>
  filter(str_detect(x, "^A"))

# A tibble: 1 × 1
  x          
  <chr>      
1 Alice Jones

Exercise 2

text = c(
  "apple",
  "219 733 8965",
  "329-293-8753",
  "Work: (579) 499-7527; Home: (543) 355 3679"
)

• Write a regular expression that will extract all of the phone numbers contained in the vector above. Expect 0, 1, 1, and 2 matches across the four inputs.

• Once that works, use capturing groups with str_match_all() to retain every phone number while separating the area code from the remaining number. Keep the matches associated with their original input string.

Python’s re module

Raw strings in Python

Python’s raw strings are written r"...", and the convention is to write every regex pattern as one.

print("\\int_0^\\infty 1/e^x")

\int_0^\infty 1/e^x

print(r"\int_0^\infty 1/e^x")

\int_0^\infty 1/e^x

import re
text = "the brown owl ate the egg"

re.sub(r"\bow", "---", text)

'the brown ---l ate the egg'

re.sub("\bow", "---", text)

'the brown owl ate the egg'

Detecting patterns

re.search() takes a single string, so the vectorized str_detect() and str_subset() become comprehensions.

text = ["The quick brown", "fox jumps over", "the lazy dog"]

[re.search("quick", s) for s in text]

[<re.Match object; span=(4, 9), match='quick'>, None, None]

[bool(re.search("quick", s)) for s in text]

[True, False, False]

[s for s in text if re.search("o", s)]

['The quick brown', 'fox jumps over', 'the lazy dog']

[bool(re.search("the", s)) for s in text]

[False, False, True]

[bool(re.search("the", s, re.IGNORECASE)) for s in text]

[True, False, True]

Match objects

re.search() returns a Match object, or None when there is no match. None is falsy, so if m: is the usual test. Positions are 0-based and the end is exclusive.

x = "fox jumps over"

m = re.search("o", x); m

<re.Match object; span=(1, 2), match='o'>

m.group()

'o'

m.start(), m.end()

(1, 2)

print(re.search("z", x))

None

re.findall("o", x)

['o', 'o']

[
  m.start()
  for m in re.finditer("o", x)
]

[1, 10]

Replacing and splitting

re.sub() plays the role of both str_replace() and str_replace_all(). Backreferences in the replacement work the same way as in stringr.

text = "the brown owl ate the egg"

re.sub(r"the", "---", text)

'--- brown owl ate --- egg'

re.sub(r"the", "---", text, count=1)

'--- brown owl ate the egg'

re.sub(r"\b(a|e|i|o|u)", "*", text)

'the brown *wl *te the *gg'

re.sub(r"^(\w+) (\w+)", r"\2, \1", "Bob Smith")

'Smith, Bob'

re.split(r"\s+", text)

['the', 'brown', 'owl', 'ate', 'the', 'egg']

Match groups in Python

A Match object gives access to the capture groups, with group(0) being the full match.

m = re.search(
  r"^(\w+) (\w+)", "Bob Smith"
)

m.group(0)

'Bob Smith'

m.group(1)

'Bob'

m.group(2)

'Smith'

m.groups()

('Bob', 'Smith')

Flags and compiled patterns

Flags are passed as an additional argument and combined with |. re.compile() stores a pattern and its flags in an object with the same methods as the re module.

text = "The cat in\nthe hat"

re.findall(r"^the", text)

[]

re.findall(r"^the", text, re.M)

['the']

re.findall(r"^the", text, re.M | re.I)

['The', 'the']

p = re.compile(r"^the", re.M | re.I)
p.findall(text)

['The', 'the']

p.sub("a", text)

'a cat in\na hat'

Regex in pandas and polars

Many .str methods in pandas and polars accept regular expressions. polars uses Rust’s regex engine, which does not support backreferences in patterns.

import pandas as pd

s = pd.Series(
  ["Bob Smith", "Alice Jones"]
)

s.str.extract(r"^(\w+) (\w+)")

       0      1
0    Bob  Smith
1  Alice  Jones

s.str.contains(r"S")

0     True
1    False
dtype: bool

d = pl.DataFrame({
  "x": ["Bob Smith", "Alice Jones"]
})

d.with_columns(
  first = pl.col("x").str.extract(r"^(\w+)"),
  s = pl.col("x").str.contains(r"S")
)

Regex in R and Python

Takeaways