Data Frames

Lecture 07

Dr. Colin Rundel

Data frames in R

Data frames

A data frame is R’s structure for heterogeneous tabular data - the table is composed of a generic vector containing one or more equal length vectors forming the columns.

(df = data.frame(
  x = 1:3,
  y = c("a", "b", "c"),
  z = TRUE              # Recycled
))

  x y    z
1 1 a TRUE
2 2 b TRUE
3 3 c TRUE

str(df)

'data.frame':   3 obs. of  3 variables:
 $ x: int  1 2 3
 $ y: chr  "a" "b" "c"
 $ z: logi  TRUE TRUE TRUE

nrow(df)

[1] 3

ncol(df)

[1] 3

Data frame structure

Beyond thelist and equal length vectors all data frames have three attributes: names (the columns), row.names, and class.

typeof(df)

[1] "list"

class(df)

[1] "data.frame"

str( attributes(df) )

List of 3
 $ names    : chr [1:3] "x" "y" "z"
 $ class    : chr "data.frame"
 $ row.names: int [1:3] 1 2 3

str( unclass(df) )

List of 3
 $ x: int [1:3] 1 2 3
 $ y: chr [1:3] "a" "b" "c"
 $ z: logi [1:3] TRUE TRUE TRUE
 - attr(*, "row.names")= int [1:3] 1 2 3

length(df)

[1] 3

names(df)

[1] "x" "y" "z"

Build your own data frame

Since it is only a list plus attributes, we can build a data frame by hand just like we constructed a factor previously.

l = list(x = 1:3, y = c("a", "b", "c"), z = c(TRUE, TRUE, TRUE))

attr(l, "class") = "data.frame"
l

[1] x y z
<0 rows> (or 0-length row.names)

attr(l, "row.names") = 1:3
l

  x y    z
1 1 a TRUE
2 2 b TRUE
3 3 c TRUE

identical(l, df)

[1] TRUE

s = structure(
  list(x = 1:3, y = c("a", "b", "c"),
       z = c(TRUE, TRUE, TRUE)),
  class = "data.frame",
  row.names = 1:3
)
s

  x y    z
1 1 a TRUE
2 2 b TRUE
3 3 c TRUE

identical(s, df)

[1] TRUE

Data frames as S3 objects

Everything that makes a data frame behave like a table comes from S3 methods dispatched on the data.frame class.

length( methods(class = "data.frame") )

[1] 64

sloop::s3_dispatch( print(df) )

=> print.data.frame
 * print.default

sloop::s3_dispatch( df[1, ] )

=> [.data.frame
   [.default
-> [ (internal)

sloop::s3_dispatch( dim(df) )

=> dim.data.frame
   dim.default
 * dim (internal)

dim(df)

[1] 3 3

dim( unclass(df) )

NULL

Recycling

data.frame() (and other construction methods) recycle shorter columns to the longest column’s length - as long as it is a whole multiple, anything else is an error.

data.frame(x = 1:4, y = "a")

  x y
1 1 a
2 2 a
3 3 a
4 4 a

data.frame(x = 1:4, y = 1:2)

data.frame(x = 1:4, y = 1:3)

Error in `data.frame()`:
! arguments imply differing number of rows: 4, 3

Subsetting

A data frame is a named list that also has a matrix like shape, so both the list and matrix subsetting rules from earlier lectures apply:

List style (one index) - df$x and df[["x"]] return a column as a vector, while df["x"] and df[c("x", "z")] return a smaller data frame.
Matrix style (two indices) - df[rows, cols] where each index can be positive or negative integers, logicals, or names, and an empty index keeps everything, e.g. df[df$x > 1, c("x", "y")].
Assignment works with all of these forms - df$w = df$x * 2 adds a column, df[df$x > 1, "y"] = "zz" replaces values, and df$z = NULL removes a column.

As with matrices, a single column is simplified to a vector, but a single row is preserved as a data frame since its values are heterogeneous - drop controls this.

df[, "x"]

[1] 1 2 3

df[, "x", drop = FALSE]

df[1, ]

  x y    z
1 1 a TRUE

df[1, , drop = TRUE] |> str()

List of 3
 $ x: int 1
 $ y: chr "a"
 $ z: logi TRUE

Exercise 1

Using base R and the penguins data frame from the palmerpenguins package,

Select the rows for Gentoo penguins with a body mass above 5000 g, keeping only the species, island, and body_mass_g columns.
Add a column body_mass_kg containing the body mass in kilograms.
Calculate the mean bill length (in mm) of the Adelie penguins, ignoring missing values.

Modern data frames

The tidyverse’s tibble package provides tbl_df (and other S3 classes and methods) as a modern extension of data frames.

library(tibble)
df = data.frame(x = 1:3, y = c("a", "b", "c"), z = TRUE)

tb = as_tibble(df)
class(tb)

[1] "tbl_df"     "tbl"        "data.frame"

inherits(tb, "data.frame")

[1] TRUE

str( unclass(tb) )

List of 3
 $ x: int [1:3] 1 2 3
 $ y: chr [1:3] "a" "b" "c"
 $ z: logi [1:3] TRUE TRUE TRUE
 - attr(*, "row.names")= int [1:3] 1 2 3

Printing

Data frames print every row and column, tibbles print what fits along with dimensions and column types.

as.data.frame(penguins)

      species    island bill_length_mm
1      Adelie Torgersen           39.1
2      Adelie Torgersen           39.5
3      Adelie Torgersen           40.3
4      Adelie Torgersen             NA
5      Adelie Torgersen           36.7
6      Adelie Torgersen           39.3
7      Adelie Torgersen           38.9
8      Adelie Torgersen           39.2
9      Adelie Torgersen           34.1
10     Adelie Torgersen           42.0
11     Adelie Torgersen           37.8
12     Adelie Torgersen           37.8
13     Adelie Torgersen           41.1
14     Adelie Torgersen           38.6
15     Adelie Torgersen           34.6
16     Adelie Torgersen           36.6
17     Adelie Torgersen           38.7
18     Adelie Torgersen           42.5
19     Adelie Torgersen           34.4
20     Adelie Torgersen           46.0
21     Adelie    Biscoe           37.8
22     Adelie    Biscoe           37.7
23     Adelie    Biscoe           35.9
24     Adelie    Biscoe           38.2
25     Adelie    Biscoe           38.8
26     Adelie    Biscoe           35.3
27     Adelie    Biscoe           40.6
28     Adelie    Biscoe           40.5
29     Adelie    Biscoe           37.9
30     Adelie    Biscoe           40.5
31     Adelie     Dream           39.5
32     Adelie     Dream           37.2
33     Adelie     Dream           39.5
34     Adelie     Dream           40.9
35     Adelie     Dream           36.4
36     Adelie     Dream           39.2
37     Adelie     Dream           38.8
38     Adelie     Dream           42.2
39     Adelie     Dream           37.6
40     Adelie     Dream           39.8
41     Adelie     Dream           36.5
42     Adelie     Dream           40.8
43     Adelie     Dream           36.0
44     Adelie     Dream           44.1
45     Adelie     Dream           37.0
46     Adelie     Dream           39.6
47     Adelie     Dream           41.1
48     Adelie     Dream           37.5
49     Adelie     Dream           36.0
50     Adelie     Dream           42.3
51     Adelie    Biscoe           39.6
52     Adelie    Biscoe           40.1
53     Adelie    Biscoe           35.0
54     Adelie    Biscoe           42.0
55     Adelie    Biscoe           34.5
56     Adelie    Biscoe           41.4
57     Adelie    Biscoe           39.0
58     Adelie    Biscoe           40.6
59     Adelie    Biscoe           36.5
60     Adelie    Biscoe           37.6
61     Adelie    Biscoe           35.7
62     Adelie    Biscoe           41.3
63     Adelie    Biscoe           37.6
64     Adelie    Biscoe           41.1
65     Adelie    Biscoe           36.4
66     Adelie    Biscoe           41.6
67     Adelie    Biscoe           35.5
68     Adelie    Biscoe           41.1
69     Adelie Torgersen           35.9
70     Adelie Torgersen           41.8
71     Adelie Torgersen           33.5
72     Adelie Torgersen           39.7
73     Adelie Torgersen           39.6
74     Adelie Torgersen           45.8
75     Adelie Torgersen           35.5
76     Adelie Torgersen           42.8
77     Adelie Torgersen           40.9
78     Adelie Torgersen           37.2
79     Adelie Torgersen           36.2
80     Adelie Torgersen           42.1
81     Adelie Torgersen           34.6
82     Adelie Torgersen           42.9
83     Adelie Torgersen           36.7
84     Adelie Torgersen           35.1
85     Adelie     Dream           37.3
86     Adelie     Dream           41.3
87     Adelie     Dream           36.3
88     Adelie     Dream           36.9
89     Adelie     Dream           38.3
90     Adelie     Dream           38.9
91     Adelie     Dream           35.7
92     Adelie     Dream           41.1
93     Adelie     Dream           34.0
94     Adelie     Dream           39.6
95     Adelie     Dream           36.2
96     Adelie     Dream           40.8
97     Adelie     Dream           38.1
98     Adelie     Dream           40.3
99     Adelie     Dream           33.1
100    Adelie     Dream           43.2
101    Adelie    Biscoe           35.0
102    Adelie    Biscoe           41.0
103    Adelie    Biscoe           37.7
104    Adelie    Biscoe           37.8
105    Adelie    Biscoe           37.9
106    Adelie    Biscoe           39.7
107    Adelie    Biscoe           38.6
108    Adelie    Biscoe           38.2
109    Adelie    Biscoe           38.1
110    Adelie    Biscoe           43.2
111    Adelie    Biscoe           38.1
112    Adelie    Biscoe           45.6
113    Adelie    Biscoe           39.7
114    Adelie    Biscoe           42.2
115    Adelie    Biscoe           39.6
116    Adelie    Biscoe           42.7
117    Adelie Torgersen           38.6
118    Adelie Torgersen           37.3
119    Adelie Torgersen           35.7
120    Adelie Torgersen           41.1
121    Adelie Torgersen           36.2
122    Adelie Torgersen           37.7
123    Adelie Torgersen           40.2
124    Adelie Torgersen           41.4
125    Adelie Torgersen           35.2
126    Adelie Torgersen           40.6
127    Adelie Torgersen           38.8
128    Adelie Torgersen           41.5
129    Adelie Torgersen           39.0
130    Adelie Torgersen           44.1
131    Adelie Torgersen           38.5
132    Adelie Torgersen           43.1
133    Adelie     Dream           36.8
134    Adelie     Dream           37.5
135    Adelie     Dream           38.1
136    Adelie     Dream           41.1
137    Adelie     Dream           35.6
138    Adelie     Dream           40.2
139    Adelie     Dream           37.0
140    Adelie     Dream           39.7
141    Adelie     Dream           40.2
142    Adelie     Dream           40.6
143    Adelie     Dream           32.1
144    Adelie     Dream           40.7
145    Adelie     Dream           37.3
146    Adelie     Dream           39.0
147    Adelie     Dream           39.2
148    Adelie     Dream           36.6
149    Adelie     Dream           36.0
150    Adelie     Dream           37.8
151    Adelie     Dream           36.0
152    Adelie     Dream           41.5
153    Gentoo    Biscoe           46.1
154    Gentoo    Biscoe           50.0
155    Gentoo    Biscoe           48.7
156    Gentoo    Biscoe           50.0
157    Gentoo    Biscoe           47.6
158    Gentoo    Biscoe           46.5
159    Gentoo    Biscoe           45.4
160    Gentoo    Biscoe           46.7
161    Gentoo    Biscoe           43.3
162    Gentoo    Biscoe           46.8
163    Gentoo    Biscoe           40.9
164    Gentoo    Biscoe           49.0
165    Gentoo    Biscoe           45.5
166    Gentoo    Biscoe           48.4
167    Gentoo    Biscoe           45.8
168    Gentoo    Biscoe           49.3
169    Gentoo    Biscoe           42.0
170    Gentoo    Biscoe           49.2
171    Gentoo    Biscoe           46.2
172    Gentoo    Biscoe           48.7
173    Gentoo    Biscoe           50.2
174    Gentoo    Biscoe           45.1
175    Gentoo    Biscoe           46.5
176    Gentoo    Biscoe           46.3
177    Gentoo    Biscoe           42.9
178    Gentoo    Biscoe           46.1
179    Gentoo    Biscoe           44.5
180    Gentoo    Biscoe           47.8
181    Gentoo    Biscoe           48.2
182    Gentoo    Biscoe           50.0
183    Gentoo    Biscoe           47.3
184    Gentoo    Biscoe           42.8
185    Gentoo    Biscoe           45.1
186    Gentoo    Biscoe           59.6
187    Gentoo    Biscoe           49.1
188    Gentoo    Biscoe           48.4
189    Gentoo    Biscoe           42.6
190    Gentoo    Biscoe           44.4
191    Gentoo    Biscoe           44.0
192    Gentoo    Biscoe           48.7
193    Gentoo    Biscoe           42.7
194    Gentoo    Biscoe           49.6
195    Gentoo    Biscoe           45.3
196    Gentoo    Biscoe           49.6
197    Gentoo    Biscoe           50.5
198    Gentoo    Biscoe           43.6
199    Gentoo    Biscoe           45.5
200    Gentoo    Biscoe           50.5
201    Gentoo    Biscoe           44.9
202    Gentoo    Biscoe           45.2
203    Gentoo    Biscoe           46.6
204    Gentoo    Biscoe           48.5
205    Gentoo    Biscoe           45.1
206    Gentoo    Biscoe           50.1
207    Gentoo    Biscoe           46.5
208    Gentoo    Biscoe           45.0
209    Gentoo    Biscoe           43.8
210    Gentoo    Biscoe           45.5
211    Gentoo    Biscoe           43.2
212    Gentoo    Biscoe           50.4
213    Gentoo    Biscoe           45.3
214    Gentoo    Biscoe           46.2
215    Gentoo    Biscoe           45.7
216    Gentoo    Biscoe           54.3
217    Gentoo    Biscoe           45.8
218    Gentoo    Biscoe           49.8
219    Gentoo    Biscoe           46.2
220    Gentoo    Biscoe           49.5
221    Gentoo    Biscoe           43.5
222    Gentoo    Biscoe           50.7
223    Gentoo    Biscoe           47.7
224    Gentoo    Biscoe           46.4
225    Gentoo    Biscoe           48.2
226    Gentoo    Biscoe           46.5
227    Gentoo    Biscoe           46.4
228    Gentoo    Biscoe           48.6
229    Gentoo    Biscoe           47.5
230    Gentoo    Biscoe           51.1
231    Gentoo    Biscoe           45.2
232    Gentoo    Biscoe           45.2
233    Gentoo    Biscoe           49.1
234    Gentoo    Biscoe           52.5
235    Gentoo    Biscoe           47.4
236    Gentoo    Biscoe           50.0
237    Gentoo    Biscoe           44.9
238    Gentoo    Biscoe           50.8
239    Gentoo    Biscoe           43.4
240    Gentoo    Biscoe           51.3
241    Gentoo    Biscoe           47.5
242    Gentoo    Biscoe           52.1
243    Gentoo    Biscoe           47.5
244    Gentoo    Biscoe           52.2
245    Gentoo    Biscoe           45.5
246    Gentoo    Biscoe           49.5
247    Gentoo    Biscoe           44.5
248    Gentoo    Biscoe           50.8
249    Gentoo    Biscoe           49.4
250    Gentoo    Biscoe           46.9
251    Gentoo    Biscoe           48.4
252    Gentoo    Biscoe           51.1
253    Gentoo    Biscoe           48.5
254    Gentoo    Biscoe           55.9
255    Gentoo    Biscoe           47.2
256    Gentoo    Biscoe           49.1
257    Gentoo    Biscoe           47.3
258    Gentoo    Biscoe           46.8
259    Gentoo    Biscoe           41.7
260    Gentoo    Biscoe           53.4
261    Gentoo    Biscoe           43.3
262    Gentoo    Biscoe           48.1
263    Gentoo    Biscoe           50.5
264    Gentoo    Biscoe           49.8
265    Gentoo    Biscoe           43.5
266    Gentoo    Biscoe           51.5
267    Gentoo    Biscoe           46.2
268    Gentoo    Biscoe           55.1
269    Gentoo    Biscoe           44.5
270    Gentoo    Biscoe           48.8
271    Gentoo    Biscoe           47.2
272    Gentoo    Biscoe             NA
273    Gentoo    Biscoe           46.8
274    Gentoo    Biscoe           50.4
275    Gentoo    Biscoe           45.2
276    Gentoo    Biscoe           49.9
277 Chinstrap     Dream           46.5
278 Chinstrap     Dream           50.0
279 Chinstrap     Dream           51.3
280 Chinstrap     Dream           45.4
281 Chinstrap     Dream           52.7
282 Chinstrap     Dream           45.2
283 Chinstrap     Dream           46.1
284 Chinstrap     Dream           51.3
285 Chinstrap     Dream           46.0
286 Chinstrap     Dream           51.3
287 Chinstrap     Dream           46.6
288 Chinstrap     Dream           51.7
289 Chinstrap     Dream           47.0
290 Chinstrap     Dream           52.0
291 Chinstrap     Dream           45.9
292 Chinstrap     Dream           50.5
293 Chinstrap     Dream           50.3
294 Chinstrap     Dream           58.0
295 Chinstrap     Dream           46.4
296 Chinstrap     Dream           49.2
297 Chinstrap     Dream           42.4
298 Chinstrap     Dream           48.5
299 Chinstrap     Dream           43.2
300 Chinstrap     Dream           50.6
301 Chinstrap     Dream           46.7
302 Chinstrap     Dream           52.0
303 Chinstrap     Dream           50.5
304 Chinstrap     Dream           49.5
305 Chinstrap     Dream           46.4
306 Chinstrap     Dream           52.8
307 Chinstrap     Dream           40.9
308 Chinstrap     Dream           54.2
309 Chinstrap     Dream           42.5
310 Chinstrap     Dream           51.0
311 Chinstrap     Dream           49.7
312 Chinstrap     Dream           47.5
313 Chinstrap     Dream           47.6
314 Chinstrap     Dream           52.0
315 Chinstrap     Dream           46.9
316 Chinstrap     Dream           53.5
317 Chinstrap     Dream           49.0
318 Chinstrap     Dream           46.2
319 Chinstrap     Dream           50.9
320 Chinstrap     Dream           45.5
321 Chinstrap     Dream           50.9
322 Chinstrap     Dream           50.8
323 Chinstrap     Dream           50.1
324 Chinstrap     Dream           49.0
325 Chinstrap     Dream           51.5
326 Chinstrap     Dream           49.8
327 Chinstrap     Dream           48.1
328 Chinstrap     Dream           51.4
329 Chinstrap     Dream           45.7
330 Chinstrap     Dream           50.7
331 Chinstrap     Dream           42.5
332 Chinstrap     Dream           52.2
333 Chinstrap     Dream           45.2
334 Chinstrap     Dream           49.3
335 Chinstrap     Dream           50.2
336 Chinstrap     Dream           45.6
337 Chinstrap     Dream           51.9
338 Chinstrap     Dream           46.8
339 Chinstrap     Dream           45.7
340 Chinstrap     Dream           55.8
341 Chinstrap     Dream           43.5
342 Chinstrap     Dream           49.6
343 Chinstrap     Dream           50.8
344 Chinstrap     Dream           50.2
    bill_depth_mm flipper_length_mm
1            18.7               181
2            17.4               186
3            18.0               195
4              NA                NA
5            19.3               193
6            20.6               190
7            17.8               181
8            19.6               195
9            18.1               193
10           20.2               190
11           17.1               186
12           17.3               180
13           17.6               182
14           21.2               191
15           21.1               198
16           17.8               185
17           19.0               195
18           20.7               197
19           18.4               184
20           21.5               194
21           18.3               174
22           18.7               180
23           19.2               189
24           18.1               185
25           17.2               180
26           18.9               187
27           18.6               183
28           17.9               187
29           18.6               172
30           18.9               180
31           16.7               178
32           18.1               178
33           17.8               188
34           18.9               184
35           17.0               195
36           21.1               196
37           20.0               190
38           18.5               180
39           19.3               181
40           19.1               184
41           18.0               182
42           18.4               195
43           18.5               186
44           19.7               196
45           16.9               185
46           18.8               190
47           19.0               182
48           18.9               179
49           17.9               190
50           21.2               191
51           17.7               186
52           18.9               188
53           17.9               190
54           19.5               200
55           18.1               187
56           18.6               191
57           17.5               186
58           18.8               193
59           16.6               181
60           19.1               194
61           16.9               185
62           21.1               195
63           17.0               185
64           18.2               192
65           17.1               184
66           18.0               192
67           16.2               195
68           19.1               188
69           16.6               190
70           19.4               198
71           19.0               190
72           18.4               190
73           17.2               196
74           18.9               197
75           17.5               190
76           18.5               195
77           16.8               191
78           19.4               184
79           16.1               187
80           19.1               195
81           17.2               189
82           17.6               196
83           18.8               187
84           19.4               193
85           17.8               191
86           20.3               194
87           19.5               190
88           18.6               189
89           19.2               189
90           18.8               190
91           18.0               202
92           18.1               205
93           17.1               185
94           18.1               186
95           17.3               187
96           18.9               208
97           18.6               190
98           18.5               196
99           16.1               178
100          18.5               192
101          17.9               192
102          20.0               203
103          16.0               183
104          20.0               190
105          18.6               193
106          18.9               184
107          17.2               199
108          20.0               190
109          17.0               181
110          19.0               197
111          16.5               198
112          20.3               191
113          17.7               193
114          19.5               197
115          20.7               191
116          18.3               196
117          17.0               188
118          20.5               199
119          17.0               189
120          18.6               189
121          17.2               187
122          19.8               198
123          17.0               176
124          18.5               202
125          15.9               186
126          19.0               199
127          17.6               191
128          18.3               195
129          17.1               191
130          18.0               210
131          17.9               190
132          19.2               197
133          18.5               193
134          18.5               199
135          17.6               187
136          17.5               190
137          17.5               191
138          20.1               200
139          16.5               185
140          17.9               193
141          17.1               193
142          17.2               187
143          15.5               188
144          17.0               190
145          16.8               192
146          18.7               185
147          18.6               190
148          18.4               184
149          17.8               195
150          18.1               193
151          17.1               187
152          18.5               201
153          13.2               211
154          16.3               230
155          14.1               210
156          15.2               218
157          14.5               215
158          13.5               210
159          14.6               211
160          15.3               219
161          13.4               209
162          15.4               215
163          13.7               214
164          16.1               216
165          13.7               214
166          14.6               213
167          14.6               210
168          15.7               217
169          13.5               210
170          15.2               221
171          14.5               209
172          15.1               222
173          14.3               218
174          14.5               215
175          14.5               213
176          15.8               215
177          13.1               215
178          15.1               215
179          14.3               216
180          15.0               215
181          14.3               210
182          15.3               220
183          15.3               222
184          14.2               209
185          14.5               207
186          17.0               230
187          14.8               220
188          16.3               220
189          13.7               213
190          17.3               219
191          13.6               208
192          15.7               208
193          13.7               208
194          16.0               225
195          13.7               210
196          15.0               216
197          15.9               222
198          13.9               217
199          13.9               210
200          15.9               225
201          13.3               213
202          15.8               215
203          14.2               210
204          14.1               220
205          14.4               210
206          15.0               225
207          14.4               217
208          15.4               220
209          13.9               208
210          15.0               220
211          14.5               208
212          15.3               224
213          13.8               208
214          14.9               221
215          13.9               214
216          15.7               231
217          14.2               219
218          16.8               230
219          14.4               214
220          16.2               229
221          14.2               220
222          15.0               223
223          15.0               216
224          15.6               221
225          15.6               221
226          14.8               217
227          15.0               216
228          16.0               230
229          14.2               209
230          16.3               220
231          13.8               215
232          16.4               223
233          14.5               212
234          15.6               221
235          14.6               212
236          15.9               224
237          13.8               212
238          17.3               228
239          14.4               218
240          14.2               218
241          14.0               212
242          17.0               230
243          15.0               218
244          17.1               228
245          14.5               212
246          16.1               224
247          14.7               214
248          15.7               226
249          15.8               216
250          14.6               222
251          14.4               203
252          16.5               225
253          15.0               219
254          17.0               228
255          15.5               215
256          15.0               228
257          13.8               216
258          16.1               215
259          14.7               210
260          15.8               219
261          14.0               208
262          15.1               209
263          15.2               216
264          15.9               229
265          15.2               213
266          16.3               230
267          14.1               217
268          16.0               230
269          15.7               217
270          16.2               222
271          13.7               214
272            NA                NA
273          14.3               215
274          15.7               222
275          14.8               212
276          16.1               213
277          17.9               192
278          19.5               196
279          19.2               193
280          18.7               188
281          19.8               197
282          17.8               198
283          18.2               178
284          18.2               197
285          18.9               195
286          19.9               198
287          17.8               193
288          20.3               194
289          17.3               185
290          18.1               201
291          17.1               190
292          19.6               201
293          20.0               197
294          17.8               181
295          18.6               190
296          18.2               195
297          17.3               181
298          17.5               191
299          16.6               187
300          19.4               193
301          17.9               195
302          19.0               197
303          18.4               200
304          19.0               200
305          17.8               191
306          20.0               205
307          16.6               187
308          20.8               201
309          16.7               187
310          18.8               203
311          18.6               195
312          16.8               199
313          18.3               195
314          20.7               210
315          16.6               192
316          19.9               205
317          19.5               210
318          17.5               187
319          19.1               196
320          17.0               196
321          17.9               196
322          18.5               201
323          17.9               190
324          19.6               212
325          18.7               187
326          17.3               198
327          16.4               199
328          19.0               201
329          17.3               193
330          19.7               203
331          17.3               187
332          18.8               197
333          16.6               191
334          19.9               203
335          18.8               202
336          19.4               194
337          19.5               206
338          16.5               189
339          17.0               195
340          19.8               207
341          18.1               202
342          18.2               193
343          19.0               210
344          18.7               198
    body_mass_g    sex year
1          3750   male 2007
2          3800 female 2007
3          3250 female 2007
4            NA   <NA> 2007
5          3450 female 2007
6          3650   male 2007
7          3625 female 2007
8          4675   male 2007
9          3475   <NA> 2007
10         4250   <NA> 2007
11         3300   <NA> 2007
12         3700   <NA> 2007
13         3200 female 2007
14         3800   male 2007
15         4400   male 2007
16         3700 female 2007
17         3450 female 2007
18         4500   male 2007
19         3325 female 2007
20         4200   male 2007
21         3400 female 2007
22         3600   male 2007
23         3800 female 2007
24         3950   male 2007
25         3800   male 2007
26         3800 female 2007
27         3550   male 2007
28         3200 female 2007
29         3150 female 2007
30         3950   male 2007
31         3250 female 2007
32         3900   male 2007
33         3300 female 2007
34         3900   male 2007
35         3325 female 2007
36         4150   male 2007
37         3950   male 2007
38         3550 female 2007
39         3300 female 2007
40         4650   male 2007
41         3150 female 2007
42         3900   male 2007
43         3100 female 2007
44         4400   male 2007
45         3000 female 2007
46         4600   male 2007
47         3425   male 2007
48         2975   <NA> 2007
49         3450 female 2007
50         4150   male 2007
51         3500 female 2008
52         4300   male 2008
53         3450 female 2008
54         4050   male 2008
55         2900 female 2008
56         3700   male 2008
57         3550 female 2008
58         3800   male 2008
59         2850 female 2008
60         3750   male 2008
61         3150 female 2008
62         4400   male 2008
63         3600 female 2008
64         4050   male 2008
65         2850 female 2008
66         3950   male 2008
67         3350 female 2008
68         4100   male 2008
69         3050 female 2008
70         4450   male 2008
71         3600 female 2008
72         3900   male 2008
73         3550 female 2008
74         4150   male 2008
75         3700 female 2008
76         4250   male 2008
77         3700 female 2008
78         3900   male 2008
79         3550 female 2008
80         4000   male 2008
81         3200 female 2008
82         4700   male 2008
83         3800 female 2008
84         4200   male 2008
85         3350 female 2008
86         3550   male 2008
87         3800   male 2008
88         3500 female 2008
89         3950   male 2008
90         3600 female 2008
91         3550 female 2008
92         4300   male 2008
93         3400 female 2008
94         4450   male 2008
95         3300 female 2008
96         4300   male 2008
97         3700 female 2008
98         4350   male 2008
99         2900 female 2008
100        4100   male 2008
101        3725 female 2009
102        4725   male 2009
103        3075 female 2009
104        4250   male 2009
105        2925 female 2009
106        3550   male 2009
107        3750 female 2009
108        3900   male 2009
109        3175 female 2009
110        4775   male 2009
111        3825 female 2009
112        4600   male 2009
113        3200 female 2009
114        4275   male 2009
115        3900 female 2009
116        4075   male 2009
117        2900 female 2009
118        3775   male 2009
119        3350 female 2009
120        3325   male 2009
121        3150 female 2009
122        3500   male 2009
123        3450 female 2009
124        3875   male 2009
125        3050 female 2009
126        4000   male 2009
127        3275 female 2009
128        4300   male 2009
129        3050 female 2009
130        4000   male 2009
131        3325 female 2009
132        3500   male 2009
133        3500 female 2009
134        4475   male 2009
135        3425 female 2009
136        3900   male 2009
137        3175 female 2009
138        3975   male 2009
139        3400 female 2009
140        4250   male 2009
141        3400 female 2009
142        3475   male 2009
143        3050 female 2009
144        3725   male 2009
145        3000 female 2009
146        3650   male 2009
147        4250   male 2009
148        3475 female 2009
149        3450 female 2009
150        3750   male 2009
151        3700 female 2009
152        4000   male 2009
153        4500 female 2007
154        5700   male 2007
155        4450 female 2007
156        5700   male 2007
157        5400   male 2007
158        4550 female 2007
159        4800 female 2007
160        5200   male 2007
161        4400 female 2007
162        5150   male 2007
163        4650 female 2007
164        5550   male 2007
165        4650 female 2007
166        5850   male 2007
167        4200 female 2007
168        5850   male 2007
169        4150 female 2007
170        6300   male 2007
171        4800 female 2007
172        5350   male 2007
173        5700   male 2007
174        5000 female 2007
175        4400 female 2007
176        5050   male 2007
177        5000 female 2007
178        5100   male 2007
179        4100   <NA> 2007
180        5650   male 2007
181        4600 female 2007
182        5550   male 2007
183        5250   male 2007
184        4700 female 2007
185        5050 female 2007
186        6050   male 2007
187        5150 female 2008
188        5400   male 2008
189        4950 female 2008
190        5250   male 2008
191        4350 female 2008
192        5350   male 2008
193        3950 female 2008
194        5700   male 2008
195        4300 female 2008
196        4750   male 2008
197        5550   male 2008
198        4900 female 2008
199        4200 female 2008
200        5400   male 2008
201        5100 female 2008
202        5300   male 2008
203        4850 female 2008
204        5300   male 2008
205        4400 female 2008
206        5000   male 2008
207        4900 female 2008
208        5050   male 2008
209        4300 female 2008
210        5000   male 2008
211        4450 female 2008
212        5550   male 2008
213        4200 female 2008
214        5300   male 2008
215        4400 female 2008
216        5650   male 2008
217        4700 female 2008
218        5700   male 2008
219        4650   <NA> 2008
220        5800   male 2008
221        4700 female 2008
222        5550   male 2008
223        4750 female 2008
224        5000   male 2008
225        5100   male 2008
226        5200 female 2008
227        4700 female 2008
228        5800   male 2008
229        4600 female 2008
230        6000   male 2008
231        4750 female 2008
232        5950   male 2008
233        4625 female 2009
234        5450   male 2009
235        4725 female 2009
236        5350   male 2009
237        4750 female 2009
238        5600   male 2009
239        4600 female 2009
240        5300   male 2009
241        4875 female 2009
242        5550   male 2009
243        4950 female 2009
244        5400   male 2009
245        4750 female 2009
246        5650   male 2009
247        4850 female 2009
248        5200   male 2009
249        4925   male 2009
250        4875 female 2009
251        4625 female 2009
252        5250   male 2009
253        4850 female 2009
254        5600   male 2009
255        4975 female 2009
256        5500   male 2009
257        4725   <NA> 2009
258        5500   male 2009
259        4700 female 2009
260        5500   male 2009
261        4575 female 2009
262        5500   male 2009
263        5000 female 2009
264        5950   male 2009
265        4650 female 2009
266        5500   male 2009
267        4375 female 2009
268        5850   male 2009
269        4875   <NA> 2009
270        6000   male 2009
271        4925 female 2009
272          NA   <NA> 2009
273        4850 female 2009
274        5750   male 2009
275        5200 female 2009
276        5400   male 2009
277        3500 female 2007
278        3900   male 2007
279        3650   male 2007
280        3525 female 2007
281        3725   male 2007
282        3950 female 2007
283        3250 female 2007
284        3750   male 2007
285        4150 female 2007
286        3700   male 2007
287        3800 female 2007
288        3775   male 2007
289        3700 female 2007
290        4050   male 2007
291        3575 female 2007
292        4050   male 2007
293        3300   male 2007
294        3700 female 2007
295        3450 female 2007
296        4400   male 2007
297        3600 female 2007
298        3400   male 2007
299        2900 female 2007
300        3800   male 2007
301        3300 female 2007
302        4150   male 2007
303        3400 female 2008
304        3800   male 2008
305        3700 female 2008
306        4550   male 2008
307        3200 female 2008
308        4300   male 2008
309        3350 female 2008
310        4100   male 2008
311        3600   male 2008
312        3900 female 2008
313        3850 female 2008
314        4800   male 2008
315        2700 female 2008
316        4500   male 2008
317        3950   male 2008
318        3650 female 2008
319        3550   male 2008
320        3500 female 2008
321        3675 female 2009
322        4450   male 2009
323        3400 female 2009
324        4300   male 2009
325        3250   male 2009
326        3675 female 2009
327        3325 female 2009
328        3950   male 2009
329        3600 female 2009
330        4050   male 2009
331        3350 female 2009
332        3450   male 2009
333        3250 female 2009
334        4050   male 2009
335        3800   male 2009
336        3525 female 2009
337        3950   male 2009
338        3650 female 2009
339        3650 female 2009
340        4000   male 2009
341        3400 female 2009
342        3775   male 2009
343        4100   male 2009
344        3775 female 2009

penguins

# A tibble: 344 × 8
  species island    bill_length_mm
  <fct>   <fct>              <dbl>
1 Adelie  Torgersen           39.1
2 Adelie  Torgersen           39.5
3 Adelie  Torgersen           40.3
4 Adelie  Torgersen           NA  
5 Adelie  Torgersen           36.7
# ℹ 339 more rows
# ℹ 5 more variables:
#   bill_depth_mm <dbl>,
#   flipper_length_mm <int>,
#   body_mass_g <int>, sex <fct>,
#   year <int>

Stricter subsetting

Tibbles never simplify with [ (there is no drop surprise), and $ does not partially match column names.

tb[, 1]

# A tibble: 3 × 1
      x
  <int>
1     1
2     2
3     3

tb[1, ]

# A tibble: 1 × 3
      x y     z    
  <int> <chr> <lgl>
1     1 a     TRUE

tb[[1]]

[1] 1 2 3

tb$x

[1] 1 2 3

penguins$sp

Warning: Unknown or uninitialised column: `sp`.

NULL

as.data.frame(penguins)$sp |> head()

[1] Adelie Adelie Adelie Adelie Adelie Adelie
Levels: Adelie Chinstrap Gentoo

Stricter construction

tibble() only recycles length one values, never coerces strings to factors, leaves column names alone, and lets later columns refer to earlier ones.

tibble(x = 1:4, y = 1:2)

Error in `tibble()`:
! Tibble columns must have compatible sizes.
• Size 4: Existing data.
• Size 2: Column `y`.
ℹ Only values of size one are recycled.

names( data.frame(`a b` = 1, `1x` = 2) )

[1] "a.b" "X1x"

names( tibble(`a b` = 1, `1x` = 2) )

[1] "a b" "1x"

data.frame(x = 1:3, y = x^2)

Error:
! object 'x' not found

tibble(x = 1:3, y = x^2)

# A tibble: 3 × 2
      x     y
  <int> <dbl>
1     1     1
2     2     4
3     3     9

Alternative construction

tibble() is a drop in replacement for data.frame()

tibble(
  x = 1:2, y = c("a", "b")
)

# A tibble: 2 × 2
      x y    
  <int> <chr>
1     1 a    
2     2 b

as_tibble() converts data frames, matrices, and lists of columns,

as_tibble(
  list(x = 1:2, y = c("a", "b"))
)

# A tibble: 2 × 2
      x y    
  <int> <chr>
1     1 a    
2     2 b

tribble() builds a tibble row by row, which is convenient for small hand-entered tables,

tribble(
  ~species,    ~n,
  "Adelie",    152,
  "Gentoo",    124,
  "Chinstrap",  68
)

# A tibble: 3 × 2
  species       n
  <chr>     <dbl>
1 Adelie      152
2 Gentoo      124
3 Chinstrap    68

Column-oriented data

Rows or columns?

A table can be stored as a collection of rows (records) or a collection of columns (variables).

Row-oriented - a list of records:

Column-oriented - a list of vectors:

recs = list(
  list(x = 1, y = "a"),
  list(x = 2, y = "b"),
  list(x = 3, y = "c")
)

cols = list(
  x = c(1, 2, 3),
  y = c("a", "b", "c")
)

Rows make records cheap: recs[[i]] is one element and a new record is one append. A variable is scattered across all $n$ records and must be gathered by a loop before anything vectorized can happen.

Columns make variables cheap: cols$x is one element and already a typed vector, so vectorized functions apply directly. A record is scattered across all $p$ columns and must be gathered and assembled.

Analysis works down variables far more often than across records, so R and pandas chose columns. Transactional databases and web APIs work one record at a time and typically chose rows.

Why columns?

Each column is one typed vector stored contiguously - exactly what vectorized operations expect. df$x * 2 and mean(df$x) work on the column directly, no unpacking of records needed.
Type information is stored once per column rather than once per value, so memory use is compact and the type of a column is known without scanning it or storing a separate schema.
Row access is the awkward direction - df[i, ] must extract one element from every column and assemble a new data frame.

Example data

The nycflights13 package contains every flight that departed from a New York City airport in 2013 - large enough for performance to matter.

library(nycflights13)
flights

# A tibble: 336,776 × 19
   year month   day dep_time sched_dep_time dep_delay arr_time sched_arr_time
  <int> <int> <int>    <int>          <int>     <dbl>    <int>          <int>
1  2013     1     1      517            515         2      830            819
2  2013     1     1      533            529         4      850            830
3  2013     1     1      542            540         2      923            850
4  2013     1     1      544            545        -1     1004           1022
5  2013     1     1      554            600        -6      812            837
# ℹ 336,771 more rows
# ℹ 11 more variables: arr_delay <dbl>, carrier <chr>, flight <int>,
#   tailnum <chr>, origin <chr>, dest <chr>, air_time <dbl>, distance <dbl>,
#   hour <dbl>, minute <dbl>, time_hour <dttm>

Row iteration in R

d = as.data.frame(flights)

by_row = function(d) {
  out = numeric(nrow(d))
  for (i in seq_len(nrow(d)))
    out[i] = d[i, "distance"] * 1.6
  out
}

by_element = function(d) {
  out = numeric(nrow(d))
  for (i in seq_len(nrow(d)))
    out[i] = d$distance[i] * 1.6
  out
}

bench::mark(
  rows       = by_row(d),
  elements   = by_element(d),
  vectorized = d$distance * 1.6
)

# A tibble: 3 × 6
  expression      min   median `itr/sec` mem_alloc `gc/sec`
  <bch:expr> <bch:tm> <bch:tm>     <dbl> <bch:byt>    <dbl>
1 rows          1.38s    1.38s     0.724   66.83MB     11.6
2 elements   110.77ms 111.38ms     8.97     2.59MB     10.8
3 vectorized  39.56µs  44.53µs 14998.       2.57MB    306.

Storage formats

The column-oriented distinction also applies to files on disk,

Format	Layout	Types	Notes
CSV / TSV	rows (text)	none, guessed	universal, human readable, slow to parse, large
JSON	rows (text)	partial	nested records, web APIs
Parquet	columns	full schema	compressed, read a subset of columns
Arrow / Feather	columns	full schema	Arrow’s memory layout on disk, fast, less compression
RDS / pickle	serialized	full	any R / Python object, only readable by that language
SQLite / DuckDB	rows / columns	full schema	database in a file, queried with SQL

File size

flights written to disk in both formats,

csv = file.path(tempdir(), "flights.csv")
pq = file.path(tempdir(), "flights.parquet")

readr::write_csv(flights, csv)
nanoparquet::write_parquet(flights, pq)

fs::file_size( c(csv, pq) )

29.62M  5.42M

Read speed

bench::mark(
  csv_base        = read.csv(csv),
  csv_readr       = readr::read_csv(csv, show_col_types = FALSE),
  csv_fread       = data.table::fread(csv),
  par_nanoparquet = nanoparquet::read_parquet(pq),
  par_arrow       = arrow::read_parquet(pq),
  check = FALSE
)

# A tibble: 5 × 6
  expression           min   median `itr/sec` mem_alloc `gc/sec`
  <bch:expr>      <bch:tm> <bch:tm>     <dbl> <bch:byt>    <dbl>
1 csv_base        618.21ms 618.21ms      1.62     228MB     3.24
2 csv_readr       574.39ms 574.39ms      1.74    49.8MB     1.74
3 csv_fread        40.74ms  42.35ms     23.6     35.6MB     7.86
4 par_nanoparquet   79.1ms  82.02ms     11.4     52.3MB    11.4 
5 par_arrow         4.97ms   5.85ms    163.      21.5MB     1.99

dplyr verbs

dplyr provides a set of functions (verbs) that each do one thing to a data frame - each corresponds to one of the vectorized operations from before.

Core single table verbs:

filter() / slice() - pick rows based on their values or position
select() / rename() / relocate() - pick, rename, or reorder columns by name
mutate() - create or modify columns
arrange() - reorder rows
summarize() / count() - reduce columns to values
group_by() / .by - make other verbs act within groups
distinct() - filter for unique rows
pull() - extract a column as a vector

dplyr rules

library(dplyr)

The first argument is always a data frame
Subsequent arguments describe what to do with its columns, referring to them by bare name
The result is always a new data frame, the input is never modified
Nothing happens in place - a sequence of verbs is a pipeline of copies (copy-on-modify makes this cheaper than it sounds)

Rules 1 and 3 are what make the verbs composable - the output of one is always a valid input for the next.

The pipe

R’s pipe |> passes the value on its left as the first argument of the function on its right, so a nested call can be written as a left to right sequence.

h( g( f(x), n = 1 ), m = 2 )

x |>
  f() |>
  g(n = 1) |>
  h(m = 2)

Since every dplyr verb takes and returns a data frame, the pipe is the natural way to chain them,

flights |>
  filter(dest == "LAX") |>
  count(carrier) |>
  arrange(desc(n))

Non-standard evaluation

Data masking

dplyr verbs capture their arguments and evaluate them with the columns in scope - data masking, a form of non-standard evaluation (NSE).

dest == "LAX"

Error:
! object 'dest' not found

flights |>
  filter(dest == "LAX", month == 5) |>
  select(month, day, carrier)

# A tibble: 1,453 × 3
  month   day carrier
  <int> <int> <chr>  
1     5     1 VX     
2     5     1 UA     
3     5     1 UA     
4     5     1 B6     
5     5     1 VX     
# ℹ 1,448 more rows

flights[flights$dest == "LAX" &
        flights$month == 5,
        c("month", "day", "carrier")]

# A tibble: 1,453 × 3
  month   day carrier
  <int> <int> <chr>  
1     5     1 VX     
2     5     1 UA     
3     5     1 UA     
4     5     1 B6     
5     5     1 VX     
# ℹ 1,448 more rows

Where names come from

Names in a masked expression are looked up in the data frame first and then in the calling environment, so columns and ordinary variables can be mixed - but a column always wins a tie.

long = 60
flights |>
  filter(dep_delay > long) |>
  select(carrier, dep_delay)

# A tibble: 26,581 × 2
  carrier dep_delay
  <chr>       <dbl>
1 MQ            101
2 AA             71
3 MQ            853
4 UA            144
5 UA            134
# ℹ 26,576 more rows

month = 5
flights |>
  filter(month == month) |>
  nrow()

[1] 336776

flights |>
  filter(month == .env$month) |>
  nrow()

[1] 28796

Why bother?

Data masking is what makes dplyr code short and readable - a column is just dep_delay, not flights$dep_delay or "dep_delay" - and it is what lets the same expression be evaluated by different backends (e.g. dbplyr, duckplyr, etc.).

The price is that the arguments are not ordinary values,

a verb cannot tell the difference between a column name and a variable holding a column name,
passing column names through your own functions needs extra machinery,
and code that constructs column names programmatically has to work a little harder.

tidyselect

Tidy selection

select(), rename(), across(), and .by use a second flavor of NSE - tidy selection where expressions describe which columns, not values.

flights |> select(year:day, dep_delay) |> names()

[1] "year"      "month"     "day"       "dep_delay"

flights |> select(starts_with("dep"), ends_with("delay")) |> names()

[1] "dep_time"  "dep_delay" "arr_delay"

flights |> select(where(is.character) | year) |> names()

[1] "carrier" "tailnum" "origin"  "dest"    "year"

flights |> select(!where(is.numeric)) |> names()

[1] "carrier"   "tailnum"   "origin"    "dest"      "time_hour"

Selection helpers

Helper	Selects
`x`, `x:y`, `c(x, y)`, `!x`, `-x`	by name, range, union, and exclusion
`starts_with()`, `ends_with()`, `contains()`, `matches()`	by prefix, suffix, substring, or regular expression
`num_range("x", 1:3)`	`x1`, `x2`, `x3`
`where(f)`	columns for which `f(column)` is `TRUE`
`everything()`, `last_col()`	all remaining columns, the last column
`all_of(chr)`, `any_of(chr)`	names in a character vector (strict / lenient)

cols = c("year", "month", "day")
flights |>
  select(all_of(cols)) |>
  names()

[1] "year"  "month" "day"

flights |>
  select(any_of(c(cols, "weight"))) |>
  names()

[1] "year"  "month" "day"

`across()`

across() brings tidy selection into the data masking verbs - it applies a function to every selected column inside mutate() or summarize().

penguins |>
  summarize(across(where(is.numeric), \(x) mean(x, na.rm = TRUE)))

# A tibble: 1 × 5
  bill_length_mm bill_depth_mm flipper_length_mm body_mass_g  year
           <dbl>         <dbl>             <dbl>       <dbl> <dbl>
1           43.9          17.2              201.       4202. 2008.

penguins |>
  summarize(across(
    ends_with("_g"),
    list(mean = \(x) mean(x, na.rm = TRUE), sd = \(x) sd(x, na.rm = TRUE))
  ))

# A tibble: 1 × 2
  body_mass_g_mean body_mass_g_sd
             <dbl>          <dbl>
1            4202.           802.

Grouped operations

Split-apply-combine

Most summaries follow the same pattern: split the rows into groups, apply a computation to each group, combine the results. group_by() records the grouping on the data frame and the verbs that follows respect the groups.

g = group_by(flights, origin, carrier)
class(g)

[1] "grouped_df" "tbl_df"     "tbl"        "data.frame"

g |> select(origin, carrier, dep_delay) |> head(n = 3)

# A tibble: 3 × 3
# Groups:   origin, carrier [3]
  origin carrier dep_delay
  <chr>  <chr>       <dbl>
1 EWR    UA              2
2 LGA    UA              4
3 JFK    AA              2

Grouped summaries

summarize() collapses each group to one row - any function that reduces a vector to a single value can be used, and multiple summaries and grouping variables can be combined.

flights |>
  filter(!is.na(dep_delay)) |>
  summarize(
    n = n(),
    delay = mean(dep_delay),
    .by = origin
  )

# A tibble: 3 × 3
  origin      n delay
  <chr>   <int> <dbl>
1 EWR    117596  15.1
2 LGA    101509  10.3
3 JFK    109416  12.1

flights |>
  group_by(origin, carrier) |>
  summarize(n = n()) |>
  arrange(desc(n))

# A tibble: 35 × 3
# Groups:   origin [3]
  origin carrier     n
  <chr>  <chr>   <int>
1 EWR    UA      46087
2 EWR    EV      43939
3 JFK    B6      42076
4 LGA    DL      23067
5 JFK    DL      20701
# ℹ 30 more rows

Grouped mutates and filters

mutate() and filter() within groups evaluate their expressions on each group’s columns separately - a summary such as sum() or max() becomes a per-group value that is recycled across that group’s rows.

flights |>
  count(origin, carrier) |>
  mutate(share = n / sum(n), .by = origin)

# A tibble: 35 × 4
  origin carrier     n   share
  <chr>  <chr>   <int>   <dbl>
1 EWR    9E       1268 0.0105 
2 EWR    AA       3487 0.0289 
3 EWR    AS        714 0.00591
4 EWR    B6       6557 0.0543 
5 EWR    DL       4342 0.0359 
# ℹ 30 more rows

flights |>
  filter(!is.na(dep_delay)) |>
  filter(
    dep_delay == max(dep_delay),
    .by = origin
  ) |>
  select(origin, carrier, dep_delay)

# A tibble: 3 × 3
  origin carrier dep_delay
  <chr>  <chr>       <dbl>
1 JFK    HA           1301
2 EWR    MQ           1126
3 LGA    DL            911

Example

Using nycflights13::flights,

How many flights to Los Angeles (LAX) did each of the legacy carriers (AA, UA, DL or US) have in May from JFK, and what was their average duration?
Which plane (check the tail numbers) flew out of each New York airport the most?
Which 5 days should you consider flying on if you want to have the lowest possible average departure delay?
Which flight has the largest arrival delay as a percentage of its scheduled air time?

Summary

Base R vs dplyr

Task	Base R	dplyr
refer to a column	`df$x` (standard evaluation)	`x` (data masking)
filter rows	`df[df$x > 1, ]`	`filter(df, x > 1)`
pick columns	`df[c("x", "y")]`	`select(df, x, y)`, `select(df, starts_with("x"))`
new column	`df$z = df$x * 2`	`mutate(df, z = x * 2)`
many columns at once	`lapply(df[cols], f)`	`mutate(df, across(all_of(cols), f))`
sort rows	`df[order(df$x), ]`	`arrange(df, x)`
summarize	`mean(df$x)`	`summarize(df, m = mean(x))`
grouped summary	`tapply(df$x, df$g, mean)`, `aggregate()`	`summarize(df, m = mean(x), .by = g)`
grouped transform	`ave(df$x, df$g)`	`mutate(df, m = mean(x), .by = g)`
column as vector	`df$x`, `df[["x"]]`	`pull(df, x)`

Takeaways

An R data frame is a list of equal length vectors with a class attribute - every table-like behavior is an S3 method
Tibbles are data frames with stricter, more predictable subsetting and construction, plus a better print method. Code written for data frames works on them via S3 dispatch.
Tables are stored by column because columns are what vectorized operations consume - filtering, transforming, and summarizing are all column-wise vector operations. Avoid row loops.
CSV is universal but has no types, parquet preserves the schema, compresses well, and reads a subset of columns cheaply. Use parquet for anything large or shared across languages.
dplyr verbs use data masking to evaluate expressions inside the data frame and tidy selection to describe sets of columns. Grouping turns the same vectorized expressions into per-group computations.