Analysis

Goals

Our goal is to analyze daily weather statistics like high/low temperatures and rainfall in downtown Austin using data from the Camp Mabry station.

Setup

library(tidyverse)
library(janitor)

Import cleaned data

Here, I am importing the cleaned weather data and using glimpse() to look at a quick overview of the data.

# Importing the clean data
weather <- read_rds("data-processed/01-weather.rds")

weather |> glimpse()

Rows: 31,522
Columns: 6
$ date <date> 1938-06-01, 1938-06-02, 1938-06-03, 1938-06-04, 1938-06-05, 1938…
$ prcp <dbl> 0.00, 0.00, 0.00, 0.40, 0.02, 0.00, 0.00, 0.00, 1.60, 0.01, 0.00,…
$ snow <dbl> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
$ snwd <dbl> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
$ tmax <dbl> 91, 94, 94, 90, 94, 92, 95, 92, 87, 90, 92, 91, 91, 91, 89, 89, 9…
$ tmin <dbl> 72, 67, 70, 68, 68, 70, 70, 76, 64, 76, 75, 71, 70, 68, 71, 70, 7…

I will print summary stats of my data.

# Printing summary
weather |> 
  summary()

      date                 prcp              snow               snwd         
 Min.   :1938-06-01   Min.   :0.00000   Min.   :0.000000   Min.   :0.000000  
 1st Qu.:1959-12-28   1st Qu.:0.00000   1st Qu.:0.000000   1st Qu.:0.000000  
 Median :1981-07-25   Median :0.00000   Median :0.000000   Median :0.000000  
 Mean   :1981-07-25   Mean   :0.09111   Mean   :0.002053   Mean   :0.002538  
 3rd Qu.:2003-02-20   3rd Qu.:0.00000   3rd Qu.:0.000000   3rd Qu.:0.000000  
 Max.   :2024-09-18   Max.   :7.55000   Max.   :6.500000   Max.   :6.000000  
                                        NA's   :7          NA's   :4         
      tmax            tmin      
 Min.   : 20.0   Min.   :-2.00  
 1st Qu.: 70.0   1st Qu.:47.00  
 Median : 82.0   Median :61.00  
 Mean   : 79.6   Mean   :58.45  
 3rd Qu.: 92.0   3rd Qu.:72.00  
 Max.   :112.0   Max.   :93.00  
                                

Check dates

I will remove incomplete years.

First, I am looking at a summary of my data.

weather$date |> summary()

        Min.      1st Qu.       Median         Mean      3rd Qu.         Max. 
"1938-06-01" "1959-12-28" "1981-07-25" "1981-07-25" "2003-02-20" "2024-09-18" 

Filter

I am filtering out 1938 and 2024 because they have incomplete data.

weather_clipped <- weather |>
  filter(year(date) > 1938, year(date) < 2024)

Date helpers

I am creating date helpers.

weather_dates <- weather_clipped |>
  mutate(
    yr = year(date),
    mo = month(date, label = TRUE),
    yday = yday(date)
  )

weather_dates

Summarize with count

In this section, I will analyze daily weather statistics in downtown Austin using the data from the Camp Mabry station. I will find the days with the most rain, the hottest and coldest days in history and which years had the most days of 100+ temperature.

Main quests

Here, I will find the answers to the following questions:

• Which days had the most rain, and how much rain are we talking about?
• What are the hottest and coldest days in history? This can be two different lists … one of hottest days and one of coldest.
• Which years had the most days of 100+ temperature? i.e., group by year and count the number of days with 100+ temps.

Most rain

Which days had the most rain, and how much rain are we talking about?

Here, I am selecting the dates and precipitation (prcp) values and sorting it in descending order to show the 20 days with the most rain.

# Dates with the most rain in downtown Austin 
weather_dates |>
  select(
    date,
    prcp
  ) |>
  arrange(prcp |> desc()) |>
  filter(prcp >= 5)

Data Takeaway: The date with the most rain in downtown Austin was Nov. 15, 2001, with 7.55 inches. Sept. 7, 2010 had the second most amount of rain with 7.04 inches and Oct. 17, 1998 had 6.24 inches of rain, making it the date with the third most amount of rain.

Hottest & coldest days

What was the hottest day in history?

Here, I am selecting the dates and maximum temperatures (tmax), arranging them in descending order and displaying the 15 hottest days in downtown Austin.

# The hottest dates in downtown Austin 
weather_dates |>
  select(
    date,
    tmax
  ) |> 
  arrange(tmax |> desc()) |>
  filter(tmax >= 108)

Data Takeaway: The hottest days in downtown Austin were Sept. 5, 2000 and Aug. 28, 2011 at 112 degrees.

What was the coldest day in history?

Here, I am selecting the dates and minimum temperatures (tmin), arranging them in ascending order and displaying the 15 coldest days recorded in downtown Austin.

# The coldest dates in downtown Austin 
weather_dates |>
  select(
    date,
    tmin
  ) |> 
  arrange(tmin) |>
  filter(tmin <= 10)

Data Takeaway: The coldest day in downtown Austin was Jan. 31, 1949 at minus 2 degrees.

Days of 100+ temperature

Which years had the most days of 100+ temperature?

Here, I am grouping the data by year, filtering it to show days with temperatures of 100 degrees or higher and using summarize() to show how many of those days occurred in each year. Then, I am arranging it in descending order.

# Years with the most days of 100+ temperature
weather_100 <- weather_dates |>
  filter(
    tmax >= 100
  ) |>
  group_by(yr) |>
  summarize(days_over_100 = n()) |>
  arrange(days_over_100 |> desc()) |>
  filter(days_over_100 >= 50)

weather_100

Data Takeaway: The year 2011 had most days with a temperature of 100 degrees and above with a total of 90 days. The year 2023 had the second highest amount with 80 days, and 2009 tied with 2022 for third place with 68 days.

Bonus quests

• What is the total number of days within each year that had any snow?
• What is the total number of days within each year below freezing (32 or below)?
• What is the total number of days within each year to never get above freezing (not above 32)?
• What is the total number of days within each year that reached 100 or higher in the month of May?

Snow

What is the total number of days within each year that had any snow?

Here, I am grouping the data by year, filtering it to show days with snow and using summarize() to show how many of those days occurred in each year. Then, I am arranging it in descending order.

# Total number of days within each year that had any snow
weather_dates |>
  filter(
    snow > 0
  ) |>
  group_by(yr) |>
  summarize(snow_days = n()) |>
  arrange(snow_days |> desc())

Data Takeaway: The year 1973 had the most days with snowfall at 5 days. The year 1985 had 4 days and 2021 had 3 days. There have been 30 years with snowfall since June 1, 1938.

Days below freezing

Total number of days within each year below freezing (32 or below).

Here, I am grouping the data by year, filtering it to show days where the minimum temperature was at or below 32 degrees and using summarize() to show how many of those days occurred in each year. Then, I am arranging it in descending order.

# Total number of days within each year below freezing
weather_dates |>
  filter(
    tmin <= 32
  ) |>
  group_by(yr) |>
  summarize(days_below_freezing = n()) |>
  arrange(days_below_freezing |> desc())

Data Takeaway: The year 1948 had the most days below freezing at 41 days. The year 1978 had 39 days and 1940 had 35 days.

Days to stay below freezing

Total number of days within each year to never get above freezing (not above 32).

Here, I am grouping the data by year, filtering it to show days where both the minimum and maximum temperatures stayed at or below 32 degrees and using summarize() to show how many of those days occurred in each year. Then, I am arranging it in descending order.

# Total number of days within each year to never get above freezing
weather_dates |>
  filter(
    tmax <= 32
  ) |>
  group_by(yr) |>
  summarize(freezing_days = n()) |>
  arrange(freezing_days |> desc())

Data Takeaway: The year 1983 had the most days to never get above freezing at 6 days. The years 1985, 1989 and 2021 each had 5 days never get above freezing.

Days to reach 100+ in May

Total number of days within each year that reached 100 or higher in the month of May.

Here, I am grouping the data by year, filtering it to only show days in May where the temperature reached 100 degrees or higher and using summarize() to show how many of those days occurred in each year. Then, I am arranging it in descending order.

# Total number of days within each year that reached 100 or higher in the month of May
weather_dates |>
  filter(
    tmax >= 100,
    month(date) == 5
  ) |>
  group_by(yr) |>
  summarize(hot_may_days = n()) |>
  arrange(hot_may_days |> desc())

Data Takeaway: The year 2011 had the most days that reached 100 degrees or higher in the month of May at 3 days. The years 1984 and 2008 each had 2 days. Additionally, the years 1998, 2003, 2004 and 2022 each had one day that reached 100 degrees or higher in the month of May.

Summarize with math

Does the poem line “Sweet April showers Do spring May flowers” mean April is typically the month with the most rain? We’ll answer this question and more.

Our goal in this next section is to continue to analyze daily weather statistics with math in downtown Austin using data from the Camp Mabry station.

Main quests

Here, I will find the answers to the following questions:

• Which years had the most total rainfall and how much? Which had the least?
• Which years had the most total snowfall and how much?
• Calculate the “average high temperature” and “average low temperature” for each year.
• Since 1990, how much rain do we typically get in January, February, etc. In other words, what is the average rainfall within each month. Logically you have work through this in two steps:
  • Get the total rain for each year/month, like 2.77 total inches in January 1939. To do that you can group by month and year and then calculate the total rain. You should end up with columns for year, month and total rain. And then …
  • … take that result and group by month and summarize to get the mean of the total rain in for each month, so you have an average of all the January’s, etc.

Rainfall

Which years had the most total rainfall and how much? Which had the least?

Here, I am analyzing which years had the most total rainfall. I am grouping the data by year, using summarize() to calculate the total precipitation (prcp) in each year, then arranging the results in descending order to show the years with the highest rainfall at the top.

# Years that had the most total rainfall
weather_dates |>
  group_by(yr) |>
  summarize(yearly_rain = sum(prcp)) |> 
  arrange(yearly_rain |> desc()) |>
  filter(yearly_rain >= 40)

Data Takeaway: Austin saw the most rain in 2015, when nearly 60 inches fell in the city. The second highest year was 2004 with 52.27 inches, followed by 1991 with 52.21 inches of rain.

Now, I am analyzing which years had the least total rainfall. I am grouping the data by year, calculating the total precipitation (prcp) for each year using summarize(), and then arranging the results in ascending order to show the years with the least rainfall at the top.

# Years that had the least total rainfall
weather_dates |>
  group_by(yr) |>
  summarize(yearly_rain = sum(prcp)) |> 
  arrange(yearly_rain)

Data Takeaway: Austin saw the least total rainfall in 1954 with just 11.42 inches. The second lowest year was 1956 with 15.41 inches, followed by 2008 with 16.07 inches of rain.

Snowfall

Which years had the most total snowfall and how much?

Here, I am analyzing which years had the most total snowfall. I am grouping the data by year and using summarize() to calculate the total snowfall (snow) for each year. Then, I am arranging the results in descending order to display the years with the highest snowfall amounts at the top.

# Years that had the most total snowfall
weather_dates |>
  group_by(yr) |>
  summarize(yearly_snow = sum(snow)) |> 
  arrange(yearly_snow |> desc()) |>
  filter(yearly_snow >= 2)

Data Takeaway: Austin saw the most total snowfall in 1985 with 8.7 inches. The second highest was 2021 with 7.9 inches, followed by 1944 with 7.0 inches of snow. Austin has only had 11 years with 2 or more inches of snow since 1938.

Average highs and lows

Calculate the “average high temperature” and “average low temperature” for each year.

Here, I am calculating the “average high temperature” for each year. I first group the data by year, then use the mean() function to calculate the average maximum temperature (tmax) for each year. Then I rounded to two decimal places and arranged the years in descending order to display the years with the highest average temperatures first.

# Average high temperature for each year
weather_dates |>
  group_by(yr) |>
  summarize(avg_high = round(mean(tmax),2)) |> 
  arrange(avg_high |> desc())

Data Takeaway: Austin saw the highest average high temperature at 84.06 degrees in 2011. The year 2023 follows with an average high of 83.48 degrees, and 2022 had the third highest at 83.08 degrees.

Here, I am calculating the “average low temperature” for each year. I first group the data by year, then use the mean() function to calculate the average minimum temperature (tmin) for each year. Then I rounded to two decimal places and arranged the years in ascending order to display the years with the lowest average temperatures first.

# Average low temperature for each year
weather_dates |>
  group_by(yr) |>
  summarize(avg_low = round(mean(tmin),2)) |> 
  arrange(avg_low)

Data Takeaway: Austin saw the lowest average high temperature in 1940 at an average of 54.31 degrees. The year 1976 follows with an average low of 56.05 degrees, and 1968 had the third lowest at 56.14 degrees.

Average monthly rain

Since 1990, how much rain do we typically get in January, February, etc. In other words, what is the average rainfall within each month?

Get the total rain for each year/month, like 2.77 total inches in January 1939. To do that I will group by month and year and then calculate the total rain.

# Total rainfall for each year and month
weather_avg_mo <- weather_dates |>
  group_by(mo, yr) |>
  summarise(mo_yr_rain = sum(prcp)) |>
  group_by(mo) |>
  summarise(avg_mo_rain = mean(mo_yr_rain))

`summarise()` has grouped output by 'mo'. You can override using the `.groups`
argument.

weather_avg_mo

Data Takeaway: May has the highest average rainfall with 4.54 inches. July has the lowest average rainfall at 1.96 inches.

Bonus quests

Bonus:

• What is the “average yearly temperature” for each year?
  • This is defined as the average of both the “average high temperature” and “average low temperature” you calculated above, for each year.
• What is the earliest date in each year with 100+ temperature?
  • Which year had the earliest date? (This is where yday and slice prove useful).
• What is the latest date (but before July 1) for a freeze each year?
  • Which year had the latest freeze date?

Average yearly temperature

What is the “average yearly temperature” for each year?

This is defined as the average of both the “average high temperature” and “average low temperature.”

# Average yearly temperature for each year
weather_dates |>
  group_by(yr) |>
  summarise(
    avg_high = mean(tmax),
    avg_low = mean(tmin),
    avg_yr_temp = sum(avg_high + avg_low) / 2
    ) |>
  arrange(avg_yr_temp |> desc())

Data Takeaway: Austin saw the highest average temperature in 2023 at about 72 degrees.

Earliest 100

What is the earliest date in each year with 100+ temperature? Which year had the earliest date? (This is where yday and slice prove useful).

# Earliest 100+ day
weather_dates |>
  filter(tmax >= 100) |>
  select(date, tmax, yr, yday) |>
  group_by(yr) |>
  slice_min(date) |>
  arrange(yday)

Data Takeaway: Austin saw the earliest day to reach 100 degrees on May, 4, 1984.

First Freeze

What is the latest date (but before July 1) for a freeze each year? Which year had the latest freeze date?

# Latest first freeze
weather_dates |>
  filter(
    tmin <= 32,
    mo > "Jun"
    ) |>
  group_by(yr) |>
  slice_min(date) |>
  arrange(yday)

Data Takeaway: The earliest date to freeze in Austin in one calendar year was October 28, 1957.

Visualizations

Here, I will find the answers to the following questions:

• In the summarize with count assignment you found the years with the most 100+ days. Plot the top years as a horizontal column/bar chart. Order the bars so the most days are at the top.
• In the summarize with math assignment you found the average rainfall by month. Plot this as a vertical column chart.
• Create a line chart that shows the yearly average low and high temperatures. Exclude any partial years (i.e. remove 1938 and probably your most current year.) In your summarize with math assignment you found the average high and average low for each year. You’ll use this data to make your line chart. It’s easiest to chart this if you have both the average high and average low in the same data frame. If you have the separate, you can either rebuild it, or bind your dataframes together.

Horizontal bar chart

In the summarize with count assignment you found the years with the most 100+ days. Plot the top years as a horizontal column/bar chart. Order the bars so the most days are at the top.

I will prepare the data to put into a bar chart.

# Preparing the data for the bar chart
top_years <- weather_100 |>
  arrange(days_over_100) |>
  mutate(yr = factor(yr, levels = yr))

I am creating the bar chart.

# Creating the bar chart
top_years |>
  ggplot(aes(x = yr, y = days_over_100)) +
  geom_col(fill = "steelblue") +
  geom_text(aes(label = days_over_100),
            vjust = 0.5,
            hjust = -0.2,
            color = "black") +
  coord_flip() + 
  labs(title = "Top Years with Most 100+ Degree Days",
       subtitle = str_wrap("Data from the Camp Mabry weather station shows that 2011 had the most days with 100+ degree weather."),
    caption = "By Gabriella Gonzales. Source: Climate Data Online",
       x = "Year",
       y = "Days Over 100°F") +
  theme_minimal()

Vertical column chart

In the summarize with math assignment you found the average rainfall by month. Plot this as a vertical column chart.

I will prepare the data and put it into a vertical column chart.

# Creating the vertical column chart
weather_avg_mo |>
  ggplot(aes(x = mo, y = avg_mo_rain)) +
  geom_col(fill = "steelblue") +
  geom_text(aes(label = round(avg_mo_rain, 1)), 
            vjust = 1.5, 
            color = "white") +
  labs(title = "Average Monthly Rainfall",
       subtitle = str_wrap("Data from the Camp Mabry weather station shows that on average, Austin gets the most rainfall in May and the lowest in January and July."),
    caption = "By Gabriella Gonzales. Source: Climate Data Online",
       x = "Month",
       y = "Average Rainfall (inches)") +
  theme_minimal()

Line chart

Create a line chart that shows the yearly average low and high temperatures.

Exclude any partial years (i.e. remove 1938 and probably your most current year.) In your summarize with math assignment you found the average high and average low for each year. You’ll use this data to make your line chart. It’s easiest to chart this if you have both the average high and average low in the same data frame. If you have the separate, you can either rebuild it, or bind your dataframes together.

I will combine average high and low temperatures into a single data frame.

# Combine average high and low temperatures into a single data frame
avg_temps <- weather_dates |>
  group_by(yr) |>
  summarize(
    avg_high = round(mean(tmax), 2),
    avg_low = round(mean(tmin), 2)
  )

avg_temps

I will pivot longer.

# Pivot longer
avg_temps_long <- avg_temps |>
  pivot_longer(
    cols = c(avg_high, avg_low),
    names_to = "temperature_type",
    values_to = "temperature")

avg_temps_long

Now, I will create my line chart.

# Creating the line chart
avg_temps_long |>
  ggplot(aes(x = yr, y = temperature, group = temperature_type)) +
  geom_line(aes(color = temperature_type)) +
  geom_point(aes(color = temperature_type)) +
  labs(
    title = "Austin average high and low temperatures on the rise",
    subtitle = str_wrap("Data from the Camp Mabry weather station shows rising average high and low temperatures in Austin."),
    caption = "By Gabriella Gonzales. Source: Climate Data Online",
    x = "Year",
    y = "Temperature (°F)",
    color = "Temperature Type"
    )  +
  scale_color_manual(
    values = c("avg_high" = "indianred", "avg_low" = "skyblue"),
    labels = c("Average High", "Average Low"))