Data Exploring and Data Wrangling - NYCFlights13 Dataset

1 data Exploring and data Wrangling - NYCF lights13 DatasetVaibhav Walvekar# Load standard librarieslibrary(tidyverse)library(nycfl ights13)## Warning: package' NYCFlights13 'was built under R version and Inspecting data :# Get detials about NYCFlights13 Dataset ?nycflights13ls("package:nycfligh ts13")?flights# Load different data points from the NYCFlights13 libraryairlines_data <- airlinesairports_data <- airportsflights_data <- flightsplanes_data <- planesweather_data <- weather The NYCFlights13 Dataset is a collection of data pertaining to different airlines flying from differentairports in NYC, also capturing flight, plane and weather specific details during the year of data was collected into these five different branches. This method of collecting data helps usto work on individual aspects of the whole large Dataset and also we can combine together multipleaspects to do some complex data analysis. There are also 3-4 database versions of NYCFlights13 datasetwhich cache the data from NYCFlights13 database in a local database, helping in joining tables onnatural keys efficient.

2 The source of flights Dataset is RITA, Bureau of transportation statistics, variables in flights Dataset represent as below:# Variables in flights Dataset ?flightsyear,month,day - Date of departuredep_time,arr_time - Actual departure and arrival times, local ,sched_arr_time - Scheduled departure and arrival times, local ,arr_delay - Departure and arrival delays, in minutes. Negative times represent early depar- ,minute - Time of scheduled departure broken into hour and - Two letter carrier abbreviation. See airlines to get name1tailnum - Plane tail numberflight - Flight numberorigin,dest - Origin and destination. See airports for additional - Amount of time spent in the airdistance - Distance flowntime_hour - Scheduled date and hour of the flight as a POSIXct date. Along with origin, can be used to joinflights data to weather data .# Inspecting flights datasetsapply(flights_data, class)## $year## [1] "integer"#### $month## [1] "integer"#### $day## [1] "integer"#### $dep_time## [1] "integer"#### $sched_dep_time## [1] "integer"#### $dep_delay## [1] "numeric"#### $arr_time## [1] "integer"#### $sched_arr_time## [1] "integer"#### $arr_delay## [1] "numeric"#### $carrier## [1] "character"#### $flight## [1] "integer"#### $tailnum## [1] "character"#### $origin## [1] "character"#### $dest2## [1] "character"#### $air_time## [1] "numeric"#### $distance## [1] "numeric"#### $hour## [1] "numeric"#### $minute## [1] "numeric"#### $time_hour## [1] "POSIXct" "POSIXt"head(flights_data)## # A tibble.

3 6 19## year month day dep_time sched_dep_time dep_delay arr_time## <int> <int> <int> <int> <int> <dbl> <int>## 1 2013 1 1 517 515 2 830## 2 2013 1 1 533 529 4 850## 3 2013 1 1 542 540 2 923## 4 2013 1 1 544 545 -1 1004## 5 2013 1 1 554 600 -6 812## 6 2013 1 1 554 558 -4 740## # .. with 12 more variables: sched_arr_time <int>, arr_delay <dbl>,## # carrier <chr>, flight <int>, tailnum <chr>, origin <chr>, dest <chr>,## # air_time <dbl>, distance <dbl>, hour <dbl>, minute <dbl>,## # time_hour <dttm>tail(flights_data,5)## # A tibble: 5 19## year month day dep_time sched_dep_time dep_delay arr_time## <int> <int> <int> <int> <int> <dbl> <int>## 1 2013 9 30 NA 1455 NA NA## 2 2013 9 30 NA 2200 NA NA## 3 2013 9 30 NA 1210 NA NA## 4 2013 9 30 NA 1159 NA NA## 5 2013 9 30 NA 840 NA NA## #.

4 With 12 more variables: sched_arr_time <int>, arr_delay <dbl>,## # carrier <chr>, flight <int>, tailnum <chr>, origin <chr>, dest <chr>,## # air_time <dbl>, distance <dbl>, hour <dbl>, minute <dbl>,## # time_hour <dttm>flights_newdata <- flights_data[order(flights_data$month,flights_data$day),]tail(flights_newdata,5)## # A tibble: 5 193## year month day dep_time sched_dep_time dep_delay arr_time## <int> <int> <int> <int> <int> <dbl> <int>## 1 2013 12 31 NA 705 NA NA## 2 2013 12 31 NA 825 NA NA## 3 2013 12 31 NA 1615 NA NA## 4 2013 12 31 NA 600 NA NA## 5 2013 12 31 NA 830 NA NA## # .. with 12 more variables: sched_arr_time <int>, arr_delay <dbl>,## # carrier <chr>, flight <int>, tailnum <chr>, origin <chr>, dest <chr>,## # air_time <dbl>, distance <dbl>, hour <dbl>, minute <dbl>,## # time_hour <dttm>dim(flights_data)## [1] 336776 19summary(flights_data)## year month day dep_time## Min.

5 :2013 Min. : Min. : Min. : 1## 1st Qu.:2013 1st Qu.: 1st Qu.: 1st Qu.: 907## Median :2013 Median : Median Median :1401## Mean :2013 Mean : Mean Mean :1349## 3rd Qu.:2013 3rd Qu. 3rd Qu. 3rd Qu.:1744## Max. :2013 Max. Max. Max. :2400## NA's :8255## sched_dep_time dep_delay arr_time sched_arr_time## Min. : 106 Min. : Min. : 1 Min. : 1## 1st Qu.: 906 1st Qu.: 1st Qu.:1104 1st Qu.:1124## Median :1359 Median : Median :1535 Median :1556## Mean :1344 Mean : Mean :1502 Mean :1536## 3rd Qu.:1729 3rd Qu.: 3rd Qu.:1940 3rd Qu.:1945## Max. :2359 Max. Max. :2400 Max. :2359## NA's :8255 NA's :8713## arr_delay carrier flight tailnum## Min. : Length:336776 Min.

6 : 1 Length:336776## 1st Qu.: Class :character 1st Qu.: 553 Class :character## Median : Mode :character Median :1496 Mode :character## Mean : Mean :1972## 3rd Qu.: 3rd Qu.:3465## Max. Max. :8500## NA's :9430## origin dest air_time distance## Length:336776 Length:336776 Min. : Min. : 17## Class :character Class :character 1st Qu.: 1st Qu.: 502## Mode :character Mode :character Median Median : 872## Mean Mean :1040## 3rd Qu. 3rd Qu.:1389## Max. Max. :4983## NA's :9430## hour minute time_hour## Min. : Min. : Min. :2013-01-01 05:00:00## 1st Qu.: 1st Qu.

7 : 1st Qu.:2013-04-04 13:00:004## Median Median Median :2013-07-03 10:00:00## Mean Mean Mean :2013-07-03 05:02:36## 3rd Qu. 3rd Qu. 3rd Qu.:2013-10-01 07:00:00## Max. Max. Max. :2013-12-31 23:00:00##str(flights_data)## Classes'tbl_df','tbl'and' ': 336776 obs. of 19 variables:## $ year : int 2013 2013 2013 2013 2013 2013 2013 2013 2013 2013 ..## $ month : int 1 1 1 1 1 1 1 1 1 1 ..## $ day : int 1 1 1 1 1 1 1 1 1 1 ..## $ dep_time : int 517 533 542 544 554 554 555 557 557 558 ..## $ sched_dep_time: int 515 529 540 545 600 558 600 600 600 600 ..## $ dep_delay : num 2 4 2 -1 -6 -4 -5 -3 -3 -2 ..## $ arr_time : int 830 850 923 1004 812 740 913 709 838 753 ..## $ sched_arr_time: int 819 830 850 1022 837 728 854 723 846 745 ..## $ arr_delay : num 11 20 33 -18 -25 12 19 -14 -8 8 ..## $ carrier : chr "UA" "UA" "AA" "B6" ..## $ flight : int 1545 1714 1141 725 461 1696 507 5708 79 301.

8 ## $ tailnum : chr "N14228" "N24211" "N619AA" "N804JB" ..## $ origin : chr "EWR" "LGA" "JFK" "JFK" ..## $ dest : chr "IAH" "IAH" "MIA" "BQN" ..## $ air_time : num 227 227 160 183 116 150 158 53 140 138 ..## $ distance : num 1400 1416 1089 1576 762 ..## $ hour : num 5 5 5 5 6 5 6 6 6 6 ..## $ minute : num 15 29 40 45 0 58 0 0 0 0 ..## $ time_hour : POSIXct, format: "2013-01-01 05:00:00" "2013-01-01 05:00:00" ..unique(flights_data$carrier)## [1] "UA" "AA" "B6" "DL" "EV" "MQ" "US" "WN" "VX" "FL" "AS" "9E" "F9" "HA"## [15] "YV" "OO"length(unique(flights_data$carrier)) ## [1] 16unique(flights_data$origin)## [1] "EWR" "LGA" "JFK"length(unique(flights_data$origin)) ## [1] 3unique(flights_data$dest)5## [1] "IAH" "MIA" "BQN" "ATL" "ORD" "FLL" "IAD" "MCO" "PBI" "TPA" "LAX"## [12] "SFO" "DFW" "BOS" "LAS" "MSP" "DTW" "RSW" "SJU" "PHX" "BWI" "CLT"## [23] "BUF" "DEN" "SNA" "MSY" "SLC" "XNA" "MKE" "SEA" "ROC" "SYR" "SRQ"## [34] "RDU" "CMH" "JAX" "CHS" "MEM" "PIT" "SAN" "DCA" "CLE" "STL" "MYR"## [45] "JAC" "MDW" "HNL" "BNA" "AUS" "BTV" "PHL" "STT" "EGE" "AVL" "PWM"## [56] "IND" "SAV" "CAK" "HOU" "LGB" "DAY" "ALB" "BDL" "MHT" "MSN" "GSO"## [67] "CVG" "BUR" "RIC" "GSP" "GRR" "MCI" "ORF" "SAT" "SDF" "PDX" "SJC"## [78] "OMA" "CRW" "OAK" "SMF" "TUL" "TYS" "OKC" "PVD" "DSM" "PSE" "BHM"## [89] "CAE" "HDN" "BZN" "MTJ" "EYW" "PSP" "ACK" "BGR" "ABQ" "ILM" "MVY"## [100] "SBN" "LEX" "CHO" "TVC" "ANC" "LGA"length(unique(flights_data$dest))## [1] 105# Number of departures getting cancelledsum( (flights_data$dep_time))

9 ## [1] 8255 After basic inspection of the Dataset we can understand that flights Dataset has 19 different variableswith 336776 rows. Inspecting the head of the Dataset , we understand that there are flights ariving anddeparting on the same day or either just arriving or departing on a given day. As from the tail of thedataset we can see that data isnt sorted, thus I have sorted the Dataset based on the month and theday of the year. From furhter inspection we can find that there are 16 different carriers flying out ofNYC airports. NYC has 3 different airports. There are 105 different destination locations to whichflights fly out of NYC airports. 8255 flights departures were cancelled as the data has Questions:1. Is there any particular trend of delays at all the airports or is it randomized? I think this question is interesting as it will help us understand if there are any particular pattern aboutthe delays. By knowing the delay pattern we can try to address the systemic causes for such delays.

10 Ifthere is no pattern we can atleast identify about some anomaly that at once caused a delay. We canalso gauge the performance of airports across the 12 months. I plan to answer this question firstly by filtering out relevant data . Using this filtered data , I will groupby month for count of delayed flights, which will help us know about any particular trends. This graphicwill also help us understand the absolute numbers with regards to delays. I also plan to understandpercent of delays per month for 2013 for each of the airports. From this visual we can make directcomparisons betweeen performances of airports across months. So I will filter out data according to departures from particular airports. Further, remove data aboutthe cancelled flights and also the flights that didnt have any delay. Thus the data will be split intothree different sets (3 airports) with only details about flights which were delayed. Further by groupingon monthly basis and taking the count of delays we can plot for all three airports and see the trends.