A plane is more than a machine that takes you from one city to another. Modern planes are data centers in their own right. Airlines have been using big data in their analysis almost since the days of the Wright Brothers. They are not the only ones, passengers have gotten savvy about flights using big data for their own own itineraries as well, even if they do not yet realize it.
That makes airline and flight plan data a great place to dive in and practice data analysis whether you’re an expert or starting out. From flight plans, to weather, to route planning, to ticket buying. Let’s look at the ways big data has shaped and is changing today’s airline industry.
Mapping Every Flight
Ever sat on a plane and watched the animation of the plane you are in flying across the map? Or checked in on a flight before driving to the airport to pick up someone? Then you’ve been using live and streaming data.
The Federal Aviation Administration requires flight plans for Commercial and international flights. Those flight plans form the backbone of a number of services that provide flight tracking information.
That has led to an explosion of sites that use the database provided by the FAA to track flights of all kinds. Aviation enthusiasts might want to track their favorite planes, airline industry researchers can see what routes are popular, and anyone curious on the exact route of their next trip can see what paths previous flights took. Some may even think they can help if they notice a break in a pattern that would signal a plane in distress.
One of the biggest sites for tracking flights is FlightAware. Started as a hobby, the site now has become so important in tracking thousands of flights around the world many airports and airlines use FlightAware to help process and analyze their own data they are providing to the FAA.
On FlightAware you can search by flight or airport and see active flights or view historical data. It will show exact paths from airport to airport taken by particular flights, routes, aircraft used, and other data. That functionality has led to a FlightAware community that builds what FlightAware started by finding new ways to analyze data.
That includes an API known as FlightXML that examines historical data. And Firehose, which tracks live data and identifies planes that may be above you right now.
FlightAware is not the only flight tracker out there. You can find other services that suits you to get started on your own tools or visualizations.
Weather or not
Aviation has been using big data for a long time but that’s nothing compared to meteorologists have been working with big data for much, much longer. And since flights are incredibly sensitive to weather ,knowing what the weather holds can lead to fewer grounded flights, costly delays, and even danger.
Recent efforts have gone into improving forecasting in order to stave off some of the worst effects of bad weather. If you’ve flown in days before a big blizzard recently you may have noticed airlines canceling flights rather than delaying them, even if you are not in the affected area.
While that can be frustrating to the passenger stuck in an airport like Minneapolis or Boston that can also save passengers in other cities from facing horrible delays because planes they would have flown in are grounded in a different city.
That ultimately cuts down on delays after a major event as well. Instead of postponing flights again and again, a new schedule can ensure planes are back in the air as soon as possible.
Bigger and better data-sets in meteorology itself has led to better predictions of weather events before they occur. That means safer flights since storms or blizzards are spotted earlier. This gives more time to passengers who need to make alternative plans.
Michigan professor Amy Cohn describes it as letting airlines be proactive about the weather and the changes it may force rather than adjusting plans on the fly (no pun intended). That allows airlines to move resources and have them in place so they can work instead of being stuck elsewhere.
Big data allows that to happen by combining two normally disparate sources of data (weather forecasts and flight plans) and seeing where they can intersect. This seems obvious in retrospect but the ability to match different disciplines is a skill on its own (more on our blog about interdisciplinary skills here).
Getting the Right Routes
With the aviation industry deregulated, airlines turned to data to map out their own flights. That is what drove airlines to adopt its hub and spoke model compared to when flights had to be approved by national flight boards who preferred more direct flights.
That also led to the rise of new carriers like Emirates and other middle eastern national airlines. Those airlines took advantage of the airspace above their respective nations. Located at the crossroads between Europe, North America, and Asia it has contributed to the explosive growth of those gulf states’ airline industries.
The same goes for low cost carriers who are providing lower cost options for travelers. This is even done with transatlantic flights with airlines IcelandAir and Wow! Airlines using Reykjavik as a hub between dozens of US and European cities, allowing people to cross the pond for a fraction of the cost of a major carrier ticket.
Getting the Golden Ticket
Okay, so that is all good information but what if you don’t work in the aviation industry or have an interest in planes? Passengers enjoy data analysis as well to find the best flight or cheapest ticket to where they want to go.
The days of going to a ticket counter at the airport and booking a flight direct from the airline are over. Ticket purchasing and check-in are being automated more and more as well turning flying into a totally digital experience except for the flight itself.
The first wave of innovation of ticket buying was the explosion of third party travel booking aggregators. Priceline, Hotwire, or Kayak can all search several airline itineraries at once. But airlines came back by reserving their lowest fares for direct bookings or even keeping their information away from aggregators entirely like Southwest airlines.
That has led some consumers to go back to the source. Combining flight trackers with ticket information from new tools like ITA Flight Matrix make it easy for anyone to research what flights are available across multiples airlines. Customers can then book the tickets direct with that airline.
But that hasn’t stopped others from working hard to finding the best prices. Sometimes described as “flight hacks”, these methods are really just different examples of using analytics to find flight options that may not be immediately clear during a regular ticket booking.
One example is hidden-city ticketing. Hidden-city ticketing is one way to save money by buying a ticket to one city leaving the airport you transfer planes. If there is a sale on flights to Omaha with a layover in Chicago then you could book that trip to get to Chicago. If you’re willing to travel light with only a carry-on then two one way flights using hidden city ticketing could be cheaper than a round trip flight.
Another option that is more straightforward is booking tickets fast. Keeping an eye out for sudden price drops occur or watching out for mistakes fares. If you can be flexible on when you fly then you can find deals. That’s the philosophy behind a service like Scott’s Cheap Flights. Scott Keyes started the site (and newsletter) after finding some great deals on his own and sharing the results.
That is the same impetus behind the founding of FlightAware. In both cases, the sites have exploded with more and more people taking advantage. But the key is that it is all done with data that is public.
Scott Keyes would be the first to tell you that he learns most of his deals by simply searching google flights and seeing what is available. The work is not always automatic but when you can combine a good database with the critical thinking skills that Scott has you can get great results. Those results speak for themselves in the case of Scott’s Cheap Flights which has seen meteoric growth since its start a few years ago.
In most of those cases the data has always been there in one form or another. The explosion of data science has led to sweeping changes across the industry. But the basics down is as important and helpful as more sophisticated big data applications.
Some of those are being explored today and may be commonplace as well. Big data may make it easier to keep track of your luggage. That’s the hope of Delta Airlines who introduced methods to track bags faster and more accurately.
Better ticket deals may be coming to some passengers as data trends that are common in other e-commerce marketplaces come to airlines. Look at a trip but decide it is not worth the money? You may get an email offering 10-20 percent off that ticket price to entice you to fly instead of gassing up the car. That’s the promise of personalized pricing which could be coming to an airline near you.
You may have heard that data drives decisions decisions but it looks like data also flies as well. That should give you plenty to think about on your next long flight.