Big Data Backbone for Advanced Analytics

Abstract

Our sponsor, Union Pacific (UP), is the largest freight-hauling railroad company in the world, operating 8,300 locomotives over 32,200 miles of rail track in 23 U.S. states. They own or lease approximately 18,000 railcars. UP has several data pipelines that process messages on the activity of their railcars into the Main Equipment Event Table. The Main Equipment Event Table contains all of UP’s railcar event data. The Finance Team within UP uses the data in this table to audit revenue and find missing billings. Currently, UP doesn’t receive messages on the activity of their railcars after they move onto a different railroad company’s rail tracks, called going “offline”. This lack of offline visibility hinders the efficiency of revenue auditing because the Finance Team has to manually search Railinc, the provider of rail data to the North American railroad industry, for missing billings and revenue. The goal of the UP MDP Cohort of 2022 is to help solve this problem by providing UP with a more complete picture of their railcars’ activity, including when it goes offline. To accomplish our goal, our solution strategy is to bring more data pertaining to offline railcar activity from Railinc into UP’s database. To implement this solution strategy, the main objective of our team will be to build a new pipeline capable of automatically extracting, transforming, and loading (ETL) approximately 6-10 million offline messages from Railinc per day. We will be working specifically with SWRPY87 messages, which are a type of Railcar Tracing (RCT) message that detail the offline activity of a railcar, including railcar location, timestamps, on road, to the road, and other railcar-specific data. Another objective of our team is to produce a data analytics report complete with measurable numbers that show the benefits of the SWRPY87 messages and their ability to increase railcar visibility.