Any municipality that experiences harsh winters has at least one common enemy — potholes. Purdue University is working with West Lafayette to help identify this problem.

“Potholes are a very significant issue in many municipalities, particularly in West Lafayette Ind., because we have a harsh winter,” Mohammad R. Jahanshahi, the Purdue University assistant professor working on the project, stated.

One-third of the roads are in mediocre condition, and the poor condition of the roads is responsible for approximately one-third of traffic fatalities. Each year U.S. motorists pay, on average, $67 billion in vehicle repairs from potholes and road conditions.

Pothole detection methods

Currently there have only been limited options for cities regarding pothole detection.

Individual drivers are able to report potholes. The most common method is a manual, visual inspection by professionals; however, this can be subjective, inefficient and time consuming. A van with high-resolution systems and sensors can be purchased to do the job. However, the vans are extremely expensive and still lack full automation.

With continuing advances in sensor technology and data analysis, Purdue University decided to attempt to develop a cost-effective pothole detection system that could be put on multiple vehicles. This would make the collection of data more efficient.

The challenges would be finding inexpensive hardware that could be connected to a system that would be able to handle such large amounts of data.

Jahanshahi had previously published papers on using an Xbox Kinect for such purposes. Since they are mass produced for gaming systems, they are reasonably priced. They are also able to judge distance and color similarly to a camera. “Kinect is a motion tracking sensor. A device that connects to an Xbox game console that tracks your motions basically,” he explained.

For pothole detection, the Purdue University EPICS group began the project by using “it as a device to scan the road surface, and when you have potholes, there would be a depression. So if we would develop algorithms to identify dual depressions, we could be able to detect and quantify potholes,” Jahanshahi illustrated. The Kinect would then connect with a GPS so that the location and severity of each pothole could be easily identified.

Issues and updates

Having worked on this project for three semesters, the group has encountered some issues, including difficulty with synchronization.

Jahanshahi described the workaround as “students looking at different microprocessors to see which one is compatible with Kinect.” Another challenge was coverage, with students soon discovering that more than one Kinect may be needed on the vehicle to capture an entire lane.

Since Kinect’s use of infrared was made for indoor use, EPICS discovered that direct sunlight would interfere and distort the data.

“Students were working to put together a system that can provide shade for the prototype because Kinect was made for indoors,” Jahanshahi said. “They would make sure there wouldn’t be any interference from sunlight.”

This work led to the design and creation of a box that would be able to block sunlight and provide a connection between the sensor system and the car. This method was first tested indoors with artificial potholes made from cardboard to determine the Kinect’s ability to measure depth. The box system was then able to be placed on  the back of a car to collect data. Testing was successful and there is currently a patent for the system.

Students have also looked into the design of garbage trucks to identify a location on the truck that would provide shade. “For the city of West Lafayette, the idea was to use this system on garbage trucks for the first phase, and once it is successful, we would be able to put this system on vehicles,” Jahanshahi said. Placed on vehicles, the data collection would be similar to crowdsourcing pothole information.

A newer version of Kinect has also been chosen for better data collection with its higher resolution and better quality. It is able to capture videos at 60 frames per second. Therefore, the depth information on the potholes is determined by the distance between each pixel and the sensor.

Dividing up the work

After those three semesters, students decided to split up the work between two Purdue University groups — EPICS and Vertically Integrated Projects, an electrical engineering group. V.I.P. will focus on work with the pothole detection hardware and software. Several sensors and processing units have been tested so they hope to use the new sensor to improve data analysis and optimize data collection this semester.

EPICS students will focus their work on developing and updating a phone app using GPS for citizens to take pictures of potholes, which will send the information to servers that the city will be able to access. Jahanshahi considered this a more efficient reporting method since it can be done quickly and will include both photos and a more precise the location.

They have also been asked to include more features, such as the ability to report other city issues like a street light being out.

“The ultimate goal is to have the system being tested for actual applications,” he continued. However, he admitted it will be a few more semesters before it is ready for use. The initial testing will be done on the students’ vehicles.

As part of their midterm review, students invited West Lafayette city officials to hear presentations on their project ideas and ask questions. In order to better meet the needs of the city, they worked with a city engineer, Marcus Smith. Eventually, the long-term goal is to expand this idea to other cities and even to the Indiana Department of Transportation.

“The objective here is to first make sure we can collect data and process it and get the results that we want in collaboration with the city of West Lafayette at lower speeds,” Jahanshahi clarified. “Once it is successful and long term, we want to integrate this system and work with INDOT so that we will be able to evaluate the surface of highways and roadways.” Once students reach this level, the greatest problem will be finding a way to deal with the large amounts of data that will come in.