How MSU Faculty and Students Build the Future of Autonomous Vehicles
Advances in autonomous vehicles are unlocking possibilities that would have seemed incredible just a few years earlier. Residents and visitors in Arlington, Texas can book a self-driving hybrid Lexus RX to travel around downtown. Companies like Waymo — the autonomous-driving subsidiary of Google’s parent company, Alphabet — have tested vehicles guided by cameras, radar and lidar sensors on highways. But transforming the ways we travel on a large scale still requires dramatic leaps in technology and shifts in policy.
Researchers at the Michigan State University College of Engineering are dedicated to evolving the necessary tools for safe and reliable self-driving vehicles. Joshua Siegel, an MSU assistant professor of computer science and engineering, predicts that electrically powered, connected and autonomous vehicles will transform everyday life.
“No matter who you are and how much you like driving, AV technology really will change the way people live and engage with the world,” Siegel said. “It enables people to live in different locations with longer commutes. It allows people to be farther from their children’s school. If you’re older and you want to age in place, an AV or ridesharing service can pick you up from home and take you where you need to go.”
An institutional commitment to innovative thinking with practical applications propels both the pioneering work in MSU research labs and the education that students receive through the online Master of Science in Electrical and Computer Engineering. MSU is a hub for multidisciplinary problem solving and a new generation of technical professionals who will further expand the frontiers of possibility in autonomous vehicles.
Fueling Smart Advances in Vehicle Automation
To reach the point where cars and trucks can navigate through traffic and weather conditions in both urban and rural environments requires many complex systems to work together seamlessly. Currently, academic researchers and private industry alike are striving to move vehicles forward along the framework laid out by the Society for Automotive Engineers.
The SAE defines six levels in the progression toward full automation:
SAE Driving Automation Levels
| Level 0 | The vehicle has no automation. It may offer driving support features to be fully supervised by the driver such as automatic emergency braking and a blind spot warning. |
| Level 1 | An advanced driver assistance system can help with steering (e.g. lane centering) or acceleration (e.g. adaptive cruise control), but not at the same time. |
| Level 2 | Under constant supervision from the driver, the assistance system is capable of controlling steering while simultaneously braking or accelerating. |
| Level 3 | Automated driving features operate without close supervision under certain conditions, such as slowly moving through heavy traffic. The driver must be ready to take over as necessary. |
| Level 4 | Automated driving features operate without supervision in controlled areas that have extensive infrastructure, e.g. a driverless taxi service that’s restricted to a specific urban environment. |
| Level 5 | Full autonomy: automated driving features operate without supervision on any road and under all conditions. |
To achieve the highest levels of autonomy, a vehicle must possess features such as computer vision that can gather detailed information about its environment in real time, a neural network trained to behave appropriately even in unexpected situations, and the decision making ability to determine when control must be transferred over to a human driver. Research at MSU lays the groundwork for AVs that are sophisticated enough to collect, analyze and respond to all the relevant data, allowing a self-driving car or truck to arrive at its destinations without incident.
Associate Professor of Electrical and Computer Engineering Daniel Morris sees safety as the top priority for his contributions in the MSU 3D Vision Lab. Morris and his students create algorithms that enable machine learning and artificial intelligence systems to make sense of the world around them for applications in agriculture and health care as well as transportation.
“Everyone wants cars that drive us safely from A to B and have no collisions, and I think the most essential part of that is perception,” Morris said. “Can vehicles detect all the other vehicles in their vicinity, determine where they are and predict their trajectories?”
According to Morris, nuanced computer vision techniques that fuse varied types of sensor input to provide detailed insights are a major prerequisite to reach high-level autonomy. If self-driving vehicles are going to operate under real-world conditions, they must be ready to cope with unexpected behavior from drivers or pedestrians as well as the possibility that its own sight will be occluded, obscuring stop signs or road markings. Guiding a car down a busy street on a foggy day requires efficiently synthesizing information from multiple sources like cameras, radar and lidar.
Morris and Professor Hayder Radha co-authored a paper with Ph.D. student Su Pang that proposed a fusion network of Camera-LiDAR Object Candidates as a possible solution to bring together these different types of input. The CLOCs would enable neural networks to detect objects based on both two-dimensional video images and 3D sensor data, leading to greater accuracy. Combined with ongoing technical improvements in the sensors themselves, this framework could pave the way for cars that reliably spot potential dangers in real time, even from a distance or with poor visibility due to weather. CLOCs has been well-received by other experts in object detection, ranking in the top three out of nearly 400 competing solutions on the field’s most popular leaderboard and number one among publicly available methods that rely on multimodal sensor fusion.
Even under the most favorable conditions, it’s extremely challenging for a machine to monitor and predict the movements of other vehicles, pedestrians, and cyclists the way that a human driver would. Pang and Radha addressed this key problem by developing a novel autonomous vehicle 3D-tracking framework specifically to handle a large number of dynamic objects in cluttered urban environments. The framework was one of only two solutions to receive an honorable mention at the 2020 Computer Vision and Pattern Recognition (CVPR) Conference’s Workshop on Autonomous Driving, outperforming entries from major corporations and tech firms.
The Power of Multidisciplinary Collaboration
Powerful computer vision algorithms are just one of the many areas of research that will ultimately put large numbers of autonomous vehicles on the road. That’s why faculty members and students in the MSU Department of Electrical and Computer Engineering prioritize sharing complementary expertise through collaborations with computer scientists, mechanical engineers and private firms.
“There’s a great community at MSU in CSE, ECE and the College of Engineering,” Siegel explained. “I work with people throughout the whole university, and they’re smart, kind people who are singularly driven to do the best that they possibly can in their field so that it has the broadest possible impact.”
With two self-driving vehicles, smart infrastructure and testing facilities available on campus, MSU has established itself as a vital center for researchers from a wide range of disciplines to experiment on ideas that are relevant to autonomous and connected vehicles. Much of their work takes place through CANVAS, short for Connected and Autonomous Networked Vehicles for Active Safety.
Radha, the founder and director of CANVAS, described the initiative as a collaborative effort to bring together diverse expertise from across the MSU College of Engineering. Researchers discover exciting ways to combine the knowledge housed in the ECE department with ideas from computer science, civil engineering and the Department of Computational Mathematics, Science and Engineering.
“The whole purpose of CANVAS is really to coordinate faculty’s efforts instead of potentially duplicating some research or missing opportunities for addressing important research problems,” Radha explained.
Infrastructure for Innovation
At research labs like the MSU Mobility Studio, students and faculty investigate a diverse array of transportation problems, such as enabling communication among vehicles and smart infrastructure, creating an appealing and functional user experience for the operating self-driving cars, and establishing cybersecurity measures that protect connected vehicles from hackers. Researchers can explore those connections by taking advantage of the infrastructure available on MSU’s campus, like networked smart traffic signals with vehicle and pedestrian sensors and the Spartan Mobility Village, which offers roads and parking lots that can be closed off for testing.
Engineers in this field may pursue careers at corporations engaged in cutting-edge efforts to produce autonomous vehicles. However, MSU offers a unique environment for researchers that prioritizes the sharing of knowledge and resources while maintaining close ties to the private sector. Siegel feels motivated by the creative freedom an academic setting offers to pivot and chase after potentially transformative new ideas, making a difference on a larger scale than is possible as a member of a large team in a corporate environment.
“When I graduate a student, they go out and change the world,” he said. “Maybe they’re the CEO of a start-up company focused on an AV technology or a consultant who advises people in industry day in and day out. So I see being in academia as a force multiplier to create greater societal change than I would ever be able to do on my own and to have a richer portfolio of experiences, either directly through my work consulting with industry or vicariously through my students.”
For Radha, another advantage of working at MSU is the weather in the state of Michigan. Seasonal changes make the region ideal for seeing how a self-driving car will operate in all different weather conditions. He co-authored a paper that examined how rain of varying intensities affected an object-detection system based in a deep learning neural network. The study found that even a light shower can wreak havoc on perception, pointing to the need for a specialized deep learning framework capable of adapting.
Research-Informed Learning
MSU’s commitment to self-driving vehicle technology greatly influences the M.S. in ECE curriculum. Students first build foundational knowledge in areas like linear control systems and analysis of stochastic systems. From there, they can explore the areas of inquiry that will propel the technical capabilities of autonomous vehicles like estimation theory, neural networks and robotics.
Specialized courses in autonomous vehicle topics offer multidisciplinary insights into issues affecting the technology’s future from visiting lecturers such as law professors and industry professionals. In Siegel’s view, research and teaching should form a “virtuous cycle of development.” He developed his course on advanced optics in autonomous vehicles specifically to be compatible with an online learning environment and makes revisions each year. The curriculum routinely evolves to account for the latest technology in sensing, computer vision, connectivity and deep learning, as well as new legislation that could impact the field’s future development.
The M.S. in ECE program as a whole has embraced the advantages of online learning. Faculty members shift between teaching modalities to illuminate concepts and provide an engaging experience. Even if students learning at a distance can’t be in the same room with the self-driving cars on campus, they get hands-on experience in programming on an automated platform by remotely controlling small robots and sharing their code via GitLab.
Making a Mark on the Future of Transportation
MSU’s richly collaborative atmosphere enables many ECE students to put what they learn to work by actively participating in research. Courses may include opportunities to work on AV-related projects and prepare a submission to a conference or journal. In research labs, faculty members encourage their students to take the lead, following their own curiosity and building out ideas that could set the course for their careers.
“We get our senior-level undergraduate and master’s students heavily involved in our research,” said Radha. “That’s one of the missions of CANVAS: to provide students ample opportunities to engage with faculty and work on some exciting, viable, and critical problems for this particular area.”
For example, with their students as the primary authors, Siegel and Morris both contributed to a paper on automated vehicles sharing the road with cyclists that appeared in the journal IEEE Transactions on Intelligent Vehicles. In another recent publication, Siegel worked with two students on creating a Physically Adversarial Intelligent Network (PAIN) that automated the generation of simulated worst-case scenarios, training a neural network to avoid collisions under a broader range of driving and environmental conditions.
While researchers have made great strides in autonomous vehicle technology over recent years, there’s still a tremendous amount of work for future ECE graduates to pursue in all areas of the field.
“The closer we get to the level of perception and reasoning that we humans are capable of, the closer we are to the point of a major, major breakthrough,” Radha explained. “Right now, we are somewhere in the early stages of making those breakthroughs.”
There are revolutionary possibilities when it comes to training neural networks to interact safely with unpredictable pedestrians, determining the most accurate and cost-effective ways of fusing sensor data, and hardening systems against cyberattacks. The online M.S. in Electrical Engineering puts technical problem solvers in touch with the resources they need to work at the forefront of advances that could dramatically change how we live, work and travel.
About Michigan State University’s Online Master of Science in Electrical & Computer Engineering
Michigan State University’s online Master of Science in Electrical & Computer Engineering program readies students to excel in the field by expanding their technical knowledge and focusing on the real problems spurring technical innovation. Courses in the online program are taught by MSU’s faculty of pioneering researchers and experienced educators. Online students can choose from two plans of study, selecting a thesis or non-thesis option as they complete a graduate education from an R1 research institution ranked among the Top 100 Global Universities by U.S. News & World Report.
Request Brochure
To learn more about Michigan State University’s online master’s programs in engineering and download a free brochure, fill out the fields below to request information. You can also call us toll-free at (888) 351-8360.