Through artificial intelligence and machine learning methods, SURP project aims to enhance shark spotting by drones.
An avid surfer, swimmer and scuba diver, Grace Nolan found a perfect way to combine her interest in ocean biology with the artificial intelligence (AI) concepts she is studying as a Cal Poly computer science student.
Nolan, who was set to spend the summer abroad in Thailand before the pandemic cancelled her plans, saw an educational alternative when she took a quick glance at the 2020 SURP (Summer Undergraduate Research Program) catalog of projects and noticed Professor Franz Kurfess’ Shark Spotting with Drones. The Southern California native was inspired by a project that aims to both protect sharks and alert lifeguards and beach goers by enhancing AI methods for the recognition of sharks and other objects in aerial surveillance video taken by drones.
“This project stood out because I have always spent time in and around the ocean and the notion of helping protect it and its inhabitants sounded very appealing to me,” Nolan said. “So, I jumped at the chance to look at the ocean in a whole different way — from the perspective of a drone. I believe that this technology will be instrumental to the way we look at the ocean, especially now that we can do so in a non-invasive way.”
During this virtual learning summer, Nolan, along with computer science students Kathir Gounder, Daniel Moore, Casey Daly and Damon Tan, and general engineering student Caroline Skae, have been analyzing video provided by the Shark Lab at Cal State University Long Beach. Nolan said the work, which has included creating an improved machine learning model with more than 1,300 annotated images of sharks, seals, boats, surfers and swimmers, has been a deep dive into the key components of modern computer science like AI, machine learning and neural networks.
“The biggest challenge for me has been learning the ropes of artificial intelligence and how the system works, in terms of image recognition,” Nolan said. “I’ve found that being thrown into the deep end with such an abstract discipline as AI has been one of the best and most rewarding ways to learn. It’s really not as intimidating a topic as the reputation that precedes it.”
Nolan said deep neural networks and machine learning fall under the umbrella of AI and form the center of the challenge.
“We are focused on training and using a deep neural network to do the image recognition needed for the project,” she said. “The network describes relationships and patterns in order to recognize objects from pixels in an image. Machine learning is the ability of the program to learn from these relationships and identify similar objects when shown new images.”
The project, which is sponsored by Nic and Sara Johnson, builds off the preliminary research conducted by students in two of Kurfess’ computer science classes, and one by his colleague, Maria Pantoja, on Deep Learning, last fall. Kurfess said the SURP team aims to assist the Cal State Long Beach researchers by using advanced computing methods to identify sharks and other objects of interest in their drone footage.
“While our initial work confirmed that our approach is viable, and the results deliver accuracy rates of about 80 to 90 percent with limited data sets and training efforts, we believe that there is significant room for improvement through a more concentrated effort by the SURP team,” Kurfess said. “The team has made impressive progress, and importantly, the students seem to be having fun with the project.”
Nolan, who said proximity to the ocean was definitely high on the list of reasons she chose Cal Poly, agrees.
“I think my favorite thing about the project has been learning from each and every team member, as we all have very different strengths and backgrounds in engineering,” Nolan said. “I’ve learned a lot about the applications of AI as well as how to work within a team of this size and skill. It’s most satisfying to see it all coming together in the final weeks as we will have a research paper and fully functioning program to show for it.”
For more information on the Shark Lab at Cal State University Long Beach, see https://www.csulb.edu/shark-lab.
For more information on SURP, see https://ceng.calpoly.edu/surp/
Drone photos courtesy of Cal State University Long Beach.
