As shown in the video, not knowing when they are near an intersection, crosswalk, or crossing button is hazardous and can cause life threatening injuries.

View my app submission and a demo of the app working

My most recent engineering project is my app, Sightly. I just recently submitted my app to the Congressional App Challenge, and won in my district.

Sightly is a mobile app that makes navigation easier for the visually impaired.

I spent a lot of time with the Visually impaired to understand their daily life.
When commuting to work or to a grocery store, for example, they struggle with not knowing that they are near a traffic light, or crosswalk, sometimes resulting in accidents that could potentially be fatal.
It made me realize that we take things for granted and I felt compelled to make their lives safer.
I used all these insights to create Sightly.

The app uses computer vision models to detect, recognize, and describe pedestrian aids(like crosswalks and traffic signals) in real time. It alerts the user through voice alerts, almost like a virtual companion. Also, they don't have to touch the screen to get the app to work and the app can work without wifi or cell phone connection, important must-have features for the visually impaired. Current solutions don’t have all of these features.

Ex. if it recognizes that a crosswalk signal is on the stop hand, the app will warn, “Don’t walk”. Likewise, if the crosswalk signal turns to the walk signal, it will say, "Walk."

To create my app, I used Apple's Xcode environment, programming in Swift. While creating my app, I learned how to program with computer vision models/libraries, like YOLO, OpenCV, and MobileVNet. I’m currently field testing my app with the Oregon Commission for the Blind.

My vision(pun intended) is to help every visually impaired person be more independent.

View my submission video and presentation below for more details and demos:

h ttps://docs.google.com/presentation/d/13URcapnZwKNKyukzwiaukopcx6c2poAesx8VmtOcjfQ/edit?usp=sharing