AI-Powered Virtual Coach Takes Swimmers to the Next Level
Deep Learning Algorithms Take the Guesswork and Frustration Out of Swimming
Yuri Zhovnirovsky needed a break from the pool. An avid swimmer and water polo player, he spent too much time in chlorinated water. After college, he decided he needed another athletic outlet. After an interlude during which Zhovnirovsky tried competitive cycling, his neighbor dragged him back to the pool. It was a reunion tinged with disappointment. The familiar swimming frustrations came back to him.
“When I got back in the pool, nothing had changed,” he says. “I still had to count my own laps and strokes. It’s the equivalent of going for a run and having to count your own steps to figure out how far you’ve gone.”
After Zhovnirovsky’s bout with cycling, he was spoiled with fitness wearables that tracked a plethora of cycling metrics—like power, cadence, speed, and elevation. “You name it, they had it,” says Zhovnirovsky, who also worked in tech. When it came to swimming, there were no wearables that matched the accuracy of those tracking running or cycling. “I thought, ‘Well, I have the technology background, and I can see a solution.’”
The solution was an elegant one: smart goggles. Zhovnirovsky founded Ciye (pronounced “see”; it’s an acronym that stands for “coach in your eye”) to create smart goggles and bring them to market. In the Smart Goggle, a “smart coach” module snaps into the left corner of the left lens. The module has a small display showing swimmers their lap times, numbers of laps, and split times (time per lap), visible with a quick leftward glance. New features to track and perfect form are also in the works. Ciye’s Smart Goggle serves all swimmers, from amateurs to Olympians.
The Smart Goggle, powered by the Ciye “smart coach” module, displays lap times, numbers of laps, and split times with a quick leftward glance. (Video credit: Ciye)
A Natural Solution
In his professional life, Zhovnirovsky is a serial entrepreneur. By the time he started swimming again, he had years of experience starting and scaling startups. He knew what was needed to develop the Ciye smart goggles solution. He also had a wide market to target. “In the US alone, there are 26 million people who go to the pool at least three times per week,” he says. And the lap-counting problems are a near-universal frustration. For example, one Ciye tester swims 9.5 kilometers (six miles) every day and before testing the Smart Goggle, she needed to mentally track her laps—all 422 of them.
Common sense told Zhovnirovsky that swimmers needed a solution integrated into something universal in the swimming world.
“It’s an inherent issue. When you ask people how many laps they swam, most people will tell you they don’t know because nobody seems to be able to keep track beyond four laps,” says Zhovnirovsky. The inability to track progress during a swim also impedes a swimmer’s ability to improve. “If you talk to swimmers, their biggest complaints are performance-related: ‘I’m changing my style, but am I getting better? Faster? Am I getting worse?’” he says. “It’s very difficult to get information about how changes affect performance.”
While some devices like swim watches have tried to solve these problems, they have major drawbacks. For starters, the watch is worn on the arm, and the motion of the arm while swimming overwhelms the sensor. They aren’t very accurate at tracking laps, and if you want to check your pace, you need to move your wrist into view, disrupting your stroke. They also can’t provide real-time feedback during a workout.
Common sense told Zhovnirovsky that swimmers needed a solution integrated into something universal in the swimming world. “Everybody who swims wears goggles,” he says. “It was a very obvious answer.”
A Smart Goggle with the Ciye module. (Image credit: Ciye)
If You Build It, They Will Wear It
As swimmers themselves, the Ciye team had a good idea of what their market would want in smart goggles. Seeing swim times, rest times, and split times during a swim, without breaking from their strokes, would allow swimmers to push or pace themselves according to their goals in real time. Ciye’s Smart Goggle also needed to be better than the major smart goggle on the market, which projected swimmers’ stats in front of them as they swam. It was attractive but obstructed swimmers’ views and caused nausea. The challenge was making the Smart Goggle a reality.
“It frees you up from the mundane counting to focus on your technique and performance. We want to be your personalized, dedicated coach in your eye.”
They needed to create a product that could accurately sense and identify the elements of a swim—when someone starts swimming, when they stop, when they turn, which stroke they’re using—and could sense those elements from a sensor on the head. When detecting movement and orientation, a nine-axis inertial measurement unit—a common sensor in wearables that wraps an accelerometer, gyroscope, and magnetometer into one device—produces noisy signals. The Ciye team needed an AI model that could tease apart the relevant signals from a convoluted mess and label specific movements, like a backstroke or freestyle.
“Data labeling is one of the most important activities in machine learning and the hardest to implement at scale,” says Zhovnirovsky. “In my experience, it is more important than the algorithm itself.” What Ciye needed didn’t already exist, so using MATLAB®, the team created a custom data labeling tool that met their needs.
In addition to running deep learning algorithms, the technology needed to be small and waterproof with a long battery life. The algorithms had to be compact yet powerful. “We had to balance the performance and complexity of the algorithms,” says Koosha Sadeghi, head of data science at Ciye.
Sadeghi needed to create an algorithm that saved on memory and computational power, placing as little demand as possible on the battery so the sensors could work to their full capacities. Sadeghi worked on one iteration of the algorithm after another using MATLAB toolboxes for deep learning and signal processing.
He trained the algorithms on data from Ciye’s community of test swimmers who would send over their data from workouts and detailed descriptions of their swims. The goggles capture information with over 99% accuracy, identifying stroke type, stroke rate, lap count, and more.
The optimization of the machine learning model for swimming style recognition. (Image credit: Ciye)
“It’s hard to have an algorithm with such high accuracy across a large population of people,” says Zhovnirovsky. “We were able to achieve this, in large part, because of MATLAB and the tools it provided.”
Once Ciye had the technology, they needed the goggles to host it. The Ciye team partnered with FINIS, Inc., which develops and sells products that improve swim technique and experience, to create the Smart Goggle, powered by a removable Ciye Coach module.
The Ciye module, roughly the size and shape of a guitar pick missing the tip, snaps into the periphery of the left lens of the goggles, which look deceptively typical. The module’s design also allows customers to easily swap and replace the plastic lenses without the expense of buying a whole new coaching solution each time.
The Ciye module. (Image credit: Ciye)
But that’s part of the beauty of the Smart Goggle. Wearing the goggles, a swimmer’s display sits out of view, visible only when their eyes dart to the left. In that glance, the swimmer sees a black screen cycling through a handful of data points, one at a time, including lap number, time per lap, rest time, and total time. When the swimmer turns and pushes off the wall to start a new lap, the goggles display the time for the previous lap and the number of strokes. “It frees you up from the mundane counting to focus on your technique and performance,” Zhovnirovsky says.
The goggles also track more detailed information about the swim, which appears in the app afterward. There, users can get a detailed breakdown of each workout, seeing their total distance, average pace, and calories burned, in addition to the real-time display data. Through the app, users can also set goals for improvement and connect with other swimmers, giving an otherwise solitary sport a social element. “The app is there to help you stay motivated and connected,” says Zhovnirovsky. “We want to be your personalized, dedicated coach in your eye.”
A Virtual Coach
The COVID-19 pandemic shook up the swimming world as it did every other facet of daily life. It also put a spotlight on the utility of virtual tools in sports. “For me, having this device during COVID was instrumental. During COVID, even though you could go to the pool, you were very much alone,” says Zhovnirovsky. “The pandemic removed whatever social aspect was there before.” No one was allowed on deck, not even coaches, and swimmers needed to wear masks before getting into the pool and when they got out. The Smart Goggle and app restored some of that social aspect and also ensured athletes could keep training.
“For getting real-time feedback while exercising, technology is going to move into the eye. Our mission is to bring virtual coaching to everybody.”
For professionals, a virtual coach wasn’t just instrumental, it was vital. During the 2020 Summer Paralympics in Tokyo, which took place in 2021, US swimmer Mallory Weggemann went without her coach, who wasn’t allowed to come to the games due to travel restrictions. But she used the Smart Goggle to record her training swims and send data back to her coach, who could then advise and support Weggemann several time zones away. Weggemann went on to win two gold medals, win one silver medal, and break two Paralympic records.
In another instance, the Ciye team brought the goggles to an Olympic training center and had the German National Team try them. “They all went faster,” says Zhovnirovsky. “Now that they had the clock in front of them, they knew when they were going slow or going fast, and that pushed them to be better.”
The Smart Goggle data can be accessed by the Ciye app. (Image credit: Ciye)
With initial success in the swim market, Ciye is bringing the virtual coach in your eye to other sports, developing smart sunglasses for runners and cyclists. The sunglasses will be able to track the terrestrial equivalents of the Smart Goggle metrics, like steps, distance, speed, and activity duration. The current prototype looks like a normal pair of sports sunglasses, but with a small battery and Ciye technology built into it. Eventually, Zhovnirovsky says that Ciye is planning to design glasses with a removable module, like the goggles.
While still a prototype, the sunglasses are fully functioning, and Zhovnirovsky hopes they’ll launch toward the end of the year. The Ciye team is also working on a biometric recording feature for the sunglasses and goggles. For the goggles, they have a patent for technology that detects heart rate through the head and are pursuing something similar with the sunglasses. In Zhovnirovsky’s vision, the sunglasses won’t only track heart rate but will also measure blood oxygen saturation and calculate VO2 max, which is the maximum amount of oxygen the body can use during intense activity and reflects cardiovascular fitness.
Among fitness wearables, Zhovnirovsky thinks products like Ciye’s will dominate in the future. “For getting real-time feedback while exercising, technology is going to move into the eye,” he says. “Our mission is to bring virtual coaching to everybody.”
Read Other Stories
AI / ROBOTICS
Combining Artificial Intelligence and Footwear Improves Diabetes Treatment
Smart Shoe for Diabetic Neuropathy Gets a Tech Upgrade
AI / SIGNAL PROCESSING
Wearable Tech Provides Data-Driven Mental Health Therapy
Psychotherapists Partner with Algorithms to Improve Mental Health Treatment
AI
Cataract Patients See Better Results with AI
AI Modeling Technique from the Automotive Industry Improves Eye Surgery Results
