001 : Low-cost computer vision system for tennis (2016)

Low cost vision based system was built to assist audio-based bounce detector, and fill missing gaps in game event tracking in fully uncorrelated manner. The system consists of 2 low-cost cameras fixed into a pole near the court. The cameras operate at 30Hz and 640x480 resolution and return images to Yocto Linux based system with custom image processing pipeline. System has special SMARC processor card which supports HW-accelerated MJPEG decoding via DMA transfers. It turned out that SMARC power is often running out during decoding, but finally customer service in Taiwan provided a kernel patch which solved the problem. The system calibrates the cameras automatically using 2D points extracted as intersection of Hough transform generated line candidates. Then multi-threaded image processing pipeline extracts ball and player positions in 2D using motion images. These "video events" are then used to generate precise trajectory, bounce coordinates, player coordinates and spin of the ball in metric tennis coordinate system. Tracks are incrementally continued through ball observations using noise-aware combinatorial approach which favors continuity, allows missing frames and respects tennis trajectory outline. Spin detection spans 2D plane across tennis court between the player coordinates, backprojects tennis ball observations into plane and fits a trajectory model. Trajectory geometry is then compared with discrete set of curves which are produced using precise physical model with spin parameter. The best matching curve then labels the spin for trajectory.

Tennis from computer vision system perspective.

Spin detection using physical model.

Automatic camera calibration.

Testing gray scale cameras.

Automatically generated court masks.

Testing back view tracking.