Latency accumulates in small purchases: the converter, the buffer, the processing, the output device, and for visuals, the render and the display. Each stage is a few milliseconds, and the sum decides whether a system feels connected to the music or trailing it.
Musicians feel latency above roughly 10 milliseconds when playing; audiences watching visuals are more forgiving, but the audiovisual binding window is finite, and reactions arriving tens of milliseconds late read as soggy even when nobody can articulate why.
The craft is spending the budget where it buys the most: tight buffers for detection, prediction (like beat tracking) to hide what cannot be eliminated.