Modern DirectX is not fully backward compatible with the installer detection logic from 2002. The game’s setup program looks for a specific registry key or DLL signature from "dx8.1." When it doesn't find it (because DirectX 9 and 10 overwrote those markers), it refuses to proceed.
If you are encountering the notorious error, this article will walk you through why it happens and how to fix it to get you back to the neon lights of Vice City. Why GTA Vice City Requires DirectX 8.1 gta vice city directx 8.1