This isn’t an Apple Music issue—it’s almost certainly Windows not switching audio output correctly or a hardware/driver problem with the headphone jack on your Surface Laptop 4.
Use a USB audio adapter (USB → 3.5mm)
Cheap, plug-and-play, and bypasses the faulty headphone jack entirely.
Works instantly on Windows 10
No driver headache
Better than repairing the jack