Phyton, not Python. 🙃

Python has a ton of machine learning libraries. I'd maybe even go so far as to say it's the de facto standard when developing AI

There's also some cuda libraries which by definition do things directly on the card

Yes... It's possible to have that. Even when it doesn't do that by default. The CPU can and still is the bottleneck in a fair few cases and you bet you can run shitty code on there.