The truly silly (and simultaneously neat and fascinating, and to a large extent, very logical and obvious) part is that US attempts to cripple Chinese ML/LLM development by putting most advanced GPU tech behind ITAR has more or less made the Chinese ML industry just write straight up better and more optimized code for the domain.
US and western companies are currently relying on throwing bleeding edge hardware at the problem (read: very obviously brute-forcing the problem), whereas Chinese companies and engineers, having to rely on comparatively less advanced (though still highly capable) hardware has meant they’re staying competitive by optimizing the software layer more effectively.
This has very interesting and likely negative long-term implications for western engineering practices in a lot of ways. And it’s also very much not a new problem - simply throwing compute at things is a mindset that became highly prevalent in the west around when The Cloud became a thing. Previously, you couldn’t just spawn another mainframe in your company’s datacenter, but when it’s as simple as aws ec2 start-instances ..., the barrier to entry for getting more compute is so low that most people were just going “hey, it’s faster”. I feel like the engineering expertise around embedded/baremetal optimization has just been largely evaporating over the last couple decades; in the vast majority of professional software domains, it’s simply not a valued skill anymore (or, more correctly, at the moment). It’s sad for me in several ways - not least of which because I did my degree in EECS, and hardware/software interplay and optimization is an area of engineering that I have always found very interesting.
The truly silly (and simultaneously neat and fascinating, and to a large extent, very logical and obvious) part is that US attempts to cripple Chinese ML/LLM development by putting most advanced GPU tech behind ITAR has more or less made the Chinese ML industry just write straight up better and more optimized code for the domain.
US and western companies are currently relying on throwing bleeding edge hardware at the problem (read: very obviously brute-forcing the problem), whereas Chinese companies and engineers, having to rely on comparatively less advanced (though still highly capable) hardware has meant they’re staying competitive by optimizing the software layer more effectively.
This has very interesting and likely negative long-term implications for western engineering practices in a lot of ways. And it’s also very much not a new problem - simply throwing compute at things is a mindset that became highly prevalent in the west around when The Cloud became a thing. Previously, you couldn’t just spawn another mainframe in your company’s datacenter, but when it’s as simple as
aws ec2 start-instances ..., the barrier to entry for getting more compute is so low that most people were just going “hey, it’s faster”. I feel like the engineering expertise around embedded/baremetal optimization has just been largely evaporating over the last couple decades; in the vast majority of professional software domains, it’s simply not a valued skill anymore (or, more correctly, at the moment). It’s sad for me in several ways - not least of which because I did my degree in EECS, and hardware/software interplay and optimization is an area of engineering that I have always found very interesting.