I think in principle, an application didn't need to know the exact format of an IP address, even if connecting directly to an IP. A simple idea that could have made application code much more IP-agnostic would have been for SOCK_ADDR_IN to take the IP in string format, not as a four-byte value. That way, lots of application code would not need to even be recompiled to move from a 4 byte IPv4 address to a 16 byte IPv6 address, whereas today they not only need to be recompiled, they need to be changed at the source level to use a new type that allows for both.
Of course, code that operates on packets, in the TCP/IP stack of the OS would have still needed to be rewritten. But that is far less code than "every application that opens a socket".
Of course, this only applies to code that uses IPs only to open connections. There's lots of application code that does more things with IPs, such as parsing, displaying, validating etc. All of this code would still need to be rewritten to accept IPv6 addresses (and its much more complex string representations), that part is inevitable.
Yeah, the big issue is that any code that took addresses from user input had to do validation to make sure addresses were valid, in allowed ranges, etc.
While the sockaddr struct allowed to to abstractly handle v4/v6 socket connections, there wasn’t a clean way to do all of that additional stuff and IP address logic leaked into all kinds of software where you wouldn’t first expect it.
Something as simple as a web app that needs to inspect proxy headers would even have it.
It also didn’t help that it became practice to explicitly not trust the addr resolution offered by the sockets API because it would do unexpected things like resolving something that looked like an integer to a uint32 and then a 4 byte V4 addr.
This is vastly oversimplifying the problem, the difference between IPv4 and IPv6 is not just the format of the address. Different protocols have different features, which is why the sockaddr_in and sockaddr_in6 types don't just differ in the address field. Plus the vast majority of network programs are using higher level abstractions, for example even in C or C++ a lot of people would be using a network library like libevent or asio to handle a lot of these details (especially if you want to write code that easily works with TLS).
Of course, code that operates on packets, in the TCP/IP stack of the OS would have still needed to be rewritten. But that is far less code than "every application that opens a socket".
Of course, this only applies to code that uses IPs only to open connections. There's lots of application code that does more things with IPs, such as parsing, displaying, validating etc. All of this code would still need to be rewritten to accept IPv6 addresses (and its much more complex string representations), that part is inevitable.