when LLMs can generate derivative works that mimic the style of prior art it is clearly infringing, and they've clearly seen the source to reproduce it

I don't think it is really that clear. A lot of learning a craft comes down to studying a body of material well enough to be able to reproduce or re-mix aspects of it. For extreme examples: museums all over the world are full of folks that study paintings by creating reproductions, possibly selling those to fund their work. Mimicry and impersonation is common in the entertainment realm, Vegas has a whole cottage industry of Elvis Impersonators. Many companies that produce goods compete by copying each other's products with slight variations in form or functionality that appeal to different user bases. There are definitely ethical concerns about LLMs, but I don't think you can quite equate consumption of a body of knowledge and then reproduction from that base as theft. It's much more complicated than that.