Intuitively, it at first seems like LLM would be great for summarization, however researchers have analyzed and evaluated LLM summarization results and concluded that LLMs do not actually even do summarization at all. Instead, they only shorten the text by removing repetition, based on statistical patterns in their training set, rather than based on an understanding of the specific given content. Again, intuitively it might seem like this is still something that might be useful to be able to do, but in practice the results are almost never useful or what the user actually wanted.

This is a fundamentally different task than doing summarization, because summarization requires understanding of the content and context, in order to identify the key information and point/purpose of the content. A statistical model of language just cannot do summarization.

Our misleading intuition about how LLMs work and how they can be used makes them even more unsuitable for it, because they seem like they are doing what the user asked, when they are actually doing something entirely different.

tl;dr summary: What they provide appears to be a summary of the given content, until you really dig in and evaluate it, then you realize it isn't really a summary of the content at all, it just looks like one.