Struggling to Summarize What I Understand in a Second Language

I’m aware that there are already many articles and videos addressing this issue, and I’ve gone through quite a few of them myself, but I still haven’t found a solution.

I’m currently learning a second language (not English), and I consider myself to be at an intermediate to upper-intermediate level. However, I run into a recurring problem: when someone speaks for more than five minutes, I can understand what they’re saying in the moment, but when I try to summarize or restate it afterward, I struggle.

To be honest, I experience the same issue even in my native language.

I suspect this might be related to what is called “memory span” in interpreting. So I’m wondering—does this stem from a limitation in my memory, or is it more of a language proficiency issue?