How do you know you have an invasion? Well, the first sign is seeing moths. The moth itself doesn't eat fiber. But their larva do. If you see moths around, be extra vigilant. You can tell you actually have a problem when you find larva poo. It looks like little grains of sand.
The fiber will have chunks missing. The larva just bites right into the fiber. There will be a bunch of poo around missing chunks as well.
You may even find a live larva. They are pretty gross.
Put all your fiber in the "infected" bag into quarantine. Even if the fiber appears to be untouched, if it was next to mothy fiber, bag it up. Then put it all in the freezer for 2-3 days. The temperature will kill off all the larva and moths. Conversely, if you have a rather hot garage during the summer, the fiber could be left there for a few days. Extreme heat will also kill off any unwanted pests.
While I was able to salvage a bunch of my fiber, I did lose a bag to moths. It was just filled with little larva poo and every few inches of the roving had a chunk missing. I learned that moth larva have good taste. They ate the angora-wool blend.