The first time I heard about doing that, I was shocked! (really) But this works for many of you, and here are some of the ways people store their individual swatches:
Silvia/@astrangerview purchased a Box Box transparent box from The Container Store. I couldn't find that exact product still available, but you get the idea. I've heard that an Aurifil thread case also works well.
Then there's Felice/Sew scatterbrained, who punched a whole in each swatch and keeps them on a ball chain to create a swatch ring, keeping them in order as they were on the card for side by side comparisons.
Velcro was the answer for Melanie/Mell Meyer, who created her own Velcro Kona Color Card. Melanie has placement sheets available for you to print out to make your own on cardstock.
And then Sheila/thought & found's idea is a hybrid of labeling fabric vs. cut-up color card in that she makes swatch cards that offer larger fabric samples. The cards are available for a small fee on Craftsy.