It generally is random loot, but I wonder if there are places with better odds for finding it.

Really though, required material for armor shouldn't be freaking random loot. I don't know what they were thinking.

Not surprisingly, I think I only find amber matrices in Mania, so that's where I'd hunt. Not much help beyond that given, as you say, random loot. Just today I picked one up from a grummite along the road in Mania and another down in the bowels of Cann (also in Mania).

Edit: Oh, and of course you only need the matrices if you want enchanted stuff made. Like many, I prefer to enchant my own gear, so the matrices are purely snazzy looking decorations to place about when decorating for me. Lol.