So you don't care that a person more than willing to bring destruction to the U.S. was trying to get the material to do so?
Okie Dokie...
Ideally, no one would have these materials.
Anyway, no, it doesn't worry me because rogue nations have been trying to go nuclear for years and years since the Western powers made it clear early in the Atomic Age that they were only going to share the information with what were considered responsible nations. Also, the material is useless in an offensive capacity without the technology and know-how to actually covert it for military use. How many nations have had the materials but had to figure it out for themselves? Look at how long it took them to do, and they had much better physicists and engineers in their employ than Iraq had under Hussein I'm sure.