In my case, I reject armchair deontological ethics in favor of ethics that have at least some grounding in science and empirical examination of the world, so the non-aggression axiom fails for the same reason that Kant's ethics and Christian ethics fail.
Can you explain when "ethics that have at least some grounding in science and empirical examination of the world" conflict with the non-agression axiom?
They may or may not conflict; I just reject it as an axiom. It's possible that there are good reasons to support a society structured around a non-aggression principle, though.
(I don't have a fully worked-out ethical theory, but I'm intrigued by the stuff Sam Harris is doing to try to reduce the gap between ethical theorizing and science.)