CORS is a server setting to tell the browser not to load its data from potentially unsafe origins. If you set a server to send access-control-allow-origin: *, then your browser will happily load these resources for you regardless of where you currently are. And chrome extensions need to be loadable from everywhere to be able to inject code or images into pages, so enabling CORS for them would defeat their main purpose. The extensions themselves might even need to bypass an existing CORS setup for the website you are currently on to fetch additional data.

From the other end, yes extensions access all page data, but pages shouldn't access extension data at all; it feels like that should be the CORS violation.

You have it backwards. For an extension to work on a page, it's data/code needs to be accessible from said page. If your extension server in chrome enforced CORS to prevent access from tabs on other websites, extensions wouldn't work anywhere.