in crosscompilation, that doesn’t matter. the toolchain for the target is used for compilation. this has no connection to the host that runs the compilation.
besides, NEON only exists on ARM, not on x86. entirely different architectures. they have nothing in common.