"Mystery" is an old name that's rarely ever explicitly mentioned in more recent Nasuverse works, but it's the only name we have for the "age equals power" concept that persists in Nasu's magic system. Magecraft is inherently linked with Mystery, which is inherently linked with the World as a whole. Those with Mystery are inherently 'magical'.
It all comes down to having a spiritual body[1]. A "spiritual body" is the one thing Rin explicitly mentions to be what is needed to harm Servants, without any vague language about it. "Servants' weapons are in spirit form like them, so they should be able to hurt them."
Also, in regards to the example of the living Nero somehow being able to affect Servants, remember that her sword, Aestus Estus, is inherently magical, and that she was noted to already be comparatively superhuman in life, in a time where everyone is already superhuman by modern standards due to the strength of the planet.
To address another misunderstanding above, Servants don't "amp" whatever weapons they hold, they make them inherently magical. The paper knife doesn't suddenly become sharper or sturdier just because a Servant is using it, it just becomes magical enough to hurt another Servant. It's the same as a mundane sword VS a sword that is exactly the same in every way except it has magical energy flowing through or around it. Just because the magic is there doesn't mean it's doing anything. There's a reason enhancement magecraft like Shirou's Reinforcement is more than just pumping something with magical energy. It's a craft. There's a technique to it that keeps it from being useless or exploding in the user's face.
[1] ""Spiritual Body" is a word that describes an existence with spiritual properties. In other words, a shape is constructed without relying on components with physical properties. On the other hand, "Material Body" means something that is identical in structure as our flesh bodies. A Material Body, just like us, cannot walk through walls, and will hurt and become wounded when beaten with an iron baton. However, a Spirit Body is capable of completely disregarding such physical interference. Attacking a Spiritual Body with a baton or sword (unless they are magical items) will only hit empty space."
- Fate/complete material III, p.018