Great point from Jim Neilsen re: the instructions found deep inside a model or agent’s instructions, which we may not prefer yet live with if we don’t know to override them:
It’s like a Trojan Horse of craft: guidelines you might never agree to explicitly are guiding LLM outputs, which means you are agreeing to them implicitly.
It is worth the effort to dig under the hood of the tools we use so that we know what we may want to change. Especially so if we continue to get results we don’t prefer.