Mastodon Feed
jonny@neuromatch.social ("jonny (nonvenomous)") wrote:
The entire security posture here, if there is one, is oriented around preventing the LLM from misbehaving, but having the entire context be user-editable and user-visible is a sort of mind-blowing neglect of a misbehaving user. even for the "fully transparent agent" they're trying to do, it makes very little sense to store the full message logs of the subagents including reasoning output. That basically gives me a full map to see what the LLM is consulting when it refuses commands. I would also bet money that the system prompts are strings in the hatch binary even though that makes no sense to have clientside, but haven't looked.