We currently default to allowing arbitrary tools and system prompts from the frontend. A bad actor can escalate their “authority” in the chain of command (see also: Model Spec – Chain of Command).

The justification is this: Because current models are already highly vulnerable to jailbreaking, you have to design your LLM tools surface with jailbreak risk in mind anyway.

There are strong developer experience (DX) benefits to:

• Defining tools in the frontend (including those that can run code in the frontend)
• Exposing application state in the system prompt (e.g., useAssistantInstructions("the user is currently on the settings page"))

That’s why our current default is to allow passing system prompts and tools directly from the frontend.

As models improve and become more resistant to jailbreaks, you might instead choose to expose sensitive tools to the LLM directly—trusting it to use them responsibly. In that scenario, you wouldn’t want arbitrary tools or system prompts from the frontend to inherit elevated authority.

Future-proofing idea

To avoid authority escalation risks in a more secure future, we could:

• Put the frontend system prompt in quotes
• Prefix any frontend-provided tools with something like UNSAFE_USER_PROVIDED_

This would make the source of authority explicit while still supporting flexible DX.

It’s unclear what the ultimate best practice will be for this use case.
If you have references or guidance on the right long-term approach, I’d be interested to see them.