Back to all posts

Moving from Trust to Verification: The Case for Jacquard

AI generated code is becoming a standard component of modern workflows, but we’re hitting a wall: auditing what that code is actually allowed to do.

AI SecurityProgramming LanguagesSoftware EngineeringMachine Learning
main thumbnail for Moving from Trust to Verification: The Case for Jacquard
main thumbnail for Moving from Trust to Verification: The Case for Jacquard
Reader Lens

Automation needs a narrow first win

The best first AI workflow is usually a repeated task with a clear input, clear output, and a human approval step.

AI-generated code is becoming a standard component of modern workflows, but we’re hitting a wall: auditing what that code is actually allowed to do. We can’t simply trust an LLM's output because a single hidden side effect—an unauthorized network call or a sneaky file write—can compromise an entire system. Jacquard, a research project by FriendMachine, addresses this by moving security from a human-review problem to a language-level constraint. It makes the "what can this touch?" question a fundamental part of the syntax.

Enforcing Permissions via Effect Signatures

Jacquard isn't just a wrapper; it’s a small programming language designed to make side effects explicit. Every function signature must declare its "outside-world" effects, such as network access, file manipulation, clock interaction, or randomness. If a function wants to do something, it has to declare it upfront. The runtime then enforces these permissions strictly: unless an effect is explicitly granted via a --allow flag at the command line, the code simply won't run. This shifts the security boundary from "hope the AI didn't do something bad" to "the runtime won't let it do something bad." Crucially, this applies even to dynamic code, ensuring that scripts generated on the fly still respect these boundaries. It also includes a library for discrete probabilistic programming, allowing for sampling and inference within this restricted environment.

Testing Across Isolated 'Worlds'

One of the more practical features of Jacquard is its ability to run the same code against different "worlds." Because effects are isolated and declared, you can swap a real network for a scripted fake or a probability model. This allows for testing how code behaves in diverse environments without actually triggering side effects in production. Additionally, the language uses content-addressed definitions. This means it ignores formatting and comments when checking for structural identity. For a developer, this is a relief; the machine only cares about the logic, not whether you put extra whitespace or a comment between lines. The goal is to keep the language small enough that an AI agent with no prior training can learn it from a single documentation file.

The Friction of Explicit Security

The real story here is that Jacquard is a research prototype (version 0.1), not a production-ready framework for your entire stack. While the ability for AI agents to learn the language from a single doc is clever, we have to be honest about the friction of migrating existing logic into such a restricted environment. There is a clear tradeoff between the safety of explicit effect signatures and the velocity of general-purpose programming. It’s not a "plug-and-play" solution for every Python script; it’s a specialized sandbox for high-stakes code where the cost of an unauthorized action outweighs the friction of declaring every single side effect. The challenge isn't just making these restrictions work—it's making them feel like a standard safety rail rather than a hurdle that slows down development.

inside paper visual for Moving from Trust to Verification: The Case for Jacquard
main thumbnail for Moving from Trust to Verification: The Case for Jacquard
Source and trust note

Built from source research and filtered through practical implementation judgment.

Reference: github.com

Got a question about how this applies to you? →

Keep reading

Follow the thread