Turn engineering knowledge into a scalable advantage with AI agents

Published via GitHub Executive Insights | Authored by Bronte van der Hoorn

When considering AI in software development, many organizations focus narrowly on code generation. But this view misses the transformative potential of AI agents across the engineering system. Our Engineering System Success Playbook (ESSP) provides the framework for understanding how AI agents can elevate quality, velocity, and developer happiness across the organization.

At GitHub, our Trade Compliance team's experience illustrates how AI agents can transform engineering systems — and the organization as a whole — when approached intentionally through the ESSP framework.

Step 1: Identifying barriers to success

One of the team’s main challenges was finding time to document complex trade controls embedded in Ruby code. This is a common engineering issue that creates knowledge silos, where non-engineering stakeholders are unable to access information that they need. The team was curious if GitHub’s Copilot coding agent could help them document the controls more efficiently, thus breaking down these knowledge silos.

"Good documentation scales our small team by making key logic accessible across GitHub. However, it's an ongoing challenge to prioritize this work. We saw experimentation with AI as a means to address this challenge." Shruti Corbett, Engineering Director

Step 2: Evaluating the path forward with a pilot

Rather than viewing AI as a one-off automation tool, the team initiated a pilot to test how effectively the agent could translate complex code into clear documentation as a potential long-term solution.

During this pilot phase, they invested time in prompt engineering to increase the quality of the agent's documentation and Mermaid diagrams from complex code structures. The team recognized that while the initial investment in refining prompts was substantial, the greatest benefit would come from scaling the agent for long-term use across the team.

"We'll benefit from this experimentation when we reuse the prompt to keep our docs current." Shruti Corbett, Engineering Director

Step 3: Implementation, monitoring, and adjustment

Our Trade Compliance engineering lead observed during the pilot that the team could make the most of the agent by sharing optimized prompts more broadly across the organization. By democratizing access to these carefully crafted prompts, they could multiply the return on their initial investment and create a compounding effect throughout the engineering system.

This engineering insight improved what might have been a localized productivity increase into an opportunity for organization-wide knowledge-sharing transformation. The team is now committed to exploring how they can better share, version, and continuously improve these prompts as organizational assets.

"By treating prompts as reusable assets, a one-time investment becomes long-term leverage, first for our team and potentially across the company." Shruti Corbett, Engineering Director

The systemic impact of AI agents in engineering systems

As they continue to use their refined pilot prompt at scale, the team anticipates that documentation will become more comprehensive and accessible. When knowledge flows more freely between teams, collaboration barriers diminish, potentially contributing to faster time to market for new features and compliance updates.

This effect may compound over time as more teams adopt similar documentation practices leveraging agents, creating an interconnected knowledge ecosystem that accelerates innovation.

Three principles for engineering leaders in the age of agents

For enterprise leaders applying GitHub’s ESSP today, our Trade Compliance case study highlights key insights:

1. Investment vs. cost mindset: Approach prompt engineering and agent experimentation as investments in knowledge assets that compound in value over time, just like well-designed code libraries.
2. Complementary partnerships: Use the ESSP framework to identify where human and AI strengths can be optimally deployed with humans providing judgment and strategy, while agents can deliver consistency and scale.
3. Systems transformation: Measure success not just in the initial experience, but in how AI agents can change how you work across your engineering system — reducing friction points that previously hindered innovation.

Engineering success through human-AI collaboration

AI's potential isn't in replacing human creativity, but in removing the toil that constrains it. For our Trade Compliance team, the creation of documentation no longer competes with development for engineering attention. This means knowledge can flow more easily between teams and may unlock even more engineering success.

It's crucial to recognize that in this case, the transformative value of AI agents emerges through scale and reuse, when refined prompts become organizational assets that teams can leverage repeatedly. This is where the engineering system truly benefits from the compounding returns on that initial effort.

Our Trade Compliance team's journey shows that when AI agents are integrated intentionally into engineering systems, they become more than tools. They can become partners in transforming how we build software together, driving better business outcomes.

---

Want to learn more about the strategic role of AI and other innovations at GitHub? Explore Executive Insights for more thought leadership on the future of technology and business.