Trace Solutions LLM-Powered Test Case Generation and Validation
Trace Solutions partnered with BTM to assess whether LLMs could improve the speed, quality, and cost-efficiency of test case generation in real engineering workflows.
The challenge
The brief was intentionally open-ended, requiring the team to scope a complex problem under conditions of uncertainty. They needed to define exactly how AI tools should be responsibly and effectively introduced into established, real-world engineering workflows without disrupting existing processes.
The solution
BTM approached the challenge systematically, beginning with rigorous model benchmarking across GPT-4, Claude 3, and Mistral 7B. This progressed into structured validation against specific requirement sets and JIRA mappings. Working iteratively with Trace Solutions practitioners, the team developed and delivered a fully functional prototype featuring document upload capabilities, embedded-prompt test generation, and direct JIRA export integration.
The outcome
Trace Solutions highly commended the team for their professionalism, strong communication, and creative contributions to the project. The company utilized the findings and the delivered prototype to directly shape their internal AI adoption strategy and inform future technology rollout planning.
“Working with BTM was fantastic. Their expertise has not only improved our workflow, but also helped us realise how AI can be applied to other areas of our business.”
