Case study · 2024

ITAF

Ideyalabs Testing Automation Framework

Turns Jira stories into self-healing browser tests that run against live applications.

Understandstory to test stepsExploremaps the live UIExecutereason-and-act loopSelf-healfixes only what brokeReuseruns become scriptsDiagnosedrift, defect or errorbuilt as an internabout 60% less testing effort Understandstory to test stepsExploremaps the live UIExecutereason-and-act loopSelf-healfixes only what brokeReuseruns become scriptsDiagnosedrift, defect or errorbuilt as aninternabout 60% lesstesting effort
How ITAF works. The coloured notes are my annotations.

What it does

ITAF hands regression testing to agents. It reads the Jira story, explores the live application in a headless browser, runs each case through a reasoning-and-action loop, repairs broken steps on its own and explains failures with evidence.

For two enterprise customers it cut the manual and automation testing effort needed by about 60% compared with conventional regression cycles, shifting testers toward validation and audit.

  • About 60% less manual and automation testing effort than conventional regression cycles
  • 2 enterprise customers

How it works

  1. Understand

    LangChain and LangGraph workflows turn Jira user stories and ticket descriptions into executable plain-English test steps.

  2. Explore

    Playwright headless browsers, rendered-screen context and normalized DOM snapshots reveal features and the current UI state.

  3. Execute

    Reasoning-and-action loops generate data for each test and run cases sequentially against the live application.

  4. Self-heal

    When the UI changes, the model repairs only the affected interactive element and keeps the validated test flow intact.

  5. Reuse

    Successful runs become reusable Playwright scripts or English steps, then expand into positive and negative scenarios.

  6. Diagnose

    Failure analysis separates ticket drift, application defects and unrecoverable errors, with evidence-backed root causes.

What I built

  • Built an AI testing platform for two enterprise customers, reducing required manual and automation testing effort by about 60% versus conventional regression cycles and shifting testers toward validation and audit.
  • Developed LangChain and LangGraph workflows that convert Jira user stories and ticket descriptions into executable plain-English test steps.
  • Used Playwright headless browsers, rendered-screen context and normalized DOM snapshots to discover features and understand the current UI state.
  • Built reasoning-and-action execution loops with per-test data generation to run test cases sequentially against live applications.
  • Implemented LLM-powered self-healing that updates only the affected interactive element while preserving the validated test flow.
  • Converted successful runs into reusable Playwright scripts or English steps and expanded them into positive and negative scenarios.
  • Added failure analysis for ticket drift, application defects and unrecoverable errors, returning evidence-backed root-cause information.
  • Built a guarded RAG assistant for ITAF guidance and testing access, using FastMCP for tools and Socket.IO for real-time updates.