feat: Ads/eng 4109 implement ralphhook for iterative agent refinement

[GangGreenTemperTatum]

[HOOKS] ralph hook

Key Changes:

• creates a ralph loop (IE here) into the SDK, both are about iterative refinement with observability, but they achieve it differently and this attempt aims to bridge a gap and follows same structure as backoff_on_error, summarize_when_long
• event-driven system by listening for GenerationEnd events and intercepting completion attempts (responses without tool calls). When the agent produces a final answer, ralph_hook uses the SDK's scorer composition (avg() to combine multiple scorers) to evaluate output quality, then returns RetryWithFeedback if the score is below threshold—injecting feedback as a user message and forcing regeneration to create an iterative refinement loop. The agent's reaction processor prioritizes these reactions, continuing until output meets the quality threshold (Finish) or max iterations is reached (Fail). Each session keeps isolated state via ULID keys, and the hook resets on StepStart to avoid interfering with multi-step reasoning.

Added:

• ralph hook and tests

---

Generated Summary:

Summary of Changes

• Introduced a new hook ralph_hook that implements iterative agent refinement based on scoring thresholds.
• Added functionality to score outputs of agent responses and provide feedback for improvements up to a specified maximum number of iterations.
• Implemented a state management system for tracking scoring history and iterations for each agent session.
• Enhanced the summarize_when_long function to include a preserve_tool_pairs option, ensuring that tool call/response pairs are kept together during summarization.

Key Modifications

• New Hook - ralph_hook:

  • Tracks iterations and scoring for agent responses.
  • Provides feedback if the score does not meet minimum requirements.
  • Supports multiple scoring functions and averages scores when multiple are provided.
  • Throws validation errors for incorrect parameter values (e.g., negative iterations, out-of-bound scores).

• Summarization Enhancement:

  • Added a new boolean parameter preserve_tool_pairs to ensure that tool call/response pairs are not orphaned during summarization.
  • Adjusted the logic to find summarization boundaries, utilizing tool-awareness if preserve_tool_pairs is set to true.

• Testing:

  • Created unit tests for the ralph_hook, covering various scenarios including convergence, multiple scorers, maximum iteration limits, and session isolation.
  • Implemented tests for the new preserve_tool_pairs functionality to validate correct behavior in different message scenarios.

Potential Impact

• These changes improve the agent's ability to refine its outputs iteratively, potentially leading to better quality responses.
• The summary improvement supports strict API requirements of external services, likely reducing API errors during tool interactions.
• The introduction of systematic tests enhances the reliability and maintainability of the features added.

This summary was generated with ❤️ by rigging