Adding InyaAI plugin #4512
Adding InyaAI plugin #4512
Conversation
|
|
📝 WalkthroughWalkthroughThis pull request introduces a new Gnani speech-to-text plugin for the LiveKit framework. The addition includes the plugin implementation, namespace packaging, configuration files, test coverage, and documentation. The plugin handles audio transcription via the Gnani Vachana API v3 with credential management, language options, and error handling. Changes
Sequence Diagram(s)sequenceDiagram
actor Client
participant STT as Gnani STT Plugin
participant AudioBuffer as Audio Buffer
participant HTTPSession as aiohttp Session
participant GnaniAPI as Gnani API
participant Response as Response Parser
Client->>STT: _recognize_impl(audio_buffer, language)
STT->>AudioBuffer: collect frames & convert to WAV bytes
STT->>STT: build multipart form (language_code, audio_file)
STT->>HTTPSession: _ensure_session() creates/reuses session
STT->>GnaniAPI: POST request with auth headers & request_id
GnaniAPI-->>HTTPSession: HTTP response (200 or error)
HTTPSession-->>STT: ClientResponse with status & JSON
alt API Error (non-200)
STT->>STT: log error details
STT-->>Client: raise APIStatusError
else Success (200)
STT->>Response: parse JSON response
alt response.success == false
STT-->>Client: raise APIConnectionError
else success == true
STT->>Response: extract transcript text
STT->>STT: _create_speech_event(text, language, request_id)
STT-->>Client: return SpeechEvent (FINAL_TRANSCRIPT)
end
end
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 inconclusive)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Fix all issues with AI agents
In `@CONTRIBUTING.md`:
- Around line 95-96: Update the incorrect plugin module name
`livekit.plugins.inyaai` to `livekit.plugins.gnani` wherever referenced (e.g.,
the two lines in CONTRIBUTING.md and the import in
tests/test_plugin_inyaai_stt.py); change the import path in the test from `from
livekit.plugins.inyaai...` to `from livekit.plugins.gnani...` and update any
test identifiers or docstrings that reference "inyaai" so they match the gnani
package naming.
In `@tests/test_plugin_inyaai_stt.py`:
- Around line 146-161: The test test_recognize_api_error should expect the HTTP
error exception raised by STT.recognize, not the network error; change the
assertion to expect APIStatusError (the exception raised for non-200 responses
in livekit.plugins.gnani.stt.STT.recognize) instead of APIConnectionError so the
test matches the implementation that throws APIStatusError for status != 200.
🧹 Nitpick comments (6)
livekit-plugins/livekit-plugins-gnani/README.md (1)
48-71: Basic usage example has incomplete imports.The example references
VoiceAssistant,silero, andttswithout showing their imports. This could confuse users trying to copy the example.📝 Suggested fix for imports
```python from livekit.agents import AutoSubscribe, JobContext, WorkerOptions, cli, llm +from livekit.agents.pipeline import VoiceAssistant from livekit.plugins import gnani +from livekit.plugins import silero +from livekit.plugins import openai as ttstests/test_plugin_inyaai_stt.py (1)
146-175: MoveAPIConnectionErrorimport to module level.
APIConnectionErroris imported multiple times inside individual test methods (lines 148, 167, 181). Moving it to the top-level imports improves readability and follows Python best practices.♻️ Suggested fix
Add to the imports at the top of the file:
from livekit import rtc from livekit.agents import stt +from livekit.agents import APIConnectionError from livekit.agents.stt import SpeechData, SpeechEventType from livekit.plugins.inyaai import STTThen remove the local imports from each test method.
livekit-plugins/livekit-plugins-gnani/pyproject.toml (1)
25-37: Consider adding Python 3.13 classifier.The classifiers list Python 3.9 through 3.12. If Python 3.13 is supported (which is likely given
>=3.9.0requirement), consider adding the classifier for discoverability on PyPI.📝 Suggested addition
"Programming Language :: Python :: 3.11", "Programming Language :: Python :: 3.12", + "Programming Language :: Python :: 3.13", "Programming Language :: Python :: 3 :: Only",livekit-plugins/livekit-plugins-gnani/setup.py (1)
31-32: Use context manager for file reading.The file handle for
README.mdis not explicitly closed. While Python's garbage collector will eventually close it, using a context manager is the recommended practice.♻️ Suggested fix
+with open("README.md", encoding="utf-8") as readme_file: + long_description = readme_file.read() + setup( name="livekit-plugins-gnani", version=about["__version__"], description="Agent Framework plugin for Gnani (Vachana) Speech-to-Text API.", - long_description=open("README.md").read(), + long_description=long_description, long_description_content_type="text/markdown",livekit-plugins/livekit-plugins-gnani/livekit/plugins/gnani/stt.py (2)
109-112: Guard against closed aiohttp sessions.
If a providedClientSessionis already closed, the current logic will reuse it and fail later with a less actionable error. Consider checking.closedand recreating (or raising explicitly).♻️ Proposed fix
- def _ensure_session(self) -> aiohttp.ClientSession: - if not self._session: - self._session = utils.http_context.http_session() - return self._session + def _ensure_session(self) -> aiohttp.ClientSession: + if self._session is None or self._session.closed: + self._session = utils.http_context.http_session() + return self._session
122-222: Normalize language before adding to form data.
IfGnaniLanguagesis a plain Enum,FormDatamay serialize the enum name rather than the language code. Coercing to.valuewhen present avoids incorrect payloads.♻️ Proposed fix
- form_data.add_field("language_code", target_language) + language_code = ( + target_language.value if hasattr(target_language, "value") else target_language + ) + form_data.add_field("language_code", language_code)
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
uv.lockis excluded by!**/*.lock
📒 Files selected for processing (14)
CONTRIBUTING.mdlivekit-plugins/livekit-plugins-gnani/README.mdlivekit-plugins/livekit-plugins-gnani/livekit/__init__.pylivekit-plugins/livekit-plugins-gnani/livekit/plugins/__init__.pylivekit-plugins/livekit-plugins-gnani/livekit/plugins/gnani/__init__.pylivekit-plugins/livekit-plugins-gnani/livekit/plugins/gnani/log.pylivekit-plugins/livekit-plugins-gnani/livekit/plugins/gnani/models.pylivekit-plugins/livekit-plugins-gnani/livekit/plugins/gnani/py.typedlivekit-plugins/livekit-plugins-gnani/livekit/plugins/gnani/stt.pylivekit-plugins/livekit-plugins-gnani/livekit/plugins/gnani/version.pylivekit-plugins/livekit-plugins-gnani/pyproject.tomllivekit-plugins/livekit-plugins-gnani/setup.pytests/docker-compose.ymltests/test_plugin_inyaai_stt.py
🧰 Additional context used
📓 Path-based instructions (1)
**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
**/*.py: Format code with ruff
Run ruff linter and auto-fix issues
Run mypy type checker in strict mode
Maintain line length of 100 characters maximum
Ensure Python 3.9+ compatibility
Use Google-style docstrings
Files:
livekit-plugins/livekit-plugins-gnani/livekit/plugins/__init__.pylivekit-plugins/livekit-plugins-gnani/livekit/plugins/gnani/models.pylivekit-plugins/livekit-plugins-gnani/livekit/plugins/gnani/version.pylivekit-plugins/livekit-plugins-gnani/livekit/plugins/gnani/stt.pytests/test_plugin_inyaai_stt.pylivekit-plugins/livekit-plugins-gnani/livekit/plugins/gnani/log.pylivekit-plugins/livekit-plugins-gnani/livekit/__init__.pylivekit-plugins/livekit-plugins-gnani/livekit/plugins/gnani/__init__.pylivekit-plugins/livekit-plugins-gnani/setup.py
🧠 Learnings (1)
📚 Learning: 2026-01-16T07:44:56.353Z
Learnt from: CR
Repo: livekit/agents PR: 0
File: AGENTS.md:0-0
Timestamp: 2026-01-16T07:44:56.353Z
Learning: Follow the Plugin System pattern where plugins in livekit-plugins/ are separate packages registered via the Plugin base class
Applied to files:
CONTRIBUTING.mdlivekit-plugins/livekit-plugins-gnani/livekit/__init__.py
🧬 Code graph analysis (2)
tests/test_plugin_inyaai_stt.py (4)
livekit-agents/livekit/agents/stt/stt.py (3)
SpeechData(53-61)SpeechEventType(32-49)SpeechEvent(70-74)tests/conftest.py (1)
job_process(19-22)livekit-agents/livekit/agents/_exceptions.py (1)
APIConnectionError(84-88)livekit-agents/livekit/agents/utils/http_context.py (1)
http_session(40-51)
livekit-plugins/livekit-plugins-gnani/livekit/plugins/gnani/__init__.py (2)
livekit-plugins/livekit-plugins-gnani/livekit/plugins/gnani/stt.py (1)
STT(46-242)livekit-agents/livekit/agents/plugin.py (2)
Plugin(13-56)register_plugin(31-36)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
- GitHub Check: type-check (3.9)
- GitHub Check: type-check (3.13)
- GitHub Check: unit-tests
🔇 Additional comments (24)
livekit-plugins/livekit-plugins-gnani/livekit/plugins/gnani/py.typed (1)
1-1: LGTM!Correct PEP 561 marker file for indicating type hint support in the package.
tests/docker-compose.yml (2)
63-65: LGTM!Environment variables for InyaAI credentials follow the established pattern used by other plugins in this compose file.
86-86: LGTM!The extra_hosts mapping for
api.vachana.aicorrectly routes the Gnani API through toxiproxy for integration testing, consistent with other service mappings.livekit-plugins/livekit-plugins-gnani/livekit/plugins/gnani/log.py (1)
1-17: LGTM!Standard logging setup following the established pattern in other LiveKit plugins. The logger name correctly matches the module path.
livekit-plugins/livekit-plugins-gnani/livekit/__init__.py (1)
1-2: LGTM!Correct namespace package setup using
pkgutil.extend_path, allowing thelivekitnamespace to be shared across multiple plugin packages. This follows the established plugin system pattern. Based on learnings, plugins in livekit-plugins/ are separate packages, and this namespace approach supports that architecture.livekit-plugins/livekit-plugins-gnani/livekit/plugins/__init__.py (1)
1-2: LGTM!Correct namespace package initialization for
livekit.plugins, consistent with the parent namespace setup.livekit-plugins/livekit-plugins-gnani/README.md (1)
1-135: LGTM!The documentation is comprehensive, covering installation, features, supported languages, configuration options, and usage examples. The API details and limitations sections are particularly helpful for users evaluating this plugin.
tests/test_plugin_inyaai_stt.py (3)
26-41: LGTM!The
audio_bufferfixture correctly creates a 1-second audio frame with proper 16-bit PCM parameters (16kHz, mono). This provides a realistic test fixture for STT recognition tests.
106-125: LGTM!The recognition success test properly mocks the API response and verifies all expected fields of the
SpeechEventincluding type, alternatives count, text, language, and confidence.
227-245: LGTM!Good coverage of all supported languages including the code-switching variant (
en-IN,hi-IN). This ensures the STT accepts the full range of documented language options.livekit-plugins/livekit-plugins-gnani/livekit/plugins/gnani/version.py (1)
1-15: LGTM!Simple and clean version module following standard Python packaging conventions.
livekit-plugins/livekit-plugins-gnani/pyproject.toml (1)
38-41: LGTM!Dependencies are appropriately specified with minimum versions, allowing flexibility for users while ensuring compatibility with the required features.
livekit-plugins/livekit-plugins-gnani/livekit/plugins/gnani/models.py (1)
17-29: LGTM!The
GnaniLanguagestype alias provides good type safety and IDE auto-completion for supported language codes. The inline comments documenting each language code are helpful.livekit-plugins/livekit-plugins-gnani/setup.py (2)
22-25: Version loading pattern is acceptable.The
exec()pattern for loading version from a separate file is a common setuptools idiom that avoids import-time side effects. This is fine for packaging purposes.
37-42: LGTM!The namespace package discovery and dependencies are correctly configured, matching the
pyproject.tomlspecification for consistency.livekit-plugins/livekit-plugins-gnani/livekit/plugins/gnani/__init__.py (4)
15-26: Clear module contract and exports.
Docstring and__all__make the public surface explicit and tidy.
34-36: Plugin initialization wiring looks good.
Constructor forwards name/version/package/logger cleanly.
39-39: Confirm import-time registration happens on the main thread.
Plugin.register_pluginraises outside the main thread; please verify this module is only imported on the main thread or registration is deferred to a controlled startup path.
41-48: Doc cleanup for unexported symbols is fine.
__pdoc__filtering aligns with the explicit__all__.livekit-plugins/livekit-plugins-gnani/livekit/plugins/gnani/stt.py (5)
41-43: Options dataclass is clean and minimal.
46-100: Credential validation and initialization look solid.
Clear env fallbacks and explicit errors.
101-107: Explicit model/provider identifiers are good.
114-120: Option updates are straightforward.
223-242: SpeechEvent assembly is clear.
✏️ Tip: You can disable this entire section by setting review_details to false in your review settings.
| -p livekit.plugins.anam \ | ||
| -p livekit.plugins.inyaai \ |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Verify the correct plugin module names in livekit-plugins/
echo "=== Listing livekit-plugins directories ==="
ls -la livekit-plugins/ 2>/dev/null | head -20
echo -e "\n=== Searching for plugin directories matching 'anam', 'inyaai', or 'gnani' ==="
fd -type d "(anam|inyaai|gnani)" livekit-plugins/
echo -e "\n=== Checking for pyproject.toml or setup.py in potential plugin dirs ==="
fd -type f "(pyproject.toml|setup.py)" livekit-plugins/ | grep -E "(anam|inyaai|gnani)" | head -20
echo -e "\n=== Contents of plugin __init__.py files ==="
fd -type f "^__init__.py$" livekit-plugins/livekit-plugins-anam --exec echo "File: {}" \; --exec head -5 {} \;
fd -type f "^__init__.py$" livekit-plugins/livekit-plugins-gnani --exec echo "File: {}" \; --exec head -5 {} \;Repository: livekit/agents
Length of output: 2358
🏁 Script executed:
#!/bin/bash
# Better search for plugin packages
echo "=== All plugin directories in livekit-plugins/ ==="
ls -1 livekit-plugins/ | grep -E "(anam|inyaai|gnani)"
echo -e "\n=== Full list of livekit-plugins directories (first 60) ==="
ls -1 livekit-plugins/
echo -e "\n=== Search for 'inyaai' anywhere in repo ==="
grep -r "inyaai" . --include="*.md" --include="*.py" --include="*.toml" 2>/dev/null | head -20
echo -e "\n=== Search for 'gnani' anywhere in repo ==="
grep -r "gnani" . --include="*.md" --include="*.py" --include="*.toml" 2>/dev/null | head -20
echo -e "\n=== View CONTRIBUTING.md around lines 95-96 ==="
sed -n '90,100p' CONTRIBUTING.mdRepository: livekit/agents
Length of output: 3735
🏁 Script executed:
#!/bin/bash
# Check gnani plugin structure and module definition
echo "=== Contents of livekit-plugins-gnani/livekit/plugins/gnani/__init__.py ==="
cat livekit-plugins/livekit-plugins-gnani/livekit/plugins/gnani/__init__.py
echo -e "\n=== Check if there are any aliases or inyaai references in gnani ==="
grep -r "inyaai" livekit-plugins/livekit-plugins-gnani/ 2>/dev/null
echo -e "\n=== Check test_plugin_inyaai_stt.py imports and assertions ==="
cat tests/test_plugin_inyaai_stt.py | head -30Repository: livekit/agents
Length of output: 2433
Change livekit.plugins.inyaai to livekit.plugins.gnani.
The plugin module is livekit.plugins.gnani (from the livekit-plugins-gnani package), not livekit.plugins.inyaai. This reference will cause mypy to fail. Note that tests/test_plugin_inyaai_stt.py also has the incorrect import path and will need to be updated.
🤖 Prompt for AI Agents
In `@CONTRIBUTING.md` around lines 95 - 96, Update the incorrect plugin module
name `livekit.plugins.inyaai` to `livekit.plugins.gnani` wherever referenced
(e.g., the two lines in CONTRIBUTING.md and the import in
tests/test_plugin_inyaai_stt.py); change the import path in the test from `from
livekit.plugins.inyaai...` to `from livekit.plugins.gnani...` and update any
test identifiers or docstrings that reference "inyaai" so they match the gnani
package naming.
| async def test_recognize_api_error(self, mock_api_credentials, audio_buffer, job_process): | ||
| """Test handling of API errors""" | ||
| from livekit.agents import APIConnectionError | ||
|
|
||
| stt_instance = STT(language="en-IN") | ||
|
|
||
| # Mock the API response with error | ||
| mock_response = MagicMock() | ||
| mock_response.status = 400 | ||
| mock_response.text = AsyncMock(return_value="Bad Request") | ||
|
|
||
| with patch.object(aiohttp.ClientSession, "post") as mock_post: | ||
| mock_post.return_value.__aenter__.return_value = mock_response | ||
|
|
||
| with pytest.raises(APIConnectionError): | ||
| await stt_instance.recognize(buffer=audio_buffer, conn_options=TEST_CONNECT_OPTIONS) |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
# Find the InyaAI STT implementation file
find . -name "*inyaai*" -type f | grep -v __pycache__Repository: livekit/agents
Length of output: 91
🏁 Script executed:
# Search for plugin-related files and STT implementations
git ls-files | grep -E "(plugin|stt)" | head -20Repository: livekit/agents
Length of output: 1031
🏁 Script executed:
# List plugin directories to find inyaai
ls -la livekit-plugins/ | grep -i inyaaiRepository: livekit/agents
Length of output: 40
🏁 Script executed:
# List all directories in livekit-plugins
ls -d livekit-plugins/livekit-*/ | head -20Repository: livekit/agents
Length of output: 864
🏁 Script executed:
# Continue listing to find inyaai
ls -d livekit-plugins/livekit-*/ | grep -i inyaaiRepository: livekit/agents
Length of output: 40
🏁 Script executed:
# Read the test file to understand imports and context
head -50 tests/test_plugin_inyaai_stt.pyRepository: livekit/agents
Length of output: 1710
🏁 Script executed:
# Find the inyaai plugin implementation
find . -path "*/plugins/inyaai*" -type f | head -20Repository: livekit/agents
Length of output: 40
🏁 Script executed:
# Search the entire codebase for "inyaai" references
rg -l "inyaai" --type pyRepository: livekit/agents
Length of output: 89
🏁 Script executed:
# Check if the import path exists
ls -la livekit-plugins/livekit-plugins-*/livekit/plugins/ | grep inyaaiRepository: livekit/agents
Length of output: 40
🏁 Script executed:
# Read the full test file to understand the context better
cat -n tests/test_plugin_inyaai_stt.pyRepository: livekit/agents
Length of output: 11934
🏁 Script executed:
# Search for the inyaai plugin import source
rg "class STT" --type py | grep -i inyaaiRepository: livekit/agents
Length of output: 40
🏁 Script executed:
# Search for where the STT class is defined in inyaai context
rg "class STT" --type py -A 5 | head -50Repository: livekit/agents
Length of output: 4486
🏁 Script executed:
# Read the Gnani STT implementation to check exception handling
cat -n livekit-plugins/livekit-plugins-gnani/livekit/plugins/gnani/stt.pyRepository: livekit/agents
Length of output: 10917
🏁 Script executed:
# Check if there's an inyaai plugin or if it's an alias for gnani
find . -path "*/plugins/inyaai*" -type dRepository: livekit/agents
Length of output: 40
🏁 Script executed:
# Check gnani plugin's __init__.py file
cat -n livekit-plugins/livekit-plugins-gnani/livekit/plugins/gnani/__init__.pyRepository: livekit/agents
Length of output: 1680
🏁 Script executed:
# Check if inyaai exists as a separate plugin directory or symlink
ls -la livekit-plugins/ | grep -i inyaaiRepository: livekit/agents
Length of output: 40
Fix incorrect exception type in test_recognize_api_error.
The test expects APIConnectionError for a 400 HTTP status response, but the implementation raises APIStatusError for non-200 status codes (see line 182-190 in livekit-plugins/livekit-plugins-gnani/livekit/plugins/gnani/stt.py). APIConnectionError is reserved for network/connection failures and application-level API errors (lines 195-198, 216-218), while APIStatusError is raised for HTTP error responses.
Update the test to expect APIStatusError instead.
🤖 Prompt for AI Agents
In `@tests/test_plugin_inyaai_stt.py` around lines 146 - 161, The test
test_recognize_api_error should expect the HTTP error exception raised by
STT.recognize, not the network error; change the assertion to expect
APIStatusError (the exception raised for non-200 responses in
livekit.plugins.gnani.stt.STT.recognize) instead of APIConnectionError so the
test matches the implementation that throws APIStatusError for status != 200.
3 participants
We have integrated Gnani's Speech-to-Text with LiveKit Agents. The service provides low-latency, high-accuracy transcription for Indian languages and accents, supporting English (Indian), Hindi, Tamil, Telugu, and more. It offers secure API key–based authentication and real-time transcription optimized for conversational and voice-based applications.
Summary by CodeRabbit
Release Notes
New Features
Documentation
Tests
✏️ Tip: You can customize this high-level summary in your review settings.