Feature/tool calling for local llm #117
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
srebrek
force-pushed
the
feature/tool-calling-for-local-llm
branch
from
08f75d2 to
06f6693
Compare
Contributor
|
After a test, I noticed that message about tool invocation is also printed by notification service - Its not what we want, LLMService should be able to ommit this message as its not "user friendly" message, Maybe some tweaks can be done in similar way as we process reasoning models, so we can apply tokenType to tool messages |
wisedev-pstach
requested changes
Jan 15, 2026
wisedev-pstach
requested changes
Jan 21, 2026
pswierad-src
requested changes
Jan 21, 2026
- Prevent tool definition duplication in the system prompt during subsequent loop iterations. - Refine system prompt to enforce format more effectively.
- Add invalid chat responses during ToolCalling to the chat, so it sees them in the next prompts. - Mark first message as processed after processing. - Removeunnecessary parsing methods. - Add messages about invalid formating to the promt so model sees what went wrong. - Format and clean the code.
- OpenAICompatibleService and LLMService use Tool Classes Defined in the Domain. - Extract Json-Tool parse logic to helper function.
srebrek
force-pushed
the
feature/tool-calling-for-local-llm
branch
from
6d49789 to
89f3382
Compare
wisedev-pstach
approved these changes
Jan 27, 2026
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Working implementation of tool calling for local LLMs.
Left TODOs about testing with other LLMs. Currently working perfectly with gemma3:4b (example 7).
Closes #111