-
Notifications
You must be signed in to change notification settings - Fork 2
Cid/attachments #581
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Cid/attachments #581
Conversation
| if name: | ||
| processed_msg["name"] = name | ||
| processed_messages.append(processed_msg) | ||
| else: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
will this cause issues?
when the content type is unknown?
| elif item_type == "file": | ||
| file_data = item.get("file", {}) | ||
| file_id = file_data.get("file_id") | ||
| file_data_b64 = file_data.get("file_data") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
what if the base64 is too big? I don't know if it will cause issues while sending like for example if the request size exceeds.
| elif file_id: | ||
| # Just reference the file ID (can't download without API call) | ||
| attachment = Attachment(name=filename) | ||
| attachment.metadata["openai_file_id"] = file_id |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if we just have openai_file_id, what will we show in the UI?
| The attachment with storage_uri populated (if upload was needed). | ||
| """ | ||
| # Skip if already uploaded | ||
| if attachment.is_uploaded(): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
should we still upload data from openai or anywhere else to our storage? they might remove that data after sometime.
| try: | ||
| from .attachment_uploader import upload_trace_attachments | ||
|
|
||
| upload_count = upload_trace_attachments(trace) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this _upload_and_publish_trace will be called when we do have an attachment to upload - right?
because if theres nothing to upload then upload_count will be 0.
Pull Request
Summary
Introduces:
Changes
Attachments
Attachmentabstraction, which represents media data uploaded to a blob store. TheAttachmentobject stores metadata and thestorageUri, which can be used by the Openlayer platform to fetch the media.attachmentsfield, which is an array ofAttachmentobjects. This allows users to log arbitrary media to a step. For example:OpenAI multimodal
attachments, this PR also instruments thetrace_openaiwrapper to parse image/audio/files in the input or output of OpenAI LLM calls.Note that if the
typeis one ofimage,audio, orfile, the other object field is anattachment, which is a serializedAttachmentobject.Context
OPEN-8683: Multimodal attachment support for the Python SDK and OPEN-8684: Enhance OpenAI tracer to support multimodal inputs/outputs
Testing