Skip to content

Conversation

@baominghelly
Copy link
Collaborator

Description

create tools for model_summary parse and mock operators execution

Test

Unit test

(base) ➜  InfiniMetrics git:(add_new_tools) python -m unittest discover tests
.....
============================================================
🚀 Starting operator performance test (2 total)...
============================================================

--- Running (1/2): test.layer1 ---
  > Status: SUCCESS, Latency: 32.4055 ms

--- Running (2/2): test.layer2 ---
  > Status: SUCCESS, Latency: 13.4484 ms

============================================================
✅ All operator tests completed.
============================================================
.
----------------------------------------------------------------------
Ran 6 tests in 0.453s

OK

Test with real data

(base) ➜  benchmark git:(add_new_tools) python parse_summary.py -i model_summary
--- 1. Parsing Phase ---
Parsing model_summary ...
Parsing complete! Found 76 operator instances in sequential order.
Test cases saved to: operator_test_cases.json
(base) ➜  benchmark git:(add_new_tools) ✗ python run_benchmark.py -i operator_test_cases.json
--- 1. Loading Test Configuration ---
Successfully loaded 76 test cases (from operator_test_cases.json)

============================================================
🚀 Starting operator performance test (76 total)...
============================================================

--- Running (1/76): transformer.wte ---
  > Status: SUCCESS, Latency: 62.6148 ms

--- Running (2/76): transformer.wpe ---
  > Status: SUCCESS, Latency: 94.1032 ms

--- Running (3/76): transformer.h.0.ln_1 ---
  > Status: SUCCESS, Latency: 82.2264 ms

...

--- Running (72/76): transformer.h.11.ln_2 ---
  > Status: SUCCESS, Latency: 65.5149 ms

--- Running (73/76): transformer.h.11.mlp.c_fc ---
  > Status: SUCCESS, Latency: 65.9573 ms

--- Running (74/76): transformer.h.11.mlp.c_proj ---
  > Status: SUCCESS, Latency: 36.6459 ms

--- Running (75/76): transformer.ln_f ---
  > Status: SUCCESS, Latency: 30.699 ms

--- Running (76/76): lm_head ---
  > Status: SUCCESS, Latency: 95.2294 ms

============================================================
✅ All operator tests completed.
============================================================
--- 3. Saving Final Report ---
Final assessment report (including latency) saved to: operator_test_results_20251121_105423.json

@baominghelly baominghelly self-assigned this Nov 21, 2025
@Chamberlain0w0
Copy link
Contributor

这个 infinimetrics/tools/ 文件夹是什么定位?跟放在 infinimetrics/ 下面其他路径的 py 文件有什么关系?

@@ -0,0 +1,131 @@
======================================================================================================================================================
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

输入文件不提交到 git repo 内

It pretends to run a test and returns a simulated latency.

(In the future, we can replace this with a real call like:
`actual_latency = infiniCore.benchmark(...)`)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个信息最好在对应要替换的代码行上面加一个 TODO,这样清楚一些,便于之后定位

@@ -0,0 +1,49 @@
import unittest
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

单元测试的部分目前感觉不是特别必要,因为逻辑都是输入输出格式的检查,但之后如果格式还有变化的话,还需要再重新改这部分

@Chamberlain0w0
Copy link
Contributor

我感觉还有一个点就是现在的逻辑过于“流式”了,观感上很像很多零碎的脚本。我感觉如果要形成一个“框架”的话,实现上最好还是面向对象一点,比如说负责读入的定义一个 class Parser,然后固定几个操作接口;中间定义一个 Excecutor,结果输出再定义一个 Reporter,中间涉及的一些处理过的数据也可以定义为结构化的数据,这样的话所有测试都可以用一套逻辑调用,扩展性更强,并且 main 的脚本里会显得更清楚一些。晚点可以再讨论一下。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants