(下面用 codex --yolo 去执行）
锁频后，参照 https://github.com/yzh119/flashinfer-dev/tree/profiling-script 去测下面实现的 speedup，以表格形式 PR/#1234 or Folder path | xxx speedup 给我
参照测试脚本 CUDA_VISIBLE_DEVICES=0 bash benchmarks/kda_6shape_gb300/run.sh
具体的 Kernel 实现: ./kda_fwd_cute/best
For python project, you should first follow the README to setup. If not specified, then use uv to create local env for dev. If it requires specific cuda version, or depends on library like (verl,sglang,vllm), then use conda instead.
which srun
| PYTHONUNBUFFERED=1 \ | |
| python3 -m verl.trainer.main_ppo \ | |
| algorithm.adv_estimator=grpo \ | |
| data.train_files="<your data>" \ | |
| data.val_files="<your val>" \ | |
| data.filter_overlong_prompts=True \ | |
| data.train_batch_size=128 \ | |
| data.val_batch_size=512 \ | |
| data.max_prompt_length=9216 \ | |
| data.max_response_length=8192 \ |
| root@batch-block1-3665:/home/ligengz/workspace/slime# bash scripts/run-qwen3-4B.sh | |
| Did not find any active Ray processes. | |
| + export PYTHONBUFFERED=16 | |
| + PYTHONBUFFERED=16 | |
| ++ nvidia-smi topo -m | |
| ++ grep -o 'NV[0-9][0-9]*' | |
| ++ wc -l | |
| + NVLINK_COUNT=56 | |
| + '[' 56 -gt 0 ']' |
| python -m sglang.launch_server \ | |
| --model Qwen/qwen3-8B --port 8000 \ | |
| --dtype auto --data-parallel-size 8 --tensor-parallel-size 1 | |
| ################################################################## | |
| lm_eval --model local-completions \ | |
| --tasks gsm8k,gsm8k_cot,aime24,aime25 \ | |
| --model_args model=Qwen/qwen3-8B,base_url=http://127.0.0.1:8000/v1/completions,num_concurrent=32,max_retries=3 | |
| // MAIN CONVERSATION | |
| {"role": "system", "content": "You are Qwen, created, by alibaba cloud, you are a helpful assistant. You are a helpful assistant."} | |
| {"role": "user", "content": "Find the prime from 1 to 10"} | |
| {"role": "assistant", "content": "Okay, so I need to find the prime numbers between 1 and 10. Let's think in parallel <Parallel>\n<Goal>\n<Outline>\n1. Find the prime numbers between 1 and 5.</Outline>\n<Outline>\n2. Find the prime numbers between 6 and 10.</Outline>\n</Goal>\n"} //OUTPUT// Note: the end of the conversation is not <|im_end|> but rather </Goal> | |
| {"role": "assistant", "content": "I will think about the prime numbers between 1 and 5 first. ....."} | |
| {"role": "assistant", "content": "I will then think about the prime numbers between 6 and 10 ....."} | |
| {"role": "assistant", "content": "After analyzing both ranges, the prime numbers between 1 and 10 are: 2, 3, 5, 7."} //OUTPUT | |
| // THREAD CONVERSATION 1 | |
| {"role": "system", "content": "You are Qwen, created, by alibaba cloud, you are a helpful assi |
| import os | |
| import json | |
| import time | |
| import concurrent.futures | |
| from tqdm import tqdm | |
| from google import genai | |
| from google.genai import types | |
| import fire | |
| def load_video_info_from_jsonl(jsonl_path): |
| import pandas as pd | |
| import time | |
| from sglang import gen, system | |
| import os, sys, os.path as osp | |
| import asyncio | |
| import openai | |
| from tqdm import tqdm | |
| import json | |
| import sys | |
| from sglang.utils import wait_for_server, print_highlight, terminate_process |
| ligengz@:~/workspace/VILA-dev$ python serving/lmsys_test.py --model openai/nvila-8b-dev --api-base http://localhost:8000 --req-per-sec 1 | |
| /home/ligengz/anaconda3/envs/hf/lib/python3.10/site-packages/transformers/tokenization_utils_base.py:1601: FutureWarning: `clean_up_tokenization_spaces` was not set. It will be set to `True` by default. This behavior will be depracted in transformers v4.45, and will be then set to `False` by default. For more details check this issue: https://github.com/huggingface/transformers/issues/31884 | |
| warnings.warn( | |
| /home/ligengz/anaconda3/envs/hf/lib/python3.10/site-packages/pydantic/main.py:347: UserWarning: Pydantic serializer warnings: | |
| Expected `str` but got `int` - serialized value may not be as expected | |
| return self.__pydantic_serializer__.to_python( | |
| Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new | |
| LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'. |
| // https://www.shj.work/tools/secha/ | |
| // cheat code for the small color testing game | |
| function processPlates() { | |
| const d = {}; | |
| const plates = document.querySelectorAll("#box span"); | |
| plates.forEach(p => { | |
| if (!(p.style.backgroundColor in d)) { |