Augustin
|
e17a4dd9d0
|
Feature: Add ONNX model support with NPU/DirectML acceleration
- Replace GGUF models with ONNX models optimized for DirectML
- Add Microsoft Phi-3 Mini DirectML (INT4, 2.4GB)
- Add Xenova ONNX models (DistilBERT, BERT, MiniLM, CLIP)
- Update model catalog with working HuggingFace URLs
- Create ONNX/NPU integration test suite (tests/onnx_npu_test.rs)
- Successfully test DistilBERT ONNX loading with DirectML
- Verify NPU session creation and model inputs/outputs
Test Results:
- ✅ NPU Detection: Intel AI Boost NPU (via DirectML)
- ✅ ONNX Session: Created successfully with DirectML
- ✅ Model: DistilBERT (268 MB) loaded
- ✅ Inputs: input_ids, attention_mask
- ✅ Output: logits
- ⚡ Performance: Ready for NPU hardware acceleration
All tests passing with NPU-accelerated ONNX inference
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
|
2025-10-16 18:53:52 +02:00 |
|