44 lines
960 B
YAML
44 lines
960 B
YAML
name: Benchmark Submission
|
|
description: Submit evaluation or benchmark results for a model configuration
|
|
labels: ["benchmark"]
|
|
|
|
body:
|
|
- type: textarea
|
|
id: results
|
|
attributes:
|
|
label: Results
|
|
value: |
|
|
### Model
|
|
|
|
Name and provider (e.g., gpt-4o, claude-3.5-sonnet, llama-3.1-70b)
|
|
|
|
### Configuration
|
|
|
|
```python
|
|
config = {
|
|
"llm": {"provider": "...", "config": {"model": "..."}},
|
|
"vector_store": {"provider": "...", "config": {...}}
|
|
}
|
|
```
|
|
|
|
### Dataset
|
|
|
|
Which dataset or eval suite did you use?
|
|
|
|
### Results
|
|
|
|
| Metric | Score |
|
|
|--------------|-------|
|
|
| Precision | |
|
|
| Recall | |
|
|
| F1 | |
|
|
| Latency (ms) | |
|
|
|
|
### Environment
|
|
|
|
- Hardware:
|
|
- mem0 version:
|
|
- Python version:
|
|
validations:
|
|
required: true
|