name: Benchmark Submission
description: Submit evaluation or benchmark results for a model configuration
labels: ["benchmark"]

body:
  - type: textarea
    id: results
    attributes:
      label: Results
      value: |
        ### Model

        Name and provider (e.g., gpt-4o, claude-3.5-sonnet, llama-3.1-70b)

        ### Configuration

        ```python
        config = {
            "llm": {"provider": "...", "config": {"model": "..."}},
            "vector_store": {"provider": "...", "config": {...}}
        }
        ```

        ### Dataset

        Which dataset or eval suite did you use?

        ### Results

        | Metric       | Score |
        |--------------|-------|
        | Precision    |       |
        | Recall       |       |
        | F1           |       |
        | Latency (ms) |       |

        ### Environment

        - Hardware:
        - mem0 version:
        - Python version:
    validations:
      required: true