/lop_eval_5__complex_engineering_test

Problem #5: Complex Engineering Project Test

Instructions

• Pass the prompt into each nano-agent AS IS, replacing MODEL_NAME with the actual model name.
• This is a comprehensive engineering task requiring multiple file operations and code generation.

Variables

PROMPT: "Create a complete mini Python project for a 'Token Counter Utility' with the following requirements:

1. Create these files with unique names using your model name:

a) 'token_counter_MODEL_NAME.py' (replace MODEL_NAME with your actual model name) - Main module with:

• A class TokenCounter with methods:
• __init__(self, model_name='MODEL_NAME')
• count_tokens(self, text: str) -> int (simple word count * 1.3 as estimate)
• get_cost(self, token_count: int) -> float (use $0.001 per 1000 tokens)
• A docstring explaining it was created by MODEL_NAME

b) 'test_token_counter_MODEL_NAME.py' - Test file with:

• At least 2 test cases for the TokenCounter class
• Tests for both count_tokens and get_cost methods
• A comment: '# Tests generated by MODEL_NAME'

c) 'README_MODEL_NAME.md' - Documentation with:

• Project title and description
• Usage example
• Installation instructions
• Footer: '---\nGenerated by MODEL_NAME for nano-agent evaluation'

d) 'usage_example_MODEL_NAME.py' - Example script that:

• Imports TokenCounter from your token_counter file
• Reads the first 100 characters from the main README.md
• Counts tokens and calculates cost
• Prints results
• Includes comment: '# Example created by MODEL_NAME'

2. After creating all files:

• List the current directory
• Read and return the content of your README_MODEL_NAME.md

3. Create a final summary file 'project_summary_MODEL_NAME.json' with: { 'model': 'MODEL_NAME', 'files_created': [<list of all files>], 'total_lines_of_code': <count>, 'project_status': 'complete', 'signature': 'Engineering test completed by MODEL_NAME' }

Return the directory listing and confirmation that all tasks were completed.

Respond with your entire JSON response structure as is."

Agents

IMPORTANT: You're calling the respective claude code sub agents - do not call the `mcp__nano-agent__prompt_nano_agent` tool directly, let the sub agent's handle that.

@agent-nano-agent-gpt-5-nano PROMPT @agent-nano-agent-gpt-5-mini PROMPT @agent-nano-agent-gpt-5 PROMPT @agent-nano-agent-claude-opus-4-1 PROMPT @agent-nano-agent-claude-opus-4 PROMPT @agent-nano-agent-claude-sonnet-4 PROMPT @agent-nano-agent-claude-3-haiku PROMPT @agent-nano-agent-gpt-oss-20b PROMPT @agent-nano-agent-gpt-oss-120b PROMPT

Expected Output

Verify each agent created a complete mini-project with all required files and that the code is functional.

IMPORTANT: All agents must will respond with this JSON structure. Don't change the structure or add any additional fields. Output it as the given structure as raw JSON for each agent with no preamble.

{
    "success": true,
    "result": "<directory listing and completion confirmation>",
    "error": null,
    "metadata": {
        ...keep all fields given
    },
    "execution_time_seconds": X.XX
}

Grading rubric

• Did the agent create all 5 required files with all specified components?
• Is the Python code syntactically correct and well-structured?
• Did the agent provide clear documentation and maintain consistency across the project?
• Do the files work together as a cohesive project?