Grades the output of AI agents and chatbots against quality criteria, so you can test an app at scale instead of reviewing answers by hand. It also checks academic papers for correctness and novelty and verifies BibTeX references for fabricated citations.
@agentscope-aiA curated collection of skills around AgentScope ecosystem and QwenPaw applications, ready to be installed into AI coding assistants like Claude Code, Cursor, and more.
@agentscope-aiUse when the user wants help with academic papers or citations but it's unclear which specific workflow fits — reviewing a paper,…
@agentscope-aiUse when the user wants to compare or benchmark multiple LLMs/agents arena-style but it's unclear which specific workflow fits —…
@agentscope-aiUse when the user wants to build an evaluation system for an LLM/agent application but doesn't know where to start — they have…
@agentscope-aiAutomatically evaluate and compare multiple AI models or agents without pre-existing test data. Generates test queries from a…
@agentscope-aiUse when the user needs to design evaluation datasets, create test cases, stratify samples, generate adversarial examples,…
@agentscope-aiBuild custom LLM evaluation pipelines using the OpenJudge framework. Covers selecting and configuring graders (LLM-based,…
@agentscope-aiReview academic papers for correctness, quality, and novelty using OpenJudge's multi-stage pipeline. Supports PDF files and LaTeX…
@agentscope-aiVerify a BibTeX file for hallucinated or fabricated references by cross-checking every entry against CrossRef, arXiv, and DBLP.…
@agentscope-aiUse when the user has evaluation principles or a dataset but needs help choosing the right graders, designing evaluation metrics,…
@agentscope-aiBenchmark LLM reference recommendation capabilities by verifying every cited paper against Crossref, PubMed, arXiv, and DBLP.…
@agentscope-aiBuild RL reward signals using the OpenJudge framework. Covers choosing between pointwise and pairwise reward strategies based on…
@agentscope-aiUse when the user has a judge/grader and human-labeled data, and wants to measure how well the judge agrees with humans, detect…
@agentscope-aiIs this you? Claim your profilePut a plugin at the top of Plugins, from $9
