/skill

Scholar Evaluation

Use this skill to review research work with a clear and repeatable rubric.

When to Use

Use this skill to:

• Review a paper, proposal, thesis chapter, or literature review.
• Check if sources support the claims tied to them.
• Review methods, study design, analysis, or limits.
• Compare two or more works with the same rubric.
• Give clear feedback for a revision.

Set the Scope

First, name the type of work:

• Data-based research paper
• Theory paper
• Technical report
• Systematic review
• Narrative review
• Research proposal
• Thesis or dissertation chapter
• Conference abstract or short paper

Then choose a review scope:

• Full: Score all parts that apply.
• Focused: Score only one or two parts, such as methods or citations.
• Compare: Score two or more works with the same rules.

State the scope before the review. Do not score a part that was not given to you. Mark it N/A.

Score Guide

Score each part from 1 to 5:

• 5: Strong, clear, careful, and ready to share.
• 4: Good, with small changes needed.
• 3: Usable, but key gaps remain.
• 2: Weak, with large changes needed.
• 1: Not sound or not clear.
• N/A: Does not apply or cannot be checked.

Use whole numbers. Give proof for each score. Do not use a score when there is too little text to judge.

For a full review, the total score is the mean of all scored parts. Leave out N/A parts. Round to one decimal place.

Review Rubric

1. Problem and Research Question

Check whether:

• The problem is clear and exact.
• The work matters.
• The scope and key beliefs are stated.
• The question fits the claimed value of the work.

2. Past Work and Context

Check whether:

• The key past work is covered.
• Sources are joined into a clear view, not just listed.
• The research gap is stated with care.
• New and key older sources are both used when needed.

3. Methods

Check whether:

• The method can answer the research question.
• The design choices have clear reasons.
• Data, people, tools, and steps are explained.
• Another person could repeat the work.
• Ethics and real-world limits are covered.

For a proposal, score the planned method. Do not ask for results that do not exist yet.

4. Data and Evidence

Check whether:

• The data source is fit for the task.
• The sample or set of texts is large and broad enough.
• Rules for adding, removing, and cleaning data are stated.
• Missing data and sources of bias are discussed.

For theory papers, judge the logic and source base instead of a data set.

5. Analysis

Check whether:

• The form of analysis fits the question and data.
• Tests, control groups, and baselines are fair.
• Error, doubt, and result stability are checked when needed.
• Other reasons for the result are considered.

Do not demand number-based tests from work that uses a sound text-based method.

6. Results and Meaning

Check whether:

• Results are shown in a clear way.
• Claims do not go past the evidence.
• Charts, tables, and measures are easy to read.
• Negative and no-effect results are reported with care.

For proposals, mark this part N/A.

7. Limits and Risks

Check whether:

• Limits are clear and tied to this work.
• Risks to cause, meaning, reach, and final claims are covered.
• Proven results are kept apart from guesses.
• Claims about other people, places, or times are not too broad.

8. Writing and Structure

Check whether:

• The main line of thought is easy to follow.
• Sections support the research question.
• Key terms and signs are defined.
• The tone is exact and fair.
• Facts, views, and guesses are clearly marked.

9. Citations

Check whether:

• Each checked source supports the claim beside it.
• First-hand sources are used when they are the best choice.
• Reviews are named as reviews.
• Preprints are named as preprints.
• Names, dates, titles, and links are correct.

If the source text is not available, do not claim that it supports the point. Mark the check as open.

Review Steps

1. State the work type and review scope.
2. Read the title, abstract, opening, charts, and ending.
3. Write down the main question and strongest claims.
4. Read the methods and results for support.
5. Check the strongest claims against their cited sources.
6. Score each part that applies.
7. Split major blockers from smaller fixes.
8. End with clear next edits in priority order.

Point to a page, section, table, figure, or quote when possible. If page numbers are missing, name the section and opening words.

Edge Cases

• Only part of the work is given: Review only that part. List what you could not check.
• No sources are given: Review the claim logic, but mark source checks as open.
• A source cannot be opened: Do not guess what it says.
• A paper uses a new field or method: Judge it by its stated goals and field rules.
• Fields use different rules: State which rules you used.
• Two reviewers may disagree: Show the text or facts behind each score.
• A claim mixes fact and opinion: Split the claim before judging it.
• A conflict of interest is clear: Note it as a risk. Do not treat it as proof that the work is false.
• Several works are compared: Use the same scope, score guide, and evidence bar for all works.
• The work is not in English: Review its meaning and logic. Do not lower a score only for language style.

Output Template

# Scholar Evaluation: <work title>

## Review Scope

- Work type: <type>
- Scope: <full | focused | compare>
- Parts not checked: <list or none>

## Overall Rating

- Overall score: <1.0-5.0 or N/A>
- Confidence: <high | medium | low>
- Summary: <3-5 short sentences>

## Scores

| Part | Score | Evidence | Fix level |
| --- | ---: | --- | --- |
| Problem and question |  |  |  |
| Past work and context |  |  |  |
| Methods |  |  |  |
| Data and evidence |  |  |  |
| Analysis |  |  |  |
| Results and meaning |  |  |  |
| Limits and risks |  |  |  |
| Writing and structure |  |  |  |
| Citations |  |  |  |

## Major Blockers

<List issues that could make the main claim unsound.>

## Smaller Issues

<List useful fixes that do not break the main claim.>

## Top Revision Steps

1. <Most urgent clear edit>
2. <Next clear edit>
3. <Next clear edit>

## Open Source Checks

<List claims or citations that still need to be checked.>

Concrete Example

Input:

Review the methods section of a survey study. It says 42 college students filled out an online form. It does not say how they were picked. The paper claims the result applies to all adults.

Output:

# Scholar Evaluation: Survey Study

## Review Scope

- Work type: Data-based research paper
- Scope: Focused
- Parts not checked: All parts except methods, data, and limits

## Overall Rating

- Overall score: 2.0
- Confidence: High
- Summary: The study gives the sample size and data tool. It does not explain how people were picked. The broad claim about all adults is not supported by a small college sample.

## Scores

| Part | Score | Evidence | Fix level |
| --- | ---: | --- | --- |
| Methods | 2 | The form is named, but the way people were picked is missing. | High |
| Data and evidence | 2 | The sample has 42 college students and may not reflect other adults. | High |
| Limits and risks | 2 | The paper does not state the risk of a narrow sample. | High |

## Major Blockers

- The sample choice is not explained.
- The claim about all adults goes past the data.

## Top Revision Steps

1. Explain how the 42 people were found and chosen.
2. Limit the claim to the group that was studied.
3. Add the small and narrow sample as a key limit.

## Open Source Checks

- None.

Common Mistakes

• Do not use scores in place of clear feedback.
• Do not lower a score for a part that is outside the review scope.
• Do not treat citation count, journal name, or author fame as proof of quality.
• Do not trust a claim just because it appears in the abstract.
• Do not make up missing facts, sources, page numbers, or results.
• Do not ask every type of work to use the same method.
• Do not rewrite the work unless the user asks for a rewrite.