publications

Ordered by date, newest first.

2026Under reviewACL 2026

Evaluating Epistemic Non-Triviality in Scientific Hypothesis Generation

Jash Shah · Knowledge & Discovery Lab, DAIICT and Oak Ridge National Laboratory

Proposed DISCERN, a diagnostic framework assessing epistemic non-triviality in LLM hypothesis generators using year-indexed citation disclosure and model-agnostic signals: semantic similarity, citation graph, and temporal baselines.
Benchmarked GPT-4o, o4-mini, Magistral, and GPT-OSS-20B on IdeaBench across three prompt conditions, measuring alignment via Spearman, Kendall, Pearson, and KL divergence, with robustness validated on NLPeer V2 scores.

the question

Can a language model tell that a research question is trivial, given everything published before it? I operationalize non-triviality as how much prior background — and how influential that background is — reduces the novelty of a target question.

Using a citation-aware dataset derived from IdeaBench, I build year-by-year background sets and watch how model judgments shift as prior work accumulates. The drift is the signal: it says whether a model can separate an incremental idea from a genuinely new one.

knowledge & discovery lab

The KDM Lab started at DA-IICT in 2015 under Prof. Sourish Dasgupta. It works on natural language understanding and the evaluation of large language models, particularly their reasoning and personalization capabilities, through theoretical frameworks and experimental setups.

The lab also explores neuro-symbolic methods for specific NLU tasks, and connects problems across linguistics, economics, political science, and the social sciences. Its motto: don't bring the baggage and boredom of specific schools of thought.