SILVA, Rafael Tavares da; SIQUEIRA, Sean Wolfgand Matsui; TIBAU, Marcelo. Implementing Knowledge Gain Measurement in Real Search Environments. In: CONCURSO DE TESES, DISSERTAÇÕES E TCCS EM SI – IC/IT – SIMPÓSIO BRASILEIRO DE SISTEMAS DE INFORMAÇÃO (SBSI), 22. , 2026, Vitória/ES. Anais […]. Porto Alegre: Sociedade Brasileira de Computação, 2026 . p. 61-70. DOI: https://doi.org/10.5753/sbsi_estendido.2026.249057.
Implementing Knowledge Gain Measurement in Real Search Environments
Authors
Rafael Tavares da Silva (UNIRIO)
Sean Wolfgand Matsui Siqueira (UNIRIO)
Marcelo Tibau (UNIRIO)
Abstract
The operationalization of learning metrics in real search environments remains an open challenge in the Searching as Learning (SaL) paradigm. While behavioral proxies offer scalability, they capture conceptual change only indirectly; structured assessments provide more direct evidence but often compromise ecological validity. The Degree of Knowledge Gain (DKG) metric addresses this tension by combining Shannon entropy with semantic similarity between queries and clicked documents to model the progressive reduction of uncertainty during search. This paper reports on two technological artifacts developed to embed DKG computation into real-world search workflows, within the scope of the CNPq project 3C-BPA: Comportamento de busca, Complexidade da informação e pensamento Crítico na Busca como um Processo de Aprendizagem. A standalone search engine prototype established the feasibility of real-time DKG computation but exposed limitations in ecological validity and operational sustainability. These were addressed by a Chrome browser extension that estimates the metric unobtrusively while users interact with their preferred search engines. To assess the extension’s applicability, an experiment was conducted combining preand post-tests with the Concurrent Think-Aloud (CTA) protocol and automated interaction logging. Preliminary results indicate that DKG is sensitive to variation in search strategy use as participants who engaged in systematic query reformulation and multi-source evaluation achieved stronger knowledge gains, while those exhibiting disorientation and limited cognitive regulation showed more modest outcomes. Beyond its empirical contributions, the study illustrates how undergraduate research participation can play a substantive role in advancing the development and application of formal learning metrics in information science.
DOI: 10.5753/sbsi_estendido.2026.249057
URL: https://sol.sbc.org.br/index.php/sbsi_estendido/article/view/41996
