An Application of the Generalized Rectangular Fuzzy Model to Critical Thinking Assessment

Igor Ya. Subbotin,Michael Gr. Voskoglou

DOI: https://doi.org/10.48550/arXiv.1601.03065

2016-01-09

Abstract:The authors apply the Generalized Rectangular Model to assessing critical thinking skills and its relations with their language competency.

Artificial Intelligence

What problem does this paper attempt to address?

Clinical reasoning performance assessment: using situated cognition theory as a conceptual framework

Joseph Rencic,Lambert W.T. Schuwirth,Larry D. Gruppen,Steven J. Durning

DOI: https://doi.org/10.1515/dx-2019-0051

2020-02-06

Diagnosis

Abstract:Abstract Developing valid assessment approaches to clinical reasoning performance has been challenging. Situated cognition theory posits that cognition (e.g. clinical reasoning) emerges from interactions between the clinician and situational (contextual) factors and recognizes an opportunity to gain deeper insights into clinical reasoning performance and its assessment through the study of these interactions. The authors apply situated cognition theory to develop a conceptual model to better understand the assessment of clinical reasoning. The model highlights how the interactions between six contextual factors, including assessee, patient, rater, and environment, assessment method, and task, can impact the outcomes of clinical reasoning performance assessment. Exploring the impact of these interactions can provide insights into the nature of clinical reasoning and its assessment. Three significant implications of this model are: (1) credible clinical reasoning performance assessment requires broad sampling of learners by expert raters in diverse workplace-based contexts; (2) contextual factors should be more explicitly defined and explored; and (3) non-linear statistical models are at times necessary to reveal the complex interactions that can impact clinical reasoning performance assessment.
A collective decision model involving vague concepts and linguistic expressions.

Yongchuan Tang

DOI: https://doi.org/10.1109/TSMCB.2007.913125

2008-01-01

Abstract:In linguistic collective decision, the main objective is to select the best alternatives using linguistic evaluations provided by multiple experts. This paper presents a collective decision model, which is able to deal with complex linguistic evaluations. In this decision model, the linguistic evaluations are represented by linguistic expressions which are the logic formulas obtained by applying logic connectives to the set of basic linguistic labels. The vagueness of each linguistic expression is implicitly captured by a semantic similarity relation rather than a fuzzy set, since each linguistic expression determines a semantic similarity distribution on the set of basic linguistic labels. The basic idea of this collective decision model is to convert the semantic similarity distributions determined by linguistic expressions into probability distributions of the corresponding linguistic expressions. The main advantage of this proposed model is its capability to deal with complex linguistic evaluations and partial semantic overlapping among neighboring linguistic labels.
Dealing with the Fuzziness of Human Reasoning

Michael Gr. Voskoglou,Igor Ya. Subbotin

DOI: https://doi.org/10.48550/arXiv.1311.5355

2013-11-21

Abstract:Reasoning, the most important human brain operation, is charactrized by a degree fuzziness. In the present paper we construct a fuzzy model for the reasoning process giving through the calculation of the possibilities of all possible individuals' profiles a quantitative/qualitative view of their behaviour during the above process and we use the centroid defuzzification technique for measuring the reasoning skills. We also present a number of classroom experiments illustrating our results in practice.

Artificial Intelligence
Fuzzy models for learning assessment

Igor Ya.Subbotin,Michel Gr. Voskoglou

DOI: https://doi.org/10.48550/arXiv.1410.4497

2014-10-17

Abstract:In the current paper a new Trapezoidal Fuzzy Model for Assessment is developed. This model is a variation of a special form of the commonly used in Fuzzy Mathematics Center of Gravity technique.

Optimization and Control
Assessment and Linear Programming under Fuzzy Conditions

Michael Voskoglou

DOI: https://doi.org/10.48550/arXiv.2011.10640

IF: 14.4

2020-11-20

Artificial Intelligence

Abstract:A new fuzzy method is developed using triangular/trapezoidal fuzzy numbers for evaluating a group's mean performance, when qualitative grades instead of numerical scores are used for assessing its members' individual performance. Also, a new technique is developed for solving Linear Programming problems with fuzzy coefficients and everyday life applications are presented to illustrate our results.
Critical Thinking Assessment in Adults: Methodology and Development Experience

A.R. Sadova,J.S. Khill,N.V. Pashchenko,K.V. Tarasova

DOI: https://doi.org/10.17759/jmfp.2022110409

2022-01-01

Современная зарубежная психология

Abstract:Critical thinking (next CT) is one of the key competencies in the modern world. Despite the variety of assessment systems for secondary and higher education, there is a lack of tests for assessing this competence for adults. The importance of adult assessment is emphasized primarily by the need for employers to have employees with developed CT in various work contexts. The article describes the procedure for creating a measurement test for students of the training course «Critical Thinking» of the «Yandex.Practice» platform and the results of assessing its psychometric quality. When creating this test, we relied on the Evidence Centered Design (ECD) approach. The test is performed in computer form and consists of tasks aimed at identifying the relevant observed behavior of the subject. The validity of the developed test, including its compliance with theoretical expectations about the factor structure, is confirmed by evidence of constructive, criterion, convergent and content validity. The limitations of this study and possible directions for future developments are considered. The result of the work is a validated test for measuring critical thinking for adults, the quality of the tasks of which is confirmed by psychometric analysis in modern mental test theory (IRT — Item Response Theory).
Applications of fuzzy logic to Case-Based Reasoning

Igor Ya. Subbotin,Michael Gr. Voskoglou

DOI: https://doi.org/10.48550/arXiv.1204.2018

2012-04-10

Abstract:The article discusses some applications of fuzzy logic ideas to formalizing of the Case-Based Reasoning (CBR) process and to measuring the effectiveness of CBR systems

Artificial Intelligence
FRoG: Evaluating Fuzzy Reasoning of Generalized Quantifiers in LLMs

Yiyuan Li,Shichao Sun,Pengfei Liu

DOI: https://doi.org/10.18653/v1/2024.emnlp-main.411

2024-01-01

Abstract:Fuzzy reasoning is vital due to the frequent use of imprecise information in daily contexts. However, the ability of current large language models (LLMs) to handle such reasoning remains largely uncharted. In this paper, we introduce a new benchmark, FRoG, for fuzzy reasoning, featuring real-world mathematical word problems that incorporate generalized quantifiers. Our experimental findings reveal that fuzzy reasoning continues to pose significant challenges for LLMs. Moreover, we find that existing methods designed to enhance reasoning do not consistently improve performance in tasks involving fuzzy logic. Additionally, our results show an inverse scaling effect in the performance of LLMs on FRoG. Interestingly, we also demonstrate that strong mathematical reasoning skills are not necessarily indicative of success on our benchmark.
FRoG: Evaluating Fuzzy Reasoning of Generalized Quantifiers in Large Language Models

Yiyuan Li,Shichao Sun,Pengfei Liu

2024-07-03

Abstract:Fuzzy reasoning is vital due to the frequent use of imprecise information in daily contexts. However, the ability of current large language models (LLMs) to handle such reasoning remains largely uncharted. In this paper, we introduce a new benchmark, FRoG, for fuzzy reasoning, featuring real-world mathematical word problems that incorporate generalized quantifiers. Our experimental findings reveal that fuzzy reasoning continues to pose significant challenges for LLMs. Moreover, we find that existing methods designed to enhance reasoning do not consistently improve performance in tasks involving fuzzy logic. Additionally, our results show an inverse scaling effect in the performance of LLMs on FRoG. Interestingly, we also demonstrate that strong mathematical reasoning skills are not necessarily indicative of success on our benchmark.

Artificial Intelligence,Computation and Language
Assessment of students’ critical thinking maturity in the process of teaching philosophy

A. I. Pavlovskij,O. V. Pavlovskaya

DOI: https://doi.org/10.17853/1994-5639-2024-5-40-66

2024-04-30

The Education and science journal

Abstract:Introduction. Developing critical thinking is one of the primary objectives of modern education. From the authors’ perspective, both the method and the content of the discipline, especially philosophy, have historically evolved through dialogue within the intellectual tradition of the era, which is valuable for fostering critical thinking. Aim. The study aimed to develop and test a methodology for assessing the role of philosophy in shaping critical thinking, considering its expertise, knowledge, and skills. Methodology and research methods. The development of critical thinking was assessed during the analysis of the philosophical text. Prior to the test, it is worth noting that the authors conducted an expert assessment of the text, which later served as a reference assessment, and developed a grading scale. Gradually, examples of typical student errors were added to the scale, facilitating further assessment. Results. Nearly all students were able to summarise the provided text by the end of the course, although with varying degrees of distortion. At the beginning of the course, approximately one-tenth of the students were unable to do it, indicating that the understanding and interpretation of a text was beyond their ability. The absence of dynamics in all three skills indicates that a student has not developed a critical thinking mindset if their results are low. Among the students, the weak and average ones exhibit the most significant overall improvement (16–20% of the class enhance their performance across all criteria). The test revealed tender points in the student’s critical thinking development: a feeble argument assessment skill and a poor understanding of the voice behind the text. Scientific novelty. The proposed assessment methodology enables the connection of texts to assess the development of critical thinking with philosophical objectivity. Practical significance. This work provides a convenient tool for monitoring critical thinking skills in each student. Measuring critical thinking skills through students’ written works may help track individual accomplishments of the learners and indicate areas of growth in the course taught.
STUDY AND APPLICATION OF GENERAL TOOL OF FUZZY COMPREHENSIVE EVALUATION

Han Xianglan,Zhang Gang,Liu Yangguang

DOI: https://doi.org/10.3969/j.issn.1000-386X.2010.02.060

2010-01-01

Abstract:On the basis of the universality and status quo of the application of fuzzy comprehensive evaluation method,to develop a general fuzzy comprehensive evaluation tool is suggested.The function structure of the system is built through the requirement analysis,and an example is used to represent the system's function.The tool is built based on the web,it can be used to quickly develop corresponding evaluation systems against different evaluation problems,and it also adapts to group decision,which is really convenient and easily for use.
Trapezoidal Fuzzy Logic Model for Learning Assessment

Igor Ya. Subbotin

DOI: https://doi.org/10.48550/arXiv.1407.0283

2014-07-01

Abstract:The current article discusses some applications of fuzzy logic to assessment of learning. We consider here a new trapezoidal fuzzy model for learning assessment.

General Mathematics
Assessing Students' Computational Thinking In A Learning By Modeling Environment

Ningyu Zhang,Gautam Biswas

DOI: https://doi.org/10.1007/978-981-13-6528-7_12

2017-01-01

Abstract:Researchers have hypothesized strong connections between Computational Thinking (CT) practices and STEM learning. However, there is a lack of consensus on what constitutes an adequate set of CT knowledge and skills. In this paper, we present an initial framework for evaluating students' CT learning. We introduce the primary CT concepts and practices that students can learn and apply in a learning by modeling environment. Our overall goal is to develop assessments that study the synergy between STEM and CT concepts in K-12 curricula. Towards this end, we discuss the results from a teacher-led classroom study we conducted on STEM- and CT-learning in our CTSiM environment.
Application of Grey Numbers to Assessment of Mathematical Modeling Capacities

Michael Gr. Voskoglou

DOI: https://doi.org/10.33122/ijtmer.v1i1.29

2018-12-08

International Journal of Trends in Mathematics Education Research

Abstract:Mathematical modeling appears today as a dynamic tool for teaching and learning mathematics, because it connects mathematics with real world applications thus increasing the student interest on the subject. In the present paper a method is developed that uses grey numbers for evaluating student mathematical modeling capacities and a classroom application is presented illustrating it. This new assessment method is useful in particular when the student performance is evaluated with linguistic characterizations (grades) and not with numerical scores, which makes non feasible the calculation of the mean value of those grades. Grey numbers are indeterminate numbers which are defined with the help of the real intervals and they have found nowadays many applications in real life, science and engineering for handling approximate data.
Generalized interval-valued intuitionistic fuzzy numbers with applications in workstation assessment

Zhi PEI,Jian-sha LU,Li ZHENG

DOI: https://doi.org/10.3969/j.issn.1000-6788.2012.10.010

2012-01-01

Abstract:Much attention has been drawn to the concepts of intuitionistic fuzzy number and its applications in fuzzy multi-attribute decision making problems, some of the scholars also extend the intuitionistic fuzzy number to generalized intuitionistic fuzzy number. However, very little research has been completed on the extension of traditional interval-valued intuitionistic fuzzy numbers. Therefore, in this paper, the definitions of generalized interval-valued intuitionistic fuzzy number are proposed, based on the concepts and properties of intuitionistic fuzzy numbers and interval-valued fuzzy numbers. In order to help solve the fuzzy multi-attribute decision making problems, the arithmetic rules on the set of generalized interval-valued intuitionistic fuzzy numbers are further introduced, and the closeness of the operators in the domain of definition is proven. Then three ranking principles are proposed so as to implement the aforementioned arithmetical methods into the numerical example of workstation assessment, which demonstrates the validity of the proposed approach.
How critically can an AI think? A framework for evaluating the quality of thinking of generative artificial intelligence

Luke Zaphir,Jason M. Lodge,Jacinta Lisec,Dom McGrath,Hassan Khosravi

2024-06-21

Abstract:Generative AI such as those with large language models have created opportunities for innovative assessment design practices. Due to recent technological developments, there is a need to know the limits and capabilities of generative AI in terms of simulating cognitive skills. Assessing student critical thinking skills has been a feature of assessment for time immemorial, but the demands of digital assessment create unique challenges for equity, academic integrity and assessment authorship. Educators need a framework for determining their assessments vulnerability to generative AI to inform assessment design practices. This paper presents a framework that explores the capabilities of the LLM ChatGPT4 application, which is the current industry benchmark. This paper presents the Mapping of questions, AI vulnerability testing, Grading, Evaluation (MAGE) framework to methodically critique their assessments within their own disciplinary contexts. This critique will provide specific and targeted indications of their questions vulnerabilities in terms of the critical thinking skills. This can go on to form the basis of assessment design for their tasks.

Artificial Intelligence
Computer-based performance approach for critical thinking assessment in children

Irina Uglanova,Ekaterina Orel,Daria Gracheva,Ksenia Tarasova

DOI: https://doi.org/10.1111/bjep.12576

Abstract:Background: Critical thinking (CT) has been identified as one of the key skills in the modern world. Despite a variety of CT assessment systems for secondary school and higher education, there is a shortage of CT assessments for children. Aims: To fill the gap, we developed a computer-based performance assessment (CPBA) of CT with automatic scoring. In the study, we analysed: (1) To what extent is the internal structure of the critical thinking assessment tool confirmed? (2) What are the patterns of students' behaviour within the CPBA? (3) To what extent is the convergent validity of the CPBA supported? Sample and methods: The sample consists of 1689 students, aged 9-11 years. To answer the research questions, the following quantitative data analysis methods were selected. The Bayesian Network was applied to investigate the internal structure. Latent Class Analysis was used to investigate the patterns of students' behaviour. To verify the convergent validity, the students were asked to solve the standardized math and language literacy tests. Results and conclusions: The theoretically expected internal structure of the CPBA was confirmed: the two-factor structure (the Analysis of information and the Making inferences subskills of CT) describes the data appropriately. Moreover, CT was described not as a list but as a sequence of cognitive operations. Four different strategies that lead to different results of CT were revealed. A weak positive correlation between the results of the CPBA and the achievement tests confirmed the test validity.
Thinking Critically about Critical Thinking: Validating the Russian HEIghten® Critical Thinking Assessment

Amy Shaw,Ou Lydia Liu,Lin Gu,Elena Kardonova,Igor Chirikov,Guirong Li,Shangfeng Hu,Ningning Yu,Liping Ma,Fei Guo,Qi Su,Jinghuan Shi,Henry Shi,Prashant Loyalka

DOI: https://doi.org/10.1080/03075079.2019.1672640

2019-01-01

Studies in Higher Education

Abstract:Critical thinking has been identified as a crucial general skill contributing to academic and career success in the twenty-first century. With the increasing demands of the modern workplace and a global trend of accountability in higher education, educators and employers pay great attention to the development of students' critical thinking skills throughout their training. Therefore, there is an urgent need worldwide for an updated and comprehensive assessment tool of college-level critical thinking. This paper reports on the preliminary validation for the Russian version of the HEIghten? Critical Thinking assessment developed by Educational Testing Service (ETS). Based on a large Russian college student sample (N?=?1060), we evaluated the psychometric quality of the items, individual and institution-level reliability, external validity, and student perceptions. Overall, the results suggested good psychometric quality, except that a few items showed low discriminating power and should be further examined with a second wave of data collection. IRT analyses revealed testlet effects and supported the essentially unidimensional structure of the measure. Appropriate correlations with external criteria provided support for the measure's convergent validity. Implications of the preliminary validation study results and the future research agenda, especially the need to collect longitudinal data, are discussed.
Analyzing Online Discussion Data for Understanding the Student's Critical Thinking

Juan Yang,Xu Du,Jui-Long Hung,Chih-Hsiung Tu

DOI: https://doi.org/10.1108/dta-04-2021-0088

IF: 1.713

2022-01-01

Data Technologies and Applications

Abstract:PurposeCritical thinking is considered important in psychological science because it enables students to make effective decisions and optimizes their performance. Aiming at the challenges and issues of understanding the student's critical thinking, the objective of this study is to analyze online discussion data through an advanced multi-feature fusion modeling (MFFM) approach for automatically and accurately understanding the student's critical thinking levels.Design/methodology/approachAn advanced MFFM approach is proposed in this study. Specifically, with considering the time-series characteristic and the high correlations between adjacent words in discussion contents, the long short-term memory–convolutional neural network (LSTM-CNN) architecture is proposed to extract deep semantic features, and then these semantic features are combined with linguistic and psychological knowledge generated by the LIWC2015 tool as the inputs of full-connected layers to automatically and accurately predict students' critical thinking levels that are hidden in online discussion data.FindingsA series of experiments with 94 students' 7,691 posts were conducted to verify the effectiveness of the proposed approach. The experimental results show that the proposed MFFM approach that combines two types of textual features outperforms baseline methods, and the semantic-based padding can further improve the prediction performance of MFFM. It can achieve 0.8205 overall accuracy and 0.6172 F1 score for the “high” category on the validation dataset. Furthermore, it is found that the semantic features extracted by LSTM-CNN are more powerful for identifying self-introduction or off-topic discussions, while the linguistic, as well as psychological features, can better distinguish the discussion posts with the highest critical thinking level.Originality/valueWith the support of the proposed MFFM approach, online teachers can conveniently and effectively understand the interaction quality of online discussions, which can support instructional decision-making to better promote the student's knowledge construction process and improve learning performance.
Assessment of competencies in the Federal state educational standard (FSES) 3++ based on a fuzzy-logical approach

O. V. Astashina

DOI: https://doi.org/10.21822/2073-6185-2021-48-4-72-80

2022-02-10

Abstract:Objective. The article discusses the methodology for assessing competencies in the educational standards of the Federal State Educational Standard 3++. Method. Competence assessment is carried out through competency indicators. To assess the competence of a complex multilayer object, the mathematical apparatus of the theory of fuzzy sets was used. Result. A step-by-step example of competency assessment is given. The process of transformation of input variables into fuzzy sets is presented - assessments of competency indicators. A trapezoidal scale of linguistic terms has been developed. The processes of fuzzification, accumulation and defuzzification are presented step by step. A final clear assessment of competence was obtained. Conclusion. The introduction of fuzzy numbers allows all calculations to be performed quite flexibly and, as a result of defuzzification, to obtain an objective assessment of the formed competence.

An Application of the Generalized Rectangular Fuzzy Model to Critical Thinking Assessment

Clinical reasoning performance assessment: using situated cognition theory as a conceptual framework

A collective decision model involving vague concepts and linguistic expressions.

Dealing with the Fuzziness of Human Reasoning

Fuzzy models for learning assessment

Assessment and Linear Programming under Fuzzy Conditions

Critical Thinking Assessment in Adults: Methodology and Development Experience

Applications of fuzzy logic to Case-Based Reasoning

FRoG: Evaluating Fuzzy Reasoning of Generalized Quantifiers in LLMs

FRoG: Evaluating Fuzzy Reasoning of Generalized Quantifiers in Large Language Models

Assessment of students’ critical thinking maturity in the process of teaching philosophy

STUDY AND APPLICATION OF GENERAL TOOL OF FUZZY COMPREHENSIVE EVALUATION

Trapezoidal Fuzzy Logic Model for Learning Assessment

Assessing Students' Computational Thinking In A Learning By Modeling Environment

Application of Grey Numbers to Assessment of Mathematical Modeling Capacities

Generalized interval-valued intuitionistic fuzzy numbers with applications in workstation assessment

How critically can an AI think? A framework for evaluating the quality of thinking of generative artificial intelligence

Computer-based performance approach for critical thinking assessment in children

Thinking Critically about Critical Thinking: Validating the Russian HEIghten® Critical Thinking Assessment

Analyzing Online Discussion Data for Understanding the Student's Critical Thinking

Assessment of competencies in the Federal state educational standard (FSES) 3++ based on a fuzzy-logical approach