Abstract:The paper "Is Complexity an Illusion?" (Bennett, 2024) provides a formalism for complexity, learning, inference, and generalization, and introduces a formal definition for a "policy". This reply shows that correct policies do not exist for a simple task of supervised multi-class classification, via mathematical proof and exhaustive search. Implications of this result are discussed, as well as possible responses and amendments to the theory.
What problem does this paper attempt to address?
The core problem that this paper attempts to solve is to verify whether the theoretical framework proposed by Bennett (2024) in his paper "Is Complexity an Illusion?" is effective for some simple tasks. Specifically, through mathematical proof and exhaustive search, the author shows that for a simple supervised multi - class classification task, according to Bennett's definition, there is no "correct policy". This reveals the potential gap between Bennett's theory and actual AI systems.
### Main problems of the paper
1. **Effectiveness of the theoretical framework**:
- Bennett's paper proposes a formal framework for complexity, learning, reasoning, and generalization, and introduces a formal definition of "policy".
- The author of this paper attempts to verify whether this framework is applicable to simple supervised multi - class classification tasks.
2. **Existence of the correct policy**:
- By constructing a specific task instance, the author proves that in this task, there is no "correct policy" in line with Bennett's definition.
- This finding challenges Bennett's theory, suggesting that his theory may not be directly applicable to actual machine - learning tasks.
### Specific task description
The author constructs a task environment with 5 states and defines four declarative programs \( f_1, f_2, f_3, f_4 \). These programs represent different sets of states respectively. The task \( \alpha \) is defined as:
- Input \( I_\alpha=\{ \{ f_1 \}, \{ f_2 \} \} \)
- Output \( O_\alpha = \{ \{ f_1, f_3 \}, \{ f_2, f_4 \} \} \)
### Mathematical proof
The author proves that there is no "correct policy" in this task through the following steps:
1. **Define the extension set**:
- \( E_{I_\alpha}=E_{\{ f_1 \}} \cup E_{\{ f_2 \}} \), that is, the extension set of the input.
- \( E_{I_\alpha} \) contains all statements that can be completed by the input.
2. **Check all possible strategies**:
- A strategy \( \pi \) is a statement in the language \( L_v \).
- It needs to satisfy \( E_{I_\alpha} \cap E_\pi = O_\alpha \) to be a correct strategy.
3. **Exhaust all strategies**:
- By exhaustively enumerating all possible strategies, the author finds that no strategy can satisfy the above condition.
### Conclusion
This paper reveals the limitations of Bennett's theory in dealing with actual machine - learning tasks. Although Bennett's theoretical framework provides profound insights at an abstract level, in practical applications, especially in multi - class classification tasks, there may be inconsistencies or difficulties in implementation.
### Potential impacts
- **Theory modification**: Further research is needed on how to adjust Bennett's theoretical framework to make it more suitable for actual machine - learning tasks.
- **Practical guidance**: For machine - learning practitioners, this paper reminds us to carefully verify the applicability of new theoretical frameworks in specific tasks when applying them.
### Summary of related formulas
- \( E_x = \{ y \in L_v : x \subseteq y \} \) (definition of the extension set)
- \( E_{I_\alpha}=E_{\{ f_1 \}} \cup E_{\{ f_2 \}} \)
- \( \pi \in \Pi_\alpha \iff E_{I_\alpha} \cap E_\pi = O_\alpha \)
Through these formulas, the author strictly proves that there is no correct policy in line with Bennett's definition in the given task.