Abstract:A novel method called mixed variable system Monte Carlo tree search (MVSMCTS) formulation is presented for optimization problems considering various types of variables with single and mixed continuous-discrete system. This method utilizes a reinforcement learning algorithm with improved Monte Carlo tree search (IMCTS) formulation. For sizing and shape optimization of truss structures, the design variables are the cross-sectional areas of the members and the nodal coordinates of the joints. MVSMCTS incorporates update process and accelerating technique for continuous variable and combined scheme for single and mixed system. Update process indicates that once a solution is determined by MCTS with automatic mesh generation in continuous space, it is used as the initial solution for next search tree. The search region should be expanded from the mid-point, which is the design variable for initial state. Accelerating technique is developed by decreasing the range of search region and the width of search tree based on the number of meshes during update process. Combined scheme means that various types of variables are coupled in only one search tree. Through several examples, it is demonstrated that this framework is suitable for mixed variable structural optimization. Moreover, the agent can find optimal solution in a reasonable time, stably generates an optimal design, and is applicable for practical engineering problems.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is the optimization of the structure with mixed variables, especially the size and shape optimization of truss structures that contain both continuous and discrete variables simultaneously. Specifically, the paper proposes a new method - the Mixed - Variable System Monte Carlo Tree Search (MVSMCTS) formulation method to deal with the challenges brought by different types of variables (such as cross - sectional area and node coordinates) in the optimization process.
### 1. **Research Background and Problem Description**
Traditional structural optimization methods face many challenges when dealing with the size and shape optimization of truss structures. This type of problem involves two different types of design variables: size variables (cross - sectional area) and shape variables (node coordinates). The combination of these variables may lead to different convergence speeds and make the problem ill - conditioned, thus increasing the optimization difficulty.
### 2. **Limitations of Existing Methods**
- **Mathematical Programming Methods**: Although they can be solved accurately, they have high computational costs when dealing with complex, multi - variable problems.
- **Meta - heuristic Algorithms**: Although they can handle complex problems, they may not be able to guarantee the global optimal solution.
- **Reinforcement Learning Methods Based on Q - learning**: They are suitable for structural design problems, but have high training costs.
### 3. **Solution Proposed in the Paper**
To solve the above problems, the authors propose the Mixed - Variable System Monte Carlo Tree Search (MVSMCTS) formulation method. The main features of this method include:
- **Update Process**: Once a solution is determined by MCTS, this solution will be used as the initial solution for the next search tree, thus accelerating convergence.
- **Acceleration Techniques**: Improve the search efficiency by reducing the scope of the search area and the width of the search tree.
- **Combination Scheme**: Consider multiple types of variables in the same search tree, thus simplifying the optimization process.
### 4. **Specific Applications**
The paper demonstrates the effectiveness of the MVSMCTS method through several numerical examples. The results show that this method can find the optimal solution within a reasonable time and stably generate the optimal design, which is suitable for practical engineering problems.
### 5. **Formula Summary**
The following are some of the key formulas involved in the paper:
#### Optimization Objective Function
\[
W(\mathbf{X}, \mathbf{Y})=\rho \sum_{i_X = 1}^{g_X}\sum_{j = 1}^{m_{i_X}}L_{i_X,j}(\mathbf{Y})
\]
#### Constraint Conditions
\[
K(\mathbf{X}, \mathbf{Y})\mathbf{u}=\mathbf{F}
\]
\[
\sigma_{min}\leq\sigma_{i_X,j}\leq\sigma_{max}, \quad i_X = 1,2,\ldots,g_X, \quad j = 1,2,\ldots,m_{i_X}
\]
\[
\delta_{min}\leq\delta_k\leq\delta_{max}, \quad k = 1,2,\ldots,c
\]
#### Reward Calculation in the Update Process
\[
r_T=\left(\frac{\alpha}{W_T}\right)^2
\]
#### Upper Confidence Bound (UCB)
\[
U_I = V_I + C\sqrt{\frac{\ln N}{n_I}}
\]
#### Geometric Decay in Acceleration Techniques
\[
\beta_{X1}=\begin{cases}b_X & \text{if } b_X \text{ is odd}\\b_X + 1 & \text{if } b_X \text{ is even}\end{cases}
\]
\[
\varphi_{Xp}=\beta_{X1}\times\gamma_X^{\left\lceil\frac{p - 1}{\epsilon_X}\right\rceil}
\]