新建
Multitask model加速GNN模型预测
GavinYi
赞 1
1
1
目录
数据集
AI4SCUP-CNS-BBB(v1)
Multitask model加速GNN模型预测
©️ Copyright 2023 @ Authors
作者：易佳炜 📨
日期：2023-07-10
共享协议：本作品采用知识共享署名-非商业性使用-相同方式共享 4.0 国际许可协议进行许可。
代码
文本
前言：
分子性质预测已经有很多notebook介绍，比如定量构效关系(QSAR)模型从0到1 & Uni-Mol入门实践（分类任务），比较了QSAR中主流模型之间的性能；图神经网络GNN用于分子性质预测的研究也十分火热，这一篇notebook里面的交互可以帮助大家更快学会使用GNN来解决实际问题。本notebook对一篇前沿研究的文献进行复现尝试。
本文将采用下面这些工具，可以点击下方链接了解基本情况：
- RDkit：开源的化学信息学和机器学习库，提供了丰富的化学工具和数据结构，支持分子处理、描述符计算、相似性搜索等功能，广泛应用于药物设计和生物信息学领域。
- Pytorch：开源的深度学习框架，提供了灵活的张量计算和自动求导功能，广泛应用于机器学习、计算机视觉和自然语言处理等领域。
- Pytorch_geometric：基于PyTorch的几何深度学习扩展库，用于简化不规则数据（如图形、点云和流形）的处理和实现各种图形神经网络模型。
代码
文本
目标
了解预测分子性质的方法
学习一个GNN应用案例
代码
文本
分子性质预测方法
1.为什么要分子性质预测呢？
分子性质预测是计算机辅助药物发现流程中最关键的任务之一，在许多下游应用中发挥重要作用。其主要目的是通过原子坐标、原子序数等分子内部信息，对分子物理、化学性质做出预测，从而使人们能够在大量候选化合物中找到符合预期性质的化合物，加快筛选和设计的速度。
以聚合物电解质为例，因聚合物电解质的低成本、安全性和制造兼容性，是下一代锂离子电池技术的重要研究方向。但是现在目前聚合物电解质受困于离子电导率低的问题，需要通过实验和原子尺度模拟来探索新的聚合物类别以解决当前材料的问题。
2.怎么进行分子性质预测？有什么方法？
密度泛函理论（DFT）/量子化学计算
DFT 可以精确预测多种分子性质，但是 DFT 非常耗时，往往需要数个小时来完成对单个分子的计算。类似地，传统的量子化学计算方法需要巨大的时间成本，加之候选化合物数量往往较为庞大，人们难以在短时间内完成分子性质预测。如果能找到一种快速而精确的深度学习模型，那么 DFT 的计算成本将不再是分子性质预测的一大障碍。
--优势：提供了精确的分子性质预测，可以直接计算电子结构和能量。
--劣势：计算复杂度高，对于大分子和复杂体系需要大量计算资源，计算速度慢。
机器学习方法
近年来，机器学习方法在分子性质预测领域取得了显著的进展。这些方法通过训练大量已知性质的分子数据，构建分子性质和分子描述符之间的关系模型。常见的机器学习方法包括支持向量机（SVM）、人工神经网络（ANN）、随机森林（RF）和梯度提升决策树（GBDT）等。
优势：计算速度快，适用于高通量虚拟筛选和大规模化学数据挖掘，具有良好的预测性能。
劣势：需要大量已知性质的训练数据，对于新颖化合物或没有足够训练数据的体系预测性能较差。
分子对接和药效团方法
这些方法常用于药物设计领域，通过评估分子与生物靶标之间的结合模式和能量来预测分子的生物活性。分子对接方法通常采用分子力学和统计学方法，如GOLD和AutoDock等；药效团方法则侧重于分子结构与生物活性之间的经验关系，如QSAR和Pharmacophore模型。
优势：提供了直观的分子与生物靶标之间的相互作用模式，有助于理解分子的作用机制和优化分子结构。
劣势：对于非靶向性质的预测（如溶解度、毒性等）适用性较差，预测精度受限于评分函数和搜索算法。
网络化学和化学信息学方法
这些方法通过挖掘已有的化学数据和知识，构建分子性质和分子结构之间的关系。常见的方法包括相似性搜索、分子指纹、化合物聚类和数据挖掘等。
优势：利用已有的化学数据和知识，可以在较短的时间内预测分子性质，适用于大规模化学数据库的处理。
劣势：依赖于已有数据和知识，对于新颖化合物或没有足够数据的体系预测性能较差。
现在方法众多，方法之间也存在交叉，总体的发展方向主要是将不同方法相互结合，以实现更高的预测精度和效率。例如，机器学习方法可以与量子化学方法相结合，利用机器学习模型加速计算过程，同时保持较高的预测精度。此外，深度学习、图神经网络等新兴的人工智能技术也正在逐渐应用于分子性质预测领域，为分子设计和新材料发现提供强大的计算支持，下面将针对图神经网络（graph neural networks，GNN）的一个案例进行展开。
代码
文本
案例说明
筛选新型高离子电导率聚合物电解质一直是分子性质预测中的研究重点，但是它的无定形性质和复杂时间尺度问题使得离子电导率的模拟非常昂贵，这极大地限制了采用高通量计算筛选方法的能力。一些聚合物具有晶体结构，此前已经有研究人员使用DFT计算对结晶聚合物进行了大规模筛选Polymer Genome，但在筛选结晶度较低的聚合物的时候还是需要分子动力学(MD)模拟来取样平衡，计算量依旧很大，例如，这篇文献Chem.materials2020用经典MD探索非晶态聚合物电解质的研究，但是只模拟了大约10种聚合物。
限制MD模拟的原因是什么？
聚合物电解质的无定形结构需要蒙特卡罗算法从随机分布中采样，使用MD模拟难以逐一遍历，初始结构对模拟的离子电导率影响很大。所以通常为了正确采样相空间并减少统计噪声，需要从独立配置开始进行多次模拟。
聚合物的缓慢弛豫需要很长的MD模拟时间来实现离子电导率的收敛(在10到100秒的数量级上)，因此每个MD模拟在计算上十分耗时
怎么减少MD模拟的计算量呢？
从机器学习的视角
假设用于ML训练的性质数据是通过确定性的、无偏的过程生成，直接进行训练，这一方案还是建立在大量高精度MD计算数据的基础之上，依然需要较大的计算量
降低单个MD模拟的精度要求，使用大量较低精度的数据用于训练来减少随机和系统误差
本文的案例来自nature communication2022 ，这一篇文献从分子动力学模拟中学习大量有偏、有噪声的数据和少量无偏数据来加速聚合物电解质的高通量计算筛选。为了减少模拟的偏差，文献对搜索区域中每种聚合物进行了一次MD模拟，并学习了聚合物之间的共享模型，以恢复从重复模拟中获得的真实属性；为了减少MD计算时间长的问题，进行了大量的短时间、非收敛的MD仿真和少量的长时间、收敛的仿真；为了从低精度模拟属性扩展到高精度模拟属性，通过多任务学习来学习两种属性之间的修正的方式。
通过这种方式构建的模型在真实属性方面的预测误差小于单个MD模拟的随机误差，同时比线性校正更好地校正了非收敛模拟的系统误差。由于模型精度的提高，成功地筛选了6247种聚合物的空间，并从空间中发现了最佳的聚合物电解质，与直接模拟每种聚合物的一次长时间模拟相比，相当于22.8倍的加速。最后，通过分析聚合物电解质在化学空间中的预测性质，得出了聚合物电解质的设计原则。
代码
文本
我们先来下载一下pytorch，pytorch_geometric和RDkit软件包。
代码
文本
[30]
! pip install torch
! pip install torch_geometric
! pip install rdkit
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple Requirement already satisfied: torch in /opt/conda/lib/python3.8/site-packages (1.13.1+cu116) Requirement already satisfied: typing-extensions in /opt/conda/lib/python3.8/site-packages (from torch) (4.5.0) WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple Collecting torch_geometric Downloading https://pypi.tuna.tsinghua.edu.cn/packages/06/a5/9f5af849c4185da5ea55f70ef17e23f93355cd4e989d82cfc8ba2d8747af/torch_geometric-2.3.1.tar.gz (661 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 661.6/661.6 kB 869.0 kB/s eta 0:00:0000:0100:01 Installing build dependencies ... done Getting requirements to build wheel ... done Preparing metadata (pyproject.toml) ... done Requirement already satisfied: tqdm in /opt/conda/lib/python3.8/site-packages (from torch_geometric) (4.64.1) Requirement already satisfied: psutil>=5.8.0 in /opt/conda/lib/python3.8/site-packages (from torch_geometric) (5.9.0) Requirement already satisfied: numpy in /opt/conda/lib/python3.8/site-packages (from torch_geometric) (1.22.4) Requirement already satisfied: scikit-learn in /opt/conda/lib/python3.8/site-packages (from torch_geometric) (1.0.2) Requirement already satisfied: scipy in /opt/conda/lib/python3.8/site-packages (from torch_geometric) (1.7.3) Requirement already satisfied: jinja2 in /opt/conda/lib/python3.8/site-packages (from torch_geometric) (3.1.2) Requirement already satisfied: pyparsing in /opt/conda/lib/python3.8/site-packages (from torch_geometric) (3.0.9) Requirement already satisfied: requests in /opt/conda/lib/python3.8/site-packages (from torch_geometric) (2.28.2) Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/lib/python3.8/site-packages (from jinja2->torch_geometric) (2.1.1) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /opt/conda/lib/python3.8/site-packages (from requests->torch_geometric) (1.26.14) Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.8/site-packages (from requests->torch_geometric) (2022.12.7) Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.8/site-packages (from requests->torch_geometric) (3.4) Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/lib/python3.8/site-packages (from requests->torch_geometric) (3.0.1) Requirement already satisfied: joblib>=0.11 in /opt/conda/lib/python3.8/site-packages (from scikit-learn->torch_geometric) (1.2.0) Requirement already satisfied: threadpoolctl>=2.0.0 in /opt/conda/lib/python3.8/site-packages (from scikit-learn->torch_geometric) (3.1.0) Building wheels for collected packages: torch_geometric Building wheel for torch_geometric (pyproject.toml) ... done Created wheel for torch_geometric: filename=torch_geometric-2.3.1-py3-none-any.whl size=910459 sha256=0d48868b8ebe32043b61cb69a6aba7f23dcd610af7ff06f53e6106fe92addbb3 Stored in directory: /root/.cache/pip/wheels/b3/90/52/6efafd02d2f57d08d544a62c543e2b8945236c720a96548617 Successfully built torch_geometric Installing collected packages: torch_geometric Successfully installed torch_geometric-2.3.1 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
代码
文本
案例的数据和模型代码来自 https://github.com/txie-93/polymernet ，运行下面的代码就可以直接下载到bohrium上。
代码
文本
[1]
! cd /data; if [ ! -e GCN ];then git clone https://github.com/txie-93/polymernet.git;fi;
fatal: destination path 'polymernet' already exists and is not an empty directory.
代码
文本
[31]
import torch
import torch_geometric
import rdkit
代码
文本
代码
文本
数据类型
代码
文本
以导电性数据为例，5ns和50ns文件夹分别包括了九个（cv_1.csv~cv_9.csv）训练数据集，cv_0.csv数据集用于验证，test.csv数据集用于测试模型性能
代码
文本
[53]
! tree /data/polymernet/data/conductivity
/data/polymernet/data/conductivity ├── 50ns │ ├── cv_0.csv │ ├── cv_1.csv │ ├── cv_2.csv │ ├── cv_3.csv │ ├── cv_4.csv │ ├── cv_5.csv │ ├── cv_6.csv │ ├── cv_7.csv │ ├── cv_8.csv │ ├── cv_9.csv │ └── test.csv ├── 50ns_extrapolate │ ├── cv_0.csv │ ├── cv_1.csv │ ├── cv_2.csv │ ├── cv_3.csv │ ├── cv_4.csv │ ├── cv_5.csv │ ├── cv_6.csv │ ├── cv_7.csv │ ├── cv_8.csv │ ├── cv_9.csv │ └── test.csv ├── 5ns │ ├── cv_0.csv │ ├── cv_1.csv │ ├── cv_2.csv │ ├── cv_3.csv │ ├── cv_4.csv │ ├── cv_5.csv │ ├── cv_6.csv │ ├── cv_7.csv │ ├── cv_8.csv │ ├── cv_9.csv │ ├── pred.csv │ └── test.csv └── cond_5ns_new_config.csv 3 directories, 35 files
代码
文本
数据集包括聚合物id、SMILE名称和MD计算的离子传导率，下面是cv_0.csv演示的内容。
代码
文本
[52]
! head /data/polymernet/data/conductivity/5ns/cv_0.csv
9-0-413616244-0,CN(CCCN(C)CCOC(=O)[Au])CCO[Cu],0.00017827816102107538 9-0-538226084-0,CC(COCCOCCOC(=O)[Au])O[Cu],0.00020509371483390544 9-0-33222100-0,C=CC(COC(=O)[Au])NCC(COC)O[Cu],0.0001301783890331634 9-0-246210842-0,CC(CCO[Cu])NC(C)C(C)(C)OC(=O)[Au],2.465121877993986e-05 9-0-413690717-0,CC(C)(CO[Cu])C(=O)NCC=CCOC(=O)[Au],4.339674367150885e-05 9-0-1119507515-0,CC(C)(NCC(CO[Cu])OC(=O)[Au])C(=O)O,4.462199095252281e-05 9-0-962050944-0,O=C([Au])NCCSCCCSCCN[Cu],0.00015008015756796058 9-0-13127562-0,CC(CNCC#CCO[Cu])OC(=O)[Au],7.655827506140518e-05 9-0-1119510779-0,CN(CCCO[Cu])C(=O)COC(=O)[Au],6.0616822240295196e-05 9-0-413632785-0,O=C([Au])OCCNC(=O)C(CC(F)F)N[Cu],2.8385849750179147e-05
代码
文本
模型架构
我们这里的复现着重对多任务模型消除误差进行讲解，他们是怎么实现的呢？
代码
文本
问题描述：常规通过反复模拟来消除误差的方法计算成本很高
解决思路：
- 将单体结构编码为图表示，采用CGCNN来学习相应聚合物的表示信息
- 假设存在一个真实目标属性，且是由聚合物的结构唯一决定的(这需要无限重复的模拟才能获得)，并且由于模拟中的随机误差，MD计算的目标属性与真实属性略有不同。这个假设可以写成，， 是MD计算得到的目标性质， 是f是一个从单体结构映射到真实聚合物性质的确定性函数，是单体结构的编码，ε是一个与无关的随机变量，平均值为零。为了学习，需要大量的数据对t进行回归
- 训练数据生成：50ns的数据太少，用5ns的数据进行补充，， 数据集包括5ns（包含876个高分子）和50ns（包含117个高分子），90%用于训练，10%用于测试
- 合并损失函数：
代码
文本
模型训练
代码
文本
[32]
! cd /data/polymernet && python single_task_train.py --log10 0 data/logp/noise_5.12
Type train csvs ['cv_1.csv', 'cv_2.csv', 'cv_3.csv', 'cv_4.csv', 'cv_5.csv', 'cv_6.csv', 'cv_7.csv', 'cv_8.csv', 'cv_9.csv'] Type val csvs ['cv_0.csv'] Type test csvs ['test.csv'] /opt/conda/lib/python3.8/site-packages/torch_geometric/deprecation.py:22: UserWarning: 'data.DataLoader' is deprecated, use 'loader.DataLoader' instead warnings.warn(out) Type pred csvs ['pred.csv'] /opt/conda/lib/python3.8/site-packages/torch_geometric/deprecation.py:22: UserWarning: 'nn.glob.GlobalAttention' is deprecated, use 'nn.aggr.AttentionalAggregation' instead warnings.warn(out) Epoch: 000, LR: 0.001000, Loss: 1.0079191, Validation MAE: 3.6561177, Best Validation MAE: 3.6561177, Test MAE: 4.2010352 Epoch: 001, LR: 0.001000, Loss: 1.0006775, Validation MAE: 3.6438458, Best Validation MAE: 3.6438458, Test MAE: 4.1901032 Epoch: 002, LR: 0.001000, Loss: 0.9876108, Validation MAE: 3.5470519, Best Validation MAE: 3.5470519, Test MAE: 4.2167472 Epoch: 003, LR: 0.001000, Loss: 0.9948496, Validation MAE: 3.5084034, Best Validation MAE: 3.5084034, Test MAE: 4.2021792 Epoch: 004, LR: 0.001000, Loss: 0.9876342, Validation MAE: 3.5290245, Best Validation MAE: 3.5084034, Test MAE: 4.2021792 Epoch: 005, LR: 0.001000, Loss: 0.9813120, Validation MAE: 3.5415621, Best Validation MAE: 3.5084034, Test MAE: 4.2021792 Epoch: 006, LR: 0.001000, Loss: 0.9888191, Validation MAE: 3.6392662, Best Validation MAE: 3.5084034, Test MAE: 4.2021792 Epoch: 007, LR: 0.001000, Loss: 0.9819395, Validation MAE: 3.5289910, Best Validation MAE: 3.5084034, Test MAE: 4.2021792 Epoch: 008, LR: 0.001000, Loss: 0.9715480, Validation MAE: 3.3927676, Best Validation MAE: 3.3927676, Test MAE: 4.2290836 Epoch: 009, LR: 0.001000, Loss: 0.9776229, Validation MAE: 3.4520381, Best Validation MAE: 3.3927676, Test MAE: 4.2290836 Epoch: 010, LR: 0.001000, Loss: 0.9711837, Validation MAE: 3.3999035, Best Validation MAE: 3.3927676, Test MAE: 4.2290836 Epoch: 011, LR: 0.001000, Loss: 0.9645516, Validation MAE: 3.5811813, Best Validation MAE: 3.3927676, Test MAE: 4.2290836 Epoch: 012, LR: 0.001000, Loss: 0.9600396, Validation MAE: 3.4825220, Best Validation MAE: 3.3927676, Test MAE: 4.2290836 Epoch: 013, LR: 0.001000, Loss: 0.9539974, Validation MAE: 3.5635794, Best Validation MAE: 3.3927676, Test MAE: 4.2290836 Epoch: 014, LR: 0.001000, Loss: 0.9542005, Validation MAE: 3.3505681, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 015, LR: 0.001000, Loss: 0.9623818, Validation MAE: 3.6469874, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 016, LR: 0.001000, Loss: 0.9434891, Validation MAE: 3.5653312, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 017, LR: 0.001000, Loss: 0.9500265, Validation MAE: 3.5653187, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 018, LR: 0.001000, Loss: 0.9396547, Validation MAE: 3.7341463, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 019, LR: 0.001000, Loss: 0.9599667, Validation MAE: 3.5531210, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 020, LR: 0.001000, Loss: 0.9397011, Validation MAE: 3.5560237, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 021, LR: 0.001000, Loss: 0.9409040, Validation MAE: 3.6011243, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 022, LR: 0.001000, Loss: 0.9391779, Validation MAE: 3.7930737, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 023, LR: 0.001000, Loss: 0.9382383, Validation MAE: 3.5025237, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 024, LR: 0.001000, Loss: 0.9282528, Validation MAE: 3.6488875, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 025, LR: 0.001000, Loss: 0.9168642, Validation MAE: 3.7349550, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 026, LR: 0.001000, Loss: 0.9277498, Validation MAE: 3.5401054, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 027, LR: 0.001000, Loss: 0.9141976, Validation MAE: 3.7549039, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 028, LR: 0.001000, Loss: 0.9179365, Validation MAE: 3.6886761, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 029, LR: 0.001000, Loss: 0.9151736, Validation MAE: 3.5446925, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 030, LR: 0.001000, Loss: 0.9313986, Validation MAE: 3.6173723, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 031, LR: 0.001000, Loss: 0.9145647, Validation MAE: 3.7440029, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 032, LR: 0.001000, Loss: 0.9063899, Validation MAE: 3.5810924, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 033, LR: 0.001000, Loss: 0.9081892, Validation MAE: 3.6536568, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 034, LR: 0.001000, Loss: 0.9054228, Validation MAE: 3.6445046, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 035, LR: 0.001000, Loss: 0.9043991, Validation MAE: 3.4813844, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 036, LR: 0.000700, Loss: 0.8722559, Validation MAE: 3.7198532, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 037, LR: 0.000700, Loss: 0.8868859, Validation MAE: 3.5370110, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 038, LR: 0.000700, Loss: 0.8838481, Validation MAE: 3.6912963, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 039, LR: 0.000700, Loss: 0.8655826, Validation MAE: 3.6946388, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 040, LR: 0.000700, Loss: 0.8704696, Validation MAE: 3.7341318, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 041, LR: 0.000700, Loss: 0.8906665, Validation MAE: 3.7045469, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 042, LR: 0.000700, Loss: 0.8713605, Validation MAE: 3.6196908, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 043, LR: 0.000700, Loss: 0.8711517, Validation MAE: 3.9700525, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 044, LR: 0.000700, Loss: 0.8684062, Validation MAE: 3.6241104, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 045, LR: 0.000700, Loss: 0.8725513, Validation MAE: 3.8277891, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 046, LR: 0.000700, Loss: 0.8501632, Validation MAE: 3.7710319, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 047, LR: 0.000700, Loss: 0.8580150, Validation MAE: 3.6325221, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 048, LR: 0.000700, Loss: 0.8855386, Validation MAE: 3.8158845, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 049, LR: 0.000700, Loss: 0.8606727, Validation MAE: 3.7606329, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 050, LR: 0.000700, Loss: 0.8496389, Validation MAE: 3.6829389, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 051, LR: 0.000700, Loss: 0.8459622, Validation MAE: 3.9607578, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 052, LR: 0.000700, Loss: 0.8471979, Validation MAE: 3.7956482, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 053, LR: 0.000700, Loss: 0.8471824, Validation MAE: 3.8839882, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 054, LR: 0.000700, Loss: 0.8422751, Validation MAE: 4.0233694, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 055, LR: 0.000700, Loss: 0.8448830, Validation MAE: 3.8016089, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 056, LR: 0.000700, Loss: 0.8354773, Validation MAE: 3.9049062, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 057, LR: 0.000490, Loss: 0.8264168, Validation MAE: 3.8100087, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 058, LR: 0.000490, Loss: 0.8152956, Validation MAE: 4.0235628, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 059, LR: 0.000490, Loss: 0.8102573, Validation MAE: 3.9019944, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 060, LR: 0.000490, Loss: 0.8008470, Validation MAE: 3.8733698, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 061, LR: 0.000490, Loss: 0.8113878, Validation MAE: 3.8259047, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 062, LR: 0.000490, Loss: 0.8013931, Validation MAE: 4.0702821, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 063, LR: 0.000490, Loss: 0.8015032, Validation MAE: 3.7568511, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 064, LR: 0.000490, Loss: 0.8172040, Validation MAE: 3.7773546, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 065, LR: 0.000490, Loss: 0.8066885, Validation MAE: 3.8747082, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 066, LR: 0.000490, Loss: 0.7883166, Validation MAE: 3.8383880, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 067, LR: 0.000490, Loss: 0.7838246, Validation MAE: 4.0326806, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 068, LR: 0.000490, Loss: 0.7969582, Validation MAE: 3.6864918, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 069, LR: 0.000490, Loss: 0.7845900, Validation MAE: 3.7906216, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 070, LR: 0.000490, Loss: 0.7829038, Validation MAE: 3.9087062, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 071, LR: 0.000490, Loss: 0.7819139, Validation MAE: 3.7927531, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 072, LR: 0.000490, Loss: 0.7840215, Validation MAE: 3.8670652, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 073, LR: 0.000490, Loss: 0.7994667, Validation MAE: 3.8563075, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 074, LR: 0.000490, Loss: 0.7813467, Validation MAE: 3.6600803, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 075, LR: 0.000490, Loss: 0.7866704, Validation MAE: 3.8279104, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 076, LR: 0.000490, Loss: 0.7795839, Validation MAE: 3.9804814, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 077, LR: 0.000490, Loss: 0.7833246, Validation MAE: 3.7486213, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 078, LR: 0.000343, Loss: 0.7679972, Validation MAE: 3.8810070, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 079, LR: 0.000343, Loss: 0.7563890, Validation MAE: 3.9723815, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 080, LR: 0.000343, Loss: 0.7472286, Validation MAE: 3.8710705, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 081, LR: 0.000343, Loss: 0.7627459, Validation MAE: 3.9238854, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 082, LR: 0.000343, Loss: 0.7445888, Validation MAE: 4.0336835, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 083, LR: 0.000343, Loss: 0.7577700, Validation MAE: 3.9431738, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 084, LR: 0.000343, Loss: 0.7419415, Validation MAE: 3.9485384, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 085, LR: 0.000343, Loss: 0.7442266, Validation MAE: 3.9471362, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 086, LR: 0.000343, Loss: 0.7491210, Validation MAE: 3.8588816, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 087, LR: 0.000343, Loss: 0.7389029, Validation MAE: 3.8613820, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 088, LR: 0.000343, Loss: 0.7502310, Validation MAE: 3.7809670, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 089, LR: 0.000343, Loss: 0.7400830, Validation MAE: 3.9083159, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 090, LR: 0.000343, Loss: 0.7329096, Validation MAE: 3.9505676, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 091, LR: 0.000343, Loss: 0.7299683, Validation MAE: 3.8970257, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 092, LR: 0.000343, Loss: 0.7186689, Validation MAE: 4.0220199, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 093, LR: 0.000343, Loss: 0.7290625, Validation MAE: 3.8766309, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 094, LR: 0.000343, Loss: 0.7268826, Validation MAE: 3.8557442, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 095, LR: 0.000343, Loss: 0.7227190, Validation MAE: 3.9192366, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 096, LR: 0.000343, Loss: 0.7435275, Validation MAE: 3.9747037, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 097, LR: 0.000343, Loss: 0.7238804, Validation MAE: 3.9896868, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 098, LR: 0.000343, Loss: 0.7185756, Validation MAE: 3.9276484, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 099, LR: 0.000240, Loss: 0.7068815, Validation MAE: 3.9280799, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 100, LR: 0.000240, Loss: 0.7074579, Validation MAE: 3.8654546, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 101, LR: 0.000240, Loss: 0.7023548, Validation MAE: 3.9281072, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 102, LR: 0.000240, Loss: 0.7036551, Validation MAE: 3.8726884, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 103, LR: 0.000240, Loss: 0.7098450, Validation MAE: 3.9093573, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 104, LR: 0.000240, Loss: 0.6945707, Validation MAE: 3.8975743, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 105, LR: 0.000240, Loss: 0.7301453, Validation MAE: 4.1634446, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 106, LR: 0.000240, Loss: 0.7113347, Validation MAE: 3.9163547, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 107, LR: 0.000240, Loss: 0.6957734, Validation MAE: 3.9996017, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 108, LR: 0.000240, Loss: 0.7044227, Validation MAE: 3.8919457, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 109, LR: 0.000240, Loss: 0.7038648, Validation MAE: 3.9520105, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 110, LR: 0.000240, Loss: 0.6957047, Validation MAE: 4.1633735, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 111, LR: 0.000240, Loss: 0.6998028, Validation MAE: 4.0879699, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 112, LR: 0.000240, Loss: 0.7032497, Validation MAE: 3.8246686, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 113, LR: 0.000240, Loss: 0.6972188, Validation MAE: 3.9644435, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 114, LR: 0.000240, Loss: 0.6896585, Validation MAE: 3.8805713, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 115, LR: 0.000240, Loss: 0.7018686, Validation MAE: 4.0455101, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 116, LR: 0.000240, Loss: 0.6815858, Validation MAE: 4.1056870, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 117, LR: 0.000240, Loss: 0.6858696, Validation MAE: 4.0583344, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 118, LR: 0.000240, Loss: 0.6893380, Validation MAE: 3.7657447, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 119, LR: 0.000240, Loss: 0.6825231, Validation MAE: 4.0918129, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 120, LR: 0.000168, Loss: 0.6733719, Validation MAE: 4.0804748, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 121, LR: 0.000168, Loss: 0.6819621, Validation MAE: 4.0468392, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 122, LR: 0.000168, Loss: 0.6712011, Validation MAE: 4.0768272, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 123, LR: 0.000168, Loss: 0.6580609, Validation MAE: 4.0289962, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 124, LR: 0.000168, Loss: 0.6784055, Validation MAE: 4.0680570, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 125, LR: 0.000168, Loss: 0.6720761, Validation MAE: 4.1805824, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 126, LR: 0.000168, Loss: 0.6776290, Validation MAE: 4.1621765, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 127, LR: 0.000168, Loss: 0.6798655, Validation MAE: 4.1058071, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 128, LR: 0.000168, Loss: 0.6657880, Validation MAE: 4.1114309, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 129, LR: 0.000168, Loss: 0.6603020, Validation MAE: 3.9554610, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 130, LR: 0.000168, Loss: 0.6657391, Validation MAE: 4.0211674, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 131, LR: 0.000168, Loss: 0.6776440, Validation MAE: 4.0346135, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 132, LR: 0.000168, Loss: 0.6644422, Validation MAE: 4.0708615, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 133, LR: 0.000168, Loss: 0.6690529, Validation MAE: 4.0751131, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 134, LR: 0.000168, Loss: 0.6601926, Validation MAE: 4.1871278, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 135, LR: 0.000168, Loss: 0.6670363, Validation MAE: 4.1127052, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 136, LR: 0.000168, Loss: 0.6654744, Validation MAE: 4.0727346, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 137, LR: 0.000168, Loss: 0.6590190, Validation MAE: 4.1580005, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 138, LR: 0.000168, Loss: 0.6527942, Validation MAE: 4.0612960, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 139, LR: 0.000168, Loss: 0.6645174, Validation MAE: 4.0821712, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 140, LR: 0.000168, Loss: 0.6470408, Validation MAE: 3.9643837, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 141, LR: 0.000118, Loss: 0.6442673, Validation MAE: 4.0181158, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 142, LR: 0.000118, Loss: 0.6666488, Validation MAE: 4.0781874, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 143, LR: 0.000118, Loss: 0.6500663, Validation MAE: 3.9807604, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 144, LR: 0.000118, Loss: 0.6568501, Validation MAE: 4.1360836, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 145, LR: 0.000118, Loss: 0.6230123, Validation MAE: 4.1538810, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 146, LR: 0.000118, Loss: 0.6456209, Validation MAE: 4.1039127, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 147, LR: 0.000118, Loss: 0.6466568, Validation MAE: 4.1248536, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 148, LR: 0.000118, Loss: 0.6401928, Validation MAE: 4.1012914, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 149, LR: 0.000118, Loss: 0.6449630, Validation MAE: 4.0908547, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 150, LR: 0.000118, Loss: 0.6446109, Validation MAE: 4.2479317, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 151, LR: 0.000118, Loss: 0.6373500, Validation MAE: 4.1172430, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 152, LR: 0.000118, Loss: 0.6335556, Validation MAE: 4.1530852, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 153, LR: 0.000118, Loss: 0.6295283, Validation MAE: 4.1520388, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 154, LR: 0.000118, Loss: 0.6503015, Validation MAE: 4.0590347, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 155, LR: 0.000118, Loss: 0.6399904, Validation MAE: 4.0828697, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 156, LR: 0.000118, Loss: 0.6395661, Validation MAE: 4.1443463, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 157, LR: 0.000118, Loss: 0.6493184, Validation MAE: 4.1759791, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 158, LR: 0.000118, Loss: 0.6326253, Validation MAE: 4.1375814, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 159, LR: 0.000118, Loss: 0.6323408, Validation MAE: 4.0305670, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 160, LR: 0.000118, Loss: 0.6422125, Validation MAE: 4.1057156, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 161, LR: 0.000118, Loss: 0.6338290, Validation MAE: 4.0978579, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 162, LR: 0.000082, Loss: 0.6452717, Validation MAE: 4.1397902, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 163, LR: 0.000082, Loss: 0.6165965, Validation MAE: 4.1016481, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 164, LR: 0.000082, Loss: 0.6311217, Validation MAE: 4.1561583, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 165, LR: 0.000082, Loss: 0.6277789, Validation MAE: 4.1458739, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 166, LR: 0.000082, Loss: 0.6284708, Validation MAE: 4.1529921, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 167, LR: 0.000082, Loss: 0.6324971, Validation MAE: 4.1121864, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 168, LR: 0.000082, Loss: 0.6203227, Validation MAE: 4.1572358, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 169, LR: 0.000082, Loss: 0.6275275, Validation MAE: 4.0932729, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 170, LR: 0.000082, Loss: 0.6231303, Validation MAE: 4.2156541, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 171, LR: 0.000082, Loss: 0.6288447, Validation MAE: 4.1107853, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 172, LR: 0.000082, Loss: 0.6154311, Validation MAE: 4.2437556, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 173, LR: 0.000082, Loss: 0.6273217, Validation MAE: 4.1920121, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 174, LR: 0.000082, Loss: 0.6338081, Validation MAE: 4.1345535, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 175, LR: 0.000082, Loss: 0.6281858, Validation MAE: 4.1264279, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 176, LR: 0.000082, Loss: 0.6359167, Validation MAE: 4.1157346, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 177, LR: 0.000082, Loss: 0.6296624, Validation MAE: 4.2017528, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 178, LR: 0.000082, Loss: 0.6267653, Validation MAE: 4.1741648, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 179, LR: 0.000082, Loss: 0.6300560, Validation MAE: 4.1523520, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 180, LR: 0.000082, Loss: 0.6266444, Validation MAE: 4.1982080, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 181, LR: 0.000082, Loss: 0.6178547, Validation MAE: 4.0928613, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 182, LR: 0.000082, Loss: 0.6289184, Validation MAE: 4.1148950, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 183, LR: 0.000058, Loss: 0.6204636, Validation MAE: 4.1953567, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 184, LR: 0.000058, Loss: 0.6278166, Validation MAE: 4.1876268, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 185, LR: 0.000058, Loss: 0.6198133, Validation MAE: 4.0880509, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 186, LR: 0.000058, Loss: 0.6014782, Validation MAE: 4.1532572, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 187, LR: 0.000058, Loss: 0.6127083, Validation MAE: 4.1246921, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 188, LR: 0.000058, Loss: 0.6151294, Validation MAE: 4.1384090, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 189, LR: 0.000058, Loss: 0.6235056, Validation MAE: 4.1228888, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 190, LR: 0.000058, Loss: 0.6047949, Validation MAE: 4.1866106, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 191, LR: 0.000058, Loss: 0.6172754, Validation MAE: 4.0808628, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 192, LR: 0.000058, Loss: 0.6151622, Validation MAE: 4.0969647, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 193, LR: 0.000058, Loss: 0.6159860, Validation MAE: 4.1094072, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 194, LR: 0.000058, Loss: 0.6062729, Validation MAE: 4.1592654, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 195, LR: 0.000058, Loss: 0.6121015, Validation MAE: 4.1680827, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 196, LR: 0.000058, Loss: 0.6146634, Validation MAE: 4.2030834, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 197, LR: 0.000058, Loss: 0.6191993, Validation MAE: 4.1134149, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 198, LR: 0.000058, Loss: 0.6116156, Validation MAE: 4.1880867, Best Validation MAE: 3.3505681, Test MAE: 4.1812692 Epoch: 199, LR: 0.000058, Loss: 0.6160542, Validation MAE: 4.1367778, Best Validation MAE: 3.3505681, Test MAE: 4.1812692
代码
文本
[35]
! cd /data/polymernet && python multi_task_train.py data/conductivity/5ns data/conductivity/50ns
Type train csvs ['cv_1.csv', 'cv_2.csv', 'cv_3.csv', 'cv_4.csv', 'cv_5.csv', 'cv_6.csv', 'cv_7.csv', 'cv_8.csv', 'cv_9.csv'] Type val csvs ['cv_0.csv'] Type test csvs ['test.csv'] Type train csvs ['cv_1.csv', 'cv_2.csv', 'cv_3.csv', 'cv_4.csv', 'cv_5.csv', 'cv_6.csv', 'cv_7.csv', 'cv_8.csv', 'cv_9.csv'] Type val csvs ['cv_0.csv'] Type test csvs ['test.csv'] /opt/conda/lib/python3.8/site-packages/torch_geometric/deprecation.py:22: UserWarning: 'data.DataLoader' is deprecated, use 'loader.DataLoader' instead warnings.warn(out) Type pred csvs ['pred.csv'] /opt/conda/lib/python3.8/site-packages/torch_geometric/deprecation.py:22: UserWarning: 'nn.glob.GlobalAttention' is deprecated, use 'nn.aggr.AttentionalAggregation' instead warnings.warn(out) Epoch: 000, LR: 0.001000, Loss: 1.0345437, Validation exp MAE: 0.2085239, Validation sim MAE: 0.2343769, Best Validation MAE: 0.2085239, Test exp MAE: 0.1632079, Test sim MAE: 0.1785585 Epoch: 001, LR: 0.001000, Loss: 0.7621363, Validation exp MAE: 0.2309920, Validation sim MAE: 0.2019072, Best Validation MAE: 0.2085239, Test exp MAE: 0.1632079, Test sim MAE: 0.1785585 Epoch: 002, LR: 0.001000, Loss: 0.5888378, Validation exp MAE: 0.1831155, Validation sim MAE: 0.1611083, Best Validation MAE: 0.1831155, Test exp MAE: 0.2048954, Test sim MAE: 0.1217799 Epoch: 003, LR: 0.001000, Loss: 0.4650620, Validation exp MAE: 0.1013253, Validation sim MAE: 0.1306380, Best Validation MAE: 0.1013253, Test exp MAE: 0.1114545, Test sim MAE: 0.0909970 Epoch: 004, LR: 0.001000, Loss: 0.4227607, Validation exp MAE: 0.1335523, Validation sim MAE: 0.1349934, Best Validation MAE: 0.1013253, Test exp MAE: 0.1114545, Test sim MAE: 0.0909970 Epoch: 005, LR: 0.001000, Loss: 0.3995860, Validation exp MAE: 0.1067551, Validation sim MAE: 0.1253388, Best Validation MAE: 0.1013253, Test exp MAE: 0.1114545, Test sim MAE: 0.0909970 Epoch: 006, LR: 0.001000, Loss: 0.4121429, Validation exp MAE: 0.0811863, Validation sim MAE: 0.1308657, Best Validation MAE: 0.0811863, Test exp MAE: 0.1115289, Test sim MAE: 0.0951228 Epoch: 007, LR: 0.001000, Loss: 0.3710645, Validation exp MAE: 0.1102458, Validation sim MAE: 0.1332774, Best Validation MAE: 0.0811863, Test exp MAE: 0.1115289, Test sim MAE: 0.0951228 Epoch: 008, LR: 0.001000, Loss: 0.3890064, Validation exp MAE: 0.1489270, Validation sim MAE: 0.1438660, Best Validation MAE: 0.0811863, Test exp MAE: 0.1115289, Test sim MAE: 0.0951228 Epoch: 009, LR: 0.001000, Loss: 0.3670784, Validation exp MAE: 0.1145498, Validation sim MAE: 0.1423334, Best Validation MAE: 0.0811863, Test exp MAE: 0.1115289, Test sim MAE: 0.0951228 Epoch: 010, LR: 0.001000, Loss: 0.3815618, Validation exp MAE: 0.1271060, Validation sim MAE: 0.1328231, Best Validation MAE: 0.0811863, Test exp MAE: 0.1115289, Test sim MAE: 0.0951228 Epoch: 011, LR: 0.001000, Loss: 0.3657936, Validation exp MAE: 0.1091851, Validation sim MAE: 0.1413889, Best Validation MAE: 0.0811863, Test exp MAE: 0.1115289, Test sim MAE: 0.0951228 Epoch: 012, LR: 0.001000, Loss: 0.3808824, Validation exp MAE: 0.0749333, Validation sim MAE: 0.1288266, Best Validation MAE: 0.0749333, Test exp MAE: 0.1111283, Test sim MAE: 0.0953215 Epoch: 013, LR: 0.001000, Loss: 0.3447186, Validation exp MAE: 0.1205600, Validation sim MAE: 0.1345140, Best Validation MAE: 0.0749333, Test exp MAE: 0.1111283, Test sim MAE: 0.0953215 Epoch: 014, LR: 0.001000, Loss: 0.3558554, Validation exp MAE: 0.1111974, Validation sim MAE: 0.1253715, Best Validation MAE: 0.0749333, Test exp MAE: 0.1111283, Test sim MAE: 0.0953215 Epoch: 015, LR: 0.001000, Loss: 0.3528100, Validation exp MAE: 0.1250694, Validation sim MAE: 0.1274108, Best Validation MAE: 0.0749333, Test exp MAE: 0.1111283, Test sim MAE: 0.0953215 Epoch: 016, LR: 0.001000, Loss: 0.3294473, Validation exp MAE: 0.0886762, Validation sim MAE: 0.1404715, Best Validation MAE: 0.0749333, Test exp MAE: 0.1111283, Test sim MAE: 0.0953215 Epoch: 017, LR: 0.001000, Loss: 0.3313783, Validation exp MAE: 0.0818454, Validation sim MAE: 0.1425812, Best Validation MAE: 0.0749333, Test exp MAE: 0.1111283, Test sim MAE: 0.0953215 Epoch: 018, LR: 0.001000, Loss: 0.3475677, Validation exp MAE: 0.1474214, Validation sim MAE: 0.1427400, Best Validation MAE: 0.0749333, Test exp MAE: 0.1111283, Test sim MAE: 0.0953215 Epoch: 019, LR: 0.001000, Loss: 0.3098766, Validation exp MAE: 0.0987498, Validation sim MAE: 0.1275526, Best Validation MAE: 0.0749333, Test exp MAE: 0.1111283, Test sim MAE: 0.0953215 Epoch: 020, LR: 0.001000, Loss: 0.3177650, Validation exp MAE: 0.0965355, Validation sim MAE: 0.1299959, Best Validation MAE: 0.0749333, Test exp MAE: 0.1111283, Test sim MAE: 0.0953215 Epoch: 021, LR: 0.001000, Loss: 0.3197838, Validation exp MAE: 0.1213780, Validation sim MAE: 0.1303299, Best Validation MAE: 0.0749333, Test exp MAE: 0.1111283, Test sim MAE: 0.0953215 Epoch: 022, LR: 0.001000, Loss: 0.3244143, Validation exp MAE: 0.0781636, Validation sim MAE: 0.1343999, Best Validation MAE: 0.0749333, Test exp MAE: 0.1111283, Test sim MAE: 0.0953215 Epoch: 023, LR: 0.001000, Loss: 0.3223155, Validation exp MAE: 0.1008880, Validation sim MAE: 0.1363778, Best Validation MAE: 0.0749333, Test exp MAE: 0.1111283, Test sim MAE: 0.0953215 Epoch: 024, LR: 0.001000, Loss: 0.3171630, Validation exp MAE: 0.1403529, Validation sim MAE: 0.1334976, Best Validation MAE: 0.0749333, Test exp MAE: 0.1111283, Test sim MAE: 0.0953215 Epoch: 025, LR: 0.001000, Loss: 0.3106532, Validation exp MAE: 0.1015714, Validation sim MAE: 0.1311367, Best Validation MAE: 0.0749333, Test exp MAE: 0.1111283, Test sim MAE: 0.0953215 Epoch: 026, LR: 0.001000, Loss: 0.3108561, Validation exp MAE: 0.1470740, Validation sim MAE: 0.1500750, Best Validation MAE: 0.0749333, Test exp MAE: 0.1111283, Test sim MAE: 0.0953215 Epoch: 027, LR: 0.001000, Loss: 0.3249191, Validation exp MAE: 0.0899896, Validation sim MAE: 0.1272192, Best Validation MAE: 0.0749333, Test exp MAE: 0.1111283, Test sim MAE: 0.0953215 Epoch: 028, LR: 0.001000, Loss: 0.3033144, Validation exp MAE: 0.1319943, Validation sim MAE: 0.1318220, Best Validation MAE: 0.0749333, Test exp MAE: 0.1111283, Test sim MAE: 0.0953215 Epoch: 029, LR: 0.001000, Loss: 0.3025165, Validation exp MAE: 0.0827142, Validation sim MAE: 0.1389472, Best Validation MAE: 0.0749333, Test exp MAE: 0.1111283, Test sim MAE: 0.0953215 Epoch: 030, LR: 0.001000, Loss: 0.3070384, Validation exp MAE: 0.0784032, Validation sim MAE: 0.1302036, Best Validation MAE: 0.0749333, Test exp MAE: 0.1111283, Test sim MAE: 0.0953215 Epoch: 031, LR: 0.001000, Loss: 0.2914352, Validation exp MAE: 0.1032034, Validation sim MAE: 0.1323858, Best Validation MAE: 0.0749333, Test exp MAE: 0.1111283, Test sim MAE: 0.0953215 Epoch: 032, LR: 0.001000, Loss: 0.3022243, Validation exp MAE: 0.1126031, Validation sim MAE: 0.1286811, Best Validation MAE: 0.0749333, Test exp MAE: 0.1111283, Test sim MAE: 0.0953215 Epoch: 033, LR: 0.001000, Loss: 0.2958803, Validation exp MAE: 0.1034565, Validation sim MAE: 0.1353392, Best Validation MAE: 0.0749333, Test exp MAE: 0.1111283, Test sim MAE: 0.0953215 Epoch: 034, LR: 0.000700, Loss: 0.2820727, Validation exp MAE: 0.0989544, Validation sim MAE: 0.1297092, Best Validation MAE: 0.0749333, Test exp MAE: 0.1111283, Test sim MAE: 0.0953215 Epoch: 035, LR: 0.000700, Loss: 0.2941067, Validation exp MAE: 0.1147409, Validation sim MAE: 0.1262908, Best Validation MAE: 0.0749333, Test exp MAE: 0.1111283, Test sim MAE: 0.0953215 Epoch: 036, LR: 0.000700, Loss: 0.2878055, Validation exp MAE: 0.0942308, Validation sim MAE: 0.1274443, Best Validation MAE: 0.0749333, Test exp MAE: 0.1111283, Test sim MAE: 0.0953215 Epoch: 037, LR: 0.000700, Loss: 0.2695617, Validation exp MAE: 0.0705967, Validation sim MAE: 0.1282461, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 038, LR: 0.000700, Loss: 0.2834594, Validation exp MAE: 0.1068955, Validation sim MAE: 0.1274615, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 039, LR: 0.000700, Loss: 0.2875639, Validation exp MAE: 0.1055211, Validation sim MAE: 0.1292633, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 040, LR: 0.000700, Loss: 0.2816099, Validation exp MAE: 0.1041406, Validation sim MAE: 0.1249658, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 041, LR: 0.000700, Loss: 0.2794974, Validation exp MAE: 0.1041297, Validation sim MAE: 0.1262152, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 042, LR: 0.000700, Loss: 0.2790304, Validation exp MAE: 0.0998004, Validation sim MAE: 0.1277096, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 043, LR: 0.000700, Loss: 0.2850172, Validation exp MAE: 0.1024477, Validation sim MAE: 0.1260490, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 044, LR: 0.000700, Loss: 0.2767454, Validation exp MAE: 0.0816588, Validation sim MAE: 0.1339998, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 045, LR: 0.000700, Loss: 0.2790704, Validation exp MAE: 0.0901146, Validation sim MAE: 0.1340300, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 046, LR: 0.000700, Loss: 0.2735670, Validation exp MAE: 0.0948497, Validation sim MAE: 0.1358932, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 047, LR: 0.000700, Loss: 0.2746204, Validation exp MAE: 0.0816151, Validation sim MAE: 0.1319026, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 048, LR: 0.000700, Loss: 0.2781470, Validation exp MAE: 0.0907208, Validation sim MAE: 0.1281495, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 049, LR: 0.000700, Loss: 0.2672222, Validation exp MAE: 0.0867386, Validation sim MAE: 0.1317510, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 050, LR: 0.000700, Loss: 0.2767498, Validation exp MAE: 0.1019476, Validation sim MAE: 0.1331053, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 051, LR: 0.000700, Loss: 0.2774096, Validation exp MAE: 0.0956031, Validation sim MAE: 0.1343142, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 052, LR: 0.000700, Loss: 0.2714596, Validation exp MAE: 0.1110973, Validation sim MAE: 0.1286374, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 053, LR: 0.000700, Loss: 0.2695404, Validation exp MAE: 0.1035774, Validation sim MAE: 0.1270299, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 054, LR: 0.000700, Loss: 0.2725442, Validation exp MAE: 0.0997318, Validation sim MAE: 0.1273963, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 055, LR: 0.000700, Loss: 0.2846387, Validation exp MAE: 0.0897470, Validation sim MAE: 0.1272819, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 056, LR: 0.000700, Loss: 0.2616036, Validation exp MAE: 0.1399168, Validation sim MAE: 0.1260562, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 057, LR: 0.000700, Loss: 0.2695524, Validation exp MAE: 0.1136007, Validation sim MAE: 0.1315913, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 058, LR: 0.000700, Loss: 0.2718794, Validation exp MAE: 0.1024684, Validation sim MAE: 0.1214172, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 059, LR: 0.000490, Loss: 0.2570125, Validation exp MAE: 0.0834174, Validation sim MAE: 0.1294514, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 060, LR: 0.000490, Loss: 0.2575272, Validation exp MAE: 0.1002661, Validation sim MAE: 0.1292439, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 061, LR: 0.000490, Loss: 0.2538297, Validation exp MAE: 0.1036892, Validation sim MAE: 0.1279095, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 062, LR: 0.000490, Loss: 0.2467176, Validation exp MAE: 0.0909141, Validation sim MAE: 0.1260746, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 063, LR: 0.000490, Loss: 0.2651386, Validation exp MAE: 0.0919451, Validation sim MAE: 0.1319084, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 064, LR: 0.000490, Loss: 0.2576376, Validation exp MAE: 0.1041686, Validation sim MAE: 0.1255854, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 065, LR: 0.000490, Loss: 0.2569370, Validation exp MAE: 0.1040731, Validation sim MAE: 0.1246774, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 066, LR: 0.000490, Loss: 0.2580902, Validation exp MAE: 0.0974179, Validation sim MAE: 0.1274739, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 067, LR: 0.000490, Loss: 0.2564593, Validation exp MAE: 0.0923134, Validation sim MAE: 0.1301545, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 068, LR: 0.000490, Loss: 0.2503035, Validation exp MAE: 0.0915544, Validation sim MAE: 0.1285340, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 069, LR: 0.000490, Loss: 0.2486897, Validation exp MAE: 0.1004011, Validation sim MAE: 0.1274015, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 070, LR: 0.000490, Loss: 0.2526861, Validation exp MAE: 0.1061895, Validation sim MAE: 0.1318743, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 071, LR: 0.000490, Loss: 0.2492533, Validation exp MAE: 0.1115816, Validation sim MAE: 0.1293328, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 072, LR: 0.000490, Loss: 0.2565800, Validation exp MAE: 0.0949644, Validation sim MAE: 0.1276782, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 073, LR: 0.000490, Loss: 0.2488834, Validation exp MAE: 0.0861691, Validation sim MAE: 0.1285133, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 074, LR: 0.000490, Loss: 0.2524851, Validation exp MAE: 0.1195085, Validation sim MAE: 0.1247636, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 075, LR: 0.000490, Loss: 0.2691395, Validation exp MAE: 0.0976775, Validation sim MAE: 0.1289039, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 076, LR: 0.000490, Loss: 0.2416658, Validation exp MAE: 0.1158061, Validation sim MAE: 0.1286872, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 077, LR: 0.000490, Loss: 0.2514028, Validation exp MAE: 0.0941719, Validation sim MAE: 0.1343500, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 078, LR: 0.000490, Loss: 0.2469643, Validation exp MAE: 0.1200689, Validation sim MAE: 0.1322145, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 079, LR: 0.000490, Loss: 0.2574402, Validation exp MAE: 0.1072626, Validation sim MAE: 0.1313684, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 080, LR: 0.000343, Loss: 0.2386085, Validation exp MAE: 0.0964768, Validation sim MAE: 0.1337447, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 081, LR: 0.000343, Loss: 0.2412063, Validation exp MAE: 0.1009604, Validation sim MAE: 0.1278475, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 082, LR: 0.000343, Loss: 0.2391631, Validation exp MAE: 0.1023692, Validation sim MAE: 0.1287730, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 083, LR: 0.000343, Loss: 0.2405429, Validation exp MAE: 0.0997062, Validation sim MAE: 0.1312115, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 084, LR: 0.000343, Loss: 0.2391960, Validation exp MAE: 0.1015120, Validation sim MAE: 0.1287041, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 085, LR: 0.000343, Loss: 0.2449168, Validation exp MAE: 0.0986252, Validation sim MAE: 0.1291131, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 086, LR: 0.000343, Loss: 0.2430836, Validation exp MAE: 0.1068822, Validation sim MAE: 0.1263174, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 087, LR: 0.000343, Loss: 0.2335134, Validation exp MAE: 0.0985573, Validation sim MAE: 0.1309216, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 088, LR: 0.000343, Loss: 0.2315630, Validation exp MAE: 0.0947469, Validation sim MAE: 0.1298381, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 089, LR: 0.000343, Loss: 0.2312731, Validation exp MAE: 0.1080402, Validation sim MAE: 0.1263011, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 090, LR: 0.000343, Loss: 0.2303419, Validation exp MAE: 0.1040517, Validation sim MAE: 0.1257139, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 091, LR: 0.000343, Loss: 0.2367129, Validation exp MAE: 0.0964111, Validation sim MAE: 0.1302703, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 092, LR: 0.000343, Loss: 0.2409649, Validation exp MAE: 0.0924742, Validation sim MAE: 0.1339724, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 093, LR: 0.000343, Loss: 0.2358200, Validation exp MAE: 0.1037882, Validation sim MAE: 0.1309672, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 094, LR: 0.000343, Loss: 0.2376599, Validation exp MAE: 0.0987423, Validation sim MAE: 0.1309931, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 095, LR: 0.000343, Loss: 0.2352106, Validation exp MAE: 0.1063199, Validation sim MAE: 0.1319156, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 096, LR: 0.000343, Loss: 0.2371163, Validation exp MAE: 0.0996852, Validation sim MAE: 0.1296025, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 097, LR: 0.000343, Loss: 0.2325720, Validation exp MAE: 0.0986595, Validation sim MAE: 0.1283659, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 098, LR: 0.000343, Loss: 0.2348674, Validation exp MAE: 0.1026102, Validation sim MAE: 0.1309305, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 099, LR: 0.000343, Loss: 0.2339204, Validation exp MAE: 0.1127662, Validation sim MAE: 0.1272161, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 100, LR: 0.000343, Loss: 0.2316172, Validation exp MAE: 0.1011707, Validation sim MAE: 0.1292522, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 101, LR: 0.000240, Loss: 0.2303676, Validation exp MAE: 0.1060450, Validation sim MAE: 0.1285655, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 102, LR: 0.000240, Loss: 0.2274003, Validation exp MAE: 0.1081966, Validation sim MAE: 0.1283794, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 103, LR: 0.000240, Loss: 0.2271597, Validation exp MAE: 0.1172808, Validation sim MAE: 0.1261711, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 104, LR: 0.000240, Loss: 0.2241290, Validation exp MAE: 0.0942063, Validation sim MAE: 0.1291134, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 105, LR: 0.000240, Loss: 0.2272026, Validation exp MAE: 0.1023666, Validation sim MAE: 0.1252258, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 106, LR: 0.000240, Loss: 0.2311450, Validation exp MAE: 0.1137988, Validation sim MAE: 0.1257715, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 107, LR: 0.000240, Loss: 0.2265888, Validation exp MAE: 0.1024335, Validation sim MAE: 0.1266857, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 108, LR: 0.000240, Loss: 0.2272156, Validation exp MAE: 0.1076545, Validation sim MAE: 0.1337377, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 109, LR: 0.000240, Loss: 0.2229345, Validation exp MAE: 0.0986066, Validation sim MAE: 0.1279205, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 110, LR: 0.000240, Loss: 0.2262168, Validation exp MAE: 0.1081511, Validation sim MAE: 0.1262166, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 111, LR: 0.000240, Loss: 0.2309479, Validation exp MAE: 0.1045203, Validation sim MAE: 0.1284022, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 112, LR: 0.000240, Loss: 0.2324993, Validation exp MAE: 0.1183608, Validation sim MAE: 0.1290305, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 113, LR: 0.000240, Loss: 0.2210516, Validation exp MAE: 0.1137586, Validation sim MAE: 0.1318286, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 114, LR: 0.000240, Loss: 0.2286415, Validation exp MAE: 0.1147179, Validation sim MAE: 0.1301395, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 115, LR: 0.000240, Loss: 0.2185029, Validation exp MAE: 0.1181584, Validation sim MAE: 0.1247029, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 116, LR: 0.000240, Loss: 0.2303736, Validation exp MAE: 0.1109142, Validation sim MAE: 0.1263697, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 117, LR: 0.000240, Loss: 0.2256568, Validation exp MAE: 0.1186651, Validation sim MAE: 0.1299488, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 118, LR: 0.000240, Loss: 0.2184038, Validation exp MAE: 0.1107373, Validation sim MAE: 0.1294881, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 119, LR: 0.000240, Loss: 0.2213137, Validation exp MAE: 0.1114412, Validation sim MAE: 0.1263530, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 120, LR: 0.000240, Loss: 0.2217483, Validation exp MAE: 0.1027068, Validation sim MAE: 0.1278970, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 121, LR: 0.000240, Loss: 0.2309746, Validation exp MAE: 0.1131793, Validation sim MAE: 0.1285029, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 122, LR: 0.000168, Loss: 0.2167757, Validation exp MAE: 0.1041008, Validation sim MAE: 0.1270315, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 123, LR: 0.000168, Loss: 0.2167450, Validation exp MAE: 0.1034754, Validation sim MAE: 0.1260565, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 124, LR: 0.000168, Loss: 0.2172238, Validation exp MAE: 0.1079014, Validation sim MAE: 0.1276787, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 125, LR: 0.000168, Loss: 0.2181756, Validation exp MAE: 0.1164059, Validation sim MAE: 0.1240862, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 126, LR: 0.000168, Loss: 0.2188587, Validation exp MAE: 0.1037641, Validation sim MAE: 0.1269999, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 127, LR: 0.000168, Loss: 0.2253183, Validation exp MAE: 0.0986779, Validation sim MAE: 0.1290090, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 128, LR: 0.000168, Loss: 0.2206542, Validation exp MAE: 0.1009622, Validation sim MAE: 0.1278787, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 129, LR: 0.000168, Loss: 0.2169620, Validation exp MAE: 0.0978638, Validation sim MAE: 0.1272136, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 130, LR: 0.000168, Loss: 0.2157376, Validation exp MAE: 0.0990593, Validation sim MAE: 0.1297195, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 131, LR: 0.000168, Loss: 0.2151686, Validation exp MAE: 0.1060926, Validation sim MAE: 0.1291823, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 132, LR: 0.000168, Loss: 0.2146812, Validation exp MAE: 0.1078018, Validation sim MAE: 0.1248667, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 133, LR: 0.000168, Loss: 0.2124639, Validation exp MAE: 0.1046678, Validation sim MAE: 0.1284582, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 134, LR: 0.000168, Loss: 0.2186321, Validation exp MAE: 0.1200126, Validation sim MAE: 0.1289376, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 135, LR: 0.000168, Loss: 0.2116498, Validation exp MAE: 0.1097524, Validation sim MAE: 0.1271504, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 136, LR: 0.000168, Loss: 0.2165718, Validation exp MAE: 0.1050347, Validation sim MAE: 0.1295763, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 137, LR: 0.000168, Loss: 0.2174620, Validation exp MAE: 0.1139272, Validation sim MAE: 0.1273266, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 138, LR: 0.000168, Loss: 0.2130072, Validation exp MAE: 0.1115513, Validation sim MAE: 0.1285435, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 139, LR: 0.000168, Loss: 0.2122483, Validation exp MAE: 0.0988785, Validation sim MAE: 0.1297341, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 140, LR: 0.000168, Loss: 0.2119490, Validation exp MAE: 0.1005130, Validation sim MAE: 0.1276783, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 141, LR: 0.000168, Loss: 0.2159282, Validation exp MAE: 0.1020839, Validation sim MAE: 0.1293081, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 142, LR: 0.000168, Loss: 0.2153908, Validation exp MAE: 0.1081724, Validation sim MAE: 0.1244630, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 143, LR: 0.000118, Loss: 0.2097387, Validation exp MAE: 0.1130651, Validation sim MAE: 0.1279813, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 144, LR: 0.000118, Loss: 0.2093170, Validation exp MAE: 0.1049887, Validation sim MAE: 0.1265782, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 145, LR: 0.000118, Loss: 0.2155242, Validation exp MAE: 0.1054771, Validation sim MAE: 0.1277189, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 146, LR: 0.000118, Loss: 0.2118213, Validation exp MAE: 0.1109797, Validation sim MAE: 0.1270982, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 147, LR: 0.000118, Loss: 0.2083771, Validation exp MAE: 0.1064296, Validation sim MAE: 0.1262796, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 148, LR: 0.000118, Loss: 0.2087086, Validation exp MAE: 0.1059319, Validation sim MAE: 0.1285590, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 149, LR: 0.000118, Loss: 0.2122555, Validation exp MAE: 0.1087195, Validation sim MAE: 0.1273418, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 150, LR: 0.000118, Loss: 0.2181127, Validation exp MAE: 0.1122776, Validation sim MAE: 0.1287158, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 151, LR: 0.000118, Loss: 0.2085037, Validation exp MAE: 0.1071072, Validation sim MAE: 0.1261056, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 152, LR: 0.000118, Loss: 0.2059550, Validation exp MAE: 0.1107631, Validation sim MAE: 0.1293787, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 153, LR: 0.000118, Loss: 0.2106734, Validation exp MAE: 0.1063876, Validation sim MAE: 0.1282631, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 154, LR: 0.000118, Loss: 0.2068352, Validation exp MAE: 0.1040212, Validation sim MAE: 0.1267419, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 155, LR: 0.000118, Loss: 0.2048880, Validation exp MAE: 0.1046259, Validation sim MAE: 0.1280994, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 156, LR: 0.000118, Loss: 0.2092975, Validation exp MAE: 0.1207428, Validation sim MAE: 0.1265057, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 157, LR: 0.000118, Loss: 0.2119779, Validation exp MAE: 0.1126619, Validation sim MAE: 0.1275983, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 158, LR: 0.000118, Loss: 0.2090878, Validation exp MAE: 0.1090649, Validation sim MAE: 0.1250047, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 159, LR: 0.000118, Loss: 0.2053891, Validation exp MAE: 0.1049195, Validation sim MAE: 0.1245771, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 160, LR: 0.000118, Loss: 0.2083954, Validation exp MAE: 0.1076963, Validation sim MAE: 0.1277609, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 161, LR: 0.000118, Loss: 0.2095846, Validation exp MAE: 0.1089924, Validation sim MAE: 0.1273702, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 162, LR: 0.000118, Loss: 0.2075743, Validation exp MAE: 0.1117140, Validation sim MAE: 0.1267416, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 163, LR: 0.000118, Loss: 0.2080345, Validation exp MAE: 0.1025570, Validation sim MAE: 0.1280851, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 164, LR: 0.000082, Loss: 0.2078768, Validation exp MAE: 0.1135504, Validation sim MAE: 0.1284237, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 165, LR: 0.000082, Loss: 0.2082996, Validation exp MAE: 0.1065278, Validation sim MAE: 0.1255299, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 166, LR: 0.000082, Loss: 0.2091099, Validation exp MAE: 0.1032422, Validation sim MAE: 0.1263998, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 167, LR: 0.000082, Loss: 0.2099110, Validation exp MAE: 0.1069931, Validation sim MAE: 0.1273691, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 168, LR: 0.000082, Loss: 0.2082767, Validation exp MAE: 0.1039928, Validation sim MAE: 0.1272455, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 169, LR: 0.000082, Loss: 0.2078815, Validation exp MAE: 0.1048181, Validation sim MAE: 0.1277320, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 170, LR: 0.000082, Loss: 0.2096921, Validation exp MAE: 0.1052326, Validation sim MAE: 0.1256558, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 171, LR: 0.000082, Loss: 0.2092916, Validation exp MAE: 0.1064304, Validation sim MAE: 0.1272209, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 172, LR: 0.000082, Loss: 0.2112029, Validation exp MAE: 0.1062825, Validation sim MAE: 0.1265221, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 173, LR: 0.000082, Loss: 0.2047702, Validation exp MAE: 0.1135328, Validation sim MAE: 0.1266222, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 174, LR: 0.000082, Loss: 0.2075551, Validation exp MAE: 0.1097093, Validation sim MAE: 0.1271926, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 175, LR: 0.000082, Loss: 0.2104130, Validation exp MAE: 0.1061968, Validation sim MAE: 0.1259796, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 176, LR: 0.000082, Loss: 0.2078665, Validation exp MAE: 0.1121001, Validation sim MAE: 0.1275107, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 177, LR: 0.000082, Loss: 0.2057661, Validation exp MAE: 0.1080116, Validation sim MAE: 0.1280922, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 178, LR: 0.000082, Loss: 0.2098187, Validation exp MAE: 0.1047841, Validation sim MAE: 0.1293075, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 179, LR: 0.000082, Loss: 0.2033553, Validation exp MAE: 0.1095139, Validation sim MAE: 0.1292216, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 180, LR: 0.000082, Loss: 0.2081784, Validation exp MAE: 0.1025588, Validation sim MAE: 0.1274031, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 181, LR: 0.000082, Loss: 0.2035020, Validation exp MAE: 0.1128896, Validation sim MAE: 0.1286992, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 182, LR: 0.000082, Loss: 0.2063688, Validation exp MAE: 0.1080282, Validation sim MAE: 0.1277065, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 183, LR: 0.000082, Loss: 0.2009240, Validation exp MAE: 0.1049314, Validation sim MAE: 0.1271990, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 184, LR: 0.000082, Loss: 0.2064473, Validation exp MAE: 0.1108786, Validation sim MAE: 0.1283870, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 185, LR: 0.000058, Loss: 0.1996267, Validation exp MAE: 0.1117041, Validation sim MAE: 0.1271350, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 186, LR: 0.000058, Loss: 0.2033004, Validation exp MAE: 0.1065130, Validation sim MAE: 0.1271097, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 187, LR: 0.000058, Loss: 0.2058551, Validation exp MAE: 0.1049562, Validation sim MAE: 0.1285289, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 188, LR: 0.000058, Loss: 0.2043116, Validation exp MAE: 0.1113621, Validation sim MAE: 0.1288133, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 189, LR: 0.000058, Loss: 0.1998187, Validation exp MAE: 0.1121635, Validation sim MAE: 0.1276991, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 190, LR: 0.000058, Loss: 0.2071426, Validation exp MAE: 0.1143158, Validation sim MAE: 0.1280454, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 191, LR: 0.000058, Loss: 0.2057870, Validation exp MAE: 0.1104960, Validation sim MAE: 0.1270911, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 192, LR: 0.000058, Loss: 0.2077483, Validation exp MAE: 0.1099412, Validation sim MAE: 0.1268022, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 193, LR: 0.000058, Loss: 0.2022617, Validation exp MAE: 0.1141023, Validation sim MAE: 0.1263785, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 194, LR: 0.000058, Loss: 0.1994772, Validation exp MAE: 0.1141199, Validation sim MAE: 0.1273328, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 195, LR: 0.000058, Loss: 0.2078581, Validation exp MAE: 0.1122510, Validation sim MAE: 0.1280742, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 196, LR: 0.000058, Loss: 0.2015395, Validation exp MAE: 0.1085090, Validation sim MAE: 0.1257389, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 197, LR: 0.000058, Loss: 0.2038589, Validation exp MAE: 0.1107233, Validation sim MAE: 0.1284379, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 198, LR: 0.000058, Loss: 0.2015415, Validation exp MAE: 0.1082368, Validation sim MAE: 0.1278548, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925 Epoch: 199, LR: 0.000058, Loss: 0.1995386, Validation exp MAE: 0.1075619, Validation sim MAE: 0.1274587, Best Validation MAE: 0.0705967, Test exp MAE: 0.1097879, Test sim MAE: 0.0913925
代码
文本
可以发现，多任务模型的Loss和MAE都比单任务模型的小，这说明多任务模型从噪声当中学到了隐藏的信息。下面简单看一看预测结果
代码
文本
[3]
import pandas as pd
import matplotlib.pyplot as plt
# 读取csv文件
data_exp = pd.read_csv('/data/polymernet/exp_test_results.csv')
# 提取需要绘制的两列数据
x_e = data.iloc[:, 1]
y_e = data.iloc[:, 2]
# 创建一个图形和坐标轴
fig, ax = plt.subplots(figsize=(6, 6))
# 绘制散点图
plt.scatter(x_e, y_e)
plt.xlabel('A')
plt.ylabel('B')
plt.title('Scatter Plot of A vs B')
# 计算对角线起点和终点
min_value = min(min(x_e), min(y_e))
max_value = max(max(x_e), max(y_e))
# 绘制对角线
ax.plot([min_value, max_value], [min_value, max_value], color='black', linestyle='--', linewidth=2, label='Diagonal Line')
# 显示图形
plt.show()
代码
文本
感兴趣的话可以看看预测结果数据集
代码
文本
[23]
# 排序并查看预测数据集
external_data = pd.read_csv("/data/polymernet/sim_pred_results.csv", index_col=1)
external_data = external_data.reset_index(drop=True)
external_data.sort_values(by="pred", ascending=True, inplace=True)
external_data.head(10)
| id | pred | target | |
|---|---|---|---|
| 1989 | 9-0-413660560-0 | -4.625328 | CC(C)(C)S(=O)(=O)CC(COC(=O)[Au])O[Cu] |
| 3700 | 9-0-941872544-0 | -4.612829 | CC(C)C(C)(O[Cu])C(C)(OC(=O)[Au])C(C)C |
| 3463 | 9-0-482407471-0 | -4.606273 | CCS(=O)(=O)C(CO[Cu])C(C)OC(=O)[Au] |
| 1979 | 9-0-941865580-0 | -4.604899 | CC(C)(C)C(OC(=O)[Au])C(C)(C)O[Cu] |
| 750 | 9-0-1119506835-0 | -4.596963 | CC(C)S(=O)(=O)NCC(COC(=O)[Au])O[Cu] |
| 541 | 9-0-1119506607-0 | -4.595191 | CC(C)S(=O)(=O)NC(CO[Cu])COC(=O)[Au] |
| 2001 | 9-0-941865231-0 | -4.593069 | CC(C)C(O[Cu])C(C)(C)COC(=O)[Au] |
| 5814 | 9-0-822835793-0 | -4.592744 | CC(C)(C)C(C#CC(OC(=O)[Au])C(C)(C)C)O[Cu] |
| 3704 | 9-0-1132276250-0 | -4.592080 | CC(C)(O[Cu])C(C)(C)OC(=O)[Au] |
| 1504 | 9-0-1132291526-0 | -4.586338 | CC(C)C(OC(=O)[Au])C(C)O[Cu] |
代码
文本
[26]
# Load external/unlabeled data set
external_data = pd.read_csv("/data/polymernet/exp_pred_results.csv", index_col=1)
external_data = external_data.reset_index(drop=True)
external_data.sort_values(by="pred", ascending=True, inplace=True)
external_data.head(-10)
# NBVAL_CHECK_OUTPUT
| id | pred | target | |
|---|---|---|---|
| 1989 | 9-0-413660560-0 | -5.182648 | CC(C)(C)S(=O)(=O)CC(COC(=O)[Au])O[Cu] |
| 3463 | 9-0-482407471-0 | -5.177245 | CCS(=O)(=O)C(CO[Cu])C(C)OC(=O)[Au] |
| 4886 | 9-0-413647697-0 | -5.169393 | CC(O[Cu])C(=O)N(C)C(C)C(C)(C)OC(=O)[Au] |
| 4994 | 9-0-413630426-0 | -5.148970 | CC(OC(=O)[Au])C(=O)N(C)C(C)CO[Cu] |
| 3761 | 9-0-413633122-0 | -5.142471 | CC(O[Cu])C(=O)N(C)CC(C)(C)OC(=O)[Au] |
| ... | ... | ... | ... |
| 2389 | 9-0-538231396-0 | -3.943169 | O=C([Au])OCCSCCOCCO[Cu] |
| 3442 | 9-0-33212808-0 | -3.942557 | C=CCN(CCO[Cu])CCOCCOC(=O)[Au] |
| 428 | 9-0-478224263-0 | -3.928301 | CN(C)N(CCO[Cu])CCOC(=O)[Au] |
| 1891 | 9-0-428454789-0 | -3.900920 | O=C([Au])OCCNCCNCCO[Cu] |
| 857 | 9-0-413611903-0 | -3.885492 | CN(CCCO[Cu])CCOCCOC(=O)[Au] |
6237 rows × 3 columns
代码
文本
模型实现了对6237个分子的搜索空间的预测，相对于直接计算，计算时间减少了22.3倍。
代码
文本
这不是终点
Uni-Mol出来以后，GNN还有优势吗？
现在主流的发展方向是什么呢？
To be continue……
代码
文本
已赞1
本文被以下合集收录
ai4s
1111
更新于 2023-09-20
7 篇0 人关注
推荐阅读
公开
DF-ODENet | 燃烧反应源项神经网络模型李晗
发布于 2024-01-02
2 赞11 转存文件
公开
定量构效关系(QSAR)模型从0到1 & Uni-Mol入门实践（分类任务）
zhengh@dp.tech
发布于 2023-06-12
31 赞85 转存文件6 评论