

透视细胞命运的迷雾:TrajectoryNet 绘制人类胚胎干细胞发育轨迹
“It is not our abilities that show what we truly are, it is our choices. ——Albus Percival Wulfric Brian Dumbledore”
在每个命运的抉择点,细胞是如何选择的呢?
在这个教程中,你将学习到 TrajectoryNet[1] 的基本原理,以及如何使用 TrajectoryNet 和 PHATE [2] (Potential of Heat-diffusion for Affinity-based Transition Embedding)的数据集来分析一个包含31,000个细胞的、为期27天的人类胚胎干细胞上(EB)分化时间序列。
本教程参考了 TrajectoryNet 原作者的讲解PPT,实际演示部分搬运自原作者提供的Notebook.
我们将遵循以下步骤:
0. TrajectoryNet的基本介绍
1. 加载10X数据
2. 预处理:过滤、标准化和转换
3. 使用PHATE嵌入数据
4. 使用TrajectoryNet建模细胞动态转换
参考文献:
- Tong, A., Huang, J., Wolf, G., van Dijk, D. & Krishnaswamy, S. TrajectoryNet: A Dynamic Optimal Transport Network for Modeling Cellular Dynamics. in Proceedings of the 37th International Conference on Machine Learning (2020). url
- Moon, K. R. et al. Visualizing structure and transitions in high-dimensional biological data. Nature Biotechnoly 37, 1482–1492 (2019). url
📖 上手指南
本文档可在 Bohrium Notebook 上直接运行。如果是在案例广场,你可以直接运行;如果是保存到自己的空间中,你可以点击界面上方按钮 开始连接,选择基础镜像 ubuntu:22.04-py3.10-pytorch2.0 及 c24_m92_2 * NVIDIA GPU B 节点配置,稍等片刻即可运行。
0.0 TrajectoryNet模型介绍
TrajectoryNet 是一个用于模拟细胞动态的动态最优传输网络,而 PHATE 是一种基于亲和性转换嵌入的热扩散潜力方法,用于可视化高维生物数据中的结构和转换。这些工具在理解细胞如何随时间发展和分化方面非常有用,特别是在大规模细胞数据集中。
下面,我们简单介绍 TrajectoryNet。
- 问题定义
在针对单细胞组学数据的分析中,一个重要的分析是找到各个细胞在分化或是演化上的先后顺序,也就是得到单细胞的拟时序(pseudotime)。拟时序(pseudotime)是一种生物学概念,用于研究单细胞生物在发育过程中所经历的时间轨迹。单细胞拟时序分析可以揭示细胞在不同阶段的功能特性和转录水平,为我们提供了深入了解细胞发育过程的机会。通过对单细胞数据的拟时序分析,我们可以描绘出细胞类型和状态的演变过程,从而为研究生物学过程,如组织发育、疾病进程和信号转导等提供重要信息。此外,拟时序分析还可以帮助我们发现潜在的细胞命运决定因素,为进一步实验研究和临床应用奠定基础。
普遍使用的单细胞的拟时序方法,通常是用已知的或是一些差异分析方法获得的标志物来判断拟时序。这样的方法存在bias,文献中也有失败案例,此外,这样的方法没有考虑真正的实验时间。
而 TrajectoryNet 利用了真实实验时间,对于每个时间点的单细胞数据,将其视为一个分布的采样结果,从而建模分布之间演化的模型。因此,TrajectoryNet 将单细胞的拟时序问题定义为一个 unbalanced dynamic transport 问题。
换而言之,TrajectoryNet 希望定义一种高效的、有意义的转换方程,建立两个分布之间的转换关系。
- 连续标准化流
通常,针对分布之间的变换,我们会想到连续标准化流(Continuous Normalizing Flows,CNF)。CNF 通过连续变换将任意复杂分布映射到简单的基础分布(如高斯分布),从而实现有效的采样和概率密度计算。通常应用于图像生成、异常检测、强化学习、自然语言处理等领域。而将最优传输作为优化目标(视为一种正则化,如下右图),将诱导出一种分布之间的最优传输路径。我们可以直观地定义最优传输中的能量最小作为正则化,而定义了能量最小作为正则化的CNF,其实就是本文介绍的 TrajectoryNet 的主体。
- 最优传输
从另一个角度来看,如果我们希望两个分布之间转换能实现能量最小化,那么我们可以要求 Wasserstein distance 最小。也就是实现最优传输(如图),其中两个分布 μ 和 ν 分别是连个边界条件。
这样两个边界条件的方程并不容易求解,如果我们将边界条件 ν 改为软性的限制加入loss中,则得到正则化的CNF(如图)。
- TrajectoryNet
再将 RNAseq 数据中的生物学先验纳入考虑,包括密度、速度和生长,TrajectoryNet 就构建了一种使用动态最优传输,从而能够模拟系统中实体的连续动态和非线性路径的方法。
最终,TrajectoryNet 将损失函数定义为动态最优传输与生物学先验的结合。
其中,密度主要考虑到 KNN 密度估计,也就是细胞必须通过状态空间中允许的部分进行转换,比如,图中展示了一个不合理的路径,并提供了密度惩罚的公式。
速度主要考虑到RNA变化的方向,也就是细胞状态空间的速度向量,这主要参考了La Manno等人在2018年的Velocyto和Volker等人的ScVelo。
生长主要考虑到在状态转换之间,可能存在一些细胞状态的消失,所以 TrajectoryNet 允许不平衡的运输,从而允许细胞“死亡”而不是将它们移动到不合理的位置。
0.1 本教程数据集说明:时间序列的人类胚胎干细胞
研究人员使用特定的培养条件和技术来培养和分化人类胚胎干细胞(hESCs)。这些细胞在特定条件下可以分化为不同类型的细胞,模拟早期人类胚胎的发育过程。研究团队收集了不同时间点的细胞样本,以研究细胞分化的动态过程。此外,他们通过单细胞测序技术(10x Genomics)来分析这些细胞,这是一种强大的技术,可以提供对每个单独细胞的深入了解。
人类胚胎体分化的时间过程如下:低通道H1人胚胎干细胞(hESCs)在用Matrigel涂层的培养皿中维持,使用DMEM/F12-N2B27培养基和FGF2补充。为了形成胚胎体,细胞用Dispase处理,分离成小团块,并在加有20%胎牛血清的培养基中进行非粘附培养,这种血清经过筛选,适用于胚胎体分化。在为期27天的分化时间序列中,每隔3天收集样本。同时还包括一个未分化的hESC样本(图S7D)。通过qPCR验证了这些胚胎体培养中主要胚层标记物的诱导(数据未显示)。对于单细胞分析,胚胎体培养物被分离,通过流式细胞术(FACS)排序以去除双细胞和死细胞,并在10x基因组仪器上处理,生成cDNA文库,随后进行测序。小规模测序确定我们已经成功收集了大约31,000个细胞的数据,这些细胞在整个时间序列中均匀分布。
0.2 相关包的安装
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple Requirement already satisfied: TrajectoryNet in /opt/mamba/lib/python3.10/site-packages (0.2.4) Requirement already satisfied: POT>=0.7.0 in /opt/mamba/lib/python3.10/site-packages (from TrajectoryNet) (0.9.1) Requirement already satisfied: matplotlib>=3.2.1 in /opt/mamba/lib/python3.10/site-packages (from TrajectoryNet) (3.8.1) Requirement already satisfied: scanpy in /opt/mamba/lib/python3.10/site-packages (from TrajectoryNet) (1.9.6) Requirement already satisfied: numpy>=1.18.4 in /opt/mamba/lib/python3.10/site-packages (from TrajectoryNet) (1.24.2) Requirement already satisfied: scikit-learn>=0.23.1 in /opt/mamba/lib/python3.10/site-packages (from TrajectoryNet) (1.3.2) Requirement already satisfied: torch>=1.5.0 in /opt/mamba/lib/python3.10/site-packages (from TrajectoryNet) (2.0.0+cu118) Requirement already satisfied: torchdiffeq==0.0.1 in /opt/mamba/lib/python3.10/site-packages (from TrajectoryNet) (0.0.1) Requirement already satisfied: scipy>=1.4.1 in /opt/mamba/lib/python3.10/site-packages (from TrajectoryNet) (1.10.1) Collecting argparse Using cached https://pypi.tuna.tsinghua.edu.cn/packages/f2/94/3af39d34be01a24a6e65433d19e107099374224905f1e0cc6bbe1fd22a2f/argparse-1.4.0-py2.py3-none-any.whl (23 kB) Requirement already satisfied: kiwisolver>=1.3.1 in /opt/mamba/lib/python3.10/site-packages (from matplotlib>=3.2.1->TrajectoryNet) (1.4.5) Requirement already satisfied: fonttools>=4.22.0 in /opt/mamba/lib/python3.10/site-packages (from matplotlib>=3.2.1->TrajectoryNet) (4.44.0) Requirement already satisfied: python-dateutil>=2.7 in /opt/mamba/lib/python3.10/site-packages (from matplotlib>=3.2.1->TrajectoryNet) (2.8.2) Requirement already satisfied: pyparsing>=2.3.1 in /opt/mamba/lib/python3.10/site-packages (from matplotlib>=3.2.1->TrajectoryNet) (3.1.1) Requirement already satisfied: pillow>=8 in /opt/mamba/lib/python3.10/site-packages (from matplotlib>=3.2.1->TrajectoryNet) (10.1.0) Requirement already satisfied: cycler>=0.10 in /opt/mamba/lib/python3.10/site-packages (from matplotlib>=3.2.1->TrajectoryNet) (0.12.1) Requirement already satisfied: packaging>=20.0 in /opt/mamba/lib/python3.10/site-packages (from matplotlib>=3.2.1->TrajectoryNet) (23.0) Requirement already satisfied: contourpy>=1.0.1 in /opt/mamba/lib/python3.10/site-packages (from matplotlib>=3.2.1->TrajectoryNet) (1.2.0) Requirement already satisfied: threadpoolctl>=2.0.0 in /opt/mamba/lib/python3.10/site-packages (from scikit-learn>=0.23.1->TrajectoryNet) (3.2.0) Requirement already satisfied: joblib>=1.1.1 in /opt/mamba/lib/python3.10/site-packages (from scikit-learn>=0.23.1->TrajectoryNet) (1.3.2) Requirement already satisfied: typing-extensions in /opt/mamba/lib/python3.10/site-packages (from torch>=1.5.0->TrajectoryNet) (4.5.0) Requirement already satisfied: jinja2 in /opt/mamba/lib/python3.10/site-packages (from torch>=1.5.0->TrajectoryNet) (3.1.2) Requirement already satisfied: networkx in /opt/mamba/lib/python3.10/site-packages (from torch>=1.5.0->TrajectoryNet) (3.0) Requirement already satisfied: filelock in /opt/mamba/lib/python3.10/site-packages (from torch>=1.5.0->TrajectoryNet) (3.10.0) Requirement already satisfied: triton==2.0.0 in /opt/mamba/lib/python3.10/site-packages (from torch>=1.5.0->TrajectoryNet) (2.0.0) Requirement already satisfied: sympy in /opt/mamba/lib/python3.10/site-packages (from torch>=1.5.0->TrajectoryNet) (1.11.1) Requirement already satisfied: cmake in /opt/mamba/lib/python3.10/site-packages (from triton==2.0.0->torch>=1.5.0->TrajectoryNet) (3.26.0) Requirement already satisfied: lit in /opt/mamba/lib/python3.10/site-packages (from triton==2.0.0->torch>=1.5.0->TrajectoryNet) (15.0.7) Requirement already satisfied: patsy in /opt/mamba/lib/python3.10/site-packages (from scanpy->TrajectoryNet) (0.5.3) Requirement already satisfied: umap-learn>=0.3.10 in /opt/mamba/lib/python3.10/site-packages (from scanpy->TrajectoryNet) (0.5.4) Requirement already satisfied: numba>=0.41.0 in /opt/mamba/lib/python3.10/site-packages (from scanpy->TrajectoryNet) (0.58.1) Requirement already satisfied: session-info in /opt/mamba/lib/python3.10/site-packages (from scanpy->TrajectoryNet) (1.0.0) Requirement already satisfied: statsmodels>=0.10.0rc2 in /opt/mamba/lib/python3.10/site-packages (from scanpy->TrajectoryNet) (0.14.0) Requirement already satisfied: seaborn!=0.13.0 in /opt/mamba/lib/python3.10/site-packages (from scanpy->TrajectoryNet) (0.12.2) Requirement already satisfied: pandas!=2.1.2,>=1.1.1 in /opt/mamba/lib/python3.10/site-packages (from scanpy->TrajectoryNet) (1.5.3) Requirement already satisfied: h5py>=3 in /opt/mamba/lib/python3.10/site-packages (from scanpy->TrajectoryNet) (3.8.0) Requirement already satisfied: tqdm in /opt/mamba/lib/python3.10/site-packages (from scanpy->TrajectoryNet) (4.64.1) Requirement already satisfied: anndata>=0.7.4 in /opt/mamba/lib/python3.10/site-packages (from scanpy->TrajectoryNet) (0.10.3) Requirement already satisfied: natsort in /opt/mamba/lib/python3.10/site-packages (from scanpy->TrajectoryNet) (8.4.0) Requirement already satisfied: exceptiongroup in /opt/mamba/lib/python3.10/site-packages (from anndata>=0.7.4->scanpy->TrajectoryNet) (1.1.3) Requirement already satisfied: array-api-compat in /opt/mamba/lib/python3.10/site-packages (from anndata>=0.7.4->scanpy->TrajectoryNet) (1.4) Requirement already satisfied: llvmlite<0.42,>=0.41.0dev0 in /opt/mamba/lib/python3.10/site-packages (from numba>=0.41.0->scanpy->TrajectoryNet) (0.41.1) Requirement already satisfied: pytz>=2020.1 in /opt/mamba/lib/python3.10/site-packages (from pandas!=2.1.2,>=1.1.1->scanpy->TrajectoryNet) (2022.7.1) Requirement already satisfied: six>=1.5 in /opt/mamba/lib/python3.10/site-packages (from python-dateutil>=2.7->matplotlib>=3.2.1->TrajectoryNet) (1.16.0) Requirement already satisfied: tbb>=2019.0 in /opt/mamba/lib/python3.10/site-packages (from umap-learn>=0.3.10->scanpy->TrajectoryNet) (2021.10.0) Requirement already satisfied: pynndescent>=0.5 in /opt/mamba/lib/python3.10/site-packages (from umap-learn>=0.3.10->scanpy->TrajectoryNet) (0.5.10) Requirement already satisfied: MarkupSafe>=2.0 in /opt/mamba/lib/python3.10/site-packages (from jinja2->torch>=1.5.0->TrajectoryNet) (2.1.2) Requirement already satisfied: stdlib-list in /opt/mamba/lib/python3.10/site-packages (from session-info->scanpy->TrajectoryNet) (0.9.0) Requirement already satisfied: mpmath>=0.19 in /opt/mamba/lib/python3.10/site-packages (from sympy->torch>=1.5.0->TrajectoryNet) (1.3.0) Installing collected packages: argparse Successfully installed argparse-1.4.0 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple Requirement already satisfied: python-magic in /opt/mamba/lib/python3.10/site-packages (0.4.27) WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1466 kB] Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1274 kB] Fetched 3078 kB in 8s (403 kB/s) Reading package lists... Done Reading package lists... Done Building dependency tree... Done Reading state information... Done libmagic1 is already the newest version (1:5.41-3ubuntu0.1). 0 upgraded, 0 newly installed, 0 to remove and 119 not upgraded. Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple Requirement already satisfied: phate in /opt/mamba/lib/python3.10/site-packages (1.0.11) Requirement already satisfied: tasklogger>=1.2 in /opt/mamba/lib/python3.10/site-packages (from phate) (1.2.0) Requirement already satisfied: numpy>=1.16.0 in /opt/mamba/lib/python3.10/site-packages (from phate) (1.24.2) Requirement already satisfied: scprep>=0.11.1 in /opt/mamba/lib/python3.10/site-packages (from phate) (1.2.3) Requirement already satisfied: s-gd2>=1.8.1 in /opt/mamba/lib/python3.10/site-packages (from phate) (1.8.1) Requirement already satisfied: scikit-learn>=0.20.0 in /opt/mamba/lib/python3.10/site-packages (from phate) (1.3.2) Requirement already satisfied: Deprecated in /opt/mamba/lib/python3.10/site-packages (from phate) (1.2.14) Requirement already satisfied: graphtools>=1.5.3 in /opt/mamba/lib/python3.10/site-packages (from phate) (1.5.3) Requirement already satisfied: future in /opt/mamba/lib/python3.10/site-packages (from phate) (0.18.3) Requirement already satisfied: scipy>=1.1.0 in /opt/mamba/lib/python3.10/site-packages (from phate) (1.10.1) Requirement already satisfied: matplotlib>=3.0 in /opt/mamba/lib/python3.10/site-packages (from phate) (3.8.1) Requirement already satisfied: pygsp>=0.5.1 in /opt/mamba/lib/python3.10/site-packages (from graphtools>=1.5.3->phate) (0.5.1) Requirement already satisfied: contourpy>=1.0.1 in /opt/mamba/lib/python3.10/site-packages (from matplotlib>=3.0->phate) (1.2.0) Requirement already satisfied: kiwisolver>=1.3.1 in /opt/mamba/lib/python3.10/site-packages (from matplotlib>=3.0->phate) (1.4.5) Requirement already satisfied: cycler>=0.10 in /opt/mamba/lib/python3.10/site-packages (from matplotlib>=3.0->phate) (0.12.1) Requirement already satisfied: pyparsing>=2.3.1 in /opt/mamba/lib/python3.10/site-packages (from matplotlib>=3.0->phate) (3.1.1) Requirement already satisfied: pillow>=8 in /opt/mamba/lib/python3.10/site-packages (from matplotlib>=3.0->phate) (10.1.0) Requirement already satisfied: python-dateutil>=2.7 in /opt/mamba/lib/python3.10/site-packages (from matplotlib>=3.0->phate) (2.8.2) Requirement already satisfied: packaging>=20.0 in /opt/mamba/lib/python3.10/site-packages (from matplotlib>=3.0->phate) (23.0) Requirement already satisfied: fonttools>=4.22.0 in /opt/mamba/lib/python3.10/site-packages (from matplotlib>=3.0->phate) (4.44.0) Requirement already satisfied: joblib>=1.1.1 in /opt/mamba/lib/python3.10/site-packages (from scikit-learn>=0.20.0->phate) (1.3.2) Requirement already satisfied: threadpoolctl>=2.0.0 in /opt/mamba/lib/python3.10/site-packages (from scikit-learn>=0.20.0->phate) (3.2.0) Requirement already satisfied: decorator>=4.3.0 in /opt/mamba/lib/python3.10/site-packages (from scprep>=0.11.1->phate) (5.1.1) Requirement already satisfied: pandas<2.1,>=0.25 in /opt/mamba/lib/python3.10/site-packages (from scprep>=0.11.1->phate) (1.5.3) Requirement already satisfied: wrapt<2,>=1.10 in /opt/mamba/lib/python3.10/site-packages (from Deprecated->phate) (1.16.0) Requirement already satisfied: pytz>=2020.1 in /opt/mamba/lib/python3.10/site-packages (from pandas<2.1,>=0.25->scprep>=0.11.1->phate) (2022.7.1) Requirement already satisfied: six>=1.5 in /opt/mamba/lib/python3.10/site-packages (from python-dateutil>=2.7->matplotlib>=3.0->phate) (1.16.0) WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple Requirement already satisfied: scprep in /opt/mamba/lib/python3.10/site-packages (1.2.3) Requirement already satisfied: pandas<2.1,>=0.25 in /opt/mamba/lib/python3.10/site-packages (from scprep) (1.5.3) Requirement already satisfied: scipy>=0.18.1 in /opt/mamba/lib/python3.10/site-packages (from scprep) (1.10.1) Requirement already satisfied: scikit-learn>=0.19.1 in /opt/mamba/lib/python3.10/site-packages (from scprep) (1.3.2) Requirement already satisfied: numpy>=1.12.0 in /opt/mamba/lib/python3.10/site-packages (from scprep) (1.24.2) Requirement already satisfied: packaging in /opt/mamba/lib/python3.10/site-packages (from scprep) (23.0) Requirement already satisfied: decorator>=4.3.0 in /opt/mamba/lib/python3.10/site-packages (from scprep) (5.1.1) Requirement already satisfied: python-dateutil>=2.8.1 in /opt/mamba/lib/python3.10/site-packages (from pandas<2.1,>=0.25->scprep) (2.8.2) Requirement already satisfied: pytz>=2020.1 in /opt/mamba/lib/python3.10/site-packages (from pandas<2.1,>=0.25->scprep) (2022.7.1) Requirement already satisfied: joblib>=1.1.1 in /opt/mamba/lib/python3.10/site-packages (from scikit-learn>=0.19.1->scprep) (1.3.2) Requirement already satisfied: threadpoolctl>=2.0.0 in /opt/mamba/lib/python3.10/site-packages (from scikit-learn>=0.19.1->scprep) (3.2.0) Requirement already satisfied: six>=1.5 in /opt/mamba/lib/python3.10/site-packages (from python-dateutil>=2.8.1->pandas<2.1,>=0.25->scprep) (1.16.0) WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple Requirement already satisfied: magic-impute in /root/.local/lib/python3.10/site-packages (3.0.0) Requirement already satisfied: tasklogger>=1.0.0 in /opt/mamba/lib/python3.10/site-packages (from magic-impute) (1.2.0) Requirement already satisfied: pandas>=0.25 in /opt/mamba/lib/python3.10/site-packages (from magic-impute) (1.5.3) Requirement already satisfied: graphtools>=1.4.0 in /opt/mamba/lib/python3.10/site-packages (from magic-impute) (1.5.3) Requirement already satisfied: future in /opt/mamba/lib/python3.10/site-packages (from magic-impute) (0.18.3) Requirement already satisfied: scipy>=1.1.0 in /opt/mamba/lib/python3.10/site-packages (from magic-impute) (1.10.1) Requirement already satisfied: numpy>=1.14.0 in /opt/mamba/lib/python3.10/site-packages (from magic-impute) (1.24.2) Requirement already satisfied: scprep>=1.0 in /opt/mamba/lib/python3.10/site-packages (from magic-impute) (1.2.3) Requirement already satisfied: scikit-learn>=0.19.1 in /opt/mamba/lib/python3.10/site-packages (from magic-impute) (1.3.2) Requirement already satisfied: matplotlib in /opt/mamba/lib/python3.10/site-packages (from magic-impute) (3.8.1) Requirement already satisfied: pygsp>=0.5.1 in /opt/mamba/lib/python3.10/site-packages (from graphtools>=1.4.0->magic-impute) (0.5.1) Requirement already satisfied: Deprecated in /opt/mamba/lib/python3.10/site-packages (from graphtools>=1.4.0->magic-impute) (1.2.14) Requirement already satisfied: python-dateutil>=2.8.1 in /opt/mamba/lib/python3.10/site-packages (from pandas>=0.25->magic-impute) (2.8.2) Requirement already satisfied: pytz>=2020.1 in /opt/mamba/lib/python3.10/site-packages (from pandas>=0.25->magic-impute) (2022.7.1) Requirement already satisfied: threadpoolctl>=2.0.0 in /opt/mamba/lib/python3.10/site-packages (from scikit-learn>=0.19.1->magic-impute) (3.2.0) Requirement already satisfied: joblib>=1.1.1 in /opt/mamba/lib/python3.10/site-packages (from scikit-learn>=0.19.1->magic-impute) (1.3.2) Requirement already satisfied: packaging in /opt/mamba/lib/python3.10/site-packages (from scprep>=1.0->magic-impute) (23.0) Requirement already satisfied: decorator>=4.3.0 in /opt/mamba/lib/python3.10/site-packages (from scprep>=1.0->magic-impute) (5.1.1) Requirement already satisfied: fonttools>=4.22.0 in /opt/mamba/lib/python3.10/site-packages (from matplotlib->magic-impute) (4.44.0) Requirement already satisfied: kiwisolver>=1.3.1 in /opt/mamba/lib/python3.10/site-packages (from matplotlib->magic-impute) (1.4.5) Requirement already satisfied: pillow>=8 in /opt/mamba/lib/python3.10/site-packages (from matplotlib->magic-impute) (10.1.0) Requirement already satisfied: cycler>=0.10 in /opt/mamba/lib/python3.10/site-packages (from matplotlib->magic-impute) (0.12.1) Requirement already satisfied: pyparsing>=2.3.1 in /opt/mamba/lib/python3.10/site-packages (from matplotlib->magic-impute) (3.1.1) Requirement already satisfied: contourpy>=1.0.1 in /opt/mamba/lib/python3.10/site-packages (from matplotlib->magic-impute) (1.2.0) Requirement already satisfied: six>=1.5 in /opt/mamba/lib/python3.10/site-packages (from python-dateutil>=2.8.1->pandas>=0.25->magic-impute) (1.16.0) Requirement already satisfied: wrapt<2,>=1.10 in /opt/mamba/lib/python3.10/site-packages (from Deprecated->graphtools>=1.4.0->magic-impute) (1.16.0) WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
--2023-11-10 18:19:06-- https://codeload.github.com/KrishnaswamyLab/TrajectoryNet/zip/refs/heads/master Resolving ga.dp.tech (ga.dp.tech)... 10.255.254.7, 10.255.254.18, 10.255.254.37 Connecting to ga.dp.tech (ga.dp.tech)|10.255.254.7|:8118... connected. Proxy request sent, awaiting response... 200 OK Length: unspecified [application/zip] Saving to: ‘master.1’ master.1 [ <=> ] 35.99M 3.96MB/s in 11s 2023-11-10 18:19:18 (3.32 MB/s) - ‘master.1’ saved [37736431] Archive: master 162e6c77728135f27ad04f1c83d78a319e79dff4 replace TrajectoryNet-master/.flake8? [y]es, [n]o, [A]ll, [N]one, [r]ename:
接下来,进入实际演示部分。
从 Mendeley Datasets 导入数据
这个胚胎体数据集来源于公开数据Mendeley数据集https://data.mendeley.com/datasets/v6n743h5ng/,本教程已经为您下载并以挂载数据集的方式挂载好了,在路径 /bohr/bio-notebook-trajectorynet-uo5o/v1 中。
这些文件是单细胞RNA测序数据的标准格式,通常由CellRanger软件生成。了解更多关于CellRanger如何生成这些文件的信息,请查看 Gene-Barcode Matrices Documentation。
以下是目录结构:
download_path
└── scRNAseq
├── scRNAseq.zip
├── T0_1A
│ ├── barcodes.tsv
│ ├── genes.tsv
│ └── matrix.mtx
├── T2_3B
│ ├── barcodes.tsv
│ ├── genes.tsv
│ └── matrix.mtx
├── T4_5C
│ ├── barcodes.tsv
│ ├── genes.tsv
│ └── matrix.mtx
├── T6_7D
│ ├── barcodes.tsv
│ ├── genes.tsv
│ └── matrix.mtx
└── T8_9E
├── barcodes.tsv
├── genes.tsv
└── matrix.mtx
/bohr/bio-notebook-trajectorynet-uo5o/v1
使用scprep导入数据到Pandas DataFrame
接下来,使用一个名为scprep的工具包来加载和操作单细胞数据。函数load_10X可以自动将10X单细胞RNA测序数据集(以及其他数据集)加载到Pandas DataFrame中。
让我们加载数据,并创建一个单一矩阵,我们可以用它来进行预处理、可视化和分析。
1. 导入数据
2. 使用 scprep.io.load_10X 导入每个样本的三个矩阵到一个DataFrame中(读取数据会花费约5分钟的时间)
注意:默认情况下,scprep.io.load_10X 使用 Pandas 的 SparseDataFrame (see Pandas docs) 来最大化内存效率。然而,这会比加载密集矩阵慢一些。要加载密集矩阵,请将 sparse=False 参数传递给 load_10X。我们使用 gene_labels = 'both' 以便在保留基因ID的唯一性的同时看到基因符号。
RP11-34P13.3 (ENSG00000243485) | FAM138A (ENSG00000237613) | OR4F5 (ENSG00000186092) | RP11-34P13.7 (ENSG00000238009) | RP11-34P13.8 (ENSG00000239945) | RP11-34P13.14 (ENSG00000239906) | RP11-34P13.9 (ENSG00000241599) | FO538757.3 (ENSG00000279928) | FO538757.2 (ENSG00000279457) | AP006222.2 (ENSG00000228463) | ... | AC007325.2 (ENSG00000277196) | BX072566.1 (ENSG00000277630) | AL354822.1 (ENSG00000278384) | AC023491.2 (ENSG00000278633) | AC004556.1 (ENSG00000276345) | AC233755.2 (ENSG00000277856) | AC233755.1 (ENSG00000275063) | AC240274.1 (ENSG00000271254) | AC213203.1 (ENSG00000277475) | FAM231B (ENSG00000268674) | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | |||||||||||||||||||||
AAACATACCAGAGG-1 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 | ... | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
AAACATTGAAAGCA-1 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | ... | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
AAACATTGAAGTGA-1 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | ... | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
AAACATTGGAGGTG-1 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | ... | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
AAACATTGGTTTCT-1 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | ... | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
5 rows × 33694 columns
3. Library 的大小筛选
我们筛选掉具有非常大或非常小库大小的细胞。对于这个数据集,库大小与样本有一定的相关性,因此我们基于每个样本进行筛选。在这种情况下,我们排除了每个样本中前20%和后20%的细胞。采用更简单、不太保守的筛选方法也能获得类似的结果。

4.合并所有的数据集,并建立一个表示每个样本的时间序列向量
A1BG (ENSG00000121410) | A1BG-AS1 (ENSG00000268895) | A1CF (ENSG00000148584) | A2M (ENSG00000175899) | A2M-AS1 (ENSG00000245105) | A2ML1 (ENSG00000166535) | A2ML1-AS1 (ENSG00000256661) | A2ML1-AS2 (ENSG00000256904) | A3GALT2 (ENSG00000184389) | A4GALT (ENSG00000128274) | ... | ZXDC (ENSG00000070476) | ZYG11A (ENSG00000203995) | ZYG11B (ENSG00000162378) | ZYX (ENSG00000159840) | ZZEF1 (ENSG00000074755) | ZZZ3 (ENSG00000036549) | bP-21264C1.2 (ENSG00000278932) | bP-2171C21.3 (ENSG00000279501) | bP-2189O9.3 (ENSG00000279579) | hsa-mir-1253 (ENSG00000272920) | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
AAACATTGAAAGCA-1_Day 00-03 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | ... | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
AAACCGTGCAGAAA-1_Day 00-03 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | ... | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
AAACCGTGGAAGGC-1_Day 00-03 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | ... | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
AAACGCACCGGTAT-1_Day 00-03 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | ... | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
AAACGCACCTATTC-1_Day 00-03 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | ... | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
5 rows × 33694 columns
过滤 I:库大小筛选
我们之前已经进行了库大小筛选,因为库大小与我们的样本之间存在强相关性。然而,如果你想要进行更简单的筛选,你可以在这里运行以下操作:
EBT_counts, sample_labels = scprep.filter.library_size_filter(EBT_counts, sample_labels, cutoff=2000)
过滤 II:将检测率较小的基因过滤
我们过滤了在少于等于10个细胞中表达的基因。
归一化
为了校正不同库大小之间的差异,我们将每个细胞的表达量除以其库大小,然后通过中位数库大小进行重新缩放。
在Python中,可以使用预处理方法 library_size_normalize()
来执行这个操作。
过滤 III:去除死细胞
死细胞很可能具有比活细胞更高的线粒体RNA表达水平。因此,我们通过消除平均具有最高线粒体RNA表达水平的细胞来去除疑似死细胞。
首先,让我们看一下线粒体基因的分布情况。

我们可以看到,在0.9分位数以上的地方,线粒体RNA的表达出现了急剧增加。因此,我们过滤掉这一部分细胞。
转换
在单细胞RNA测序分析中,数据通常会进行 log 转换。通常需要添加一些小值以避免取 log(0)。本教程中,我们完全避免了这个问题,而是采用了平方根变换。平方根函数具有与 log 函数类似的形式,而且在0处更加稳定。
3.1 实例化PHATE估计器
PHATE模型的API与Scikit Learn的API相似。首先,您需要使用适用于给定数据集的参数来实例化一个PHATE估计器对象。然后,您可以使用 fit
和 fit_transform
函数来生成嵌入。要获取更多信息,请查看 PHATE的readthedocs页面。
现在我们只使用默认参数,但以下参数可以进行调整(请阅读文档 phate.readthedocs.io 以了解更多信息):
knn
:最近邻居的数量(默认值:5)。如果您的PHATE嵌入看起来非常不连通,可以增加此值(例如,设为20)。如果您的数据集非常大(例如,>100k个细胞),还应考虑增加knn
。decay
:Alpha衰减(默认值:15)。减小decay
会增加图中的连接性,增加decay
会减少连接性。这很少需要调整。将其设置为None
以获得k最近邻居核。t
:操作符幂次数(默认值:'auto')。这等于对数据进行的平滑处理次数。默认情况下自动选择,但如果您的嵌入缺乏结构,可以增加它,或者如果结构看起来太紧凑,可以减小它。gamma
:信息距离常数(默认值:1)。gamma=1
给出PHATE对数潜力,但其他信息距离也可能很有趣。如果大部分点似乎集中在绘图的一个部分,可以尝试gamma=0
。
由于我们正在寻找细节的结构,而且我们预计一些轨迹可能会稀疏,因此我们可能希望将 knn 从默认值5减小,并将 t 从自动值21(如上面的输出中所示)降低。对于单细胞RNA测序,如果您正在寻找微妙的结构,可以将 knn 设为3或4,如果您有数十万个细胞,也可以将其设为30或40。我们还会将 alpha 降低到15,部分抵消由于减小 knn 而导致的连接性减少。
Calculating PHATE... Running PHATE on 16821 observations and 17845 variables. Calculating graph and diffusion operator... Calculating PCA... Calculated PCA in 43.01 seconds. Calculating KNN search... Calculated KNN search in 9.08 seconds. Calculating affinities... Calculated affinities in 0.90 seconds. Calculated graph and diffusion operator in 55.26 seconds. Calculating landmark operator... Calculating SVD... Calculated SVD in 2.47 seconds. Calculating KMeans... Calculated KMeans in 2.98 seconds. Calculated landmark operator in 6.91 seconds. Calculating optimal t... Automatically selected t = 19 Calculated optimal t in 1.31 seconds. Calculating diffusion potential... Calculated diffusion potential in 0.20 seconds. Calculating metric MDS... Calculated metric MDS in 5.78 seconds. Calculated PHATE in 69.49 seconds.

4. 使用TrajectoryNet建模细胞动态转换
本节内容将它展示了如何在基因空间中使用TrajectoryNet计算细胞转换轨迹。
本节计算的backward_trajectories.npy
是一个形状为 [timepoints, cells, pcs]
的数组,与论文中使用的相同,使用以下命令生成:
python main.py --save [SAVE_DIR] --dataset EB-PCA --top_k_reg 0.1 --training_noise 0.0 --max_dim 5
这基本上是默认设置,带有少量密度正则化(称为 top_k_reg)。这将计算模型并保存一些检查点,最终的权重存储在 checkpt.pt
中。在论文中进行实验的主要参数是 --top_k_reg
(密度正则化)和 --vecint
(速度正则化)。
然后我们运行:
python eval.py --save [SAVE_DIR] --dataset EB-PCA --top_k_reg 0.1 --training_noise 0.0 --max_dim 5
这将从保存的模型中在 [SAVE_DIR]
目录中创建 backwards_trajectories.npy
。
完整的训练运行输出保存在 results/fig8_results/
中。
这些轨迹将在最终时间点中的点向后积分到起始时间点,共计100个均匀分布的时间点。也就是说,基于一些已有的隐空间中的嵌入(也就是降维后的数据),如PCA的结果,TrajectoryNet构建了这个嵌入数据的基因空间中的动态轨迹。
4.1 运行 TrajectoryNet,进行轨迹推断
📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢
注意!4.1 小节运行时间可能超过 1h ,运行结果已经存在了,因此你可以选择不运行本节!
📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢
mkdir: cannot create directory ‘notebook_results’: File exists /data/bioinfo/TrajectoryNet/TrajectoryNet-master/TrajectoryNet/main.py """ main.py Learns ODE from scrna data """ import os import matplotlib import matplotlib.pyplot as plt import numpy as np import time import torch import torch.nn.functional as F import torch.optim as optim from TrajectoryNet.lib.growth_net import GrowthNet from TrajectoryNet.lib import utils from TrajectoryNet.lib.visualize_flow import visualize_transform from TrajectoryNet.lib.viz_scrna import ( save_trajectory, trajectory_to_video, save_vectors, ) from TrajectoryNet.lib.viz_scrna import save_trajectory_density # from train_misc import standard_normal_logprob from TrajectoryNet.train_misc import ( set_cnf_options, count_nfe, count_parameters, count_total_time, add_spectral_norm, spectral_norm_power_iteration, create_regularization_fns, get_regularization, append_regularization_to_log, build_model_tabular, ) from TrajectoryNet import dataset from TrajectoryNet.parse import parser matplotlib.use("Agg") def get_transforms(device, args, model, integration_times): """ Given a list of integration points, returns a function giving integration times """ def sample_fn(z, logpz=None): int_list = [ torch.tensor([it - args.time_scale, it]).type(torch.float32).to(device) for it in integration_times ] if logpz is not None: # TODO this works right? for it in int_list: z, logpz = model(z, logpz, integration_times=it, reverse=True) return z, logpz else: for it in int_list: z = model(z, integration_times=it, reverse=True) return z def density_fn(x, logpx=None): int_list = [ torch.tensor([it - args.time_scale, it]).type(torch.float32).to(device) for it in integration_times[::-1] ] if logpx is not None: for it in int_list: x, logpx = model(x, logpx, integration_times=it, reverse=False) return x, logpx else: for it in int_list: x = model(x, integration_times=it, reverse=False) return x return sample_fn, density_fn def compute_loss(device, args, model, growth_model, logger, full_data): """ Compute loss by integrating backwards from the last time step At each time step integrate back one time step, and concatenate that to samples of the empirical distribution at that previous timestep repeating over and over to calculate the likelihood of samples in later timepoints iteratively, making sure that the ODE is evaluated at every time step to calculate those later points. The growth model is a single model of time independent cell growth / death rate defined as a variation from uniform. """ # Backward pass accumulating losses, previous state and deltas deltas = [] zs = [] z = None interp_loss = 0.0 for i, (itp, tp) in enumerate(zip(args.int_tps[::-1], args.timepoints[::-1])): # tp counts down from last integration_times = torch.tensor([itp - args.time_scale, itp]) integration_times = integration_times.type(torch.float32).to(device) # integration_times.requires_grad = True # load data and add noise idx = args.data.sample_index(args.batch_size, tp) x = args.data.get_data()[idx] if args.training_noise > 0.0: x += np.random.randn(*x.shape) * args.training_noise x = torch.from_numpy(x).type(torch.float32).to(device) if i > 0: x = torch.cat((z, x)) zs.append(z) zero = torch.zeros(x.shape[0], 1).to(x) # transform to previous timepoint z, delta_logp = model(x, zero, integration_times=integration_times) deltas.append(delta_logp) # Straightline regularization # Integrate to random point at time t and assert close to (1 - t) * end + t * start if args.interp_reg: t = np.random.rand() int_t = torch.tensor([itp - t * args.time_scale, itp]) int_t = int_t.type(torch.float32).to(device) int_x = model(x, integration_times=int_t) int_x = int_x.detach() actual_int_x = x * (1 - t) + z * t interp_loss += F.mse_loss(int_x, actual_int_x) if args.interp_reg: print("interp_loss", interp_loss) logpz = args.data.base_density()(z) # build growth rates if args.use_growth: growthrates = [torch.ones_like(logpz)] for z_state, tp in zip(zs[::-1], args.timepoints[:-1]): # Full state includes time parameter to growth_model time_state = tp * torch.ones(z_state.shape[0], 1).to(z_state) full_state = torch.cat([z_state, time_state], 1) growthrates.append(growth_model(full_state)) # Accumulate losses losses = [] logps = [logpz] for i, delta_logp in enumerate(deltas[::-1]): logpx = logps[-1] - delta_logp if args.use_growth: logpx += torch.log(torch.clamp(growthrates[i], 1e-4, 1e4)) logps.append(logpx[: -args.batch_size]) losses.append(-torch.mean(logpx[-args.batch_size :])) losses = torch.stack(losses) weights = torch.ones_like(losses).to(logpx) if args.leaveout_timepoint >= 0: weights[args.leaveout_timepoint] = 0 losses = torch.mean(losses * weights) # Direction regularization if args.vecint: similarity_loss = 0 for i, (itp, tp) in enumerate(zip(args.int_tps, args.timepoints)): itp = torch.tensor(itp).type(torch.float32).to(device) idx = args.data.sample_index(args.batch_size, tp) x = args.data.get_data()[idx] v = args.data.get_velocity()[idx] x = torch.from_numpy(x).type(torch.float32).to(device) v = torch.from_numpy(v).type(torch.float32).to(device) x += torch.randn_like(x) * 0.1 # Only penalizes at the time / place of visible samples direction = -model.chain[0].odefunc.odefunc.diffeq(itp, x) if args.use_magnitude: similarity_loss += torch.mean(F.mse_loss(direction, v)) else: similarity_loss -= torch.mean(F.cosine_similarity(direction, v)) logger.info(similarity_loss) losses += similarity_loss * args.vecint # Density regularization if args.top_k_reg > 0: density_loss = 0 tp_z_map = dict(zip(args.timepoints[:-1], zs[::-1])) if args.leaveout_timepoint not in tp_z_map: idx = args.data.sample_index(args.batch_size, tp) x = args.data.get_data()[idx] if args.training_noise > 0.0: x += np.random.randn(*x.shape) * args.training_noise x = torch.from_numpy(x).type(torch.float32).to(device) t = np.random.rand() int_t = torch.tensor([itp - t * args.time_scale, itp]) int_t = int_t.type(torch.float32).to(device) int_x = model(x, integration_times=int_t) samples_05 = int_x else: # If we are leaving out a timepoint the regularize there samples_05 = tp_z_map[args.leaveout_timepoint] # Calculate distance to 5 closest neighbors # WARNING: This currently fails in the backward pass with cuda on pytorch < 1.4.0 # works on CPU. Fixed in pytorch 1.5.0 # RuntimeError: CUDA error: invalid configuration argument # The workaround is to run on cpu on pytorch <= 1.4.0 or upgrade cdist = torch.cdist(samples_05, full_data) values, _ = torch.topk(cdist, 5, dim=1, largest=False, sorted=False) # Hinge loss hinge_value = 0.1 values -= hinge_value values[values < 0] = 0 density_loss = torch.mean(values) print("Density Loss", density_loss.item()) losses += density_loss * args.top_k_reg losses += interp_loss return losses def train( device, args, model, growth_model, regularization_coeffs, regularization_fns, logger ): optimizer = optim.Adam( model.parameters(), lr=args.lr, weight_decay=args.weight_decay ) time_meter = utils.RunningAverageMeter(0.93) loss_meter = utils.RunningAverageMeter(0.93) nfef_meter = utils.RunningAverageMeter(0.93) nfeb_meter = utils.RunningAverageMeter(0.93) tt_meter = utils.RunningAverageMeter(0.93) full_data = ( torch.from_numpy( args.data.get_data()[args.data.get_times() != args.leaveout_timepoint] ) .type(torch.float32) .to(device) ) best_loss = float("inf") if args.use_growth: growth_model.eval() end = time.time() for itr in range(1, args.niters + 1): model.train() optimizer.zero_grad() # Train if args.spectral_norm: spectral_norm_power_iteration(model, 1) loss = compute_loss(device, args, model, growth_model, logger, full_data) loss_meter.update(loss.item()) if len(regularization_coeffs) > 0: # Only regularize on the last timepoint reg_states = get_regularization(model, regularization_coeffs) reg_loss = sum( reg_state * coeff for reg_state, coeff in zip(reg_states, regularization_coeffs) if coeff != 0 ) loss = loss + reg_loss total_time = count_total_time(model) nfe_forward = count_nfe(model) loss.backward() optimizer.step() # Eval nfe_total = count_nfe(model) nfe_backward = nfe_total - nfe_forward nfef_meter.update(nfe_forward) nfeb_meter.update(nfe_backward) time_meter.update(time.time() - end) tt_meter.update(total_time) log_message = ( "Iter {:04d} | Time {:.4f}({:.4f}) | Loss {:.6f}({:.6f}) |" " NFE Forward {:.0f}({:.1f})" " | NFE Backward {:.0f}({:.1f})".format( itr, time_meter.val, time_meter.avg, loss_meter.val, loss_meter.avg, nfef_meter.val, nfef_meter.avg, nfeb_meter.val, nfeb_meter.avg, ) ) if len(regularization_coeffs) > 0: log_message = append_regularization_to_log( log_message, regularization_fns, reg_states ) logger.info(log_message) if itr % args.val_freq == 0 or itr == args.niters: with torch.no_grad(): train_eval( device, args, model, growth_model, itr, best_loss, logger, full_data ) if itr % args.viz_freq == 0: if args.data.get_shape()[0] > 2: logger.warning("Skipping vis as data dimension is >2") else: with torch.no_grad(): visualize(device, args, model, itr) if itr % args.save_freq == 0: chkpt = { "state_dict": model.state_dict(), } if args.use_growth: chkpt.update({"growth_state_dict": growth_model.state_dict()}) utils.save_checkpoint( chkpt, args.save, epoch=itr, ) end = time.time() logger.info("Training has finished.") def train_eval(device, args, model, growth_model, itr, best_loss, logger, full_data): model.eval() test_loss = compute_loss(device, args, model, growth_model, logger, full_data) test_nfe = count_nfe(model) log_message = "[TEST] Iter {:04d} | Test Loss {:.6f} |" " NFE {:.0f}".format( itr, test_loss, test_nfe ) logger.info(log_message) utils.makedirs(args.save) with open(os.path.join(args.save, "train_eval.csv"), "a") as f: import csv writer = csv.writer(f) writer.writerow((itr, test_loss)) if test_loss.item() < best_loss: best_loss = test_loss.item() chkpt = { "state_dict": model.state_dict(), } if args.use_growth: chkpt.update({"growth_state_dict": growth_model.state_dict()}) torch.save( chkpt, os.path.join(args.save, "checkpt.pth"), ) def visualize(device, args, model, itr): model.eval() for i, tp in enumerate(args.timepoints): idx = args.data.sample_index(args.viz_batch_size, tp) p_samples = args.data.get_data()[idx] sample_fn, density_fn = get_transforms( device, args, model, args.int_tps[: i + 1] ) plt.figure(figsize=(9, 3)) visualize_transform( p_samples, args.data.base_sample(), args.data.base_density(), transform=sample_fn, inverse_transform=density_fn, samples=True, npts=100, device=device, ) fig_filename = os.path.join( args.save, "figs", "{:04d}_{:01d}.jpg".format(itr, i) ) utils.makedirs(os.path.dirname(fig_filename)) plt.savefig(fig_filename) plt.close() def plot_output(device, args, model): save_traj_dir = os.path.join(args.save, "trajectory") # logger.info('Plotting trajectory to {}'.format(save_traj_dir)) data_samples = args.data.get_data()[args.data.sample_index(2000, 0)] np.random.seed(42) start_points = args.data.base_sample()(1000, 2) # idx = args.data.sample_index(50, 0) # start_points = args.data.get_data()[idx] # start_points = torch.from_numpy(start_points).type(torch.float32) save_vectors( args.data.base_density(), model, start_points, args.data.get_data(), args.data.get_times(), args.save, skip_first=(not args.data.known_base_density()), device=device, end_times=args.int_tps, ntimes=100, ) save_trajectory( args.data.base_density(), args.data.base_sample(), model, data_samples, save_traj_dir, device=device, end_times=args.int_tps, ntimes=25, ) density_dir = os.path.join(args.save, "density2") save_trajectory_density( args.data.base_density(), model, data_samples, density_dir, device=device, end_times=args.int_tps, ntimes=25, memory=0.1, ) if args.save_movie: trajectory_to_video(save_traj_dir) trajectory_to_video(density_dir) def main(args): # logger print(args.no_display_loss) utils.makedirs(args.save) logger = utils.get_logger( logpath=os.path.join(args.save, "logs"), filepath=os.path.abspath(__file__), displaying=~args.no_display_loss, ) if args.layer_type == "blend": logger.info("!! Setting time_scale from None to 1.0 for Blend layers.") args.time_scale = 1.0 logger.info(args) device = torch.device( "cuda:" + str(args.gpu) if torch.cuda.is_available() else "cpu" ) if args.use_cpu: device = torch.device("cpu") args.data = dataset.SCData.factory(args.dataset, args) args.timepoints = args.data.get_unique_times() # Use maximum timepoint to establish integration_times # as some timepoints may be left out for validation etc. args.int_tps = (np.arange(max(args.timepoints) + 1) + 1.0) * args.time_scale regularization_fns, regularization_coeffs = create_regularization_fns(args) model = build_model_tabular(args, args.data.get_shape()[0], regularization_fns).to( device ) growth_model = None if args.use_growth: if args.leaveout_timepoint == -1: growth_model_path = "../data/externel/growth_model_v2.ckpt" elif args.leaveout_timepoint in [1, 2, 3]: assert args.max_dim == 5 growth_model_path = "../data/growth/model_%d" % args.leaveout_timepoint else: print("WARNING: Cannot use growth with this timepoint") growth_model = torch.load(growth_model_path, map_location=device) if args.spectral_norm: add_spectral_norm(model) set_cnf_options(args, model) if args.test: state_dict = torch.load(args.save + "/checkpt.pth", map_location=device) model.load_state_dict(state_dict["state_dict"]) # if "growth_state_dict" not in state_dict: # print("error growth model note in save") # growth_model = None # else: # checkpt = torch.load(args.save + "/checkpt.pth", map_location=device) # growth_model.load_state_dict(checkpt["growth_state_dict"]) # TODO can we load the arguments from the save? # eval_utils.generate_samples( # device, args, model, growth_model, timepoint=args.leaveout_timepoint # ) # with torch.no_grad(): # evaluate(device, args, model, growth_model) # exit() else: logger.info(model) n_param = count_parameters(model) logger.info("Number of trainable parameters: {}".format(n_param)) train( device, args, model, growth_model, regularization_coeffs, regularization_fns, logger, ) if args.data.data.shape[1] == 2: plot_output(device, args, model) if __name__ == "__main__": args = parser.parse_args() main(args) Namespace(test=False, dataset='EB-PCA', use_growth=False, use_density=False, leaveout_timepoint=-1, layer_type='concatsquash', max_dim=5, dims='64-64-64', num_blocks=1, time_scale=0.5, train_T=True, divergence_fn='brute_force', nonlinearity='tanh', stochastic=False, alpha=0.0, solver='dopri5', atol=1e-05, rtol=1e-05, step_size=None, test_solver=None, test_atol=None, test_rtol=None, residual=False, rademacher=False, spectral_norm=False, batch_norm=False, bn_lag=0, niters=10000, num_workers=8, batch_size=1000, test_batch_size=1000, viz_batch_size=2000, lr=0.001, weight_decay=1e-05, l1int=None, l2int=None, sl2int=None, dl2int=None, dtl2int=None, JFrobint=None, JdiagFrobint=None, JoffdiagFrobint=None, vecint=None, use_magnitude=False, interp_reg=None, save='../results/fig8_results/', save_freq=1000, viz_freq=100, viz_freq_growth=100, val_freq=100, log_freq=10, gpu=0, use_cpu=False, no_display_loss=True, top_k_reg=0.1, training_noise=0.0, embedding_name='pca', whiten=False) SequentialFlow( (chain): ModuleList( (0): CNF( (odefunc): RegularizedODEfunc( (odefunc): ODEfunc( (diffeq): ODEnet( (layers): ModuleList( (0): ConcatSquashLinear( (_layer): Linear(in_features=5, out_features=64, bias=True) (_hyper_bias): Linear(in_features=1, out_features=64, bias=False) (_hyper_gate): Linear(in_features=1, out_features=64, bias=True) ) (1-2): 2 x ConcatSquashLinear( (_layer): Linear(in_features=64, out_features=64, bias=True) (_hyper_bias): Linear(in_features=1, out_features=64, bias=False) (_hyper_gate): Linear(in_features=1, out_features=64, bias=True) ) (3): ConcatSquashLinear( (_layer): Linear(in_features=64, out_features=5, bias=True) (_hyper_bias): Linear(in_features=1, out_features=5, bias=False) (_hyper_gate): Linear(in_features=1, out_features=5, bias=True) ) ) (activation_fns): ModuleList( (0-2): 3 x Tanh() ) ) ) ) ) ) ) Number of trainable parameters: 9621 Iter 0001 | Time 2.5157(2.5157) | Loss 10.659542(10.659542) | NFE Forward 14(14.0) | NFE Backward 138(138.0) Iter 0002 | Time 1.7738(2.4638) | Loss 10.414104(10.642361) | NFE Forward 14(14.0) | NFE Backward 138(138.0) Iter 0003 | Time 1.7819(2.4161) | Loss 10.223143(10.613016) | NFE Forward 14(14.0) | NFE Backward 138(138.0) Iter 0004 | Time 1.7488(2.3694) | Loss 10.002523(10.570282) | NFE Forward 14(14.0) | NFE Backward 138(138.0) Iter 0005 | Time 1.7800(2.3281) | Loss 9.876067(10.521687) | NFE Forward 14(14.0) | NFE Backward 138(138.0) Iter 0006 | Time 1.7440(2.2872) | Loss 9.736125(10.466697) | NFE Forward 14(14.0) | NFE Backward 138(138.0) Iter 0007 | Time 1.7443(2.2492) | Loss 9.479450(10.397590) | NFE Forward 14(14.0) | NFE Backward 138(138.0) Iter 0008 | Time 1.7452(2.2139) | Loss 9.319873(10.322150) | NFE Forward 14(14.0) | NFE Backward 138(138.0) Iter 0009 | Time 1.7454(2.1811) | Loss 9.203041(10.243812) | NFE Forward 14(14.0) | NFE Backward 138(138.0) Iter 0010 | Time 1.7762(2.1528) | Loss 9.020422(10.158175) | NFE Forward 14(14.0) | NFE Backward 138(138.0) Iter 0011 | Time 1.7768(2.1265) | Loss 8.881227(10.068789) | NFE Forward 14(14.0) | NFE Backward 138(138.0) Iter 0012 | Time 1.7458(2.0998) | Loss 8.806569(9.980433) | NFE Forward 14(14.0) | NFE Backward 138(138.0) Iter 0013 | Time 1.7445(2.0749) | Loss 8.714346(9.891807) | NFE Forward 14(14.0) | NFE Backward 138(138.0) Iter 0014 | Time 1.7518(2.0523) | Loss 8.550598(9.797922) | NFE Forward 14(14.0) | NFE Backward 138(138.0) Iter 0015 | Time 1.7746(2.0329) | Loss 8.479905(9.705661) | NFE Forward 14(14.0) | NFE Backward 138(138.0) Iter 0016 | Time 1.7731(2.0147) | Loss 8.392025(9.613707) | NFE Forward 14(14.0) | NFE Backward 138(138.0) Iter 0017 | Time 1.7746(1.9979) | Loss 8.293889(9.521319) | NFE Forward 14(14.0) | NFE Backward 138(138.0) Iter 0018 | Time 1.7722(1.9821) | Loss 8.228414(9.430816) | NFE Forward 14(14.0) | NFE Backward 138(138.0) Iter 0019 | Time 1.7472(1.9657) | Loss 8.193113(9.344177) | NFE Forward 14(14.0) | NFE Backward 138(138.0) Iter 0020 | Time 1.7801(1.9527) | Loss 8.169664(9.261961) | NFE Forward 14(14.0) | NFE Backward 138(138.0) Iter 0021 | Time 1.7516(1.9386) | Loss 7.971884(9.171656) | NFE Forward 14(14.0) | NFE Backward 138(138.0) Iter 0022 | Time 1.7812(1.9276) | Loss 7.917172(9.083842) | NFE Forward 14(14.0) | NFE Backward 138(138.0) Iter 0023 | Time 1.7546(1.9155) | Loss 7.947235(9.004279) | NFE Forward 14(14.0) | NFE Backward 138(138.0) Iter 0024 | Time 1.7852(1.9063) | Loss 7.843535(8.923027) | NFE Forward 14(14.0) | NFE Backward 138(138.0) Iter 0025 | Time 1.7558(1.8958) | Loss 7.834055(8.846799) | NFE Forward 14(14.0) | NFE Backward 138(138.0) Iter 0026 | Time 1.8131(1.8900) | Loss 7.764037(8.771006) | NFE Forward 14(14.0) | NFE Backward 144(138.4) Iter 0027 | Time 1.8149(1.8848) | Loss 7.755235(8.699902) | NFE Forward 14(14.0) | NFE Backward 144(138.8) Iter 0028 | Time 1.8182(1.8801) | Loss 7.645949(8.626125) | NFE Forward 14(14.0) | NFE Backward 144(139.2) Iter 0029 | Time 1.8317(1.8767) | Loss 7.637852(8.556946) | NFE Forward 14(14.0) | NFE Backward 144(139.5) Iter 0030 | Time 1.8405(1.8742) | Loss 7.651845(8.493589) | NFE Forward 14(14.0) | NFE Backward 144(139.8) Iter 0031 | Time 1.8105(1.8697) | Loss 7.644297(8.434138) | NFE Forward 14(14.0) | NFE Backward 144(140.1) Iter 0032 | Time 1.8151(1.8659) | Loss 7.538164(8.371420) | NFE Forward 14(14.0) | NFE Backward 144(140.4) Iter 0033 | Time 1.6958(1.8540) | Loss 7.546865(8.313701) | NFE Forward 14(14.0) | NFE Backward 132(139.8) Iter 0034 | Time 1.6848(1.8421) | Loss 7.485286(8.255712) | NFE Forward 14(14.0) | NFE Backward 132(139.3) Iter 0035 | Time 1.6602(1.8294) | Loss 7.427310(8.197724) | NFE Forward 14(14.0) | NFE Backward 132(138.7) Iter 0036 | Time 1.7769(1.8257) | Loss 7.418245(8.143161) | NFE Forward 14(14.0) | NFE Backward 144(139.1) Iter 0037 | Time 1.5710(1.8079) | Loss 7.405585(8.091530) | NFE Forward 14(14.0) | NFE Backward 120(137.8) Iter 0038 | Time 1.6831(1.7992) | Loss 7.388819(8.042341) | NFE Forward 14(14.0) | NFE Backward 132(137.4) Iter 0039 | Time 1.7688(1.7970) | Loss 7.337600(7.993009) | NFE Forward 14(14.0) | NFE Backward 138(137.4) Iter 0040 | Time 1.6253(1.7850) | Loss 7.359450(7.948660) | NFE Forward 14(14.0) | NFE Backward 126(136.6) Iter 0041 | Time 1.5989(1.7720) | Loss 7.334590(7.905675) | NFE Forward 14(14.0) | NFE Backward 126(135.9) Iter 0042 | Time 1.6254(1.7617) | Loss 7.355865(7.867188) | NFE Forward 14(14.0) | NFE Backward 126(135.2) Iter 0043 | Time 1.6261(1.7522) | Loss 7.295446(7.827166) | NFE Forward 14(14.0) | NFE Backward 126(134.5) Iter 0044 | Time 1.4825(1.7333) | Loss 7.303817(7.790532) | NFE Forward 14(14.0) | NFE Backward 114(133.1) Iter 0045 | Time 1.6520(1.7277) | Loss 7.322201(7.757749) | NFE Forward 14(14.0) | NFE Backward 126(132.6) Iter 0046 | Time 1.5095(1.7124) | Loss 7.202320(7.718869) | NFE Forward 14(14.0) | NFE Backward 114(131.3) Iter 0047 | Time 1.5357(1.7000) | Loss 7.252171(7.686200) | NFE Forward 14(14.0) | NFE Backward 114(130.1) Iter 0048 | Time 1.4807(1.6847) | Loss 7.246337(7.655409) | NFE Forward 14(14.0) | NFE Backward 114(129.0) Iter 0049 | Time 1.6015(1.6788) | Loss 7.211323(7.624323) | NFE Forward 14(14.0) | NFE Backward 126(128.8) Iter 0050 | Time 1.6546(1.6771) | Loss 7.217341(7.595835) | NFE Forward 14(14.0) | NFE Backward 126(128.6) Iter 0051 | Time 1.5112(1.6655) | Loss 7.198859(7.568046) | NFE Forward 14(14.0) | NFE Backward 114(127.5) Iter 0052 | Time 1.5392(1.6567) | Loss 7.126257(7.537121) | NFE Forward 14(14.0) | NFE Backward 114(126.6) Iter 0053 | Time 1.5390(1.6485) | Loss 7.172043(7.511566) | NFE Forward 14(14.0) | NFE Backward 114(125.7) Iter 0054 | Time 1.3957(1.6308) | Loss 7.161476(7.487059) | NFE Forward 14(14.0) | NFE Backward 102(124.1) Iter 0055 | Time 1.5380(1.6243) | Loss 7.147567(7.463295) | NFE Forward 14(14.0) | NFE Backward 114(123.4) Iter 0056 | Time 1.5612(1.6199) | Loss 7.127177(7.439767) | NFE Forward 14(14.0) | NFE Backward 114(122.7) Iter 0057 | Time 1.3913(1.6039) | Loss 7.104535(7.416300) | NFE Forward 14(14.0) | NFE Backward 102(121.2) Iter 0058 | Time 1.4186(1.5909) | Loss 7.100783(7.394214) | NFE Forward 14(14.0) | NFE Backward 102(119.9) Iter 0059 | Time 1.5641(1.5890) | Loss 7.092834(7.373117) | NFE Forward 14(14.0) | NFE Backward 114(119.5) Iter 0060 | Time 1.5917(1.5892) | Loss 7.097281(7.353809) | NFE Forward 14(14.0) | NFE Backward 114(119.1) Iter 0061 | Time 1.5664(1.5876) | Loss 7.080300(7.334663) | NFE Forward 14(14.0) | NFE Backward 114(118.7) Iter 0062 | Time 1.4728(1.5796) | Loss 7.106249(7.318674) | NFE Forward 14(14.0) | NFE Backward 102(117.6) Iter 0063 | Time 1.4443(1.5701) | Loss 7.080820(7.302025) | NFE Forward 14(14.0) | NFE Backward 102(116.5) Iter 0064 | Time 1.4704(1.5631) | Loss 7.086061(7.286907) | NFE Forward 14(14.0) | NFE Backward 102(115.5) Iter 0065 | Time 1.4465(1.5550) | Loss 7.067868(7.271574) | NFE Forward 14(14.0) | NFE Backward 102(114.5) Iter 0066 | Time 1.4762(1.5494) | Loss 7.039664(7.255341) | NFE Forward 14(14.0) | NFE Backward 102(113.7) Iter 0067 | Time 1.5869(1.5521) | Loss 7.052956(7.241174) | NFE Forward 14(14.0) | NFE Backward 114(113.7) Iter 0068 | Time 1.5327(1.5507) | Loss 7.045708(7.227491) | NFE Forward 14(14.0) | NFE Backward 114(113.7) Iter 0069 | Time 1.5615(1.5515) | Loss 7.059662(7.215743) | NFE Forward 14(14.0) | NFE Backward 114(113.7) Iter 0070 | Time 1.4462(1.5441) | Loss 7.026733(7.202512) | NFE Forward 14(14.0) | NFE Backward 102(112.9) Iter 0071 | Time 1.5962(1.5477) | Loss 7.005133(7.188696) | NFE Forward 14(14.0) | NFE Backward 114(113.0) Iter 0072 | Time 1.4739(1.5426) | Loss 7.010466(7.176220) | NFE Forward 20(14.4) | NFE Backward 102(112.2) Iter 0073 | Time 1.4455(1.5358) | Loss 7.043934(7.166960) | NFE Forward 14(14.4) | NFE Backward 102(111.5) Iter 0074 | Time 1.5594(1.5374) | Loss 6.982516(7.154049) | NFE Forward 14(14.4) | NFE Backward 114(111.7) Iter 0075 | Time 1.5588(1.5389) | Loss 6.981989(7.142004) | NFE Forward 14(14.3) | NFE Backward 114(111.8) Iter 0076 | Time 1.4472(1.5325) | Loss 6.969278(7.129914) | NFE Forward 14(14.3) | NFE Backward 102(111.1) Iter 0077 | Time 1.4452(1.5264) | Loss 6.988391(7.120007) | NFE Forward 14(14.3) | NFE Backward 102(110.5) Iter 0078 | Time 1.4445(1.5207) | Loss 6.954866(7.108447) | NFE Forward 14(14.3) | NFE Backward 102(109.9) Iter 0079 | Time 1.5321(1.5215) | Loss 6.987312(7.099968) | NFE Forward 14(14.3) | NFE Backward 108(109.8) Iter 0080 | Time 1.4721(1.5180) | Loss 6.936859(7.088550) | NFE Forward 14(14.2) | NFE Backward 102(109.2) Iter 0081 | Time 1.4672(1.5145) | Loss 6.946655(7.078617) | NFE Forward 20(14.6) | NFE Backward 102(108.7) Iter 0082 | Time 1.4958(1.5131) | Loss 6.945632(7.069308) | NFE Forward 20(15.0) | NFE Backward 102(108.3) Iter 0083 | Time 1.4430(1.5082) | Loss 6.920443(7.058888) | NFE Forward 14(14.9) | NFE Backward 102(107.8) Iter 0084 | Time 1.5584(1.5117) | Loss 6.952826(7.051464) | NFE Forward 14(14.9) | NFE Backward 114(108.2) Iter 0085 | Time 1.5861(1.5170) | Loss 6.860269(7.038080) | NFE Forward 20(15.2) | NFE Backward 114(108.7) Iter 0086 | Time 1.5576(1.5198) | Loss 6.911075(7.029190) | NFE Forward 14(15.1) | NFE Backward 114(109.0) Iter 0087 | Time 1.5304(1.5205) | Loss 6.870991(7.018116) | NFE Forward 14(15.1) | NFE Backward 114(109.4) Iter 0088 | Time 1.5583(1.5232) | Loss 6.872494(7.007922) | NFE Forward 14(15.0) | NFE Backward 114(109.7) Iter 0089 | Time 1.5638(1.5260) | Loss 6.879437(6.998928) | NFE Forward 14(14.9) | NFE Backward 114(110.0) Iter 0090 | Time 1.5899(1.5305) | Loss 6.847491(6.988328) | NFE Forward 14(14.9) | NFE Backward 114(110.3) Iter 0091 | Time 1.4454(1.5245) | Loss 6.846432(6.978395) | NFE Forward 14(14.8) | NFE Backward 102(109.7) Iter 0092 | Time 1.4714(1.5208) | Loss 6.838494(6.968602) | NFE Forward 14(14.7) | NFE Backward 102(109.2) Iter 0093 | Time 1.6102(1.5271) | Loss 6.859857(6.960990) | NFE Forward 20(15.1) | NFE Backward 114(109.5) Iter 0094 | Time 1.4421(1.5211) | Loss 6.817008(6.950911) | NFE Forward 14(15.0) | NFE Backward 102(109.0) Iter 0095 | Time 1.5582(1.5237) | Loss 6.762992(6.937757) | NFE Forward 14(15.0) | NFE Backward 114(109.3) Iter 0096 | Time 1.4429(1.5181) | Loss 6.799840(6.928103) | NFE Forward 14(14.9) | NFE Backward 102(108.8) Iter 0097 | Time 1.5573(1.5208) | Loss 6.759663(6.916312) | NFE Forward 14(14.8) | NFE Backward 114(109.2) Iter 0098 | Time 1.4709(1.5173) | Loss 6.774218(6.906365) | NFE Forward 14(14.8) | NFE Backward 102(108.7) Iter 0099 | Time 1.5874(1.5222) | Loss 6.751725(6.895540) | NFE Forward 14(14.7) | NFE Backward 114(109.0) Iter 0100 | Time 1.5927(1.5272) | Loss 6.744266(6.884951) | NFE Forward 14(14.7) | NFE Backward 114(109.4) [TEST] Iter 0100 | Test Loss 6.757403 | NFE 14 Skipping vis as data dimension is >2 Iter 0101 | Time 1.4416(1.5212) | Loss 6.730072(6.874110) | NFE Forward 14(14.6) | NFE Backward 102(108.9) Iter 0102 | Time 1.5608(1.5239) | Loss 6.781401(6.867620) | NFE Forward 14(14.6) | NFE Backward 114(109.2) Iter 0103 | Time 1.4436(1.5183) | Loss 6.690221(6.855202) | NFE Forward 14(14.5) | NFE Backward 102(108.7) Iter 0104 | Time 1.5919(1.5235) | Loss 6.717668(6.845575) | NFE Forward 14(14.5) | NFE Backward 114(109.1) Iter 0105 | Time 1.5634(1.5263) | Loss 6.661384(6.832681) | NFE Forward 14(14.5) | NFE Backward 114(109.4) Iter 0106 | Time 1.4485(1.5208) | Loss 6.663709(6.820853) | NFE Forward 14(14.4) | NFE Backward 102(108.9) Iter 0107 | Time 1.5609(1.5236) | Loss 6.664721(6.809924) | NFE Forward 14(14.4) | NFE Backward 114(109.3) Iter 0108 | Time 1.5605(1.5262) | Loss 6.666580(6.799890) | NFE Forward 14(14.4) | NFE Backward 114(109.6) Iter 0109 | Time 1.5631(1.5288) | Loss 6.653235(6.789624) | NFE Forward 14(14.3) | NFE Backward 114(109.9) Iter 0110 | Time 1.5643(1.5313) | Loss 6.611124(6.777129) | NFE Forward 14(14.3) | NFE Backward 114(110.2) Iter 0111 | Time 1.4752(1.5274) | Loss 6.665914(6.769344) | NFE Forward 14(14.3) | NFE Backward 102(109.6) Iter 0112 | Time 1.5609(1.5297) | Loss 6.637329(6.760103) | NFE Forward 14(14.3) | NFE Backward 114(109.9) Iter 0113 | Time 1.4736(1.5258) | Loss 6.561581(6.746206) | NFE Forward 14(14.3) | NFE Backward 102(109.4) Iter 0114 | Time 1.5631(1.5284) | Loss 6.595440(6.735653) | NFE Forward 14(14.2) | NFE Backward 114(109.7) Iter 0115 | Time 1.6160(1.5345) | Loss 6.575023(6.724409) | NFE Forward 20(14.6) | NFE Backward 114(110.0) Iter 0116 | Time 1.6231(1.5407) | Loss 6.557133(6.712699) | NFE Forward 14(14.6) | NFE Backward 120(110.7) Iter 0117 | Time 1.6167(1.5460) | Loss 6.610967(6.705578) | NFE Forward 14(14.6) | NFE Backward 114(110.9) Iter 0118 | Time 1.6735(1.5550) | Loss 6.556471(6.695140) | NFE Forward 14(14.5) | NFE Backward 120(111.6) Iter 0119 | Time 1.6722(1.5632) | Loss 6.557287(6.685491) | NFE Forward 20(14.9) | NFE Backward 120(112.2) Iter 0120 | Time 1.6719(1.5708) | Loss 6.572733(6.677598) | NFE Forward 20(15.3) | NFE Backward 120(112.7) Iter 0121 | Time 1.6721(1.5779) | Loss 6.509355(6.665821) | NFE Forward 20(15.6) | NFE Backward 120(113.2) Iter 0122 | Time 1.6983(1.5863) | Loss 6.570173(6.659125) | NFE Forward 20(15.9) | NFE Backward 120(113.7) Iter 0123 | Time 1.7015(1.5944) | Loss 6.515835(6.649095) | NFE Forward 20(16.2) | NFE Backward 120(114.1) Iter 0124 | Time 1.6976(1.6016) | Loss 6.561468(6.642961) | NFE Forward 20(16.5) | NFE Backward 120(114.5) Iter 0125 | Time 1.7547(1.6123) | Loss 6.531626(6.635168) | NFE Forward 20(16.7) | NFE Backward 126(115.3) Iter 0126 | Time 1.7585(1.6225) | Loss 6.546728(6.628977) | NFE Forward 20(16.9) | NFE Backward 126(116.1) Iter 0127 | Time 1.7593(1.6321) | Loss 6.519599(6.621320) | NFE Forward 20(17.1) | NFE Backward 126(116.8) Iter 0128 | Time 1.8165(1.6450) | Loss 6.551576(6.616438) | NFE Forward 20(17.3) | NFE Backward 132(117.9) Iter 0129 | Time 1.9370(1.6655) | Loss 6.485641(6.607282) | NFE Forward 20(17.5) | NFE Backward 144(119.7) Iter 0130 | Time 1.7560(1.6718) | Loss 6.509239(6.600419) | NFE Forward 20(17.7) | NFE Backward 126(120.1) Iter 0131 | Time 1.8161(1.6819) | Loss 6.515427(6.594470) | NFE Forward 20(17.9) | NFE Backward 132(121.0) Iter 0132 | Time 1.7034(1.6834) | Loss 6.493918(6.587431) | NFE Forward 20(18.0) | NFE Backward 120(120.9) Iter 0133 | Time 1.8742(1.6968) | Loss 6.502629(6.581495) | NFE Forward 20(18.2) | NFE Backward 138(122.1) Iter 0134 | Time 1.8169(1.7052) | Loss 6.506875(6.576272) | NFE Forward 20(18.3) | NFE Backward 132(122.8) Iter 0135 | Time 1.7559(1.7087) | Loss 6.521713(6.572453) | NFE Forward 20(18.4) | NFE Backward 126(123.0) Iter 0136 | Time 1.7603(1.7123) | Loss 6.471766(6.565405) | NFE Forward 20(18.5) | NFE Backward 126(123.2) Iter 0137 | Time 1.8170(1.7197) | Loss 6.508933(6.561452) | NFE Forward 20(18.6) | NFE Backward 132(123.8) Iter 0138 | Time 1.8198(1.7267) | Loss 6.466777(6.554824) | NFE Forward 20(18.7) | NFE Backward 132(124.4) Iter 0139 | Time 1.8176(1.7330) | Loss 6.476082(6.549312) | NFE Forward 20(18.8) | NFE Backward 132(124.9) Iter 0140 | Time 1.8197(1.7391) | Loss 6.466837(6.543539) | NFE Forward 20(18.9) | NFE Backward 132(125.4) Iter 0141 | Time 1.8222(1.7449) | Loss 6.481664(6.539208) | NFE Forward 20(19.0) | NFE Backward 132(125.9) Iter 0142 | Time 1.7637(1.7462) | Loss 6.454792(6.533299) | NFE Forward 20(19.0) | NFE Backward 126(125.9) Iter 0143 | Time 1.7670(1.7477) | Loss 6.506814(6.531445) | NFE Forward 20(19.1) | NFE Backward 126(125.9) Iter 0144 | Time 1.7658(1.7490) | Loss 6.445365(6.525419) | NFE Forward 20(19.2) | NFE Backward 126(125.9) Iter 0145 | Time 1.8328(1.7548) | Loss 6.456510(6.520596) | NFE Forward 20(19.2) | NFE Backward 132(126.3) Iter 0146 | Time 1.7657(1.7556) | Loss 6.398702(6.512063) | NFE Forward 20(19.3) | NFE Backward 126(126.3) Iter 0147 | Time 1.8253(1.7605) | Loss 6.427059(6.506113) | NFE Forward 20(19.3) | NFE Backward 132(126.7) Iter 0148 | Time 1.7663(1.7609) | Loss 6.395658(6.498381) | NFE Forward 20(19.4) | NFE Backward 126(126.7) Iter 0149 | Time 1.8318(1.7658) | Loss 6.411268(6.492283) | NFE Forward 20(19.4) | NFE Backward 132(127.0) Iter 0150 | Time 1.7668(1.7659) | Loss 6.440076(6.488628) | NFE Forward 20(19.5) | NFE Backward 126(127.0) Iter 0151 | Time 1.8275(1.7702) | Loss 6.432609(6.484707) | NFE Forward 20(19.5) | NFE Backward 132(127.3) Iter 0152 | Time 1.7731(1.7704) | Loss 6.419179(6.480120) | NFE Forward 20(19.5) | NFE Backward 126(127.2) Iter 0153 | Time 1.9503(1.7830) | Loss 6.403602(6.474764) | NFE Forward 20(19.6) | NFE Backward 144(128.4) Iter 0154 | Time 1.7751(1.7825) | Loss 6.399434(6.469491) | NFE Forward 20(19.6) | NFE Backward 126(128.2) Iter 0155 | Time 1.8320(1.7859) | Loss 6.413655(6.465582) | NFE Forward 20(19.6) | NFE Backward 132(128.5) Iter 0156 | Time 1.8393(1.7897) | Loss 6.348957(6.457418) | NFE Forward 20(19.7) | NFE Backward 132(128.7) Iter 0157 | Time 1.7762(1.7887) | Loss 6.378783(6.451914) | NFE Forward 20(19.7) | NFE Backward 126(128.5) Iter 0158 | Time 1.8374(1.7921) | Loss 6.354018(6.445061) | NFE Forward 20(19.7) | NFE Backward 132(128.8) Iter 0159 | Time 1.9540(1.8035) | Loss 6.381140(6.440587) | NFE Forward 20(19.7) | NFE Backward 144(129.9) Iter 0160 | Time 1.8391(1.8060) | Loss 6.399700(6.437725) | NFE Forward 20(19.7) | NFE Backward 132(130.0) Iter 0161 | Time 1.8392(1.8083) | Loss 6.396057(6.434808) | NFE Forward 20(19.8) | NFE Backward 132(130.1) Iter 0162 | Time 1.8376(1.8103) | Loss 6.297676(6.425209) | NFE Forward 20(19.8) | NFE Backward 132(130.3) Iter 0163 | Time 1.9557(1.8205) | Loss 6.326290(6.418284) | NFE Forward 20(19.8) | NFE Backward 144(131.2) Iter 0164 | Time 1.9579(1.8301) | Loss 6.346894(6.413287) | NFE Forward 20(19.8) | NFE Backward 144(132.1) Iter 0165 | Time 1.9573(1.8390) | Loss 6.322408(6.406925) | NFE Forward 20(19.8) | NFE Backward 144(133.0) Iter 0166 | Time 1.8687(1.8411) | Loss 6.353167(6.403162) | NFE Forward 20(19.8) | NFE Backward 132(132.9) Iter 0167 | Time 1.8431(1.8412) | Loss 6.319182(6.397284) | NFE Forward 20(19.8) | NFE Backward 132(132.8) Iter 0168 | Time 1.9626(1.8497) | Loss 6.289198(6.389718) | NFE Forward 20(19.9) | NFE Backward 144(133.6) Iter 0169 | Time 1.9596(1.8574) | Loss 6.308218(6.384013) | NFE Forward 20(19.9) | NFE Backward 144(134.3) Iter 0170 | Time 1.9545(1.8642) | Loss 6.292892(6.377634) | NFE Forward 20(19.9) | NFE Backward 144(135.0) Iter 0171 | Time 1.9534(1.8705) | Loss 6.281129(6.370879) | NFE Forward 20(19.9) | NFE Backward 144(135.6) Iter 0172 | Time 1.8378(1.8682) | Loss 6.247387(6.362235) | NFE Forward 20(19.9) | NFE Backward 132(135.4) Iter 0173 | Time 1.9560(1.8743) | Loss 6.250247(6.354395) | NFE Forward 20(19.9) | NFE Backward 144(136.0) Iter 0174 | Time 1.9545(1.8799) | Loss 6.277206(6.348992) | NFE Forward 20(19.9) | NFE Backward 144(136.6) Iter 0175 | Time 1.9538(1.8851) | Loss 6.247478(6.341886) | NFE Forward 20(19.9) | NFE Backward 144(137.1) Iter 0176 | Time 1.9549(1.8900) | Loss 6.230546(6.334092) | NFE Forward 20(19.9) | NFE Backward 144(137.6) Iter 0177 | Time 1.9588(1.8948) | Loss 6.250772(6.328260) | NFE Forward 20(19.9) | NFE Backward 144(138.0) Iter 0178 | Time 1.9560(1.8991) | Loss 6.219993(6.320681) | NFE Forward 20(19.9) | NFE Backward 144(138.4) Iter 0179 | Time 1.9548(1.9030) | Loss 6.240479(6.315067) | NFE Forward 20(19.9) | NFE Backward 144(138.8) Iter 0180 | Time 1.9583(1.9069) | Loss 6.208460(6.307605) | NFE Forward 20(19.9) | NFE Backward 144(139.2) Iter 0181 | Time 1.9002(1.9064) | Loss 6.203256(6.300300) | NFE Forward 20(19.9) | NFE Backward 138(139.1) Iter 0182 | Time 1.8970(1.9057) | Loss 6.201762(6.293402) | NFE Forward 20(19.9) | NFE Backward 138(139.0) Iter 0183 | Time 1.8998(1.9053) | Loss 6.200816(6.286921) | NFE Forward 20(20.0) | NFE Backward 138(139.0) Iter 0184 | Time 1.9565(1.9089) | Loss 6.171173(6.278819) | NFE Forward 20(20.0) | NFE Backward 144(139.3) Iter 0185 | Time 1.8959(1.9080) | Loss 6.163555(6.270751) | NFE Forward 20(20.0) | NFE Backward 138(139.2) Iter 0186 | Time 1.8976(1.9073) | Loss 6.132141(6.261048) | NFE Forward 20(20.0) | NFE Backward 138(139.1) Iter 0187 | Time 1.8965(1.9065) | Loss 6.175517(6.255061) | NFE Forward 20(20.0) | NFE Backward 138(139.0) Iter 0188 | Time 1.8996(1.9060) | Loss 6.093469(6.243749) | NFE Forward 20(20.0) | NFE Backward 138(139.0) Iter 0189 | Time 1.8994(1.9056) | Loss 6.119079(6.235022) | NFE Forward 20(20.0) | NFE Backward 138(138.9) Iter 0190 | Time 1.8971(1.9050) | Loss 6.100731(6.225622) | NFE Forward 20(20.0) | NFE Backward 138(138.8) Iter 0191 | Time 1.9566(1.9086) | Loss 6.093403(6.216367) | NFE Forward 20(20.0) | NFE Backward 144(139.2) Iter 0192 | Time 1.9395(1.9107) | Loss 6.109213(6.208866) | NFE Forward 20(20.0) | NFE Backward 144(139.5) Iter 0193 | Time 1.8802(1.9086) | Loss 6.052131(6.197894) | NFE Forward 20(20.0) | NFE Backward 138(139.4) Iter 0194 | Time 1.9375(1.9106) | Loss 6.091786(6.190467) | NFE Forward 20(20.0) | NFE Backward 144(139.8) Iter 0195 | Time 1.8814(1.9086) | Loss 6.029783(6.179219) | NFE Forward 20(20.0) | NFE Backward 138(139.6) Iter 0196 | Time 1.8846(1.9069) | Loss 6.024668(6.168400) | NFE Forward 20(20.0) | NFE Backward 138(139.5) Iter 0197 | Time 1.8810(1.9051) | Loss 6.025043(6.158365) | NFE Forward 20(20.0) | NFE Backward 138(139.4) Iter 0198 | Time 1.8797(1.9033) | Loss 6.004670(6.147607) | NFE Forward 20(20.0) | NFE Backward 138(139.3) Iter 0199 | Time 1.8818(1.9018) | Loss 5.993629(6.136828) | NFE Forward 20(20.0) | NFE Backward 138(139.2) Iter 0200 | Time 1.9407(1.9045) | Loss 5.984060(6.126134) | NFE Forward 20(20.0) | NFE Backward 144(139.6) [TEST] Iter 0200 | Test Loss 5.955402 | NFE 20 Skipping vis as data dimension is >2 Iter 0201 | Time 1.9349(1.9067) | Loss 6.008430(6.117895) | NFE Forward 20(20.0) | NFE Backward 144(139.9) Iter 0202 | Time 1.8842(1.9051) | Loss 5.943787(6.105708) | NFE Forward 20(20.0) | NFE Backward 138(139.7) Iter 0203 | Time 1.8906(1.9041) | Loss 5.956751(6.095281) | NFE Forward 20(20.0) | NFE Backward 138(139.6) Iter 0204 | Time 1.9488(1.9072) | Loss 5.912519(6.082487) | NFE Forward 20(20.0) | NFE Backward 144(139.9) Iter 0205 | Time 1.9472(1.9100) | Loss 5.932334(6.071977) | NFE Forward 20(20.0) | NFE Backward 144(140.2) Iter 0206 | Time 1.9476(1.9126) | Loss 5.899208(6.059883) | NFE Forward 20(20.0) | NFE Backward 144(140.5) Iter 0207 | Time 1.9642(1.9162) | Loss 5.890640(6.048036) | NFE Forward 20(20.0) | NFE Backward 144(140.7) Iter 0208 | Time 1.9782(1.9206) | Loss 5.911300(6.038464) | NFE Forward 20(20.0) | NFE Backward 144(140.9) Iter 0209 | Time 1.9636(1.9236) | Loss 5.906520(6.029228) | NFE Forward 20(20.0) | NFE Backward 144(141.2) Iter 0210 | Time 1.9655(1.9265) | Loss 5.888303(6.019363) | NFE Forward 20(20.0) | NFE Backward 144(141.4) Iter 0211 | Time 1.9630(1.9291) | Loss 5.848775(6.007422) | NFE Forward 20(20.0) | NFE Backward 144(141.5) Iter 0212 | Time 1.9642(1.9315) | Loss 5.877208(5.998307) | NFE Forward 20(20.0) | NFE Backward 144(141.7) Iter 0213 | Time 1.9635(1.9338) | Loss 5.826449(5.986277) | NFE Forward 20(20.0) | NFE Backward 144(141.9) Iter 0214 | Time 1.9640(1.9359) | Loss 5.836379(5.975784) | NFE Forward 20(20.0) | NFE Backward 144(142.0) Iter 0215 | Time 1.9673(1.9381) | Loss 5.836134(5.966009) | NFE Forward 20(20.0) | NFE Backward 144(142.2) Iter 0216 | Time 1.9620(1.9398) | Loss 5.846865(5.957669) | NFE Forward 20(20.0) | NFE Backward 144(142.3) Iter 0217 | Time 1.9638(1.9414) | Loss 5.827922(5.948586) | NFE Forward 20(20.0) | NFE Backward 144(142.4) Iter 0218 | Time 1.9660(1.9432) | Loss 5.787573(5.937316) | NFE Forward 20(20.0) | NFE Backward 144(142.5) Iter 0219 | Time 1.9643(1.9446) | Loss 5.789978(5.927002) | NFE Forward 20(20.0) | NFE Backward 144(142.6) Iter 0220 | Time 1.9633(1.9460) | Loss 5.791696(5.917530) | NFE Forward 20(20.0) | NFE Backward 144(142.7) Iter 0221 | Time 1.9639(1.9472) | Loss 5.794932(5.908949) | NFE Forward 20(20.0) | NFE Backward 144(142.8) Iter 0222 | Time 1.9739(1.9491) | Loss 5.804194(5.901616) | NFE Forward 20(20.0) | NFE Backward 144(142.9) Iter 0223 | Time 1.9696(1.9505) | Loss 5.789808(5.893789) | NFE Forward 20(20.0) | NFE Backward 144(143.0) Iter 0224 | Time 1.9660(1.9516) | Loss 5.799236(5.887171) | NFE Forward 20(20.0) | NFE Backward 144(143.0) Iter 0225 | Time 1.9623(1.9523) | Loss 5.788774(5.880283) | NFE Forward 20(20.0) | NFE Backward 144(143.1) Iter 0226 | Time 1.9626(1.9531) | Loss 5.791587(5.874074) | NFE Forward 20(20.0) | NFE Backward 144(143.2) Iter 0227 | Time 1.9664(1.9540) | Loss 5.742929(5.864894) | NFE Forward 20(20.0) | NFE Backward 144(143.2) Iter 0228 | Time 1.9638(1.9547) | Loss 5.746096(5.856578) | NFE Forward 20(20.0) | NFE Backward 144(143.3) Iter 0229 | Time 1.9645(1.9554) | Loss 5.756027(5.849540) | NFE Forward 20(20.0) | NFE Backward 144(143.3) Iter 0230 | Time 1.9658(1.9561) | Loss 5.734987(5.841521) | NFE Forward 20(20.0) | NFE Backward 144(143.4) Iter 0231 | Time 1.9671(1.9569) | Loss 5.755117(5.835473) | NFE Forward 20(20.0) | NFE Backward 144(143.4) Iter 0232 | Time 1.9652(1.9575) | Loss 5.743825(5.829057) | NFE Forward 20(20.0) | NFE Backward 144(143.5) Iter 0233 | Time 1.9660(1.9580) | Loss 5.760581(5.824264) | NFE Forward 20(20.0) | NFE Backward 144(143.5) Iter 0234 | Time 1.9656(1.9586) | Loss 5.743564(5.818615) | NFE Forward 20(20.0) | NFE Backward 144(143.5) Iter 0235 | Time 1.9522(1.9581) | Loss 5.721539(5.811820) | NFE Forward 20(20.0) | NFE Backward 144(143.6) Iter 0236 | Time 1.9471(1.9574) | Loss 5.738442(5.806683) | NFE Forward 20(20.0) | NFE Backward 144(143.6) Iter 0237 | Time 1.9497(1.9568) | Loss 5.760239(5.803432) | NFE Forward 20(20.0) | NFE Backward 144(143.6) Iter 0238 | Time 1.9498(1.9563) | Loss 5.730726(5.798343) | NFE Forward 20(20.0) | NFE Backward 144(143.7) Iter 0239 | Time 1.9529(1.9561) | Loss 5.695800(5.791165) | NFE Forward 20(20.0) | NFE Backward 144(143.7) Iter 0240 | Time 1.9475(1.9555) | Loss 5.739498(5.787548) | NFE Forward 20(20.0) | NFE Backward 144(143.7) Iter 0241 | Time 1.9484(1.9550) | Loss 5.741899(5.784353) | NFE Forward 20(20.0) | NFE Backward 144(143.7) Iter 0242 | Time 1.9485(1.9545) | Loss 5.690208(5.777762) | NFE Forward 20(20.0) | NFE Backward 144(143.7) Iter 0243 | Time 1.9489(1.9542) | Loss 5.714485(5.773333) | NFE Forward 20(20.0) | NFE Backward 144(143.8) Iter 0244 | Time 1.9480(1.9537) | Loss 5.695996(5.767919) | NFE Forward 20(20.0) | NFE Backward 144(143.8) Iter 0245 | Time 2.0067(1.9574) | Loss 5.687671(5.762302) | NFE Forward 20(20.0) | NFE Backward 150(144.2) Iter 0246 | Time 1.9457(1.9566) | Loss 5.723979(5.759619) | NFE Forward 20(20.0) | NFE Backward 144(144.2) Iter 0247 | Time 1.9424(1.9556) | Loss 5.665385(5.753023) | NFE Forward 20(20.0) | NFE Backward 144(144.2) True Warning: Clipping dimensionality to 5 Density Loss 0.13210906088352203 Density Loss 0.13177283108234406 Density Loss 0.13476777076721191 Density Loss 0.1338060349225998 Density Loss 0.1291569620370865 Density Loss 0.13306009769439697 Density Loss 0.12396083772182465 Density Loss 0.1287621706724167 Density Loss 0.12588725984096527 Density Loss 0.12763330340385437 Density Loss 0.1281084269285202 Density Loss 0.1258411407470703 Density Loss 0.12026677280664444 Density Loss 0.12425815314054489 Density Loss 0.12457025796175003 Density Loss 0.13219855725765228 Density Loss 0.12235518544912338 Density Loss 0.12184897065162659 Density Loss 0.12086571007966995 Density Loss 0.1250736117362976 Density Loss 0.12421943247318268 Density Loss 0.12107440829277039 Density Loss 0.12442752718925476 Density Loss 0.12812486290931702 Density Loss 0.12955373525619507 Density Loss 0.1260351836681366 Density Loss 0.12576425075531006 Density Loss 0.13068576157093048 Density Loss 0.12661434710025787 Density Loss 0.12533815205097198 Density Loss 0.12406262010335922 Density Loss 0.12458495795726776 Density Loss 0.1241413876414299 Density Loss 0.12588103115558624 Density Loss 0.12132642418146133 Density Loss 0.12045449018478394 Density Loss 0.12681932747364044 Density Loss 0.12720371782779694 Density Loss 0.12255080044269562 Density Loss 0.12313717603683472 Density Loss 0.12283317744731903 Density Loss 0.12400723248720169 Density Loss 0.12161631882190704 Density Loss 0.12905146181583405 Density Loss 0.1224924623966217 Density Loss 0.126572847366333 Density Loss 0.12356250733137131 Density Loss 0.12627045810222626 Density Loss 0.1229015439748764 Density Loss 0.12180852890014648 Density Loss 0.12114052474498749 Density Loss 0.12267707288265228 Density Loss 0.1217079684138298 Density Loss 0.12335262447595596 Density Loss 0.12521761655807495 Density Loss 0.12082555890083313 Density Loss 0.12364137917757034 Density Loss 0.12202716618776321 Density Loss 0.12441632896661758 Density Loss 0.12129644304513931 Density Loss 0.12355612218379974 Density Loss 0.1280321478843689 Density Loss 0.11998212337493896 Density Loss 0.12590830028057098 Density Loss 0.12624408304691315 Density Loss 0.1243106871843338 Density Loss 0.12483764439821243 Density Loss 0.1251819133758545 Density Loss 0.12884408235549927 Density Loss 0.13178078830242157 Density Loss 0.1364227682352066 Density Loss 0.13489042222499847 Density Loss 0.13656190037727356 Density Loss 0.13873964548110962 Density Loss 0.14349383115768433 Density Loss 0.14285632967948914 Density Loss 0.1452075093984604 Density Loss 0.1434512585401535 Density Loss 0.14673298597335815 Density Loss 0.14892059564590454 Density Loss 0.14648188650608063 Density Loss 0.15790502727031708 Density Loss 0.15567292273044586 Density Loss 0.15841728448867798 Density Loss 0.1585211604833603 Density Loss 0.16040600836277008 Density Loss 0.16300846636295319 Density Loss 0.16010990738868713 Density Loss 0.15948082506656647 Density Loss 0.16416704654693604 Density Loss 0.16687709093093872 Density Loss 0.1634892225265503 Density Loss 0.17030315101146698 Density Loss 0.16662199795246124 Density Loss 0.17173396050930023 Density Loss 0.173207625746727 Density Loss 0.1700994372367859 Density Loss 0.17332561314105988 Density Loss 0.17419323325157166 Density Loss 0.17386016249656677 Density Loss 0.17397728562355042 Density Loss 0.1766740381717682 Density Loss 0.1725229024887085 Density Loss 0.17776483297348022 Density Loss 0.17332381010055542 Density Loss 0.17938381433486938 Density Loss 0.1727907955646515 Density Loss 0.17741598188877106 Density Loss 0.17660531401634216 Density Loss 0.18063461780548096 Density Loss 0.17581675946712494 Density Loss 0.1740906983613968 Density Loss 0.18104977905750275 Density Loss 0.18201279640197754 Density Loss 0.18528343737125397 Density Loss 0.17664803564548492 Density Loss 0.1780485361814499 Density Loss 0.17820623517036438 Density Loss 0.18353348970413208 Density Loss 0.18185938894748688 Density Loss 0.18259193003177643 Density Loss 0.18024155497550964 Density Loss 0.18166832625865936 Density Loss 0.18775421380996704 Density Loss 0.18606041371822357 Density Loss 0.18969933688640594 Density Loss 0.1903911828994751 Density Loss 0.18941891193389893 Density Loss 0.18416056036949158 Density Loss 0.18604899942874908 Density Loss 0.19093191623687744 Density Loss 0.19077396392822266 Density Loss 0.1912817656993866 Density Loss 0.1871187686920166 Density Loss 0.19259744882583618 Density Loss 0.19321554899215698 Density Loss 0.19609031081199646 Density Loss 0.1949804276227951 Density Loss 0.19519491493701935 Density Loss 0.1946553885936737 Density Loss 0.19345279037952423 Density Loss 0.19717340171337128 Density Loss 0.19932135939598083 Density Loss 0.19662316143512726 Density Loss 0.2010144740343094 Density Loss 0.19336120784282684 Density Loss 0.19601237773895264 Density Loss 0.19427387416362762 Density Loss 0.20014826953411102 Density Loss 0.19784684479236603 Density Loss 0.204414501786232 Density Loss 0.20006339251995087 Density Loss 0.20250999927520752 Density Loss 0.2033420354127884 Density Loss 0.2017551213502884 Density Loss 0.201350599527359 Density Loss 0.20471803843975067 Density Loss 0.20120105147361755 Density Loss 0.20251737534999847 Density Loss 0.20760761201381683 Density Loss 0.20572875440120697 Density Loss 0.2055394947528839 Density Loss 0.2048395276069641 Density Loss 0.20618973672389984 Density Loss 0.2063029259443283 Density Loss 0.20209474861621857 Density Loss 0.2062987983226776 Density Loss 0.20615936815738678 Density Loss 0.20386581122875214 Density Loss 0.20877547562122345 Density Loss 0.20289288461208344 Density Loss 0.20340687036514282 Density Loss 0.20798222720623016 Density Loss 0.20993642508983612 Density Loss 0.20885932445526123 Density Loss 0.2052297741174698 Density Loss 0.21205377578735352 Density Loss 0.20586071908473969 Density Loss 0.21368622779846191 Density Loss 0.2105390578508377 Density Loss 0.21313060820102692 Density Loss 0.2034730613231659 Density Loss 0.2153327763080597 Density Loss 0.20490145683288574 Density Loss 0.21675819158554077 Density Loss 0.21777768433094025 Density Loss 0.2153950184583664 Density Loss 0.20924828946590424 Density Loss 0.20823007822036743 Density Loss 0.21218529343605042 Density Loss 0.21140624582767487 Density Loss 0.21272489428520203 Density Loss 0.21404828131198883 Density Loss 0.21711650490760803 Density Loss 0.22505559027194977 Density Loss 0.22212326526641846 Density Loss 0.21326109766960144 Density Loss 0.22446562349796295 Density Loss 0.22760140895843506 Density Loss 0.22760163247585297 Density Loss 0.23005948960781097 Density Loss 0.2304525524377823 Density Loss 0.2244047373533249 Density Loss 0.22177568078041077 Density Loss 0.2244587391614914 Density Loss 0.23909872770309448 Density Loss 0.23314803838729858 Density Loss 0.23573081195354462 Density Loss 0.23463402688503265 Density Loss 0.23549355566501617 Density Loss 0.24589335918426514 Density Loss 0.24826495349407196 Density Loss 0.24288344383239746 Density Loss 0.254038006067276 Density Loss 0.2622949779033661 Density Loss 0.26235926151275635 Density Loss 0.2532086968421936 Density Loss 0.266194224357605 Density Loss 0.27509447932243347 Density Loss 0.2720476984977722 Density Loss 0.2767185568809509 Density Loss 0.281005859375 Density Loss 0.28180545568466187 Density Loss 0.29104095697402954 Density Loss 0.2907642126083374 Density Loss 0.2859564423561096 Density Loss 0.2988934814929962 Density Loss 0.29294484853744507 Density Loss 0.29926684498786926 Density Loss 0.30478692054748535 Density Loss 0.3089229166507721 Density Loss 0.3091104328632355 Density Loss 0.3119639456272125 Density Loss 0.31412985920906067 Density Loss 0.3204321563243866 Density Loss 0.31585025787353516 Density Loss 0.3111954629421234 Density Loss 0.32244062423706055 Density Loss 0.3177626430988312 Density Loss 0.328848272562027 Density Loss 0.3272128701210022 Density Loss 0.33174842596054077 Density Loss 0.3313232362270355 Density Loss 0.32638847827911377 Density Loss 0.33731961250305176 Density Loss 0.3345792293548584 Density Loss 0.3313305377960205 Density Loss 0.33830657601356506 Density Loss 0.34077557921409607 Iter 0248 | Time 2.0939(1.9653) | Loss 5.685529(5.748298) | NFE Forward 20(20.0) | NFE Backward 144(144.2) Iter 0249 | Time 2.0001(1.9677) | Loss 5.669011(5.742748) | NFE Forward 20(20.0) | NFE Backward 150(144.6) Iter 0250 | Time 1.9442(1.9661) | Loss 5.708893(5.740378) | NFE Forward 20(20.0) | NFE Backward 144(144.5) Iter 0251 | Time 1.9678(1.9662) | Loss 5.664450(5.735063) | NFE Forward 26(20.4) | NFE Backward 144(144.5) Iter 0252 | Time 1.9696(1.9664) | Loss 5.680000(5.731209) | NFE Forward 26(20.8) | NFE Backward 144(144.5) Iter 0253 | Time 2.0308(1.9710) | Loss 5.628992(5.724054) | NFE Forward 26(21.2) | NFE Backward 150(144.9) Iter 0254 | Time 1.9703(1.9709) | Loss 5.667716(5.720110) | NFE Forward 26(21.5) | NFE Backward 144(144.8) Iter 0255 | Time 2.0270(1.9748) | Loss 5.654430(5.715513) | NFE Forward 26(21.8) | NFE Backward 150(145.2) Iter 0256 | Time 2.0277(1.9785) | Loss 5.657428(5.711447) | NFE Forward 26(22.1) | NFE Backward 150(145.5) Iter 0257 | Time 1.9678(1.9778) | Loss 5.689026(5.709877) | NFE Forward 26(22.4) | NFE Backward 144(145.4) Iter 0258 | Time 1.9458(1.9755) | Loss 5.649311(5.705638) | NFE Forward 20(22.2) | NFE Backward 144(145.3) Iter 0259 | Time 2.0038(1.9775) | Loss 5.663151(5.702663) | NFE Forward 20(22.1) | NFE Backward 150(145.6) Iter 0260 | Time 1.9716(1.9771) | Loss 5.682167(5.701229) | NFE Forward 26(22.3) | NFE Backward 144(145.5) Iter 0261 | Time 1.9738(1.9769) | Loss 5.656106(5.698070) | NFE Forward 26(22.6) | NFE Backward 144(145.4) Iter 0262 | Time 2.0285(1.9805) | Loss 5.623555(5.692854) | NFE Forward 26(22.8) | NFE Backward 150(145.7) Iter 0263 | Time 2.0034(1.9821) | Loss 5.659452(5.690516) | NFE Forward 20(22.6) | NFE Backward 150(146.0) Iter 0264 | Time 2.0280(1.9853) | Loss 5.690794(5.690535) | NFE Forward 26(22.9) | NFE Backward 150(146.3) Iter 0265 | Time 2.0264(1.9882) | Loss 5.660279(5.688417) | NFE Forward 26(23.1) | NFE Backward 150(146.6) Iter 0266 | Time 2.0274(1.9909) | Loss 5.653003(5.685938) | NFE Forward 26(23.3) | NFE Backward 150(146.8) Iter 0267 | Time 2.0275(1.9935) | Loss 5.621610(5.681435) | NFE Forward 26(23.5) | NFE Backward 150(147.0) Iter 0268 | Time 2.0288(1.9960) | Loss 5.629742(5.677817) | NFE Forward 26(23.7) | NFE Backward 150(147.2) Iter 0269 | Time 2.0058(1.9966) | Loss 5.605756(5.672773) | NFE Forward 20(23.4) | NFE Backward 150(147.4) Iter 0270 | Time 2.0373(1.9995) | Loss 5.623728(5.669340) | NFE Forward 26(23.6) | NFE Backward 150(147.6) Iter 0271 | Time 2.0367(2.0021) | Loss 5.607391(5.665003) | NFE Forward 26(23.8) | NFE Backward 150(147.8) Iter 0272 | Time 2.0354(2.0044) | Loss 5.644848(5.663592) | NFE Forward 26(23.9) | NFE Backward 150(147.9) Iter 0273 | Time 2.0358(2.0066) | Loss 5.625104(5.660898) | NFE Forward 26(24.1) | NFE Backward 150(148.1) Iter 0274 | Time 2.0372(2.0088) | Loss 5.636712(5.659205) | NFE Forward 26(24.2) | NFE Backward 150(148.2) Iter 0275 | Time 2.0129(2.0090) | Loss 5.608507(5.655656) | NFE Forward 20(23.9) | NFE Backward 150(148.3) Iter 0276 | Time 2.0396(2.0112) | Loss 5.608768(5.652374) | NFE Forward 26(24.0) | NFE Backward 150(148.5) Iter 0277 | Time 2.0145(2.0114) | Loss 5.602769(5.648902) | NFE Forward 20(23.8) | NFE Backward 150(148.6) Iter 0278 | Time 2.0104(2.0113) | Loss 5.594454(5.645090) | NFE Forward 20(23.5) | NFE Backward 150(148.7) Iter 0279 | Time 1.9983(2.0104) | Loss 5.612619(5.642817) | NFE Forward 20(23.3) | NFE Backward 150(148.8) Iter 0280 | Time 1.9915(2.0091) | Loss 5.583425(5.638660) | NFE Forward 20(23.0) | NFE Backward 150(148.8) Iter 0281 | Time 2.0160(2.0096) | Loss 5.543221(5.631979) | NFE Forward 26(23.2) | NFE Backward 150(148.9) Iter 0282 | Time 1.9954(2.0086) | Loss 5.597652(5.629576) | NFE Forward 20(23.0) | NFE Backward 150(149.0) Iter 0283 | Time 1.9991(2.0079) | Loss 5.576221(5.625841) | NFE Forward 20(22.8) | NFE Backward 150(149.1) Iter 0284 | Time 1.9939(2.0070) | Loss 5.597573(5.623863) | NFE Forward 20(22.6) | NFE Backward 150(149.1) Iter 0285 | Time 2.0055(2.0068) | Loss 5.576615(5.620555) | NFE Forward 20(22.4) | NFE Backward 150(149.2) Iter 0286 | Time 2.0104(2.0071) | Loss 5.561300(5.616407) | NFE Forward 20(22.3) | NFE Backward 150(149.3) Iter 0287 | Time 2.0005(2.0066) | Loss 5.548493(5.611654) | NFE Forward 20(22.1) | NFE Backward 150(149.3) Iter 0288 | Time 2.0019(2.0063) | Loss 5.587155(5.609939) | NFE Forward 20(21.9) | NFE Backward 150(149.4) Iter 0289 | Time 2.0008(2.0059) | Loss 5.573034(5.607355) | NFE Forward 20(21.8) | NFE Backward 150(149.4) Iter 0290 | Time 2.0585(2.0096) | Loss 5.557947(5.603897) | NFE Forward 20(21.7) | NFE Backward 156(149.9) Iter 0291 | Time 2.0611(2.0132) | Loss 5.567557(5.601353) | NFE Forward 20(21.6) | NFE Backward 156(150.3) Iter 0292 | Time 2.0574(2.0163) | Loss 5.583779(5.600123) | NFE Forward 20(21.5) | NFE Backward 156(150.7) Iter 0293 | Time 2.0584(2.0193) | Loss 5.564404(5.597622) | NFE Forward 20(21.4) | NFE Backward 156(151.1) Iter 0294 | Time 2.0602(2.0221) | Loss 5.560371(5.595015) | NFE Forward 20(21.3) | NFE Backward 156(151.4) Iter 0295 | Time 2.0693(2.0254) | Loss 5.589104(5.594601) | NFE Forward 20(21.2) | NFE Backward 156(151.7) Iter 0296 | Time 2.0739(2.0288) | Loss 5.598058(5.594843) | NFE Forward 20(21.1) | NFE Backward 156(152.0) Iter 0297 | Time 2.0722(2.0319) | Loss 5.538700(5.590913) | NFE Forward 20(21.0) | NFE Backward 156(152.3) Iter 0298 | Time 2.0822(2.0354) | Loss 5.573984(5.589728) | NFE Forward 20(20.9) | NFE Backward 156(152.6) Iter 0299 | Time 2.0762(2.0382) | Loss 5.562727(5.587838) | NFE Forward 20(20.9) | NFE Backward 156(152.8) Iter 0300 | Time 2.0751(2.0408) | Loss 5.560860(5.585950) | NFE Forward 20(20.8) | NFE Backward 156(153.0) [TEST] Iter 0300 | Test Loss 5.537739 | NFE 20 Skipping vis as data dimension is >2 Iter 0301 | Time 2.0719(2.0430) | Loss 5.561346(5.584227) | NFE Forward 20(20.8) | NFE Backward 156(153.2) Iter 0302 | Time 2.0737(2.0451) | Loss 5.537640(5.580966) | NFE Forward 20(20.7) | NFE Backward 156(153.4) Iter 0303 | Time 2.0761(2.0473) | Loss 5.544763(5.578432) | NFE Forward 20(20.7) | NFE Backward 156(153.6) Iter 0304 | Time 2.0569(2.0480) | Loss 5.597000(5.579732) | NFE Forward 20(20.6) | NFE Backward 156(153.8) Iter 0305 | Time 2.0645(2.0491) | Loss 5.557058(5.578145) | NFE Forward 20(20.6) | NFE Backward 156(153.9) Iter 0306 | Time 2.0574(2.0497) | Loss 5.539016(5.575406) | NFE Forward 20(20.5) | NFE Backward 156(154.1) Iter 0307 | Time 2.0574(2.0503) | Loss 5.552461(5.573799) | NFE Forward 20(20.5) | NFE Backward 156(154.2) Iter 0308 | Time 2.0596(2.0509) | Loss 5.541531(5.571541) | NFE Forward 20(20.5) | NFE Backward 156(154.3) Iter 0309 | Time 2.0584(2.0514) | Loss 5.556806(5.570509) | NFE Forward 20(20.4) | NFE Backward 156(154.5) Iter 0310 | Time 2.0564(2.0518) | Loss 5.534858(5.568014) | NFE Forward 20(20.4) | NFE Backward 156(154.6) Iter 0311 | Time 2.0591(2.0523) | Loss 5.579520(5.568819) | NFE Forward 20(20.4) | NFE Backward 156(154.7) Iter 0312 | Time 2.0573(2.0526) | Loss 5.509792(5.564687) | NFE Forward 20(20.3) | NFE Backward 156(154.8) Iter 0313 | Time 2.0573(2.0530) | Loss 5.535476(5.562642) | NFE Forward 20(20.3) | NFE Backward 156(154.8) Iter 0314 | Time 2.0565(2.0532) | Loss 5.550681(5.561805) | NFE Forward 20(20.3) | NFE Backward 156(154.9) Iter 0315 | Time 2.0573(2.0535) | Loss 5.540405(5.560307) | NFE Forward 20(20.3) | NFE Backward 156(155.0) Iter 0316 | Time 2.0802(2.0554) | Loss 5.529691(5.558164) | NFE Forward 26(20.7) | NFE Backward 156(155.1) Iter 0317 | Time 2.0567(2.0555) | Loss 5.529434(5.556153) | NFE Forward 20(20.6) | NFE Backward 156(155.1) Iter 0318 | Time 2.0560(2.0555) | Loss 5.513088(5.553138) | NFE Forward 20(20.6) | NFE Backward 156(155.2) Iter 0319 | Time 2.0569(2.0556) | Loss 5.543017(5.552430) | NFE Forward 20(20.5) | NFE Backward 156(155.3) Iter 0320 | Time 2.0582(2.0558) | Loss 5.511023(5.549531) | NFE Forward 20(20.5) | NFE Backward 156(155.3) Iter 0321 | Time 2.0557(2.0558) | Loss 5.530949(5.548231) | NFE Forward 20(20.5) | NFE Backward 156(155.4) Iter 0322 | Time 2.0580(2.0559) | Loss 5.526533(5.546712) | NFE Forward 20(20.4) | NFE Backward 156(155.4) Iter 0323 | Time 2.0568(2.0560) | Loss 5.555581(5.547333) | NFE Forward 20(20.4) | NFE Backward 156(155.4) Iter 0324 | Time 2.0583(2.0561) | Loss 5.513851(5.544989) | NFE Forward 20(20.4) | NFE Backward 156(155.5) Iter 0325 | Time 2.0574(2.0562) | Loss 5.509975(5.542538) | NFE Forward 20(20.4) | NFE Backward 156(155.5) Iter 0326 | Time 2.0827(2.0581) | Loss 5.463199(5.536984) | NFE Forward 26(20.7) | NFE Backward 156(155.5) Iter 0327 | Time 2.0863(2.0601) | Loss 5.502748(5.534588) | NFE Forward 26(21.1) | NFE Backward 156(155.6) Iter 0328 | Time 2.0591(2.0600) | Loss 5.489775(5.531451) | NFE Forward 20(21.0) | NFE Backward 156(155.6) Iter 0329 | Time 2.0570(2.0598) | Loss 5.536750(5.531822) | NFE Forward 20(21.0) | NFE Backward 156(155.6) Iter 0330 | Time 2.0583(2.0597) | Loss 5.474711(5.527824) | NFE Forward 20(20.9) | NFE Backward 156(155.7) Iter 0331 | Time 2.0850(2.0615) | Loss 5.520307(5.527298) | NFE Forward 26(21.3) | NFE Backward 156(155.7) Iter 0332 | Time 2.0841(2.0630) | Loss 5.533433(5.527727) | NFE Forward 26(21.6) | NFE Backward 156(155.7) Iter 0333 | Time 2.0571(2.0626) | Loss 5.483296(5.524617) | NFE Forward 20(21.5) | NFE Backward 156(155.7) Iter 0334 | Time 2.0996(2.0652) | Loss 5.480460(5.521526) | NFE Forward 26(21.8) | NFE Backward 156(155.7) Iter 0335 | Time 2.0824(2.0664) | Loss 5.514124(5.521008) | NFE Forward 26(22.1) | NFE Backward 156(155.8) Iter 0336 | Time 2.0598(2.0660) | Loss 5.481862(5.518268) | NFE Forward 20(21.9) | NFE Backward 156(155.8) Iter 0337 | Time 2.0939(2.0679) | Loss 5.505810(5.517396) | NFE Forward 26(22.2) | NFE Backward 156(155.8) Iter 0338 | Time 2.0625(2.0675) | Loss 5.448216(5.512553) | NFE Forward 20(22.1) | NFE Backward 156(155.8) Iter 0339 | Time 2.0612(2.0671) | Loss 5.504679(5.512002) | NFE Forward 20(21.9) | NFE Backward 156(155.8) Iter 0340 | Time 2.1153(2.0705) | Loss 5.447509(5.507487) | NFE Forward 26(22.2) | NFE Backward 156(155.8) Iter 0341 | Time 2.0670(2.0702) | Loss 5.465243(5.504530) | NFE Forward 20(22.1) | NFE Backward 156(155.8) Iter 0342 | Time 2.0937(2.0719) | Loss 5.485644(5.503208) | NFE Forward 20(21.9) | NFE Backward 156(155.9) Iter 0343 | Time 2.0913(2.0732) | Loss 5.484897(5.501926) | NFE Forward 20(21.8) | NFE Backward 156(155.9) Iter 0344 | Time 2.0910(2.0745) | Loss 5.460620(5.499035) | NFE Forward 20(21.7) | NFE Backward 156(155.9) Iter 0345 | Time 2.0911(2.0756) | Loss 5.434469(5.494515) | NFE Forward 20(21.5) | NFE Backward 156(155.9) Iter 0346 | Time 2.0901(2.0767) | Loss 5.520014(5.496300) | NFE Forward 20(21.4) | NFE Backward 156(155.9) Iter 0347 | Time 2.0907(2.0776) | Loss 5.431917(5.491794) | NFE Forward 20(21.3) | NFE Backward 156(155.9) Iter 0348 | Time 2.0948(2.0788) | Loss 5.472907(5.490471) | NFE Forward 20(21.2) | NFE Backward 156(155.9) Iter 0349 | Time 2.0959(2.0800) | Loss 5.504851(5.491478) | NFE Forward 20(21.1) | NFE Backward 156(155.9) Iter 0350 | Time 2.0951(2.0811) | Loss 5.436507(5.487630) | NFE Forward 20(21.1) | NFE Backward 156(155.9) Iter 0351 | Time 2.1285(2.0844) | Loss 5.462539(5.485874) | NFE Forward 26(21.4) | NFE Backward 156(155.9) Iter 0352 | Time 2.2285(2.0945) | Loss 5.459304(5.484014) | NFE Forward 26(21.7) | NFE Backward 156(155.9) Iter 0353 | Time 2.1303(2.0970) | Loss 5.448079(5.481498) | NFE Forward 26(22.0) | NFE Backward 156(155.9) Iter 0354 | Time 2.1025(2.0974) | Loss 5.453594(5.479545) | NFE Forward 20(21.9) | NFE Backward 156(155.9) Iter 0355 | Time 2.1069(2.0981) | Loss 5.416075(5.475102) | NFE Forward 20(21.8) | NFE Backward 156(155.9) Iter 0356 | Time 2.1081(2.0988) | Loss 5.431089(5.472021) | NFE Forward 20(21.6) | NFE Backward 156(155.9) Iter 0357 | Time 2.1077(2.0994) | Loss 5.410117(5.467688) | NFE Forward 20(21.5) | NFE Backward 156(156.0) Iter 0358 | Time 2.1294(2.1015) | Loss 5.413502(5.463895) | NFE Forward 26(21.8) | NFE Backward 156(156.0) Iter 0359 | Time 2.1031(2.1016) | Loss 5.444116(5.462510) | NFE Forward 20(21.7) | NFE Backward 156(156.0) Iter 0360 | Time 2.1328(2.1038) | Loss 5.406072(5.458560) | NFE Forward 26(22.0) | NFE Backward 156(156.0) Iter 0361 | Time 2.1053(2.1039) | Loss 5.425965(5.456278) | NFE Forward 20(21.9) | NFE Backward 156(156.0) Iter 0362 | Time 2.1661(2.1082) | Loss 5.412292(5.453199) | NFE Forward 20(21.7) | NFE Backward 162(156.4) Iter 0363 | Time 2.1051(2.1080) | Loss 5.399038(5.449408) | NFE Forward 20(21.6) | NFE Backward 156(156.4) Iter 0364 | Time 2.1891(2.1137) | Loss 5.385902(5.444962) | NFE Forward 26(21.9) | NFE Backward 162(156.8) Iter 0365 | Time 2.1340(2.1151) | Loss 5.395072(5.441470) | NFE Forward 26(22.2) | NFE Backward 156(156.7) Iter 0366 | Time 2.1910(2.1204) | Loss 5.408018(5.439128) | NFE Forward 26(22.5) | NFE Backward 162(157.1) Iter 0367 | Time 2.1646(2.1235) | Loss 5.402430(5.436560) | NFE Forward 20(22.3) | NFE Backward 162(157.4) Iter 0368 | Time 2.1648(2.1264) | Loss 5.418814(5.435317) | NFE Forward 20(22.1) | NFE Backward 162(157.7) Iter 0369 | Time 2.1900(2.1309) | Loss 5.403148(5.433065) | NFE Forward 26(22.4) | NFE Backward 162(158.0) Iter 0370 | Time 2.1601(2.1329) | Loss 5.399229(5.430697) | NFE Forward 20(22.2) | NFE Backward 162(158.3) Iter 0371 | Time 2.1832(2.1364) | Loss 5.387803(5.427694) | NFE Forward 26(22.5) | NFE Backward 162(158.6) Iter 0372 | Time 2.1562(2.1378) | Loss 5.347780(5.422100) | NFE Forward 20(22.3) | NFE Backward 162(158.8) Iter 0373 | Time 2.1832(2.1410) | Loss 5.386788(5.419628) | NFE Forward 26(22.6) | NFE Backward 162(159.0) Iter 0374 | Time 2.1556(2.1420) | Loss 5.405633(5.418649) | NFE Forward 20(22.4) | NFE Backward 162(159.2) Iter 0375 | Time 2.1567(2.1430) | Loss 5.363051(5.414757) | NFE Forward 20(22.2) | NFE Backward 162(159.4) Iter 0376 | Time 2.1612(2.1443) | Loss 5.354948(5.410570) | NFE Forward 20(22.1) | NFE Backward 162(159.6) Iter 0377 | Time 2.1528(2.1449) | Loss 5.372182(5.407883) | NFE Forward 20(21.9) | NFE Backward 162(159.8) Iter 0378 | Time 2.1528(2.1455) | Loss 5.396619(5.407095) | NFE Forward 20(21.8) | NFE Backward 162(159.9) Iter 0379 | Time 2.1760(2.1476) | Loss 5.327524(5.401525) | NFE Forward 26(22.1) | NFE Backward 162(160.1) Iter 0380 | Time 2.1526(2.1479) | Loss 5.350325(5.397941) | NFE Forward 20(21.9) | NFE Backward 162(160.2) Iter 0381 | Time 2.1549(2.1484) | Loss 5.365336(5.395658) | NFE Forward 20(21.8) | NFE Backward 162(160.3) Iter 0382 | Time 2.2755(2.1573) | Loss 5.347785(5.392307) | NFE Forward 20(21.7) | NFE Backward 174(161.3) Iter 0383 | Time 2.2861(2.1663) | Loss 5.389565(5.392115) | NFE Forward 26(22.0) | NFE Backward 168(161.8) Iter 0384 | Time 2.2810(2.1744) | Loss 5.339841(5.388456) | NFE Forward 20(21.8) | NFE Backward 174(162.6) Iter 0385 | Time 2.2768(2.1815) | Loss 5.352577(5.385945) | NFE Forward 20(21.7) | NFE Backward 174(163.4) Iter 0386 | Time 2.2778(2.1883) | Loss 5.336533(5.382486) | NFE Forward 20(21.6) | NFE Backward 174(164.2) Iter 0387 | Time 2.2766(2.1945) | Loss 5.350140(5.380222) | NFE Forward 20(21.5) | NFE Backward 174(164.8) Iter 0388 | Time 2.2782(2.2003) | Loss 5.372801(5.379702) | NFE Forward 20(21.4) | NFE Backward 174(165.5) Iter 0389 | Time 2.3020(2.2074) | Loss 5.287089(5.373219) | NFE Forward 26(21.7) | NFE Backward 174(166.1) Iter 0390 | Time 2.2771(2.2123) | Loss 5.361524(5.372400) | NFE Forward 20(21.6) | NFE Backward 174(166.6) Iter 0391 | Time 2.2768(2.2168) | Loss 5.320208(5.368747) | NFE Forward 20(21.5) | NFE Backward 174(167.2) Iter 0392 | Time 2.2819(2.2214) | Loss 5.317879(5.365186) | NFE Forward 20(21.4) | NFE Backward 174(167.6) Iter 0393 | Time 2.3045(2.2272) | Loss 5.277724(5.359064) | NFE Forward 26(21.7) | NFE Backward 174(168.1) Iter 0394 | Time 2.3048(2.2326) | Loss 5.295471(5.354612) | NFE Forward 26(22.0) | NFE Backward 174(168.5) Iter 0395 | Time 2.3047(2.2377) | Loss 5.325189(5.352553) | NFE Forward 26(22.3) | NFE Backward 174(168.9) Iter 0396 | Time 2.3098(2.2427) | Loss 5.298845(5.348793) | NFE Forward 26(22.5) | NFE Backward 174(169.2) Iter 0397 | Time 2.2785(2.2452) | Loss 5.272722(5.343468) | NFE Forward 20(22.4) | NFE Backward 174(169.6) Iter 0398 | Time 2.2787(2.2476) | Loss 5.337760(5.343069) | NFE Forward 20(22.2) | NFE Backward 174(169.9) Iter 0399 | Time 2.2770(2.2496) | Loss 5.311019(5.340825) | NFE Forward 20(22.0) | NFE Backward 174(170.2) Iter 0400 | Time 2.2794(2.2517) | Loss 5.319466(5.339330) | NFE Forward 20(21.9) | NFE Backward 174(170.4) [TEST] Iter 0400 | Test Loss 5.249241 | NFE 26 Skipping vis as data dimension is >2 Iter 0401 | Time 2.3036(2.2553) | Loss 5.289839(5.335866) | NFE Forward 26(22.2) | NFE Backward 174(170.7) Iter 0402 | Time 2.3325(2.2607) | Loss 5.259589(5.330526) | NFE Forward 26(22.5) | NFE Backward 174(170.9) Iter 0403 | Time 2.2809(2.2622) | Loss 5.247334(5.324703) | NFE Forward 20(22.3) | NFE Backward 174(171.1) Iter 0404 | Time 2.3053(2.2652) | Loss 5.278488(5.321468) | NFE Forward 20(22.1) | NFE Backward 174(171.3) Iter 0405 | Time 2.3064(2.2681) | Loss 5.271868(5.317996) | NFE Forward 20(22.0) | NFE Backward 174(171.5) Iter 0406 | Time 2.3063(2.2707) | Loss 5.283293(5.315567) | NFE Forward 26(22.3) | NFE Backward 174(171.7) Iter 0407 | Time 2.3261(2.2746) | Loss 5.308787(5.315092) | NFE Forward 26(22.5) | NFE Backward 174(171.9) Iter 0408 | Time 2.2990(2.2763) | Loss 5.268854(5.311855) | NFE Forward 20(22.3) | NFE Backward 174(172.0) Iter 0409 | Time 2.3237(2.2796) | Loss 5.219579(5.305396) | NFE Forward 20(22.2) | NFE Backward 174(172.1) Iter 0410 | Time 2.3299(2.2832) | Loss 5.213999(5.298998) | NFE Forward 20(22.0) | NFE Backward 180(172.7) Iter 0411 | Time 2.2701(2.2822) | Loss 5.248075(5.295434) | NFE Forward 20(21.9) | NFE Backward 174(172.8) Iter 0412 | Time 2.3560(2.2874) | Loss 5.216464(5.289906) | NFE Forward 20(21.8) | NFE Backward 180(173.3) Iter 0413 | Time 2.3816(2.2940) | Loss 5.216811(5.284789) | NFE Forward 26(22.0) | NFE Backward 180(173.8) Iter 0414 | Time 2.3837(2.3003) | Loss 5.274034(5.284036) | NFE Forward 26(22.3) | NFE Backward 180(174.2) Iter 0415 | Time 2.3819(2.3060) | Loss 5.271420(5.283153) | NFE Forward 26(22.6) | NFE Backward 180(174.6) Iter 0416 | Time 2.3759(2.3109) | Loss 5.249748(5.280815) | NFE Forward 26(22.8) | NFE Backward 180(175.0) Iter 0417 | Time 2.3789(2.3157) | Loss 5.198930(5.275083) | NFE Forward 26(23.0) | NFE Backward 180(175.3) Iter 0418 | Time 2.3882(2.3207) | Loss 5.238145(5.272497) | NFE Forward 26(23.3) | NFE Backward 180(175.7) Iter 0419 | Time 2.3766(2.3246) | Loss 5.194855(5.267062) | NFE Forward 26(23.4) | NFE Backward 180(176.0) Iter 0420 | Time 2.3749(2.3282) | Loss 5.213845(5.263337) | NFE Forward 26(23.6) | NFE Backward 180(176.2) Iter 0421 | Time 2.3767(2.3316) | Loss 5.234905(5.261347) | NFE Forward 26(23.8) | NFE Backward 180(176.5) Iter 0422 | Time 2.3734(2.3345) | Loss 5.215339(5.258126) | NFE Forward 26(23.9) | NFE Backward 180(176.8) Iter 0423 | Time 2.3729(2.3372) | Loss 5.222168(5.255609) | NFE Forward 26(24.1) | NFE Backward 180(177.0) Iter 0424 | Time 2.3720(2.3396) | Loss 5.219851(5.253106) | NFE Forward 26(24.2) | NFE Backward 180(177.2) Iter 0425 | Time 2.3714(2.3418) | Loss 5.158823(5.246506) | NFE Forward 26(24.3) | NFE Backward 180(177.4) Iter 0426 | Time 2.3714(2.3439) | Loss 5.196484(5.243005) | NFE Forward 26(24.5) | NFE Backward 180(177.6) Iter 0427 | Time 2.3970(2.3476) | Loss 5.210701(5.240743) | NFE Forward 26(24.6) | NFE Backward 180(177.7) Iter 0428 | Time 2.3993(2.3512) | Loss 5.210165(5.238603) | NFE Forward 26(24.7) | NFE Backward 180(177.9) Iter 0429 | Time 2.3992(2.3546) | Loss 5.164100(5.233388) | NFE Forward 26(24.8) | NFE Backward 180(178.0) Iter 0430 | Time 2.3939(2.3574) | Loss 5.217740(5.232292) | NFE Forward 26(24.8) | NFE Backward 180(178.2) Iter 0431 | Time 2.3994(2.3603) | Loss 5.190958(5.229399) | NFE Forward 26(24.9) | NFE Backward 180(178.3) Iter 0432 | Time 2.3981(2.3629) | Loss 5.164476(5.224854) | NFE Forward 26(25.0) | NFE Backward 180(178.4) Iter 0433 | Time 2.3955(2.3652) | Loss 5.149021(5.219546) | NFE Forward 26(25.1) | NFE Backward 180(178.5) Iter 0434 | Time 2.4026(2.3678) | Loss 5.152526(5.214855) | NFE Forward 26(25.1) | NFE Backward 180(178.6) Iter 0435 | Time 2.4042(2.3704) | Loss 5.171482(5.211819) | NFE Forward 26(25.2) | NFE Backward 180(178.7) Iter 0436 | Time 2.3984(2.3724) | Loss 5.219259(5.212339) | NFE Forward 26(25.3) | NFE Backward 180(178.8) Iter 0437 | Time 2.3973(2.3741) | Loss 5.144687(5.207604) | NFE Forward 26(25.3) | NFE Backward 180(178.9) Iter 0438 | Time 2.4005(2.3759) | Loss 5.136404(5.202620) | NFE Forward 26(25.4) | NFE Backward 180(179.0) Iter 0439 | Time 2.3970(2.3774) | Loss 5.146794(5.198712) | NFE Forward 26(25.4) | NFE Backward 180(179.1) Iter 0440 | Time 2.4564(2.3830) | Loss 5.166458(5.196454) | NFE Forward 26(25.4) | NFE Backward 186(179.5) Iter 0441 | Time 2.4177(2.3854) | Loss 5.127325(5.191615) | NFE Forward 26(25.5) | NFE Backward 180(179.6) Iter 0442 | Time 2.4366(2.3890) | Loss 5.102553(5.185381) | NFE Forward 32(25.9) | NFE Backward 180(179.6) Iter 0443 | Time 2.5688(2.4016) | Loss 5.139082(5.182140) | NFE Forward 32(26.4) | NFE Backward 186(180.1) Iter 0444 | Time 2.4321(2.4037) | Loss 5.115035(5.177443) | NFE Forward 32(26.8) | NFE Backward 180(180.0) Iter 0445 | Time 2.4281(2.4054) | Loss 5.112051(5.172865) | NFE Forward 32(27.1) | NFE Backward 180(180.0) Iter 0446 | Time 2.4276(2.4070) | Loss 5.119226(5.169110) | NFE Forward 26(27.0) | NFE Backward 180(180.0) Iter 0447 | Time 2.4074(2.4070) | Loss 5.119718(5.165653) | NFE Forward 26(27.0) | NFE Backward 180(180.0) Iter 0448 | Time 2.4773(2.4119) | Loss 5.116934(5.162243) | NFE Forward 32(27.3) | NFE Backward 186(180.5) Iter 0449 | Time 2.3927(2.4106) | Loss 5.075800(5.156192) | NFE Forward 26(27.2) | NFE Backward 180(180.4) Iter 0450 | Time 2.3923(2.4093) | Loss 5.088017(5.151419) | NFE Forward 26(27.1) | NFE Backward 180(180.4) Iter 0451 | Time 2.4177(2.4099) | Loss 5.137006(5.150410) | NFE Forward 32(27.5) | NFE Backward 180(180.4) Iter 0452 | Time 2.4499(2.4127) | Loss 5.106774(5.147356) | NFE Forward 26(27.4) | NFE Backward 186(180.8) Iter 0453 | Time 2.4520(2.4154) | Loss 5.083057(5.142855) | NFE Forward 26(27.3) | NFE Backward 186(181.1) Iter 0454 | Time 2.4748(2.4196) | Loss 5.117283(5.141065) | NFE Forward 32(27.6) | NFE Backward 186(181.5) Iter 0455 | Time 2.4771(2.4236) | Loss 5.100429(5.138220) | NFE Forward 32(27.9) | NFE Backward 186(181.8) Iter 0456 | Time 2.5012(2.4290) | Loss 5.117259(5.136753) | NFE Forward 32(28.2) | NFE Backward 186(182.1) Iter 0457 | Time 2.4182(2.4283) | Loss 5.082797(5.132976) | NFE Forward 32(28.5) | NFE Backward 180(181.9) Iter 0458 | Time 2.4578(2.4303) | Loss 5.067832(5.128416) | NFE Forward 32(28.7) | NFE Backward 180(181.8) Iter 0459 | Time 2.4762(2.4336) | Loss 5.052147(5.123077) | NFE Forward 32(28.9) | NFE Backward 186(182.1) Iter 0460 | Time 2.4698(2.4361) | Loss 5.058591(5.118563) | NFE Forward 32(29.2) | NFE Backward 186(182.4) Iter 0461 | Time 2.4670(2.4383) | Loss 5.085937(5.116279) | NFE Forward 32(29.4) | NFE Backward 186(182.6) Iter 0462 | Time 2.4646(2.4401) | Loss 5.068943(5.112966) | NFE Forward 32(29.5) | NFE Backward 186(182.9) Iter 0463 | Time 2.4690(2.4421) | Loss 5.067272(5.109767) | NFE Forward 32(29.7) | NFE Backward 186(183.1) Iter 0464 | Time 2.4319(2.4414) | Loss 5.080828(5.107741) | NFE Forward 26(29.5) | NFE Backward 186(183.3) Iter 0465 | Time 2.3978(2.4384) | Loss 5.077271(5.105609) | NFE Forward 32(29.6) | NFE Backward 180(183.1) Iter 0466 | Time 2.4291(2.4377) | Loss 5.071556(5.103225) | NFE Forward 26(29.4) | NFE Backward 186(183.3) Iter 0467 | Time 2.3999(2.4351) | Loss 5.082212(5.101754) | NFE Forward 32(29.6) | NFE Backward 180(183.0) Iter 0468 | Time 2.4326(2.4349) | Loss 5.054872(5.098472) | NFE Forward 26(29.3) | NFE Backward 186(183.2) Iter 0469 | Time 2.4815(2.4381) | Loss 5.051657(5.095195) | NFE Forward 32(29.5) | NFE Backward 186(183.4) Iter 0470 | Time 2.4026(2.4357) | Loss 5.087147(5.094632) | NFE Forward 32(29.7) | NFE Backward 180(183.2) Iter 0471 | Time 2.4019(2.4333) | Loss 5.075002(5.093258) | NFE Forward 32(29.8) | NFE Backward 180(183.0) Iter 0472 | Time 2.4007(2.4310) | Loss 5.010445(5.087461) | NFE Forward 32(30.0) | NFE Backward 180(182.8) Iter 0473 | Time 2.4016(2.4290) | Loss 5.041224(5.084224) | NFE Forward 32(30.1) | NFE Backward 180(182.6) Iter 0474 | Time 2.4538(2.4307) | Loss 5.073129(5.083447) | NFE Forward 32(30.3) | NFE Backward 180(182.4) Iter 0475 | Time 2.3953(2.4282) | Loss 5.046418(5.080855) | NFE Forward 32(30.4) | NFE Backward 174(181.8) Iter 0476 | Time 2.4511(2.4298) | Loss 5.025108(5.076953) | NFE Forward 32(30.5) | NFE Backward 180(181.7) Iter 0477 | Time 2.4543(2.4315) | Loss 5.024568(5.073286) | NFE Forward 32(30.6) | NFE Backward 180(181.6) Iter 0478 | Time 2.4490(2.4328) | Loss 4.991920(5.067590) | NFE Forward 32(30.7) | NFE Backward 180(181.4) Iter 0479 | Time 2.5080(2.4380) | Loss 5.027802(5.064805) | NFE Forward 32(30.8) | NFE Backward 186(181.8) Iter 0480 | Time 2.4521(2.4390) | Loss 5.046018(5.063490) | NFE Forward 32(30.9) | NFE Backward 180(181.6) Iter 0481 | Time 2.4528(2.4400) | Loss 5.034821(5.061483) | NFE Forward 32(31.0) | NFE Backward 180(181.5) Iter 0482 | Time 2.3911(2.4365) | Loss 5.039774(5.059964) | NFE Forward 32(31.0) | NFE Backward 174(181.0) Iter 0483 | Time 2.4752(2.4393) | Loss 5.032221(5.058022) | NFE Forward 32(31.1) | NFE Backward 180(180.9) Iter 0484 | Time 2.4737(2.4417) | Loss 5.023918(5.055634) | NFE Forward 32(31.2) | NFE Backward 180(180.9) Iter 0485 | Time 2.3993(2.4387) | Loss 4.976318(5.050082) | NFE Forward 32(31.2) | NFE Backward 174(180.4) Iter 0486 | Time 2.4574(2.4400) | Loss 4.958096(5.043643) | NFE Forward 32(31.3) | NFE Backward 180(180.4) Iter 0487 | Time 2.4571(2.4412) | Loss 5.026014(5.042409) | NFE Forward 32(31.3) | NFE Backward 180(180.3) Iter 0488 | Time 2.4019(2.4385) | Loss 4.989119(5.038679) | NFE Forward 32(31.4) | NFE Backward 174(179.9) Iter 0489 | Time 2.4591(2.4399) | Loss 5.003662(5.036228) | NFE Forward 32(31.4) | NFE Backward 180(179.9) Iter 0490 | Time 2.4604(2.4413) | Loss 5.045918(5.036906) | NFE Forward 32(31.5) | NFE Backward 180(179.9) Iter 0491 | Time 2.3998(2.4384) | Loss 4.963475(5.031766) | NFE Forward 32(31.5) | NFE Backward 174(179.5) Iter 0492 | Time 2.4321(2.4380) | Loss 4.987168(5.028644) | NFE Forward 26(31.1) | NFE Backward 180(179.5) Iter 0493 | Time 2.4005(2.4354) | Loss 5.006474(5.027092) | NFE Forward 32(31.2) | NFE Backward 174(179.1) Iter 0494 | Time 2.4579(2.4369) | Loss 5.021466(5.026698) | NFE Forward 32(31.2) | NFE Backward 180(179.2) Iter 0495 | Time 2.4408(2.4372) | Loss 4.994740(5.024461) | NFE Forward 32(31.3) | NFE Backward 180(179.3) Iter 0496 | Time 2.4646(2.4391) | Loss 5.004718(5.023079) | NFE Forward 32(31.3) | NFE Backward 180(179.3) Iter 0497 | Time 2.4513(2.4400) | Loss 5.003932(5.021739) | NFE Forward 32(31.4) | NFE Backward 180(179.4) Iter 0498 | Time 2.3900(2.4365) | Loss 4.985498(5.019202) | NFE Forward 32(31.4) | NFE Backward 174(179.0) Density Loss 0.33056142926216125 Density Loss 0.3342806100845337 Density Loss 0.3430637717247009 Density Loss 0.33398711681365967 Density Loss 0.3495252728462219 Density Loss 0.3357544243335724 Density Loss 0.33347177505493164 Density Loss 0.3377465307712555 Density Loss 0.3271194398403168 Density Loss 0.3394697904586792 Density Loss 0.33881619572639465 Density Loss 0.34066474437713623 Density Loss 0.3440743386745453 Density Loss 0.33804449439048767 Density Loss 0.3410717248916626 Density Loss 0.3297233283519745 Density Loss 0.3447830080986023 Density Loss 0.3413858413696289 Density Loss 0.34782448410987854 Density Loss 0.33606964349746704 Density Loss 0.34323203563690186 Density Loss 0.34040552377700806 Density Loss 0.3409649431705475 Density Loss 0.34052175283432007 Density Loss 0.343374639749527 Density Loss 0.349914014339447 Density Loss 0.34011417627334595 Density Loss 0.3434253931045532 Density Loss 0.345670610666275 Density Loss 0.34449100494384766 Density Loss 0.34808796644210815 Density Loss 0.3481149673461914 Density Loss 0.3517858684062958 Density Loss 0.34114372730255127 Density Loss 0.3494010865688324 Density Loss 0.34092068672180176 Density Loss 0.34063073992729187 Density Loss 0.34465163946151733 Density Loss 0.348513662815094 Density Loss 0.3527269959449768 Density Loss 0.3363023102283478 Density Loss 0.3404964804649353 Density Loss 0.3464694619178772 Density Loss 0.34371256828308105 Density Loss 0.3421710729598999 Density Loss 0.34055474400520325 Density Loss 0.35042595863342285 Density Loss 0.34820976853370667 Density Loss 0.35125747323036194 Density Loss 0.339542955160141 Density Loss 0.3426421284675598 Density Loss 0.3492335379123688 Density Loss 0.3490980267524719 Density Loss 0.34223783016204834 Density Loss 0.3403252363204956 Density Loss 0.3411365747451782 Density Loss 0.34951797127723694 Density Loss 0.3399015963077545 Density Loss 0.3450671434402466 Density Loss 0.34834346175193787 Density Loss 0.35028520226478577 Density Loss 0.3432237207889557 Density Loss 0.3441218137741089 Density Loss 0.3481502830982208 Density Loss 0.34155479073524475 Density Loss 0.3395906686782837 Density Loss 0.3453766405582428 Density Loss 0.34051379561424255 Density Loss 0.3582109212875366 Density Loss 0.3450428247451782 Density Loss 0.34704363346099854 Density Loss 0.3462802767753601 Density Loss 0.3416333496570587 Density Loss 0.3474321663379669 Density Loss 0.35549792647361755 Density Loss 0.33799442648887634 Density Loss 0.34329086542129517 Density Loss 0.34385809302330017 Density Loss 0.3587783873081207 Density Loss 0.34163880348205566 Density Loss 0.34252607822418213 Density Loss 0.34546738862991333 Density Loss 0.3559201657772064 Density Loss 0.33887410163879395 Density Loss 0.3558359146118164 Density Loss 0.3502699136734009 Density Loss 0.34439677000045776 Density Loss 0.3490707576274872 Density Loss 0.35888466238975525 Density Loss 0.355656236410141 Density Loss 0.34885141253471375 Density Loss 0.3611695468425751 Density Loss 0.3594261705875397 Density Loss 0.35415124893188477 Density Loss 0.35377755761146545 Density Loss 0.35215020179748535 Density Loss 0.35463884472846985 Density Loss 0.34492743015289307 Density Loss 0.3556059002876282 Density Loss 0.3481394052505493 Density Loss 0.3484189212322235 Density Loss 0.35319674015045166 Density Loss 0.3588373064994812 Density Loss 0.3529534637928009 Density Loss 0.3542386591434479 Density Loss 0.34747040271759033 Density Loss 0.3376157283782959 Density Loss 0.3513616621494293 Density Loss 0.3511822521686554 Density Loss 0.351575642824173 Density Loss 0.3529316186904907 Density Loss 0.36379939317703247 Density Loss 0.35334882140159607 Density Loss 0.3474750816822052 Density Loss 0.35721129179000854 Density Loss 0.35947367548942566 Density Loss 0.3560446798801422 Density Loss 0.3501538932323456 Density Loss 0.35141536593437195 Density Loss 0.35794568061828613 Density Loss 0.3557520806789398 Density Loss 0.3508467972278595 Density Loss 0.3579273819923401 Density Loss 0.35399046540260315 Density Loss 0.3537355959415436 Density Loss 0.3457679748535156 Density Loss 0.3521207869052887 Density Loss 0.35782045125961304 Density Loss 0.34854599833488464 Density Loss 0.35187259316444397 Density Loss 0.3485312759876251 Density Loss 0.36326682567596436 Density Loss 0.34403926134109497 Density Loss 0.35601258277893066 Density Loss 0.35870328545570374 Density Loss 0.34673506021499634 Density Loss 0.35169899463653564 Density Loss 0.3626113533973694 Density Loss 0.3599590063095093 Density Loss 0.35173550248146057 Density Loss 0.35515037178993225 Density Loss 0.3594522476196289 Density Loss 0.35986843705177307 Density Loss 0.36258333921432495 Density Loss 0.3595811724662781 Density Loss 0.36772629618644714 Density Loss 0.3511419892311096 Density Loss 0.36051180958747864 Density Loss 0.3594870865345001 Density Loss 0.3597182631492615 Density Loss 0.3635668158531189 Density Loss 0.3573795258998871 Density Loss 0.35763269662857056 Density Loss 0.36254334449768066 Density Loss 0.3552294671535492 Density Loss 0.36158016324043274 Density Loss 0.3602466583251953 Density Loss 0.35587796568870544 Density Loss 0.3751535713672638 Density Loss 0.3676300644874573 Density Loss 0.3609134554862976 Density Loss 0.3680764436721802 Density Loss 0.3628658056259155 Density Loss 0.3575982451438904 Density Loss 0.36263930797576904 Density Loss 0.36303040385246277 Density Loss 0.3601185083389282 Density Loss 0.37070536613464355 Density Loss 0.3620213568210602 Density Loss 0.37116187810897827 Density Loss 0.3601396977901459 Density Loss 0.3683118522167206 Density Loss 0.3645257353782654 Density Loss 0.367088258266449 Density Loss 0.3621387183666229 Density Loss 0.36209696531295776 Density Loss 0.3727242648601532 Density Loss 0.3630576729774475 Density Loss 0.3664869964122772 Density Loss 0.37067481875419617 Density Loss 0.3696717917919159 Density Loss 0.36787810921669006 Density Loss 0.36432307958602905 Density Loss 0.3721081614494324 Density Loss 0.3742915987968445 Density Loss 0.3643566966056824 Density Loss 0.3645118772983551 Density Loss 0.36820006370544434 Density Loss 0.3728235065937042 Density Loss 0.3739452064037323 Density Loss 0.3749653697013855 Density Loss 0.3749805986881256 Density Loss 0.37143099308013916 Density Loss 0.37281206250190735 Density Loss 0.3797115981578827 Density Loss 0.37909817695617676 Density Loss 0.3687250316143036 Density Loss 0.37015148997306824 Density Loss 0.36698657274246216 Density Loss 0.3698808252811432 Density Loss 0.37351271510124207 Density Loss 0.3840036988258362 Density Loss 0.3775433301925659 Density Loss 0.3661201298236847 Density Loss 0.37474799156188965 Density Loss 0.3780530095100403 Density Loss 0.37426459789276123 Density Loss 0.374433308839798 Density Loss 0.36904382705688477 Density Loss 0.37929341197013855 Density Loss 0.37272509932518005 Density Loss 0.3724250793457031 Density Loss 0.3745363652706146 Density Loss 0.3695428967475891 Density Loss 0.3746533989906311 Density Loss 0.38222503662109375 Density Loss 0.367110937833786 Density Loss 0.3701258599758148 Density Loss 0.3803837299346924 Density Loss 0.37517890334129333 Density Loss 0.38463863730430603 Density Loss 0.37302133440971375 Density Loss 0.3767459988594055 Density Loss 0.36433789134025574 Density Loss 0.37666791677474976 Density Loss 0.38324350118637085 Density Loss 0.3736036419868469 Density Loss 0.386203408241272 Density Loss 0.3822644054889679 Density Loss 0.38550013303756714 Density Loss 0.36720573902130127 Density Loss 0.3739098906517029 Density Loss 0.37568968534469604 Density Loss 0.37876617908477783 Density Loss 0.3743564486503601 Density Loss 0.3861066401004791 Density Loss 0.3802948594093323 Density Loss 0.38332873582839966 Density Loss 0.3840266764163971 Density Loss 0.37605932354927063 Density Loss 0.3716777265071869 Density Loss 0.3752495050430298 Density Loss 0.37542229890823364 Density Loss 0.3817208409309387 Density Loss 0.3741617202758789 Density Loss 0.3799871504306793 Density Loss 0.3738763630390167 Density Loss 0.3850192427635193 Density Loss 0.3855670690536499 Density Loss 0.3762757182121277 Density Loss 0.3773749768733978 Density Loss 0.37870341539382935 Density Loss 0.3862733840942383 Iter 0499 | Time 2.3922(2.4334) | Loss 5.000800(5.017914) | NFE Forward 32(31.5) | NFE Backward 174(178.6) Iter 0500 | Time 2.4018(2.4312) | Loss 5.039250(5.019407) | NFE Forward 32(31.5) | NFE Backward 174(178.3) [TEST] Iter 0500 | Test Loss 4.980343 | NFE 32 Skipping vis as data dimension is >2 Iter 0501 | Time 2.4453(2.4322) | Loss 4.970253(5.015967) | NFE Forward 32(31.5) | NFE Backward 180(178.4) Iter 0502 | Time 2.4504(2.4334) | Loss 5.018943(5.016175) | NFE Forward 32(31.6) | NFE Backward 180(178.5) Iter 0503 | Time 2.4455(2.4343) | Loss 4.959968(5.012240) | NFE Forward 32(31.6) | NFE Backward 180(178.6) Iter 0504 | Time 2.3832(2.4307) | Loss 4.955378(5.008260) | NFE Forward 32(31.6) | NFE Backward 174(178.3) Iter 0505 | Time 2.3834(2.4274) | Loss 4.997303(5.007493) | NFE Forward 32(31.7) | NFE Backward 174(178.0) Iter 0506 | Time 2.3822(2.4242) | Loss 4.950933(5.003534) | NFE Forward 32(31.7) | NFE Backward 174(177.7) Iter 0507 | Time 2.4411(2.4254) | Loss 4.949750(4.999769) | NFE Forward 32(31.7) | NFE Backward 180(177.9) Iter 0508 | Time 2.4512(2.4272) | Loss 4.982060(4.998529) | NFE Forward 32(31.7) | NFE Backward 180(178.0) Iter 0509 | Time 2.3812(2.4240) | Loss 4.990338(4.997956) | NFE Forward 32(31.7) | NFE Backward 174(177.8) Iter 0510 | Time 2.3542(2.4191) | Loss 4.962291(4.995460) | NFE Forward 26(31.3) | NFE Backward 174(177.5) Iter 0511 | Time 2.3779(2.4162) | Loss 4.922564(4.990357) | NFE Forward 32(31.4) | NFE Backward 174(177.2) Iter 0512 | Time 2.4372(2.4177) | Loss 4.946499(4.987287) | NFE Forward 32(31.4) | NFE Backward 180(177.4) Iter 0513 | Time 2.3801(2.4151) | Loss 4.903219(4.981402) | NFE Forward 32(31.5) | NFE Backward 174(177.2) Iter 0514 | Time 2.4352(2.4165) | Loss 4.931636(4.977918) | NFE Forward 32(31.5) | NFE Backward 180(177.4) Iter 0515 | Time 2.4349(2.4178) | Loss 4.941983(4.975403) | NFE Forward 32(31.5) | NFE Backward 180(177.6) Iter 0516 | Time 2.3777(2.4150) | Loss 4.923591(4.971776) | NFE Forward 32(31.6) | NFE Backward 174(177.3) Iter 0517 | Time 2.4417(2.4168) | Loss 4.934100(4.969139) | NFE Forward 32(31.6) | NFE Backward 180(177.5) Iter 0518 | Time 2.3836(2.4145) | Loss 4.949921(4.967793) | NFE Forward 32(31.6) | NFE Backward 174(177.3) Iter 0519 | Time 2.4382(2.4162) | Loss 5.002295(4.970209) | NFE Forward 32(31.7) | NFE Backward 180(177.5) Iter 0520 | Time 2.3533(2.4118) | Loss 4.951230(4.968880) | NFE Forward 26(31.3) | NFE Backward 174(177.2) Iter 0521 | Time 2.3806(2.4096) | Loss 4.927615(4.965991) | NFE Forward 32(31.3) | NFE Backward 174(177.0) Iter 0522 | Time 2.3794(2.4075) | Loss 4.983617(4.967225) | NFE Forward 32(31.4) | NFE Backward 174(176.8) Iter 0523 | Time 2.4383(2.4096) | Loss 4.952325(4.966182) | NFE Forward 32(31.4) | NFE Backward 180(177.0) Iter 0524 | Time 2.4100(2.4097) | Loss 4.936164(4.964081) | NFE Forward 26(31.0) | NFE Backward 180(177.2) Iter 0525 | Time 2.3877(2.4081) | Loss 4.897400(4.959413) | NFE Forward 26(30.7) | NFE Backward 180(177.4) Iter 0526 | Time 2.4365(2.4101) | Loss 4.943783(4.958319) | NFE Forward 32(30.8) | NFE Backward 180(177.6) Iter 0527 | Time 2.3822(2.4081) | Loss 4.920797(4.955693) | NFE Forward 32(30.9) | NFE Backward 174(177.3) Iter 0528 | Time 2.3815(2.4063) | Loss 4.930736(4.953946) | NFE Forward 32(30.9) | NFE Backward 174(177.1) Iter 0529 | Time 2.3580(2.4029) | Loss 4.922330(4.951733) | NFE Forward 26(30.6) | NFE Backward 174(176.9) Iter 0530 | Time 2.3829(2.4015) | Loss 4.904763(4.948445) | NFE Forward 32(30.7) | NFE Backward 174(176.7) Iter 0531 | Time 2.3864(2.4004) | Loss 4.932128(4.947303) | NFE Forward 32(30.8) | NFE Backward 174(176.5) Iter 0532 | Time 2.4443(2.4035) | Loss 4.936165(4.946523) | NFE Forward 32(30.9) | NFE Backward 180(176.7) Iter 0533 | Time 2.4201(2.4047) | Loss 4.903244(4.943493) | NFE Forward 32(30.9) | NFE Backward 174(176.6) Iter 0534 | Time 2.3619(2.4017) | Loss 4.929209(4.942494) | NFE Forward 26(30.6) | NFE Backward 174(176.4) Iter 0535 | Time 2.3613(2.3989) | Loss 4.906195(4.939953) | NFE Forward 26(30.3) | NFE Backward 174(176.2) Iter 0536 | Time 2.4139(2.3999) | Loss 4.928374(4.939142) | NFE Forward 26(30.0) | NFE Backward 180(176.5) Iter 0537 | Time 2.3598(2.3971) | Loss 4.924852(4.938142) | NFE Forward 26(29.7) | NFE Backward 174(176.3) Iter 0538 | Time 2.3658(2.3949) | Loss 4.911867(4.936303) | NFE Forward 26(29.4) | NFE Backward 174(176.1) Iter 0539 | Time 2.3575(2.3923) | Loss 4.911384(4.934558) | NFE Forward 26(29.2) | NFE Backward 174(176.0) Iter 0540 | Time 2.3589(2.3900) | Loss 4.935035(4.934592) | NFE Forward 26(29.0) | NFE Backward 174(175.9) Iter 0541 | Time 2.3847(2.3896) | Loss 4.939582(4.934941) | NFE Forward 32(29.2) | NFE Backward 174(175.7) Iter 0542 | Time 2.3636(2.3878) | Loss 4.949921(4.935990) | NFE Forward 26(29.0) | NFE Backward 174(175.6) Iter 0543 | Time 2.3584(2.3857) | Loss 4.930307(4.935592) | NFE Forward 26(28.8) | NFE Backward 174(175.5) Iter 0544 | Time 2.3578(2.3838) | Loss 4.948982(4.936529) | NFE Forward 26(28.6) | NFE Backward 174(175.4) Iter 0545 | Time 2.3589(2.3820) | Loss 4.904546(4.934290) | NFE Forward 26(28.4) | NFE Backward 174(175.3) Iter 0546 | Time 2.3533(2.3800) | Loss 4.896293(4.931630) | NFE Forward 26(28.2) | NFE Backward 174(175.2) Iter 0547 | Time 2.4387(2.3841) | Loss 4.935405(4.931895) | NFE Forward 32(28.5) | NFE Backward 180(175.5) Iter 0548 | Time 2.3787(2.3837) | Loss 4.856141(4.926592) | NFE Forward 32(28.7) | NFE Backward 174(175.4) Iter 0549 | Time 2.3769(2.3833) | Loss 4.902629(4.924915) | NFE Forward 32(29.0) | NFE Backward 174(175.3) Iter 0550 | Time 2.3536(2.3812) | Loss 4.901504(4.923276) | NFE Forward 26(28.8) | NFE Backward 174(175.2) Iter 0551 | Time 2.3550(2.3794) | Loss 4.905555(4.922035) | NFE Forward 26(28.6) | NFE Backward 174(175.1) Iter 0552 | Time 2.3607(2.3780) | Loss 4.894563(4.920112) | NFE Forward 26(28.4) | NFE Backward 174(175.1) Iter 0553 | Time 2.3535(2.3763) | Loss 4.924101(4.920391) | NFE Forward 26(28.2) | NFE Backward 174(175.0) Iter 0554 | Time 2.3556(2.3749) | Loss 4.900963(4.919031) | NFE Forward 26(28.1) | NFE Backward 174(174.9) Iter 0555 | Time 2.3543(2.3734) | Loss 4.890364(4.917025) | NFE Forward 26(27.9) | NFE Backward 174(174.9) Iter 0556 | Time 2.3548(2.3721) | Loss 4.890397(4.915161) | NFE Forward 26(27.8) | NFE Backward 174(174.8) Iter 0557 | Time 2.3539(2.3709) | Loss 4.924588(4.915821) | NFE Forward 26(27.7) | NFE Backward 174(174.7) Iter 0558 | Time 2.3578(2.3699) | Loss 4.907970(4.915271) | NFE Forward 26(27.5) | NFE Backward 174(174.7) Iter 0559 | Time 2.3608(2.3693) | Loss 4.903455(4.914444) | NFE Forward 26(27.4) | NFE Backward 174(174.6) Iter 0560 | Time 2.3529(2.3682) | Loss 4.900703(4.913482) | NFE Forward 26(27.3) | NFE Backward 174(174.6) Iter 0561 | Time 2.3539(2.3672) | Loss 4.886419(4.911588) | NFE Forward 26(27.2) | NFE Backward 174(174.6) Iter 0562 | Time 2.3542(2.3662) | Loss 4.896128(4.910506) | NFE Forward 26(27.2) | NFE Backward 174(174.5) Iter 0563 | Time 2.3550(2.3655) | Loss 4.873224(4.907896) | NFE Forward 26(27.1) | NFE Backward 174(174.5) Iter 0564 | Time 2.3533(2.3646) | Loss 4.868642(4.905148) | NFE Forward 26(27.0) | NFE Backward 174(174.4) Iter 0565 | Time 2.3536(2.3638) | Loss 4.907229(4.905294) | NFE Forward 26(26.9) | NFE Backward 174(174.4) Iter 0566 | Time 2.3546(2.3632) | Loss 4.868525(4.902720) | NFE Forward 26(26.9) | NFE Backward 174(174.4) Iter 0567 | Time 2.3569(2.3628) | Loss 4.901663(4.902646) | NFE Forward 26(26.8) | NFE Backward 174(174.4) Iter 0568 | Time 2.4139(2.3663) | Loss 4.895747(4.902163) | NFE Forward 26(26.7) | NFE Backward 180(174.8) Iter 0569 | Time 2.4226(2.3703) | Loss 4.888276(4.901191) | NFE Forward 26(26.7) | NFE Backward 180(175.1) Iter 0570 | Time 2.4409(2.3752) | Loss 4.863374(4.898544) | NFE Forward 32(27.1) | NFE Backward 180(175.5) Iter 0571 | Time 2.4156(2.3781) | Loss 4.883993(4.897525) | NFE Forward 26(27.0) | NFE Backward 180(175.8) Iter 0572 | Time 2.4126(2.3805) | Loss 4.896073(4.897423) | NFE Forward 26(26.9) | NFE Backward 180(176.1) Iter 0573 | Time 2.4138(2.3828) | Loss 4.879975(4.896202) | NFE Forward 26(26.9) | NFE Backward 180(176.4) Iter 0574 | Time 2.4139(2.3850) | Loss 4.879798(4.895054) | NFE Forward 26(26.8) | NFE Backward 180(176.6) Iter 0575 | Time 2.4135(2.3870) | Loss 4.864902(4.892943) | NFE Forward 26(26.7) | NFE Backward 180(176.8) Iter 0576 | Time 2.4157(2.3890) | Loss 4.861309(4.890729) | NFE Forward 26(26.7) | NFE Backward 180(177.1) Iter 0577 | Time 2.4152(2.3908) | Loss 4.855332(4.888251) | NFE Forward 26(26.6) | NFE Backward 180(177.3) Iter 0578 | Time 2.4130(2.3924) | Loss 4.870275(4.886993) | NFE Forward 26(26.6) | NFE Backward 180(177.5) Iter 0579 | Time 2.4136(2.3939) | Loss 4.868814(4.885720) | NFE Forward 26(26.6) | NFE Backward 180(177.6) Iter 0580 | Time 2.4170(2.3955) | Loss 4.848928(4.883145) | NFE Forward 26(26.5) | NFE Backward 180(177.8) Iter 0581 | Time 2.4134(2.3967) | Loss 4.904493(4.884639) | NFE Forward 26(26.5) | NFE Backward 180(178.0) Iter 0582 | Time 2.4168(2.3981) | Loss 4.898924(4.885639) | NFE Forward 26(26.4) | NFE Backward 180(178.1) Iter 0583 | Time 2.4177(2.3995) | Loss 4.877103(4.885042) | NFE Forward 26(26.4) | NFE Backward 180(178.2) Iter 0584 | Time 2.4195(2.4009) | Loss 4.833281(4.881418) | NFE Forward 26(26.4) | NFE Backward 180(178.4) Iter 0585 | Time 2.4131(2.4018) | Loss 4.836526(4.878276) | NFE Forward 26(26.4) | NFE Backward 180(178.5) Iter 0586 | Time 2.4403(2.4045) | Loss 4.916377(4.880943) | NFE Forward 32(26.8) | NFE Backward 180(178.6) Iter 0587 | Time 2.4138(2.4051) | Loss 4.866986(4.879966) | NFE Forward 26(26.7) | NFE Backward 180(178.7) Iter 0588 | Time 2.4168(2.4059) | Loss 4.820110(4.875776) | NFE Forward 26(26.7) | NFE Backward 180(178.8) Iter 0589 | Time 2.4144(2.4065) | Loss 4.856689(4.874440) | NFE Forward 26(26.6) | NFE Backward 180(178.9) Iter 0590 | Time 2.4154(2.4071) | Loss 4.877529(4.874656) | NFE Forward 26(26.6) | NFE Backward 180(178.9) Iter 0591 | Time 2.4144(2.4077) | Loss 4.889664(4.875707) | NFE Forward 26(26.5) | NFE Backward 180(179.0) Iter 0592 | Time 2.4168(2.4083) | Loss 4.846986(4.873696) | NFE Forward 26(26.5) | NFE Backward 180(179.1) Iter 0593 | Time 2.4169(2.4089) | Loss 4.835933(4.871053) | NFE Forward 26(26.5) | NFE Backward 180(179.1) Iter 0594 | Time 2.4180(2.4095) | Loss 4.845952(4.869296) | NFE Forward 26(26.4) | NFE Backward 180(179.2) Iter 0595 | Time 2.4266(2.4107) | Loss 4.876643(4.869810) | NFE Forward 26(26.4) | NFE Backward 180(179.3) Iter 0596 | Time 2.4430(2.4130) | Loss 4.847040(4.868216) | NFE Forward 32(26.8) | NFE Backward 180(179.3) Iter 0597 | Time 2.4134(2.4130) | Loss 4.842511(4.866417) | NFE Forward 26(26.7) | NFE Backward 180(179.4) Iter 0598 | Time 2.4154(2.4132) | Loss 4.828727(4.863779) | NFE Forward 26(26.7) | NFE Backward 180(179.4) Iter 0599 | Time 2.4400(2.4151) | Loss 4.812532(4.860191) | NFE Forward 32(27.1) | NFE Backward 180(179.4) Iter 0600 | Time 2.4412(2.4169) | Loss 4.816317(4.857120) | NFE Forward 26(27.0) | NFE Backward 180(179.5) [TEST] Iter 0600 | Test Loss 4.838513 | NFE 26 Skipping vis as data dimension is >2 Iter 0601 | Time 2.4408(2.4186) | Loss 4.839795(4.855907) | NFE Forward 26(26.9) | NFE Backward 180(179.5) Iter 0602 | Time 2.4444(2.4204) | Loss 4.884872(4.857935) | NFE Forward 26(26.8) | NFE Backward 180(179.6) Iter 0603 | Time 2.4255(2.4207) | Loss 4.825031(4.855632) | NFE Forward 26(26.8) | NFE Backward 180(179.6) Iter 0604 | Time 2.4786(2.4248) | Loss 4.823422(4.853377) | NFE Forward 32(27.2) | NFE Backward 180(179.6) Iter 0605 | Time 2.4532(2.4268) | Loss 4.834447(4.852052) | NFE Forward 26(27.1) | NFE Backward 180(179.6) Iter 0606 | Time 2.4781(2.4304) | Loss 4.795221(4.848074) | NFE Forward 32(27.4) | NFE Backward 180(179.7) Iter 0607 | Time 2.4786(2.4337) | Loss 4.867915(4.849463) | NFE Forward 32(27.7) | NFE Backward 180(179.7) Iter 0608 | Time 2.4936(2.4379) | Loss 4.807065(4.846495) | NFE Forward 32(28.0) | NFE Backward 180(179.7) Iter 0609 | Time 2.4768(2.4407) | Loss 4.812575(4.844120) | NFE Forward 32(28.3) | NFE Backward 180(179.7) Iter 0610 | Time 2.4799(2.4434) | Loss 4.827189(4.842935) | NFE Forward 32(28.6) | NFE Backward 180(179.8) Iter 0611 | Time 2.4641(2.4449) | Loss 4.816154(4.841061) | NFE Forward 32(28.8) | NFE Backward 180(179.8) Iter 0612 | Time 2.4691(2.4466) | Loss 4.807623(4.838720) | NFE Forward 32(29.0) | NFE Backward 180(179.8) Iter 0613 | Time 2.5243(2.4520) | Loss 4.791735(4.835431) | NFE Forward 32(29.2) | NFE Backward 186(180.2) Iter 0614 | Time 2.4989(2.4553) | Loss 4.779813(4.831538) | NFE Forward 26(29.0) | NFE Backward 186(180.6) Iter 0615 | Time 2.5242(2.4601) | Loss 4.783416(4.828169) | NFE Forward 32(29.2) | NFE Backward 186(181.0) Iter 0616 | Time 2.5250(2.4646) | Loss 4.858166(4.830269) | NFE Forward 32(29.4) | NFE Backward 186(181.4) Iter 0617 | Time 2.5241(2.4688) | Loss 4.813563(4.829100) | NFE Forward 32(29.6) | NFE Backward 186(181.7) Iter 0618 | Time 2.5258(2.4728) | Loss 4.831250(4.829250) | NFE Forward 32(29.8) | NFE Backward 186(182.0) Iter 0619 | Time 2.5110(2.4755) | Loss 4.804504(4.827518) | NFE Forward 32(29.9) | NFE Backward 186(182.3) Iter 0620 | Time 2.5398(2.4800) | Loss 4.828302(4.827573) | NFE Forward 32(30.1) | NFE Backward 186(182.5) Iter 0621 | Time 2.5381(2.4840) | Loss 4.788766(4.824856) | NFE Forward 32(30.2) | NFE Backward 186(182.8) Iter 0622 | Time 2.5385(2.4878) | Loss 4.825687(4.824915) | NFE Forward 32(30.3) | NFE Backward 186(183.0) Iter 0623 | Time 2.5468(2.4920) | Loss 4.795219(4.822836) | NFE Forward 32(30.4) | NFE Backward 186(183.2) Iter 0624 | Time 2.5522(2.4962) | Loss 4.860134(4.825447) | NFE Forward 32(30.6) | NFE Backward 186(183.4) Iter 0625 | Time 2.5623(2.5008) | Loss 4.836402(4.826214) | NFE Forward 32(30.7) | NFE Backward 186(183.6) Iter 0626 | Time 2.5641(2.5053) | Loss 4.798772(4.824293) | NFE Forward 32(30.8) | NFE Backward 186(183.7) Iter 0627 | Time 2.5602(2.5091) | Loss 4.813421(4.823532) | NFE Forward 32(30.8) | NFE Backward 186(183.9) Iter 0628 | Time 2.5535(2.5122) | Loss 4.809001(4.822514) | NFE Forward 32(30.9) | NFE Backward 186(184.1) Iter 0629 | Time 2.5497(2.5148) | Loss 4.812720(4.821829) | NFE Forward 32(31.0) | NFE Backward 186(184.2) Iter 0630 | Time 2.5507(2.5173) | Loss 4.804223(4.820596) | NFE Forward 32(31.1) | NFE Backward 186(184.3) Iter 0631 | Time 2.6449(2.5263) | Loss 4.765191(4.816718) | NFE Forward 32(31.1) | NFE Backward 186(184.4) Iter 0632 | Time 2.5538(2.5282) | Loss 4.790802(4.814904) | NFE Forward 32(31.2) | NFE Backward 186(184.5) Iter 0633 | Time 2.5481(2.5296) | Loss 4.844099(4.816948) | NFE Forward 32(31.2) | NFE Backward 186(184.6) Iter 0634 | Time 2.5498(2.5310) | Loss 4.829321(4.817814) | NFE Forward 32(31.3) | NFE Backward 186(184.7) Iter 0635 | Time 2.5547(2.5327) | Loss 4.779773(4.815151) | NFE Forward 32(31.3) | NFE Backward 186(184.8) Iter 0636 | Time 2.5534(2.5341) | Loss 4.796067(4.813815) | NFE Forward 32(31.4) | NFE Backward 186(184.9) Iter 0637 | Time 2.5499(2.5352) | Loss 4.832262(4.815106) | NFE Forward 32(31.4) | NFE Backward 186(185.0) Iter 0638 | Time 2.5513(2.5363) | Loss 4.806302(4.814490) | NFE Forward 32(31.5) | NFE Backward 186(185.1) Iter 0639 | Time 2.5498(2.5373) | Loss 4.796994(4.813265) | NFE Forward 32(31.5) | NFE Backward 186(185.1) Iter 0640 | Time 2.5544(2.5385) | Loss 4.808084(4.812902) | NFE Forward 32(31.5) | NFE Backward 186(185.2) Iter 0641 | Time 2.5580(2.5399) | Loss 4.806823(4.812477) | NFE Forward 32(31.6) | NFE Backward 186(185.2) Iter 0642 | Time 2.5706(2.5420) | Loss 4.802147(4.811754) | NFE Forward 32(31.6) | NFE Backward 186(185.3) Iter 0643 | Time 2.5533(2.5428) | Loss 4.767859(4.808681) | NFE Forward 32(31.6) | NFE Backward 186(185.3) Iter 0644 | Time 2.5565(2.5438) | Loss 4.774682(4.806301) | NFE Forward 32(31.7) | NFE Backward 186(185.4) Iter 0645 | Time 2.5531(2.5444) | Loss 4.747845(4.802209) | NFE Forward 32(31.7) | NFE Backward 186(185.4) Iter 0646 | Time 2.5562(2.5452) | Loss 4.810699(4.802804) | NFE Forward 32(31.7) | NFE Backward 186(185.5) Iter 0647 | Time 2.5589(2.5462) | Loss 4.807237(4.803114) | NFE Forward 32(31.7) | NFE Backward 186(185.5) Iter 0648 | Time 2.5598(2.5471) | Loss 4.748694(4.799305) | NFE Forward 32(31.7) | NFE Backward 186(185.5) Iter 0649 | Time 2.5624(2.5482) | Loss 4.795764(4.799057) | NFE Forward 32(31.8) | NFE Backward 186(185.6) Iter 0650 | Time 2.5549(2.5487) | Loss 4.763824(4.796590) | NFE Forward 32(31.8) | NFE Backward 186(185.6) Iter 0651 | Time 2.5559(2.5492) | Loss 4.758220(4.793904) | NFE Forward 32(31.8) | NFE Backward 186(185.6) Iter 0652 | Time 2.5570(2.5497) | Loss 4.729554(4.789400) | NFE Forward 32(31.8) | NFE Backward 186(185.7) Iter 0653 | Time 2.5534(2.5500) | Loss 4.782605(4.788924) | NFE Forward 32(31.8) | NFE Backward 186(185.7) Iter 0654 | Time 2.5543(2.5503) | Loss 4.788897(4.788922) | NFE Forward 32(31.8) | NFE Backward 186(185.7) Iter 0655 | Time 2.5635(2.5512) | Loss 4.790551(4.789036) | NFE Forward 32(31.8) | NFE Backward 186(185.7) Iter 0656 | Time 2.5363(2.5502) | Loss 4.837354(4.792419) | NFE Forward 32(31.9) | NFE Backward 186(185.7) Iter 0657 | Time 2.5420(2.5496) | Loss 4.736359(4.788494) | NFE Forward 32(31.9) | NFE Backward 186(185.8) Iter 0658 | Time 2.5408(2.5490) | Loss 4.821864(4.790830) | NFE Forward 32(31.9) | NFE Backward 186(185.8) Iter 0659 | Time 2.5425(2.5485) | Loss 4.773426(4.789612) | NFE Forward 32(31.9) | NFE Backward 186(185.8) Iter 0660 | Time 2.5384(2.5478) | Loss 4.783272(4.789168) | NFE Forward 32(31.9) | NFE Backward 186(185.8) Iter 0661 | Time 2.5398(2.5473) | Loss 4.809199(4.790570) | NFE Forward 32(31.9) | NFE Backward 186(185.8) Iter 0662 | Time 2.5382(2.5466) | Loss 4.776723(4.789601) | NFE Forward 32(31.9) | NFE Backward 186(185.8) Iter 0663 | Time 2.5414(2.5463) | Loss 4.733506(4.785674) | NFE Forward 32(31.9) | NFE Backward 186(185.8) Iter 0664 | Time 2.5442(2.5461) | Loss 4.755322(4.783550) | NFE Forward 32(31.9) | NFE Backward 186(185.9) Iter 0665 | Time 2.5400(2.5457) | Loss 4.732025(4.779943) | NFE Forward 32(31.9) | NFE Backward 186(185.9) Iter 0666 | Time 2.5445(2.5456) | Loss 4.817101(4.782544) | NFE Forward 32(31.9) | NFE Backward 186(185.9) Iter 0667 | Time 2.5406(2.5453) | Loss 4.736100(4.779293) | NFE Forward 32(31.9) | NFE Backward 186(185.9) Iter 0668 | Time 2.5403(2.5449) | Loss 4.765929(4.778357) | NFE Forward 32(31.9) | NFE Backward 186(185.9) Iter 0669 | Time 2.5422(2.5447) | Loss 4.736880(4.775454) | NFE Forward 32(31.9) | NFE Backward 186(185.9) Iter 0670 | Time 2.5433(2.5446) | Loss 4.784191(4.776066) | NFE Forward 32(31.9) | NFE Backward 186(185.9) Iter 0671 | Time 2.5536(2.5452) | Loss 4.766878(4.775422) | NFE Forward 32(32.0) | NFE Backward 186(185.9) Iter 0672 | Time 2.5479(2.5454) | Loss 4.765064(4.774697) | NFE Forward 32(32.0) | NFE Backward 186(185.9) Iter 0673 | Time 2.5438(2.5453) | Loss 4.741936(4.772404) | NFE Forward 32(32.0) | NFE Backward 186(185.9) Iter 0674 | Time 2.5396(2.5449) | Loss 4.757749(4.771378) | NFE Forward 32(32.0) | NFE Backward 186(185.9) Iter 0675 | Time 2.5534(2.5455) | Loss 4.729947(4.768478) | NFE Forward 32(32.0) | NFE Backward 186(185.9) Iter 0676 | Time 2.5512(2.5459) | Loss 4.760894(4.767947) | NFE Forward 32(32.0) | NFE Backward 186(185.9) Iter 0677 | Time 2.5498(2.5462) | Loss 4.810967(4.770959) | NFE Forward 32(32.0) | NFE Backward 186(185.9) Iter 0678 | Time 2.5540(2.5467) | Loss 4.730913(4.768155) | NFE Forward 32(32.0) | NFE Backward 186(185.9) Iter 0679 | Time 2.5647(2.5480) | Loss 4.739574(4.766155) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0680 | Time 2.5520(2.5483) | Loss 4.785549(4.767512) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0681 | Time 2.5515(2.5485) | Loss 4.787257(4.768894) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0682 | Time 2.5500(2.5486) | Loss 4.723143(4.765692) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0683 | Time 2.5527(2.5489) | Loss 4.760483(4.765327) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0684 | Time 2.5545(2.5493) | Loss 4.728657(4.762760) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0685 | Time 2.5613(2.5501) | Loss 4.745853(4.761577) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0686 | Time 2.5552(2.5505) | Loss 4.738367(4.759952) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0687 | Time 2.5559(2.5509) | Loss 4.782442(4.761526) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0688 | Time 2.5572(2.5513) | Loss 4.734272(4.759619) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0689 | Time 2.5606(2.5520) | Loss 4.730934(4.757611) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0690 | Time 2.5595(2.5525) | Loss 4.765209(4.758142) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0691 | Time 2.5559(2.5527) | Loss 4.732094(4.756319) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0692 | Time 2.5792(2.5546) | Loss 4.794508(4.758992) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0693 | Time 2.5524(2.5544) | Loss 4.774494(4.760077) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0694 | Time 2.5524(2.5543) | Loss 4.742290(4.758832) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0695 | Time 2.5566(2.5544) | Loss 4.769546(4.759582) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0696 | Time 2.5557(2.5545) | Loss 4.725620(4.757205) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0697 | Time 2.5535(2.5545) | Loss 4.758493(4.757295) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0698 | Time 2.5572(2.5546) | Loss 4.743593(4.756336) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0699 | Time 2.5598(2.5550) | Loss 4.754664(4.756219) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0700 | Time 2.5540(2.5549) | Loss 4.720950(4.753750) | NFE Forward 32(32.0) | NFE Backward 186(186.0) [TEST] Iter 0700 | Test Loss 4.730682 | NFE 32 Skipping vis as data dimension is >2 Iter 0701 | Time 2.5526(2.5548) | Loss 4.704379(4.750294) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0702 | Time 2.5884(2.5571) | Loss 4.744310(4.749875) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0703 | Time 2.5626(2.5575) | Loss 4.732353(4.748649) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0704 | Time 2.5658(2.5581) | Loss 4.703542(4.745491) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0705 | Time 2.5592(2.5582) | Loss 4.745003(4.745457) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0706 | Time 2.5615(2.5584) | Loss 4.727102(4.744172) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0707 | Time 2.5581(2.5584) | Loss 4.746023(4.744302) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0708 | Time 2.5520(2.5579) | Loss 4.764412(4.745709) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0709 | Time 2.5387(2.5566) | Loss 4.763384(4.746947) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0710 | Time 2.5643(2.5571) | Loss 4.761900(4.747993) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0711 | Time 2.5349(2.5556) | Loss 4.699147(4.744574) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0712 | Time 2.5388(2.5544) | Loss 4.768498(4.746249) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0713 | Time 2.5635(2.5550) | Loss 4.706963(4.743499) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0714 | Time 2.5622(2.5555) | Loss 4.738999(4.743184) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0715 | Time 2.5584(2.5557) | Loss 4.737166(4.742763) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0716 | Time 2.5610(2.5561) | Loss 4.717165(4.740971) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0717 | Time 2.5359(2.5547) | Loss 4.758938(4.742228) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0718 | Time 2.5648(2.5554) | Loss 4.663561(4.736722) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0719 | Time 2.5405(2.5544) | Loss 4.757828(4.738199) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0720 | Time 2.5344(2.5530) | Loss 4.718338(4.736809) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0721 | Time 2.5334(2.5516) | Loss 4.714887(4.735274) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0722 | Time 2.5367(2.5505) | Loss 4.681723(4.731526) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0723 | Time 2.5364(2.5496) | Loss 4.733985(4.731698) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0724 | Time 2.5317(2.5483) | Loss 4.735361(4.731954) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0725 | Time 2.5652(2.5495) | Loss 4.712944(4.730624) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0726 | Time 2.5645(2.5505) | Loss 4.646559(4.724739) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0727 | Time 2.5616(2.5513) | Loss 4.730014(4.725108) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0728 | Time 2.5590(2.5519) | Loss 4.729544(4.725419) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0729 | Time 2.5351(2.5507) | Loss 4.698377(4.723526) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0730 | Time 2.5637(2.5516) | Loss 4.713428(4.722819) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0731 | Time 2.5600(2.5522) | Loss 4.675150(4.719482) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0732 | Time 2.5591(2.5527) | Loss 4.663092(4.715535) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0733 | Time 2.5393(2.5517) | Loss 4.666181(4.712080) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0734 | Time 2.5593(2.5523) | Loss 4.694521(4.710851) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0735 | Time 2.5361(2.5511) | Loss 4.715218(4.711157) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0736 | Time 2.5641(2.5520) | Loss 4.729486(4.712440) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0737 | Time 2.5589(2.5525) | Loss 4.714523(4.712586) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0738 | Time 2.5609(2.5531) | Loss 4.748360(4.715090) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0739 | Time 2.5615(2.5537) | Loss 4.693156(4.713554) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0740 | Time 2.5600(2.5541) | Loss 4.739811(4.715392) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0741 | Time 2.5616(2.5546) | Loss 4.736269(4.716854) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0742 | Time 2.5652(2.5554) | Loss 4.720840(4.717133) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0743 | Time 2.5755(2.5568) | Loss 4.689800(4.715220) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0744 | Time 2.5621(2.5572) | Loss 4.710569(4.714894) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0745 | Time 2.5675(2.5579) | Loss 4.706969(4.714339) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0746 | Time 2.5369(2.5564) | Loss 4.684198(4.712229) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0747 | Time 2.5639(2.5569) | Loss 4.731931(4.713608) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Density Loss 0.37270602583885193 Density Loss 0.382041871547699 Density Loss 0.3784896433353424 Density Loss 0.3771039843559265 Density Loss 0.3778480589389801 Density Loss 0.36793166399002075 Density Loss 0.37873387336730957 Density Loss 0.381743848323822 Density Loss 0.37768763303756714 Density Loss 0.3780282735824585 Density Loss 0.3907093107700348 Density Loss 0.38167810440063477 Density Loss 0.37672197818756104 Density Loss 0.37867632508277893 Density Loss 0.3879319131374359 Density Loss 0.3764607608318329 Density Loss 0.37841683626174927 Density Loss 0.3831029236316681 Density Loss 0.38086625933647156 Density Loss 0.37563756108283997 Density Loss 0.37994205951690674 Density Loss 0.38277387619018555 Density Loss 0.3746994435787201 Density Loss 0.3903951644897461 Density Loss 0.3763657510280609 Density Loss 0.38743215799331665 Density Loss 0.3871108889579773 Density Loss 0.3701678216457367 Density Loss 0.3815653622150421 Density Loss 0.3830493688583374 Density Loss 0.38136810064315796 Density Loss 0.38561248779296875 Density Loss 0.3845670223236084 Density Loss 0.38334035873413086 Density Loss 0.37422990798950195 Density Loss 0.38085371255874634 Density Loss 0.3765237331390381 Density Loss 0.37756428122520447 Density Loss 0.37515825033187866 Density Loss 0.39383643865585327 Density Loss 0.3819665312767029 Density Loss 0.3866230249404907 Density Loss 0.37774690985679626 Density Loss 0.3854832351207733 Density Loss 0.39171892404556274 Density Loss 0.387381374835968 Density Loss 0.37656542658805847 Density Loss 0.3834843039512634 Density Loss 0.38816770911216736 Density Loss 0.39327141642570496 Density Loss 0.38678106665611267 Density Loss 0.3783244490623474 Density Loss 0.37589311599731445 Density Loss 0.37880000472068787 Density Loss 0.388197124004364 Density Loss 0.37247616052627563 Density Loss 0.37668576836586 Density Loss 0.3922196328639984 Density Loss 0.37630781531333923 Density Loss 0.3890778124332428 Density Loss 0.3911206126213074 Density Loss 0.38614946603775024 Density Loss 0.38878774642944336 Density Loss 0.37506628036499023 Density Loss 0.38830801844596863 Density Loss 0.38170161843299866 Density Loss 0.37730830907821655 Density Loss 0.3945411145687103 Density Loss 0.38653796911239624 Density Loss 0.39135798811912537 Density Loss 0.3829972743988037 Density Loss 0.3820481598377228 Density Loss 0.37870994210243225 Density Loss 0.3834321200847626 Density Loss 0.3913589417934418 Density Loss 0.4008595943450928 Density Loss 0.37869101762771606 Density Loss 0.37284141778945923 Density Loss 0.3777552843093872 Density Loss 0.39824846386909485 Density Loss 0.3732772469520569 Density Loss 0.38041502237319946 Density Loss 0.39160484075546265 Density Loss 0.37406301498413086 Density Loss 0.3960438668727875 Density Loss 0.3923437297344208 Density Loss 0.38828179240226746 Density Loss 0.39043354988098145 Density Loss 0.38743534684181213 Density Loss 0.3797842264175415 Density Loss 0.39056894183158875 Density Loss 0.3822542130947113 Density Loss 0.38130345940589905 Density Loss 0.3885250389575958 Density Loss 0.3937722444534302 Density Loss 0.39123499393463135 Density Loss 0.3866942822933197 Density Loss 0.38498276472091675 Density Loss 0.40154340863227844 Density Loss 0.38702312111854553 Density Loss 0.3987887501716614 Density Loss 0.38826948404312134 Density Loss 0.38025984168052673 Density Loss 0.38376763463020325 Density Loss 0.38494300842285156 Density Loss 0.39026880264282227 Density Loss 0.3951303958892822 Density Loss 0.3929232954978943 Density Loss 0.391252338886261 Density Loss 0.37309515476226807 Density Loss 0.39374271035194397 Density Loss 0.3886706531047821 Density Loss 0.387592077255249 Density Loss 0.3841540813446045 Density Loss 0.3957439959049225 Density Loss 0.3877934515476227 Density Loss 0.39452269673347473 Density Loss 0.38598501682281494 Density Loss 0.40001603960990906 Density Loss 0.3952161967754364 Density Loss 0.3888600468635559 Density Loss 0.38154953718185425 Density Loss 0.38497722148895264 Density Loss 0.40454238653182983 Density Loss 0.39404886960983276 Density Loss 0.3853103518486023 Density Loss 0.39114394783973694 Density Loss 0.413472980260849 Density Loss 0.3983972668647766 Density Loss 0.38415372371673584 Density Loss 0.38002312183380127 Density Loss 0.396415650844574 Density Loss 0.39004388451576233 Density Loss 0.3875497877597809 Density Loss 0.39958831667900085 Density Loss 0.3850298821926117 Density Loss 0.37815216183662415 Density Loss 0.39261987805366516 Density Loss 0.3924159109592438 Density Loss 0.38726845383644104 Density Loss 0.3973895311355591 Density Loss 0.39389970898628235 Density Loss 0.4009617865085602 Density Loss 0.38897088170051575 Density Loss 0.39072659611701965 Density Loss 0.4019591212272644 Density Loss 0.3975820243358612 Density Loss 0.38013315200805664 Density Loss 0.39968276023864746 Density Loss 0.39483702182769775 Density Loss 0.3973986804485321 Density Loss 0.40376850962638855 Density Loss 0.40029144287109375 Density Loss 0.4043067395687103 Density Loss 0.39705780148506165 Density Loss 0.4024614095687866 Density Loss 0.39903098344802856 Density Loss 0.395405113697052 Density Loss 0.41097375750541687 Density Loss 0.3980000615119934 Density Loss 0.3961201012134552 Density Loss 0.3958694338798523 Density Loss 0.40116941928863525 Density Loss 0.39196622371673584 Density Loss 0.4039151668548584 Density Loss 0.4026358425617218 Density Loss 0.38537895679473877 Density Loss 0.3985097408294678 Density Loss 0.4016866385936737 Density Loss 0.3933990001678467 Density Loss 0.40080371499061584 Density Loss 0.39735522866249084 Density Loss 0.3909198045730591 Density Loss 0.39363303780555725 Density Loss 0.4043467342853546 Density Loss 0.396941214799881 Density Loss 0.38899511098861694 Density Loss 0.4035729467868805 Density Loss 0.39829298853874207 Density Loss 0.39218154549598694 Density Loss 0.4039570093154907 Density Loss 0.3963412046432495 Density Loss 0.3942188024520874 Density Loss 0.4011414051055908 Density Loss 0.40322351455688477 Density Loss 0.40041255950927734 Density Loss 0.40146347880363464 Density Loss 0.3954765200614929 Density Loss 0.3947635591030121 Density Loss 0.39568808674812317 Density Loss 0.4078460931777954 Density Loss 0.4087505638599396 Density Loss 0.3906497061252594 Density Loss 0.40244266390800476 Density Loss 0.3962973654270172 Density Loss 0.39431536197662354 Density Loss 0.4069017171859741 Density Loss 0.3952160179615021 Density Loss 0.4080410599708557 Density Loss 0.4054029881954193 Density Loss 0.3982178270816803 Density Loss 0.40149441361427307 Density Loss 0.4105270504951477 Density Loss 0.40216824412345886 Density Loss 0.4043779671192169 Density Loss 0.39710789918899536 Density Loss 0.39841949939727783 Density Loss 0.4036193788051605 Density Loss 0.4161200225353241 Density Loss 0.41772380471229553 Density Loss 0.4079228341579437 Density Loss 0.4161437749862671 Density Loss 0.40805715322494507 Density Loss 0.41513681411743164 Density Loss 0.41232073307037354 Density Loss 0.40419113636016846 Density Loss 0.402901828289032 Density Loss 0.40438830852508545 Density Loss 0.41130882501602173 Density Loss 0.4114045202732086 Density Loss 0.40964236855506897 Density Loss 0.3952566087245941 Density Loss 0.3988342583179474 Density Loss 0.40400463342666626 Density Loss 0.3979824483394623 Density Loss 0.41482284665107727 Density Loss 0.4121607840061188 Density Loss 0.4112860858440399 Density Loss 0.41429686546325684 Density Loss 0.40316224098205566 Density Loss 0.4115401804447174 Density Loss 0.4079354405403137 Density Loss 0.40172553062438965 Density Loss 0.41647058725357056 Density Loss 0.4007073640823364 Density Loss 0.41965723037719727 Density Loss 0.4149686396121979 Density Loss 0.42077893018722534 Density Loss 0.4060744643211365 Density Loss 0.4077649414539337 Density Loss 0.3930310606956482 Density Loss 0.4176250994205475 Density Loss 0.41808021068573 Density Loss 0.3987770080566406 Density Loss 0.4140177071094513 Density Loss 0.4050677716732025 Density Loss 0.41712430119514465 Density Loss 0.40290188789367676 Density Loss 0.41294172406196594 Density Loss 0.4072449207305908 Density Loss 0.40661439299583435 Density Loss 0.4112662971019745 Iter 0748 | Time 2.5610(2.5572) | Loss 4.674620(4.710879) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0749 | Time 2.5716(2.5582) | Loss 4.698936(4.710043) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0750 | Time 2.5635(2.5586) | Loss 4.690918(4.708705) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0751 | Time 2.5686(2.5593) | Loss 4.711659(4.708911) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0752 | Time 2.5603(2.5594) | Loss 4.719291(4.709638) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0753 | Time 2.5617(2.5595) | Loss 4.720787(4.710418) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0754 | Time 2.5590(2.5595) | Loss 4.662562(4.707068) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0755 | Time 2.5580(2.5594) | Loss 4.702570(4.706754) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0756 | Time 2.5618(2.5596) | Loss 4.687392(4.705398) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0757 | Time 2.5651(2.5599) | Loss 4.657485(4.702044) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0758 | Time 2.5568(2.5597) | Loss 4.704589(4.702222) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0759 | Time 2.5585(2.5596) | Loss 4.695499(4.701752) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0760 | Time 2.5599(2.5597) | Loss 4.686641(4.700694) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0761 | Time 2.5584(2.5596) | Loss 4.689493(4.699910) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0762 | Time 2.5567(2.5594) | Loss 4.709836(4.700605) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0763 | Time 2.5579(2.5593) | Loss 4.704689(4.700891) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0764 | Time 2.5605(2.5594) | Loss 4.689427(4.700088) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0765 | Time 2.5596(2.5594) | Loss 4.685608(4.699075) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0766 | Time 2.5628(2.5596) | Loss 4.736187(4.701672) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0767 | Time 2.5601(2.5596) | Loss 4.662461(4.698928) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0768 | Time 2.5642(2.5600) | Loss 4.691298(4.698394) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0769 | Time 2.5632(2.5602) | Loss 4.652005(4.695146) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0770 | Time 2.5589(2.5601) | Loss 4.698010(4.695347) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0771 | Time 2.5556(2.5598) | Loss 4.673377(4.693809) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0772 | Time 2.5619(2.5599) | Loss 4.677528(4.692669) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0773 | Time 2.5572(2.5597) | Loss 4.695305(4.692854) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0774 | Time 2.5594(2.5597) | Loss 4.724692(4.695082) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0775 | Time 2.5568(2.5595) | Loss 4.649195(4.691870) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0776 | Time 2.5593(2.5595) | Loss 4.708619(4.693043) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0777 | Time 2.5600(2.5595) | Loss 4.636453(4.689081) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0778 | Time 2.5766(2.5607) | Loss 4.720212(4.691261) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0779 | Time 2.5750(2.5617) | Loss 4.704694(4.692201) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0780 | Time 2.5768(2.5628) | Loss 4.672777(4.690841) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0781 | Time 2.5761(2.5637) | Loss 4.696205(4.691217) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0782 | Time 2.5733(2.5644) | Loss 4.655893(4.688744) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0783 | Time 2.5784(2.5654) | Loss 4.641618(4.685445) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0784 | Time 2.5760(2.5661) | Loss 4.693915(4.686038) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0785 | Time 2.5747(2.5667) | Loss 4.689657(4.686291) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0786 | Time 2.5725(2.5671) | Loss 4.655389(4.684128) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0787 | Time 2.5740(2.5676) | Loss 4.673023(4.683351) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0788 | Time 2.5741(2.5681) | Loss 4.677020(4.682908) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0789 | Time 2.6044(2.5706) | Loss 4.639319(4.679856) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0790 | Time 2.5751(2.5709) | Loss 4.663329(4.678700) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0791 | Time 2.5786(2.5714) | Loss 4.676671(4.678558) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0792 | Time 2.5757(2.5718) | Loss 4.648180(4.676431) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0793 | Time 2.5751(2.5720) | Loss 4.656558(4.675040) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0794 | Time 2.5740(2.5721) | Loss 4.688497(4.675982) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0795 | Time 2.5760(2.5724) | Loss 4.680855(4.676323) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0796 | Time 2.5803(2.5730) | Loss 4.656240(4.674917) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0797 | Time 2.5714(2.5728) | Loss 4.695768(4.676377) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0798 | Time 2.6007(2.5748) | Loss 4.655221(4.674896) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0799 | Time 2.5728(2.5747) | Loss 4.637170(4.672255) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0800 | Time 2.5747(2.5747) | Loss 4.631171(4.669379) | NFE Forward 32(32.0) | NFE Backward 186(186.0) [TEST] Iter 0800 | Test Loss 4.676871 | NFE 32 Skipping vis as data dimension is >2 Iter 0801 | Time 2.5736(2.5746) | Loss 4.649311(4.667974) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0802 | Time 2.5854(2.5753) | Loss 4.702480(4.670390) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0803 | Time 2.5796(2.5756) | Loss 4.611154(4.666243) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0804 | Time 2.5763(2.5757) | Loss 4.654765(4.665440) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0805 | Time 2.5733(2.5755) | Loss 4.700031(4.667861) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0806 | Time 2.5905(2.5766) | Loss 4.674522(4.668327) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0807 | Time 2.5843(2.5771) | Loss 4.659611(4.667717) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0808 | Time 2.5854(2.5777) | Loss 4.647048(4.666270) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0809 | Time 2.5841(2.5781) | Loss 4.688427(4.667821) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0810 | Time 2.5835(2.5785) | Loss 4.658952(4.667201) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0811 | Time 2.5846(2.5789) | Loss 4.670353(4.667421) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0812 | Time 2.6068(2.5809) | Loss 4.662439(4.667073) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0813 | Time 2.5783(2.5807) | Loss 4.669324(4.667230) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0814 | Time 2.6004(2.5821) | Loss 4.656133(4.666453) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0815 | Time 2.5690(2.5812) | Loss 4.651401(4.665400) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0816 | Time 2.5933(2.5820) | Loss 4.623647(4.662477) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0817 | Time 2.5661(2.5809) | Loss 4.596869(4.657884) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0818 | Time 2.6016(2.5824) | Loss 4.672848(4.658932) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0819 | Time 2.5735(2.5817) | Loss 4.608129(4.655376) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0820 | Time 2.5914(2.5824) | Loss 4.630888(4.653661) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0821 | Time 2.5922(2.5831) | Loss 4.622427(4.651475) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0822 | Time 2.5668(2.5820) | Loss 4.648979(4.651300) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0823 | Time 2.5673(2.5809) | Loss 4.611723(4.648530) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0824 | Time 2.5923(2.5817) | Loss 4.628806(4.647149) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0825 | Time 2.5930(2.5825) | Loss 4.633686(4.646207) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0826 | Time 2.5942(2.5833) | Loss 4.677064(4.648367) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0827 | Time 2.5676(2.5822) | Loss 4.611080(4.645757) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0828 | Time 2.5929(2.5830) | Loss 4.645463(4.645736) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0829 | Time 2.5863(2.5832) | Loss 4.680958(4.648202) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0830 | Time 2.5706(2.5823) | Loss 4.632587(4.647109) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0831 | Time 2.5666(2.5812) | Loss 4.614697(4.644840) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0832 | Time 2.6539(2.5863) | Loss 4.678977(4.647229) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0833 | Time 2.5644(2.5848) | Loss 4.693970(4.650501) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0834 | Time 2.5669(2.5835) | Loss 4.650959(4.650533) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0835 | Time 2.5973(2.5845) | Loss 4.636183(4.649529) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0836 | Time 2.5699(2.5835) | Loss 4.653123(4.649780) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0837 | Time 2.5666(2.5823) | Loss 4.659140(4.650436) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0838 | Time 2.6075(2.5841) | Loss 4.625899(4.648718) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0839 | Time 2.6044(2.5855) | Loss 4.616862(4.646488) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0840 | Time 2.5939(2.5861) | Loss 4.631101(4.645411) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0841 | Time 2.6005(2.5871) | Loss 4.597785(4.642077) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0842 | Time 2.6068(2.5885) | Loss 4.639114(4.641870) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0843 | Time 2.5750(2.5875) | Loss 4.638449(4.641630) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0844 | Time 2.6004(2.5884) | Loss 4.661570(4.643026) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0845 | Time 2.6003(2.5892) | Loss 4.632027(4.642256) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0846 | Time 2.6002(2.5900) | Loss 4.617195(4.640502) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0847 | Time 2.5995(2.5907) | Loss 4.599194(4.637610) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0848 | Time 2.6109(2.5921) | Loss 4.576862(4.633358) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0849 | Time 2.6106(2.5934) | Loss 4.599902(4.631016) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0850 | Time 2.6122(2.5947) | Loss 4.660584(4.633086) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0851 | Time 2.6056(2.5955) | Loss 4.599568(4.630740) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0852 | Time 2.6080(2.5963) | Loss 4.667662(4.633324) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0853 | Time 2.6033(2.5968) | Loss 4.638175(4.633664) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0854 | Time 2.6064(2.5975) | Loss 4.643799(4.634373) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0855 | Time 2.6012(2.5978) | Loss 4.628683(4.633975) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0856 | Time 2.6042(2.5982) | Loss 4.622497(4.633171) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0857 | Time 2.6047(2.5987) | Loss 4.642835(4.633848) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0858 | Time 2.6010(2.5988) | Loss 4.637456(4.634100) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0859 | Time 2.6049(2.5992) | Loss 4.626681(4.633581) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0860 | Time 2.5950(2.5989) | Loss 4.632084(4.633476) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0861 | Time 2.5960(2.5987) | Loss 4.596617(4.630896) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0862 | Time 2.5983(2.5987) | Loss 4.619281(4.630083) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0863 | Time 2.5950(2.5984) | Loss 4.594022(4.627559) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0864 | Time 2.5995(2.5985) | Loss 4.618296(4.626910) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0865 | Time 2.6112(2.5994) | Loss 4.607152(4.625527) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0866 | Time 2.6005(2.5995) | Loss 4.601938(4.623876) | NFE Forward 32(32.0) | NFE Backward 186(186.0) Iter 0867 | Time 2.6586(2.6036) | Loss 4.615934(4.623320) | NFE Forward 32(32.0) | NFE Backward 192(186.4) Iter 0868 | Time 2.5965(2.6031) | Loss 4.641735(4.624609) | NFE Forward 32(32.0) | NFE Backward 186(186.4) Iter 0869 | Time 2.5990(2.6028) | Loss 4.619409(4.624245) | NFE Forward 32(32.0) | NFE Backward 186(186.4) Iter 0870 | Time 2.5972(2.6024) | Loss 4.605617(4.622941) | NFE Forward 32(32.0) | NFE Backward 186(186.3) Iter 0871 | Time 2.5974(2.6021) | Loss 4.600851(4.621395) | NFE Forward 32(32.0) | NFE Backward 186(186.3) Iter 0872 | Time 2.5998(2.6019) | Loss 4.580026(4.618499) | NFE Forward 32(32.0) | NFE Backward 186(186.3) Iter 0873 | Time 2.6137(2.6028) | Loss 4.619385(4.618561) | NFE Forward 32(32.0) | NFE Backward 186(186.3) Iter 0874 | Time 2.6124(2.6034) | Loss 4.631147(4.619442) | NFE Forward 32(32.0) | NFE Backward 186(186.3) Iter 0875 | Time 2.6205(2.6046) | Loss 4.665291(4.622651) | NFE Forward 32(32.0) | NFE Backward 186(186.2) Iter 0876 | Time 2.7639(2.6158) | Loss 4.618814(4.622383) | NFE Forward 32(32.0) | NFE Backward 198(187.1) Iter 0877 | Time 2.6253(2.6164) | Loss 4.635691(4.623314) | NFE Forward 32(32.0) | NFE Backward 186(187.0) Iter 0878 | Time 2.6228(2.6169) | Loss 4.654740(4.625514) | NFE Forward 32(32.0) | NFE Backward 186(186.9) Iter 0879 | Time 2.6240(2.6174) | Loss 4.611419(4.624528) | NFE Forward 32(32.0) | NFE Backward 186(186.9) Iter 0880 | Time 2.6860(2.6222) | Loss 4.610539(4.623548) | NFE Forward 32(32.0) | NFE Backward 192(187.2) Iter 0881 | Time 2.6847(2.6266) | Loss 4.652838(4.625599) | NFE Forward 32(32.0) | NFE Backward 192(187.5) Iter 0882 | Time 2.6309(2.6269) | Loss 4.610694(4.624555) | NFE Forward 32(32.0) | NFE Backward 186(187.4) Iter 0883 | Time 2.6260(2.6268) | Loss 4.592422(4.622306) | NFE Forward 32(32.0) | NFE Backward 186(187.3) Iter 0884 | Time 2.6855(2.6309) | Loss 4.573494(4.618889) | NFE Forward 32(32.0) | NFE Backward 192(187.7) Iter 0885 | Time 2.6233(2.6304) | Loss 4.631715(4.619787) | NFE Forward 32(32.0) | NFE Backward 186(187.5) Iter 0886 | Time 2.6208(2.6297) | Loss 4.632702(4.620691) | NFE Forward 32(32.0) | NFE Backward 186(187.4) Iter 0887 | Time 2.6278(2.6296) | Loss 4.576525(4.617599) | NFE Forward 32(32.0) | NFE Backward 186(187.3) Iter 0888 | Time 2.6356(2.6300) | Loss 4.640747(4.619220) | NFE Forward 32(32.0) | NFE Backward 186(187.2) Iter 0889 | Time 2.6222(2.6295) | Loss 4.604590(4.618196) | NFE Forward 32(32.0) | NFE Backward 186(187.2) Iter 0890 | Time 2.6322(2.6297) | Loss 4.623794(4.618588) | NFE Forward 32(32.0) | NFE Backward 186(187.1) Iter 0891 | Time 2.6601(2.6318) | Loss 4.560741(4.614538) | NFE Forward 32(32.0) | NFE Backward 186(187.0) Iter 0892 | Time 2.6943(2.6362) | Loss 4.565881(4.611132) | NFE Forward 32(32.0) | NFE Backward 192(187.4) Iter 0893 | Time 2.6184(2.6349) | Loss 4.577891(4.608805) | NFE Forward 32(32.0) | NFE Backward 186(187.3) Iter 0894 | Time 2.6210(2.6339) | Loss 4.554774(4.605023) | NFE Forward 32(32.0) | NFE Backward 186(187.2) Iter 0895 | Time 2.7405(2.6414) | Loss 4.632188(4.606925) | NFE Forward 32(32.0) | NFE Backward 198(187.9) Iter 0896 | Time 2.6810(2.6442) | Loss 4.610398(4.607168) | NFE Forward 32(32.0) | NFE Backward 192(188.2) Iter 0897 | Time 2.6170(2.6423) | Loss 4.586990(4.605755) | NFE Forward 32(32.0) | NFE Backward 186(188.1) Iter 0898 | Time 2.6212(2.6408) | Loss 4.569597(4.603224) | NFE Forward 32(32.0) | NFE Backward 186(187.9) Iter 0899 | Time 2.6333(2.6403) | Loss 4.588427(4.602189) | NFE Forward 32(32.0) | NFE Backward 186(187.8) Iter 0900 | Time 2.6995(2.6444) | Loss 4.559232(4.599182) | NFE Forward 32(32.0) | NFE Backward 192(188.1) [TEST] Iter 0900 | Test Loss 4.628522 | NFE 32 Skipping vis as data dimension is >2 Iter 0901 | Time 2.7392(2.6510) | Loss 4.615647(4.600334) | NFE Forward 32(32.0) | NFE Backward 198(188.8) Iter 0902 | Time 2.7428(2.6575) | Loss 4.641807(4.603237) | NFE Forward 32(32.0) | NFE Backward 198(189.4) Iter 0903 | Time 2.6245(2.6552) | Loss 4.606071(4.603436) | NFE Forward 32(32.0) | NFE Backward 186(189.2) Iter 0904 | Time 2.7444(2.6614) | Loss 4.592448(4.602666) | NFE Forward 32(32.0) | NFE Backward 198(189.8) Iter 0905 | Time 2.7426(2.6671) | Loss 4.566234(4.600116) | NFE Forward 32(32.0) | NFE Backward 198(190.4) Iter 0906 | Time 2.6856(2.6684) | Loss 4.613245(4.601035) | NFE Forward 32(32.0) | NFE Backward 192(190.5) Iter 0907 | Time 2.6170(2.6648) | Loss 4.590179(4.600275) | NFE Forward 32(32.0) | NFE Backward 186(190.2) Iter 0908 | Time 2.5974(2.6601) | Loss 4.600008(4.600257) | NFE Forward 32(32.0) | NFE Backward 186(189.9) Iter 0909 | Time 2.7156(2.6640) | Loss 4.609648(4.600914) | NFE Forward 32(32.0) | NFE Backward 198(190.4) Iter 0910 | Time 2.5858(2.6585) | Loss 4.601633(4.600964) | NFE Forward 32(32.0) | NFE Backward 186(190.1) Iter 0911 | Time 2.6958(2.6611) | Loss 4.637633(4.603531) | NFE Forward 32(32.0) | NFE Backward 198(190.7) Iter 0912 | Time 2.6396(2.6596) | Loss 4.642970(4.606292) | NFE Forward 32(32.0) | NFE Backward 192(190.8) Iter 0913 | Time 2.5791(2.6540) | Loss 4.613534(4.606799) | NFE Forward 32(32.0) | NFE Backward 186(190.4) Iter 0914 | Time 2.5814(2.6489) | Loss 4.609339(4.606977) | NFE Forward 32(32.0) | NFE Backward 186(190.1) Iter 0915 | Time 2.5804(2.6441) | Loss 4.592804(4.605984) | NFE Forward 32(32.0) | NFE Backward 186(189.8) Iter 0916 | Time 2.5860(2.6400) | Loss 4.562557(4.602945) | NFE Forward 32(32.0) | NFE Backward 186(189.6) Iter 0917 | Time 2.7157(2.6453) | Loss 4.597420(4.602558) | NFE Forward 32(32.0) | NFE Backward 198(190.2) Iter 0918 | Time 2.7142(2.6501) | Loss 4.620861(4.603839) | NFE Forward 32(32.0) | NFE Backward 198(190.7) Iter 0919 | Time 2.5942(2.6462) | Loss 4.600482(4.603604) | NFE Forward 32(32.0) | NFE Backward 186(190.4) Iter 0920 | Time 2.6111(2.6438) | Loss 4.598246(4.603229) | NFE Forward 32(32.0) | NFE Backward 186(190.1) Iter 0921 | Time 2.5960(2.6404) | Loss 4.547814(4.599350) | NFE Forward 32(32.0) | NFE Backward 186(189.8) Iter 0922 | Time 2.6092(2.6382) | Loss 4.591323(4.598788) | NFE Forward 32(32.0) | NFE Backward 186(189.5) Iter 0923 | Time 2.7548(2.6464) | Loss 4.586133(4.597902) | NFE Forward 32(32.0) | NFE Backward 198(190.1) Iter 0924 | Time 2.5926(2.6426) | Loss 4.596945(4.597835) | NFE Forward 32(32.0) | NFE Backward 186(189.8) Iter 0925 | Time 2.5921(2.6391) | Loss 4.569735(4.595868) | NFE Forward 32(32.0) | NFE Backward 186(189.6) Iter 0926 | Time 2.7093(2.6440) | Loss 4.624093(4.597844) | NFE Forward 32(32.0) | NFE Backward 198(190.2) Iter 0927 | Time 2.6069(2.6414) | Loss 4.558831(4.595113) | NFE Forward 32(32.0) | NFE Backward 186(189.9) Iter 0928 | Time 2.7262(2.6473) | Loss 4.574031(4.593637) | NFE Forward 32(32.0) | NFE Backward 198(190.4) Iter 0929 | Time 2.7257(2.6528) | Loss 4.594827(4.593721) | NFE Forward 32(32.0) | NFE Backward 198(191.0) Iter 0930 | Time 2.6056(2.6495) | Loss 4.592668(4.593647) | NFE Forward 32(32.0) | NFE Backward 186(190.6) Iter 0931 | Time 2.7235(2.6547) | Loss 4.561351(4.591386) | NFE Forward 32(32.0) | NFE Backward 198(191.1) Iter 0932 | Time 2.6755(2.6562) | Loss 4.629200(4.594033) | NFE Forward 32(32.0) | NFE Backward 192(191.2) Iter 0933 | Time 2.7280(2.6612) | Loss 4.606547(4.594909) | NFE Forward 32(32.0) | NFE Backward 198(191.7) Iter 0934 | Time 2.7230(2.6655) | Loss 4.599806(4.595252) | NFE Forward 32(32.0) | NFE Backward 198(192.1) Iter 0935 | Time 2.7799(2.6735) | Loss 4.581087(4.594260) | NFE Forward 32(32.0) | NFE Backward 204(192.9) Iter 0936 | Time 2.7368(2.6780) | Loss 4.622571(4.596242) | NFE Forward 32(32.0) | NFE Backward 198(193.3) Iter 0937 | Time 2.7372(2.6821) | Loss 4.553566(4.593255) | NFE Forward 32(32.0) | NFE Backward 198(193.6) Iter 0938 | Time 2.6283(2.6783) | Loss 4.609794(4.594413) | NFE Forward 32(32.0) | NFE Backward 186(193.1) Iter 0939 | Time 2.6071(2.6733) | Loss 4.568819(4.592621) | NFE Forward 32(32.0) | NFE Backward 186(192.6) Iter 0940 | Time 2.7282(2.6772) | Loss 4.550532(4.589675) | NFE Forward 32(32.0) | NFE Backward 198(193.0) Iter 0941 | Time 2.7257(2.6806) | Loss 4.564356(4.587902) | NFE Forward 32(32.0) | NFE Backward 198(193.3) Iter 0942 | Time 2.7261(2.6838) | Loss 4.580209(4.587364) | NFE Forward 32(32.0) | NFE Backward 198(193.7) Iter 0943 | Time 2.7238(2.6866) | Loss 4.601198(4.588332) | NFE Forward 32(32.0) | NFE Backward 198(194.0) Iter 0944 | Time 2.7263(2.6893) | Loss 4.620930(4.590614) | NFE Forward 32(32.0) | NFE Backward 198(194.2) Iter 0945 | Time 2.7266(2.6920) | Loss 4.594364(4.590877) | NFE Forward 32(32.0) | NFE Backward 198(194.5) Iter 0946 | Time 2.7213(2.6940) | Loss 4.562251(4.588873) | NFE Forward 32(32.0) | NFE Backward 198(194.7) Iter 0947 | Time 2.7246(2.6961) | Loss 4.588236(4.588828) | NFE Forward 32(32.0) | NFE Backward 198(195.0) Iter 0948 | Time 2.7260(2.6982) | Loss 4.617544(4.590838) | NFE Forward 32(32.0) | NFE Backward 198(195.2) Iter 0949 | Time 2.7283(2.7003) | Loss 4.602602(4.591662) | NFE Forward 32(32.0) | NFE Backward 198(195.4) Iter 0950 | Time 2.7237(2.7020) | Loss 4.558817(4.589363) | NFE Forward 32(32.0) | NFE Backward 198(195.6) Iter 0951 | Time 2.7313(2.7040) | Loss 4.581663(4.588824) | NFE Forward 32(32.0) | NFE Backward 198(195.7) Iter 0952 | Time 2.7248(2.7055) | Loss 4.540500(4.585441) | NFE Forward 32(32.0) | NFE Backward 198(195.9) Iter 0953 | Time 2.7270(2.7070) | Loss 4.570562(4.584400) | NFE Forward 32(32.0) | NFE Backward 198(196.0) Iter 0954 | Time 2.7269(2.7084) | Loss 4.550517(4.582028) | NFE Forward 32(32.0) | NFE Backward 198(196.2) Iter 0955 | Time 2.7371(2.7104) | Loss 4.587893(4.582438) | NFE Forward 32(32.0) | NFE Backward 198(196.3) Iter 0956 | Time 2.7255(2.7115) | Loss 4.598687(4.583576) | NFE Forward 32(32.0) | NFE Backward 198(196.4) Iter 0957 | Time 2.7238(2.7123) | Loss 4.561402(4.582024) | NFE Forward 32(32.0) | NFE Backward 198(196.5) Iter 0958 | Time 2.7262(2.7133) | Loss 4.585487(4.582266) | NFE Forward 32(32.0) | NFE Backward 198(196.6) Iter 0959 | Time 2.7251(2.7141) | Loss 4.573889(4.581680) | NFE Forward 32(32.0) | NFE Backward 198(196.7) Iter 0960 | Time 2.7810(2.7188) | Loss 4.568540(4.580760) | NFE Forward 32(32.0) | NFE Backward 204(197.2) Iter 0961 | Time 2.7259(2.7193) | Loss 4.551837(4.578735) | NFE Forward 32(32.0) | NFE Backward 198(197.3) Iter 0962 | Time 2.7845(2.7239) | Loss 4.565935(4.577839) | NFE Forward 32(32.0) | NFE Backward 204(197.8) Iter 0963 | Time 2.7262(2.7240) | Loss 4.602060(4.579535) | NFE Forward 32(32.0) | NFE Backward 198(197.8) Iter 0964 | Time 2.7915(2.7287) | Loss 4.522926(4.575572) | NFE Forward 32(32.0) | NFE Backward 204(198.2) Iter 0965 | Time 2.7482(2.7301) | Loss 4.509095(4.570919) | NFE Forward 32(32.0) | NFE Backward 198(198.2) Iter 0966 | Time 2.7300(2.7301) | Loss 4.595165(4.572616) | NFE Forward 32(32.0) | NFE Backward 198(198.2) Iter 0967 | Time 2.7254(2.7298) | Loss 4.587974(4.573691) | NFE Forward 32(32.0) | NFE Backward 198(198.2) Iter 0968 | Time 2.7243(2.7294) | Loss 4.572640(4.573617) | NFE Forward 32(32.0) | NFE Backward 198(198.2) Iter 0969 | Time 2.7287(2.7293) | Loss 4.527419(4.570384) | NFE Forward 32(32.0) | NFE Backward 198(198.2) Iter 0970 | Time 2.7256(2.7291) | Loss 4.601124(4.572535) | NFE Forward 32(32.0) | NFE Backward 198(198.1) Iter 0971 | Time 2.7298(2.7291) | Loss 4.563646(4.571913) | NFE Forward 32(32.0) | NFE Backward 198(198.1) Iter 0972 | Time 2.7830(2.7329) | Loss 4.543092(4.569896) | NFE Forward 32(32.0) | NFE Backward 204(198.5) Iter 0973 | Time 2.7882(2.7368) | Loss 4.569229(4.569849) | NFE Forward 32(32.0) | NFE Backward 204(198.9) Iter 0974 | Time 2.7275(2.7361) | Loss 4.593182(4.571482) | NFE Forward 32(32.0) | NFE Backward 198(198.9) Iter 0975 | Time 2.7290(2.7356) | Loss 4.524783(4.568213) | NFE Forward 32(32.0) | NFE Backward 198(198.8) Iter 0976 | Time 2.7427(2.7361) | Loss 4.569974(4.568337) | NFE Forward 32(32.0) | NFE Backward 198(198.7) Iter 0977 | Time 2.7440(2.7367) | Loss 4.549925(4.567048) | NFE Forward 32(32.0) | NFE Backward 198(198.7) Iter 0978 | Time 2.7888(2.7403) | Loss 4.606359(4.569800) | NFE Forward 32(32.0) | NFE Backward 204(199.1) Iter 0979 | Time 2.7324(2.7398) | Loss 4.591538(4.571321) | NFE Forward 32(32.0) | NFE Backward 198(199.0) Iter 0980 | Time 2.7874(2.7431) | Loss 4.625446(4.575110) | NFE Forward 32(32.0) | NFE Backward 204(199.3) Iter 0981 | Time 2.7857(2.7461) | Loss 4.596976(4.576641) | NFE Forward 32(32.0) | NFE Backward 204(199.7) Iter 0982 | Time 2.7283(2.7448) | Loss 4.540820(4.574133) | NFE Forward 32(32.0) | NFE Backward 198(199.5) Iter 0983 | Time 2.7309(2.7439) | Loss 4.563597(4.573396) | NFE Forward 32(32.0) | NFE Backward 198(199.4) Iter 0984 | Time 2.7344(2.7432) | Loss 4.542668(4.571245) | NFE Forward 32(32.0) | NFE Backward 198(199.3) Iter 0985 | Time 2.7287(2.7422) | Loss 4.510025(4.566959) | NFE Forward 32(32.0) | NFE Backward 198(199.2) Iter 0986 | Time 2.7886(2.7454) | Loss 4.513718(4.563232) | NFE Forward 32(32.0) | NFE Backward 204(199.6) Iter 0987 | Time 2.7265(2.7441) | Loss 4.528303(4.560787) | NFE Forward 32(32.0) | NFE Backward 198(199.5) Iter 0988 | Time 2.7866(2.7471) | Loss 4.571104(4.561510) | NFE Forward 32(32.0) | NFE Backward 204(199.8) Iter 0989 | Time 2.7255(2.7456) | Loss 4.572364(4.562269) | NFE Forward 32(32.0) | NFE Backward 198(199.7) Iter 0990 | Time 2.7417(2.7453) | Loss 4.582394(4.563678) | NFE Forward 32(32.0) | NFE Backward 198(199.5) Iter 0991 | Time 2.8024(2.7493) | Loss 4.544945(4.562367) | NFE Forward 32(32.0) | NFE Backward 204(199.9) Iter 0992 | Time 2.7479(2.7492) | Loss 4.591417(4.564400) | NFE Forward 32(32.0) | NFE Backward 198(199.7) Iter 0993 | Time 2.7433(2.7488) | Loss 4.524448(4.561604) | NFE Forward 32(32.0) | NFE Backward 198(199.6) Iter 0994 | Time 2.7418(2.7483) | Loss 4.565515(4.561877) | NFE Forward 32(32.0) | NFE Backward 198(199.5) Iter 0995 | Time 2.8087(2.7525) | Loss 4.561641(4.561861) | NFE Forward 32(32.0) | NFE Backward 204(199.8) Iter 0996 | Time 2.8009(2.7559) | Loss 4.561960(4.561868) | NFE Forward 32(32.0) | NFE Backward 204(200.1) Iter 0997 | Time 2.8014(2.7591) | Loss 4.572731(4.562628) | NFE Forward 32(32.0) | NFE Backward 204(200.4) Density Loss 0.41956639289855957 Density Loss 0.400348424911499 Density Loss 0.41030028462409973 Density Loss 0.39142152667045593 Density Loss 0.41392529010772705 Density Loss 0.4176879823207855 Density Loss 0.41037556529045105 Density Loss 0.4031701982021332 Density Loss 0.41450798511505127 Density Loss 0.41370370984077454 Density Loss 0.40607839822769165 Density Loss 0.40714511275291443 Density Loss 0.41438817977905273 Density Loss 0.39539968967437744 Density Loss 0.42885345220565796 Density Loss 0.40864717960357666 Density Loss 0.40404441952705383 Density Loss 0.41303691267967224 Density Loss 0.4206524193286896 Density Loss 0.4140646457672119 Density Loss 0.4211447536945343 Density Loss 0.4110080897808075 Density Loss 0.4072974622249603 Density Loss 0.42506423592567444 Density Loss 0.40886202454566956 Density Loss 0.4167785048484802 Density Loss 0.41967079043388367 Density Loss 0.4191971719264984 Density Loss 0.4051292836666107 Density Loss 0.39866432547569275 Density Loss 0.41316482424736023 Density Loss 0.41697943210601807 Density Loss 0.404948890209198 Density Loss 0.40889498591423035 Density Loss 0.4222969710826874 Density Loss 0.4091481864452362 Density Loss 0.41323134303092957 Density Loss 0.4139295220375061 Density Loss 0.4259176552295685 Density Loss 0.42233026027679443 Density Loss 0.4097784161567688 Density Loss 0.4337080419063568 Density Loss 0.4222075045108795 Density Loss 0.40391284227371216 Density Loss 0.41103944182395935 Density Loss 0.4220496118068695 Density Loss 0.42235901951789856 Density Loss 0.4203159213066101 Density Loss 0.40522268414497375 Density Loss 0.4184694290161133 Density Loss 0.405798077583313 Density Loss 0.42142337560653687 Density Loss 0.4108923375606537 Density Loss 0.41580936312675476 Density Loss 0.41934365034103394 Density Loss 0.42170071601867676 Density Loss 0.41695961356163025 Density Loss 0.4065488278865814 Density Loss 0.4364114999771118 Density Loss 0.42080605030059814 Density Loss 0.419898122549057 Density Loss 0.42231059074401855 Density Loss 0.4203777313232422 Density Loss 0.4191795885562897 Density Loss 0.4216364622116089 Density Loss 0.40784603357315063 Density Loss 0.42753446102142334 Density Loss 0.42155417799949646 Density Loss 0.4174886643886566 Density Loss 0.40898269414901733 Density Loss 0.4255756139755249 Density Loss 0.4315449595451355 Density Loss 0.4293544888496399 Density Loss 0.42632240056991577 Density Loss 0.40815046429634094 Density Loss 0.4118505120277405 Density Loss 0.43051961064338684 Density Loss 0.42665809392929077 Density Loss 0.43049919605255127 Density Loss 0.41998007893562317 Density Loss 0.42547473311424255 Density Loss 0.43292152881622314 Density Loss 0.4195672273635864 Density Loss 0.4153519570827484 Density Loss 0.41658318042755127 Density Loss 0.41977497935295105 Density Loss 0.42669007182121277 Density Loss 0.42507895827293396 Density Loss 0.4318493902683258 Density Loss 0.4241548776626587 Density Loss 0.42210838198661804 Density Loss 0.4257117211818695 Density Loss 0.4291073679924011 Density Loss 0.42262473702430725 Density Loss 0.4299566149711609 Density Loss 0.41712841391563416 Density Loss 0.4205993115901947 Density Loss 0.4312349557876587 Density Loss 0.4434499740600586 Density Loss 0.4279860258102417 Density Loss 0.4351498782634735 Density Loss 0.41797885298728943 Density Loss 0.42689648270606995 Density Loss 0.441977322101593 Density Loss 0.42824530601501465 Density Loss 0.4273308515548706 Density Loss 0.4254894554615021 Density Loss 0.4421374797821045 Density Loss 0.4168483316898346 Density Loss 0.43339085578918457 Density Loss 0.44268855452537537 Density Loss 0.43354833126068115 Density Loss 0.42810505628585815 Density Loss 0.4283676743507385 Density Loss 0.4326910972595215 Density Loss 0.4270854890346527 Density Loss 0.43844226002693176 Density Loss 0.42532458901405334 Density Loss 0.4211476445198059 Density Loss 0.4286503791809082 Density Loss 0.4146074056625366 Density Loss 0.42286059260368347 Density Loss 0.4364018440246582 Density Loss 0.4324667751789093 Density Loss 0.4239119589328766 Density Loss 0.43678122758865356 Density Loss 0.42705219984054565 Density Loss 0.421652615070343 Density Loss 0.4193687438964844 Density Loss 0.42037007212638855 Density Loss 0.42323675751686096 Density Loss 0.42766883969306946 Density Loss 0.42798933386802673 Density Loss 0.43695423007011414 Density Loss 0.4475032091140747 Density Loss 0.43505075573921204 Density Loss 0.4345524311065674 Density Loss 0.42121511697769165 Density Loss 0.4323328137397766 Density Loss 0.42097577452659607 Density Loss 0.43410995602607727 Density Loss 0.4317871332168579 Density Loss 0.42632338404655457 Density Loss 0.4356110692024231 Density Loss 0.4309076964855194 Density Loss 0.43012815713882446 Density Loss 0.4286089241504669 Density Loss 0.4273931384086609 Density Loss 0.4426097571849823 Density Loss 0.44073328375816345 Density Loss 0.4410654604434967 Density Loss 0.42759785056114197 Density Loss 0.4472585916519165 Density Loss 0.41969504952430725 Density Loss 0.4329858720302582 Density Loss 0.43348410725593567 Density Loss 0.43029090762138367 Density Loss 0.4376709461212158 Density Loss 0.43164560198783875 Density Loss 0.4421582520008087 Density Loss 0.43101438879966736 Density Loss 0.4340737760066986 Density Loss 0.43591785430908203 Density Loss 0.43591395020484924 Density Loss 0.41946449875831604 Density Loss 0.44309574365615845 Density Loss 0.4410938322544098 Density Loss 0.4378035068511963 Density Loss 0.43501025438308716 Density Loss 0.441244512796402 Density Loss 0.4248485267162323 Density Loss 0.43800750374794006 Density Loss 0.4332834482192993 Density Loss 0.44245022535324097 Density Loss 0.4358176290988922 Density Loss 0.4416983425617218 Density Loss 0.43425917625427246 Density Loss 0.4409603774547577 Density Loss 0.4444236755371094 Density Loss 0.4444008767604828 Density Loss 0.4414699971675873 Density Loss 0.43203645944595337 Density Loss 0.44200581312179565 Density Loss 0.4456728994846344 Density Loss 0.43163496255874634 Density Loss 0.43812161684036255 Density Loss 0.44432634115219116 Density Loss 0.44473591446876526 Density Loss 0.44555047154426575 Density Loss 0.4384961724281311 Density Loss 0.4526212215423584 Density Loss 0.4364148676395416 Density Loss 0.4435241222381592 Density Loss 0.437578409910202 Density Loss 0.4231720566749573 Density Loss 0.45250019431114197 Density Loss 0.43990159034729004 Density Loss 0.4265674352645874 Density Loss 0.4410835802555084 Density Loss 0.4441390931606293 Density Loss 0.43451014161109924 Density Loss 0.4451174736022949 Density Loss 0.4502880275249481 Density Loss 0.4428701102733612 Density Loss 0.43126624822616577 Density Loss 0.43725359439849854 Density Loss 0.4313686490058899 Density Loss 0.42290887236595154 Density Loss 0.4492443799972534 Density Loss 0.4325845539569855 Density Loss 0.42008379101753235 Density Loss 0.4407268464565277 Density Loss 0.43706029653549194 Density Loss 0.45039159059524536 Density Loss 0.44708874821662903 Density Loss 0.4393995404243469 Density Loss 0.43571677803993225 Density Loss 0.4358876347541809 Density Loss 0.4401392936706543 Density Loss 0.4484124779701233 Density Loss 0.45450302958488464 Density Loss 0.4455207288265228 Density Loss 0.45966365933418274 Density Loss 0.4460254907608032 Density Loss 0.4491479992866516 Density Loss 0.4413047730922699 Density Loss 0.4409564733505249 Density Loss 0.4411602318286896 Density Loss 0.45452094078063965 Density Loss 0.4551241993904114 Density Loss 0.43189096450805664 Density Loss 0.44770145416259766 Density Loss 0.4266737699508667 Density Loss 0.44610658288002014 Density Loss 0.45802196860313416 Density Loss 0.44263777136802673 Density Loss 0.4540007710456848 Density Loss 0.44975730776786804 Density Loss 0.4472540020942688 Density Loss 0.4624861180782318 Density Loss 0.4472923278808594 Density Loss 0.46404892206192017 Density Loss 0.4498467743396759 Density Loss 0.44867321848869324 Density Loss 0.45152026414871216 Density Loss 0.4462090730667114 Density Loss 0.4408535063266754 Density Loss 0.44302502274513245 Density Loss 0.4461447596549988 Density Loss 0.4370726943016052 Density Loss 0.4332694709300995 Density Loss 0.4535726308822632 Iter 0998 | Time 2.8036(2.7622) | Loss 4.594349(4.564849) | NFE Forward 32(32.0) | NFE Backward 204(200.6) Iter 0999 | Time 2.7809(2.7635) | Loss 4.537234(4.562916) | NFE Forward 32(32.0) | NFE Backward 204(200.9) Iter 1000 | Time 2.7239(2.7608) | Loss 4.572667(4.563598) | NFE Forward 32(32.0) | NFE Backward 198(200.7) [TEST] Iter 1000 | Test Loss 4.565606 | NFE 32 Skipping vis as data dimension is >2 Iter 1001 | Time 2.7770(2.7619) | Loss 4.514576(4.560167) | NFE Forward 32(32.0) | NFE Backward 204(200.9) Iter 1002 | Time 2.7782(2.7630) | Loss 4.575545(4.561243) | NFE Forward 32(32.0) | NFE Backward 204(201.1) Iter 1003 | Time 2.7133(2.7595) | Loss 4.597102(4.563753) | NFE Forward 32(32.0) | NFE Backward 198(200.9) Iter 1004 | Time 2.7686(2.7602) | Loss 4.563985(4.563770) | NFE Forward 32(32.0) | NFE Backward 204(201.1) Iter 1005 | Time 2.7735(2.7611) | Loss 4.587363(4.565421) | NFE Forward 32(32.0) | NFE Backward 204(201.3) Iter 1006 | Time 2.7718(2.7619) | Loss 4.565230(4.565408) | NFE Forward 32(32.0) | NFE Backward 204(201.5) Iter 1007 | Time 2.7727(2.7626) | Loss 4.554226(4.564625) | NFE Forward 32(32.0) | NFE Backward 204(201.7) Iter 1008 | Time 2.7779(2.7637) | Loss 4.521390(4.561599) | NFE Forward 32(32.0) | NFE Backward 204(201.8) Iter 1009 | Time 2.7159(2.7604) | Loss 4.517414(4.558506) | NFE Forward 32(32.0) | NFE Backward 198(201.6) Iter 1010 | Time 2.7152(2.7572) | Loss 4.545860(4.557621) | NFE Forward 32(32.0) | NFE Backward 198(201.3) Iter 1011 | Time 2.7137(2.7542) | Loss 4.591354(4.559982) | NFE Forward 32(32.0) | NFE Backward 198(201.1) Iter 1012 | Time 2.7716(2.7554) | Loss 4.540809(4.558640) | NFE Forward 32(32.0) | NFE Backward 204(201.3) Iter 1013 | Time 2.7746(2.7567) | Loss 4.605666(4.561932) | NFE Forward 32(32.0) | NFE Backward 204(201.5) Iter 1014 | Time 2.7757(2.7580) | Loss 4.557418(4.561616) | NFE Forward 32(32.0) | NFE Backward 204(201.7) Iter 1015 | Time 2.7699(2.7589) | Loss 4.564585(4.561823) | NFE Forward 32(32.0) | NFE Backward 204(201.8) Iter 1016 | Time 2.7227(2.7563) | Loss 4.546104(4.560723) | NFE Forward 32(32.0) | NFE Backward 198(201.6) Iter 1017 | Time 2.7746(2.7576) | Loss 4.562832(4.560871) | NFE Forward 32(32.0) | NFE Backward 204(201.7) Iter 1018 | Time 2.7737(2.7587) | Loss 4.507828(4.557158) | NFE Forward 32(32.0) | NFE Backward 204(201.9) Iter 1019 | Time 2.7202(2.7560) | Loss 4.556908(4.557140) | NFE Forward 32(32.0) | NFE Backward 198(201.6) Iter 1020 | Time 2.7287(2.7541) | Loss 4.570930(4.558106) | NFE Forward 32(32.0) | NFE Backward 198(201.4) Iter 1021 | Time 2.7728(2.7554) | Loss 4.546783(4.557313) | NFE Forward 32(32.0) | NFE Backward 204(201.5) Iter 1022 | Time 2.7795(2.7571) | Loss 4.589585(4.559572) | NFE Forward 32(32.0) | NFE Backward 204(201.7) Iter 1023 | Time 2.7807(2.7588) | Loss 4.522407(4.556970) | NFE Forward 32(32.0) | NFE Backward 204(201.9) Iter 1024 | Time 2.7239(2.7563) | Loss 4.570228(4.557898) | NFE Forward 32(32.0) | NFE Backward 198(201.6) Iter 1025 | Time 2.7229(2.7540) | Loss 4.568040(4.558608) | NFE Forward 32(32.0) | NFE Backward 198(201.4) Iter 1026 | Time 2.7786(2.7557) | Loss 4.543559(4.557555) | NFE Forward 32(32.0) | NFE Backward 204(201.5) Iter 1027 | Time 2.7801(2.7574) | Loss 4.525725(4.555327) | NFE Forward 32(32.0) | NFE Backward 204(201.7) Iter 1028 | Time 2.7302(2.7555) | Loss 4.576235(4.556790) | NFE Forward 32(32.0) | NFE Backward 198(201.5) Iter 1029 | Time 2.7863(2.7577) | Loss 4.581320(4.558508) | NFE Forward 32(32.0) | NFE Backward 204(201.6) Iter 1030 | Time 2.7839(2.7595) | Loss 4.589354(4.560667) | NFE Forward 32(32.0) | NFE Backward 204(201.8) Iter 1031 | Time 2.7730(2.7605) | Loss 4.589346(4.562674) | NFE Forward 32(32.0) | NFE Backward 204(201.9) Iter 1032 | Time 2.7705(2.7612) | Loss 4.553514(4.562033) | NFE Forward 32(32.0) | NFE Backward 204(202.1) Iter 1033 | Time 2.7729(2.7620) | Loss 4.494319(4.557293) | NFE Forward 32(32.0) | NFE Backward 204(202.2) Iter 1034 | Time 2.7769(2.7630) | Loss 4.555759(4.557186) | NFE Forward 32(32.0) | NFE Backward 204(202.4) Iter 1035 | Time 2.7663(2.7633) | Loss 4.556180(4.557115) | NFE Forward 32(32.0) | NFE Backward 204(202.5) Iter 1036 | Time 2.7109(2.7596) | Loss 4.522163(4.554669) | NFE Forward 32(32.0) | NFE Backward 198(202.2) Iter 1037 | Time 2.7645(2.7599) | Loss 4.522335(4.552405) | NFE Forward 32(32.0) | NFE Backward 204(202.3) Iter 1038 | Time 2.7658(2.7603) | Loss 4.543546(4.551785) | NFE Forward 32(32.0) | NFE Backward 204(202.4) Iter 1039 | Time 2.7637(2.7606) | Loss 4.509645(4.548835) | NFE Forward 32(32.0) | NFE Backward 204(202.5) Iter 1040 | Time 2.7634(2.7608) | Loss 4.567635(4.550151) | NFE Forward 32(32.0) | NFE Backward 204(202.6) Iter 1041 | Time 2.7104(2.7572) | Loss 4.542739(4.549632) | NFE Forward 32(32.0) | NFE Backward 198(202.3) Iter 1042 | Time 2.7624(2.7576) | Loss 4.544460(4.549270) | NFE Forward 32(32.0) | NFE Backward 204(202.4) Iter 1043 | Time 2.7602(2.7578) | Loss 4.512158(4.546673) | NFE Forward 32(32.0) | NFE Backward 204(202.5) Iter 1044 | Time 2.7624(2.7581) | Loss 4.512099(4.544252) | NFE Forward 32(32.0) | NFE Backward 204(202.6) Iter 1045 | Time 2.7601(2.7583) | Loss 4.536722(4.543725) | NFE Forward 32(32.0) | NFE Backward 204(202.7) Iter 1046 | Time 2.7596(2.7583) | Loss 4.527181(4.542567) | NFE Forward 32(32.0) | NFE Backward 204(202.8) Iter 1047 | Time 2.7583(2.7583) | Loss 4.559706(4.543767) | NFE Forward 32(32.0) | NFE Backward 204(202.9) Iter 1048 | Time 2.7896(2.7605) | Loss 4.632055(4.549947) | NFE Forward 38(32.4) | NFE Backward 204(203.0) Iter 1049 | Time 2.7045(2.7566) | Loss 4.488336(4.545634) | NFE Forward 32(32.4) | NFE Backward 198(202.6) Iter 1050 | Time 2.7618(2.7570) | Loss 4.552024(4.546082) | NFE Forward 32(32.4) | NFE Backward 204(202.7) Iter 1051 | Time 2.7689(2.7578) | Loss 4.570754(4.547809) | NFE Forward 32(32.3) | NFE Backward 204(202.8) Iter 1052 | Time 2.7708(2.7587) | Loss 4.553167(4.548184) | NFE Forward 32(32.3) | NFE Backward 204(202.9) Iter 1053 | Time 2.7088(2.7552) | Loss 4.508487(4.545405) | NFE Forward 32(32.3) | NFE Backward 198(202.6) Iter 1054 | Time 2.7611(2.7556) | Loss 4.557537(4.546254) | NFE Forward 32(32.3) | NFE Backward 204(202.7) Iter 1055 | Time 2.8479(2.7621) | Loss 4.510732(4.543768) | NFE Forward 38(32.7) | NFE Backward 210(203.2) Iter 1056 | Time 2.7670(2.7624) | Loss 4.536531(4.543261) | NFE Forward 32(32.6) | NFE Backward 204(203.2) Iter 1057 | Time 2.7619(2.7624) | Loss 4.555174(4.544095) | NFE Forward 32(32.6) | NFE Backward 204(203.3) Iter 1058 | Time 2.7878(2.7642) | Loss 4.553329(4.544741) | NFE Forward 38(33.0) | NFE Backward 204(203.3) Iter 1059 | Time 2.7595(2.7639) | Loss 4.518323(4.542892) | NFE Forward 32(32.9) | NFE Backward 204(203.4) Iter 1060 | Time 2.7620(2.7637) | Loss 4.523531(4.541537) | NFE Forward 32(32.8) | NFE Backward 204(203.4) Iter 1061 | Time 2.7868(2.7653) | Loss 4.522912(4.540233) | NFE Forward 38(33.2) | NFE Backward 204(203.5) Iter 1062 | Time 2.7078(2.7613) | Loss 4.489423(4.536676) | NFE Forward 32(33.1) | NFE Backward 198(203.1) Iter 1063 | Time 2.7733(2.7621) | Loss 4.543707(4.537169) | NFE Forward 32(33.0) | NFE Backward 204(203.1) Iter 1064 | Time 2.7595(2.7620) | Loss 4.538571(4.537267) | NFE Forward 32(33.0) | NFE Backward 204(203.2) Iter 1065 | Time 2.7625(2.7620) | Loss 4.578029(4.540120) | NFE Forward 32(32.9) | NFE Backward 204(203.3) Iter 1066 | Time 2.8174(2.7659) | Loss 4.496967(4.537099) | NFE Forward 32(32.8) | NFE Backward 210(203.7) Iter 1067 | Time 2.7026(2.7614) | Loss 4.573649(4.539658) | NFE Forward 32(32.8) | NFE Backward 198(203.3) Iter 1068 | Time 2.8508(2.7677) | Loss 4.568003(4.541642) | NFE Forward 38(33.1) | NFE Backward 210(203.8) Iter 1069 | Time 2.8541(2.7737) | Loss 4.511767(4.539551) | NFE Forward 38(33.5) | NFE Backward 210(204.2) Iter 1070 | Time 2.7347(2.7710) | Loss 4.551649(4.540398) | NFE Forward 38(33.8) | NFE Backward 198(203.8) Iter 1071 | Time 2.7643(2.7705) | Loss 4.467531(4.535297) | NFE Forward 32(33.7) | NFE Backward 204(203.8) Iter 1072 | Time 2.7033(2.7658) | Loss 4.534340(4.535230) | NFE Forward 32(33.6) | NFE Backward 198(203.4) Iter 1073 | Time 2.7649(2.7658) | Loss 4.520100(4.534171) | NFE Forward 32(33.4) | NFE Backward 204(203.4) Iter 1074 | Time 2.8242(2.7699) | Loss 4.522561(4.533358) | NFE Forward 32(33.3) | NFE Backward 210(203.9) Iter 1075 | Time 2.7620(2.7693) | Loss 4.507596(4.531555) | NFE Forward 32(33.2) | NFE Backward 204(203.9) Iter 1076 | Time 2.7617(2.7688) | Loss 4.508928(4.529971) | NFE Forward 32(33.2) | NFE Backward 204(203.9) Iter 1077 | Time 2.8212(2.7724) | Loss 4.499390(4.527830) | NFE Forward 32(33.1) | NFE Backward 210(204.3) Iter 1078 | Time 2.8517(2.7780) | Loss 4.552518(4.529558) | NFE Forward 38(33.4) | NFE Backward 210(204.7) Iter 1079 | Time 2.8505(2.7831) | Loss 4.549869(4.530980) | NFE Forward 38(33.7) | NFE Backward 210(205.1) Iter 1080 | Time 2.7648(2.7818) | Loss 4.508342(4.529396) | NFE Forward 32(33.6) | NFE Backward 204(205.0) Iter 1081 | Time 2.7332(2.7784) | Loss 4.563317(4.531770) | NFE Forward 38(33.9) | NFE Backward 198(204.5) Iter 1082 | Time 2.7884(2.7791) | Loss 4.508441(4.530137) | NFE Forward 38(34.2) | NFE Backward 204(204.5) Iter 1083 | Time 2.8216(2.7821) | Loss 4.526504(4.529883) | NFE Forward 32(34.1) | NFE Backward 210(204.9) Iter 1084 | Time 2.8503(2.7868) | Loss 4.542248(4.530748) | NFE Forward 38(34.3) | NFE Backward 210(205.2) Iter 1085 | Time 2.8875(2.7939) | Loss 4.542603(4.531578) | NFE Forward 38(34.6) | NFE Backward 210(205.6) Iter 1086 | Time 2.8502(2.7978) | Loss 4.501306(4.529459) | NFE Forward 38(34.8) | NFE Backward 210(205.9) Iter 1087 | Time 2.7619(2.7953) | Loss 4.466123(4.525025) | NFE Forward 32(34.6) | NFE Backward 204(205.8) Iter 1088 | Time 2.8511(2.7992) | Loss 4.490609(4.522616) | NFE Forward 38(34.9) | NFE Backward 210(206.1) Iter 1089 | Time 2.8651(2.8038) | Loss 4.528549(4.523032) | NFE Forward 38(35.1) | NFE Backward 210(206.3) Iter 1090 | Time 2.8520(2.8072) | Loss 4.532950(4.523726) | NFE Forward 38(35.3) | NFE Backward 210(206.6) Iter 1091 | Time 2.8153(2.8078) | Loss 4.505532(4.522452) | NFE Forward 38(35.5) | NFE Backward 204(206.4) Iter 1092 | Time 2.8228(2.8088) | Loss 4.530762(4.523034) | NFE Forward 32(35.2) | NFE Backward 210(206.7) Iter 1093 | Time 2.8481(2.8116) | Loss 4.487484(4.520546) | NFE Forward 38(35.4) | NFE Backward 210(206.9) Iter 1094 | Time 2.7662(2.8084) | Loss 4.570267(4.524026) | NFE Forward 32(35.2) | NFE Backward 204(206.7) Iter 1095 | Time 2.8767(2.8132) | Loss 4.550589(4.525885) | NFE Forward 38(35.4) | NFE Backward 210(206.9) Iter 1096 | Time 2.8542(2.8161) | Loss 4.519147(4.525414) | NFE Forward 32(35.1) | NFE Backward 210(207.1) Iter 1097 | Time 2.8495(2.8184) | Loss 4.548004(4.526995) | NFE Forward 38(35.3) | NFE Backward 210(207.3) Iter 1098 | Time 2.7612(2.8144) | Loss 4.523193(4.526729) | NFE Forward 32(35.1) | NFE Backward 204(207.1) Iter 1099 | Time 2.8518(2.8170) | Loss 4.523011(4.526469) | NFE Forward 38(35.3) | NFE Backward 210(207.3) Iter 1100 | Time 2.7584(2.8129) | Loss 4.524315(4.526318) | NFE Forward 38(35.5) | NFE Backward 198(206.7) [TEST] Iter 1100 | Test Loss 4.528575 | NFE 38 Skipping vis as data dimension is >2 Iter 1101 | Time 2.8754(2.8173) | Loss 4.538145(4.527146) | NFE Forward 38(35.7) | NFE Backward 210(206.9) Iter 1102 | Time 2.8797(2.8216) | Loss 4.541679(4.528163) | NFE Forward 38(35.8) | NFE Backward 210(207.1) Iter 1103 | Time 2.8015(2.8202) | Loss 4.524684(4.527920) | NFE Forward 32(35.6) | NFE Backward 204(206.9) Iter 1104 | Time 2.9168(2.8270) | Loss 4.525517(4.527751) | NFE Forward 38(35.7) | NFE Backward 210(207.1) Iter 1105 | Time 2.9817(2.8378) | Loss 4.515893(4.526921) | NFE Forward 38(35.9) | NFE Backward 216(207.7) Iter 1106 | Time 2.9827(2.8480) | Loss 4.515845(4.526146) | NFE Forward 38(36.0) | NFE Backward 216(208.3) Iter 1107 | Time 2.8639(2.8491) | Loss 4.534083(4.526702) | NFE Forward 32(35.8) | NFE Backward 210(208.4) Iter 1108 | Time 2.8674(2.8504) | Loss 4.569757(4.529715) | NFE Forward 32(35.5) | NFE Backward 210(208.5) Iter 1109 | Time 2.8939(2.8534) | Loss 4.519157(4.528976) | NFE Forward 38(35.7) | NFE Backward 210(208.6) Iter 1110 | Time 2.9758(2.8620) | Loss 4.548557(4.530347) | NFE Forward 38(35.8) | NFE Backward 216(209.2) Iter 1111 | Time 2.9209(2.8661) | Loss 4.500707(4.528272) | NFE Forward 38(36.0) | NFE Backward 210(209.2) Iter 1112 | Time 2.9204(2.8699) | Loss 4.520455(4.527725) | NFE Forward 38(36.1) | NFE Backward 210(209.3) Iter 1113 | Time 2.8908(2.8714) | Loss 4.523555(4.527433) | NFE Forward 38(36.3) | NFE Backward 210(209.3) Iter 1114 | Time 2.9174(2.8746) | Loss 4.451326(4.522106) | NFE Forward 38(36.4) | NFE Backward 210(209.4) Iter 1115 | Time 2.8921(2.8758) | Loss 4.542595(4.523540) | NFE Forward 32(36.1) | NFE Backward 210(209.4) Iter 1116 | Time 2.9228(2.8791) | Loss 4.539339(4.524646) | NFE Forward 38(36.2) | NFE Backward 210(209.5) Iter 1117 | Time 2.9201(2.8820) | Loss 4.523835(4.524589) | NFE Forward 38(36.3) | NFE Backward 210(209.5) Iter 1118 | Time 2.9163(2.8844) | Loss 4.522086(4.524414) | NFE Forward 38(36.5) | NFE Backward 210(209.5) Iter 1119 | Time 2.9182(2.8867) | Loss 4.512846(4.523604) | NFE Forward 38(36.6) | NFE Backward 210(209.6) Iter 1120 | Time 2.9799(2.8933) | Loss 4.478127(4.520421) | NFE Forward 38(36.7) | NFE Backward 216(210.0) Iter 1121 | Time 2.9816(2.8994) | Loss 4.529918(4.521085) | NFE Forward 38(36.8) | NFE Backward 216(210.4) Iter 1122 | Time 2.9150(2.9005) | Loss 4.518353(4.520894) | NFE Forward 38(36.8) | NFE Backward 210(210.4) Iter 1123 | Time 2.9735(2.9056) | Loss 4.522241(4.520988) | NFE Forward 38(36.9) | NFE Backward 216(210.8) Iter 1124 | Time 2.9128(2.9061) | Loss 4.514284(4.520519) | NFE Forward 38(37.0) | NFE Backward 210(210.7) Iter 1125 | Time 2.9150(2.9068) | Loss 4.486502(4.518138) | NFE Forward 38(37.1) | NFE Backward 210(210.7) Iter 1126 | Time 2.9281(2.9083) | Loss 4.516208(4.518003) | NFE Forward 38(37.1) | NFE Backward 210(210.6) Iter 1127 | Time 2.9153(2.9087) | Loss 4.520564(4.518182) | NFE Forward 38(37.2) | NFE Backward 210(210.6) Iter 1128 | Time 2.9169(2.9093) | Loss 4.501484(4.517013) | NFE Forward 38(37.3) | NFE Backward 210(210.6) Iter 1129 | Time 2.9176(2.9099) | Loss 4.493052(4.515336) | NFE Forward 38(37.3) | NFE Backward 210(210.5) Iter 1130 | Time 2.9269(2.9111) | Loss 4.543677(4.517320) | NFE Forward 38(37.4) | NFE Backward 210(210.5) Iter 1131 | Time 2.9228(2.9119) | Loss 4.523383(4.517744) | NFE Forward 38(37.4) | NFE Backward 210(210.4) Iter 1132 | Time 2.7788(2.9026) | Loss 4.496749(4.516275) | NFE Forward 32(37.0) | NFE Backward 198(209.6) Iter 1133 | Time 2.9235(2.9040) | Loss 4.518143(4.516405) | NFE Forward 38(37.1) | NFE Backward 210(209.6) Iter 1134 | Time 2.9747(2.9090) | Loss 4.544599(4.518379) | NFE Forward 38(37.2) | NFE Backward 216(210.0) Iter 1135 | Time 2.9249(2.9101) | Loss 4.489592(4.516364) | NFE Forward 38(37.2) | NFE Backward 210(210.0) Iter 1136 | Time 2.9789(2.9149) | Loss 4.504163(4.515510) | NFE Forward 38(37.3) | NFE Backward 216(210.5) Iter 1137 | Time 2.9196(2.9153) | Loss 4.460423(4.511654) | NFE Forward 38(37.3) | NFE Backward 210(210.4) Iter 1138 | Time 2.9746(2.9194) | Loss 4.555848(4.514747) | NFE Forward 38(37.4) | NFE Backward 216(210.8) Iter 1139 | Time 2.9208(2.9195) | Loss 4.542709(4.516705) | NFE Forward 38(37.4) | NFE Backward 210(210.8) Iter 1140 | Time 2.9205(2.9196) | Loss 4.537355(4.518150) | NFE Forward 38(37.5) | NFE Backward 210(210.7) Iter 1141 | Time 2.9868(2.9243) | Loss 4.507532(4.517407) | NFE Forward 38(37.5) | NFE Backward 216(211.1) Iter 1142 | Time 2.9781(2.9281) | Loss 4.492004(4.515629) | NFE Forward 38(37.5) | NFE Backward 216(211.4) Iter 1143 | Time 2.9210(2.9276) | Loss 4.537669(4.517172) | NFE Forward 38(37.6) | NFE Backward 210(211.3) Iter 1144 | Time 2.8730(2.9237) | Loss 4.513054(4.516883) | NFE Forward 38(37.6) | NFE Backward 204(210.8) Iter 1145 | Time 2.9172(2.9233) | Loss 4.512102(4.516549) | NFE Forward 38(37.6) | NFE Backward 210(210.8) Iter 1146 | Time 2.8004(2.9147) | Loss 4.472503(4.513465) | NFE Forward 38(37.6) | NFE Backward 198(209.9) Iter 1147 | Time 2.9215(2.9152) | Loss 4.507669(4.513060) | NFE Forward 38(37.7) | NFE Backward 210(209.9) Iter 1148 | Time 2.9156(2.9152) | Loss 4.515608(4.513238) | NFE Forward 38(37.7) | NFE Backward 210(209.9) Iter 1149 | Time 2.9188(2.9154) | Loss 4.496051(4.512035) | NFE Forward 38(37.7) | NFE Backward 210(209.9) Iter 1150 | Time 2.9161(2.9155) | Loss 4.523712(4.512852) | NFE Forward 38(37.7) | NFE Backward 210(209.9) Iter 1151 | Time 2.9154(2.9155) | Loss 4.474625(4.510176) | NFE Forward 38(37.8) | NFE Backward 210(209.9) Iter 1152 | Time 2.9151(2.9155) | Loss 4.519102(4.510801) | NFE Forward 38(37.8) | NFE Backward 210(209.9) Iter 1153 | Time 2.9144(2.9154) | Loss 4.520514(4.511481) | NFE Forward 38(37.8) | NFE Backward 210(209.9) Iter 1154 | Time 2.9201(2.9157) | Loss 4.454560(4.507497) | NFE Forward 38(37.8) | NFE Backward 210(209.9) Iter 1155 | Time 2.9167(2.9158) | Loss 4.502367(4.507138) | NFE Forward 38(37.8) | NFE Backward 210(209.9) Iter 1156 | Time 2.9173(2.9159) | Loss 4.528269(4.508617) | NFE Forward 38(37.8) | NFE Backward 210(209.9) Iter 1157 | Time 2.9173(2.9160) | Loss 4.462116(4.505362) | NFE Forward 38(37.8) | NFE Backward 210(209.9) Iter 1158 | Time 2.9156(2.9160) | Loss 4.501666(4.505103) | NFE Forward 38(37.9) | NFE Backward 210(209.9) Iter 1159 | Time 2.9718(2.9199) | Loss 4.454255(4.501544) | NFE Forward 38(37.9) | NFE Backward 216(210.4) Iter 1160 | Time 2.9141(2.9195) | Loss 4.519133(4.502775) | NFE Forward 38(37.9) | NFE Backward 210(210.3) Iter 1161 | Time 2.9785(2.9236) | Loss 4.502234(4.502737) | NFE Forward 38(37.9) | NFE Backward 216(210.7) Iter 1162 | Time 2.7995(2.9149) | Loss 4.558053(4.506609) | NFE Forward 38(37.9) | NFE Backward 198(209.8) Iter 1163 | Time 2.9778(2.9193) | Loss 4.479885(4.504738) | NFE Forward 38(37.9) | NFE Backward 216(210.3) Iter 1164 | Time 2.9175(2.9192) | Loss 4.453631(4.501161) | NFE Forward 38(37.9) | NFE Backward 210(210.3) Iter 1165 | Time 2.9147(2.9189) | Loss 4.524029(4.502762) | NFE Forward 38(37.9) | NFE Backward 210(210.2) Iter 1166 | Time 2.9145(2.9186) | Loss 4.531660(4.504785) | NFE Forward 38(37.9) | NFE Backward 210(210.2) Iter 1167 | Time 2.9208(2.9187) | Loss 4.486846(4.503529) | NFE Forward 38(37.9) | NFE Backward 210(210.2) Iter 1168 | Time 2.9176(2.9186) | Loss 4.519719(4.504662) | NFE Forward 38(37.9) | NFE Backward 210(210.2) Iter 1169 | Time 2.9147(2.9184) | Loss 4.499124(4.504274) | NFE Forward 38(37.9) | NFE Backward 210(210.2) Iter 1170 | Time 2.9156(2.9182) | Loss 4.487756(4.503118) | NFE Forward 38(37.9) | NFE Backward 210(210.2) Iter 1171 | Time 2.9192(2.9182) | Loss 4.536762(4.505473) | NFE Forward 38(37.9) | NFE Backward 210(210.2) Iter 1172 | Time 2.9754(2.9222) | Loss 4.491795(4.504516) | NFE Forward 38(37.9) | NFE Backward 216(210.6) Iter 1173 | Time 2.9127(2.9216) | Loss 4.506409(4.504648) | NFE Forward 38(37.9) | NFE Backward 210(210.5) Iter 1174 | Time 2.9183(2.9213) | Loss 4.520177(4.505735) | NFE Forward 38(38.0) | NFE Backward 210(210.5) Iter 1175 | Time 2.9119(2.9207) | Loss 4.478849(4.503853) | NFE Forward 38(38.0) | NFE Backward 210(210.5) Iter 1176 | Time 2.9158(2.9203) | Loss 4.544164(4.506675) | NFE Forward 38(38.0) | NFE Backward 210(210.4) Iter 1177 | Time 2.9371(2.9215) | Loss 4.481516(4.504914) | NFE Forward 38(38.0) | NFE Backward 210(210.4) Iter 1178 | Time 2.9152(2.9211) | Loss 4.469662(4.502446) | NFE Forward 38(38.0) | NFE Backward 210(210.4) Iter 1179 | Time 2.9130(2.9205) | Loss 4.510532(4.503012) | NFE Forward 38(38.0) | NFE Backward 210(210.3) Iter 1180 | Time 2.9728(2.9242) | Loss 4.511880(4.503633) | NFE Forward 38(38.0) | NFE Backward 216(210.7) Iter 1181 | Time 2.9140(2.9235) | Loss 4.512070(4.504224) | NFE Forward 38(38.0) | NFE Backward 210(210.7) Iter 1182 | Time 2.9801(2.9274) | Loss 4.471908(4.501962) | NFE Forward 38(38.0) | NFE Backward 216(211.1) Iter 1183 | Time 2.9150(2.9266) | Loss 4.531281(4.504014) | NFE Forward 38(38.0) | NFE Backward 210(211.0) Iter 1184 | Time 2.9178(2.9259) | Loss 4.512589(4.504614) | NFE Forward 38(38.0) | NFE Backward 210(210.9) Iter 1185 | Time 2.9726(2.9292) | Loss 4.485399(4.503269) | NFE Forward 38(38.0) | NFE Backward 216(211.3) Iter 1186 | Time 2.9749(2.9324) | Loss 4.478105(4.501508) | NFE Forward 38(38.0) | NFE Backward 216(211.6) Iter 1187 | Time 2.9184(2.9314) | Loss 4.493775(4.500966) | NFE Forward 38(38.0) | NFE Backward 210(211.5) Iter 1188 | Time 2.9774(2.9346) | Loss 4.514885(4.501941) | NFE Forward 38(38.0) | NFE Backward 216(211.8) Iter 1189 | Time 2.9139(2.9332) | Loss 4.499242(4.501752) | NFE Forward 38(38.0) | NFE Backward 210(211.7) Iter 1190 | Time 2.9736(2.9360) | Loss 4.512989(4.502538) | NFE Forward 38(38.0) | NFE Backward 216(212.0) Iter 1191 | Time 2.9166(2.9347) | Loss 4.531407(4.504559) | NFE Forward 38(38.0) | NFE Backward 210(211.8) Iter 1192 | Time 2.9719(2.9373) | Loss 4.477264(4.502648) | NFE Forward 38(38.0) | NFE Backward 216(212.1) Iter 1193 | Time 2.9166(2.9358) | Loss 4.553344(4.506197) | NFE Forward 38(38.0) | NFE Backward 210(212.0) Iter 1194 | Time 2.9835(2.9392) | Loss 4.495901(4.505476) | NFE Forward 38(38.0) | NFE Backward 216(212.3) Iter 1195 | Time 2.9784(2.9419) | Loss 4.490796(4.504449) | NFE Forward 38(38.0) | NFE Backward 216(212.5) Iter 1196 | Time 2.9826(2.9448) | Loss 4.466456(4.501789) | NFE Forward 38(38.0) | NFE Backward 216(212.8) Iter 1197 | Time 2.9806(2.9473) | Loss 4.503176(4.501886) | NFE Forward 38(38.0) | NFE Backward 216(213.0) Iter 1198 | Time 2.9173(2.9452) | Loss 4.474424(4.499964) | NFE Forward 38(38.0) | NFE Backward 210(212.8) Iter 1199 | Time 2.9186(2.9433) | Loss 4.505722(4.500367) | NFE Forward 38(38.0) | NFE Backward 210(212.6) Iter 1200 | Time 2.9167(2.9414) | Loss 4.503331(4.500575) | NFE Forward 38(38.0) | NFE Backward 210(212.4) [TEST] Iter 1200 | Test Loss 4.462631 | NFE 38 Skipping vis as data dimension is >2 Iter 1201 | Time 2.9724(2.9436) | Loss 4.441529(4.496441) | NFE Forward 38(38.0) | NFE Backward 216(212.7) Iter 1202 | Time 2.9220(2.9421) | Loss 4.493988(4.496270) | NFE Forward 38(38.0) | NFE Backward 210(212.5) Iter 1203 | Time 2.9135(2.9401) | Loss 4.476356(4.494876) | NFE Forward 38(38.0) | NFE Backward 210(212.3) Iter 1204 | Time 2.9152(2.9384) | Loss 4.489758(4.494517) | NFE Forward 38(38.0) | NFE Backward 210(212.1) Iter 1205 | Time 2.9178(2.9369) | Loss 4.468195(4.492675) | NFE Forward 38(38.0) | NFE Backward 210(212.0) Iter 1206 | Time 2.9159(2.9354) | Loss 4.487465(4.492310) | NFE Forward 38(38.0) | NFE Backward 210(211.9) Iter 1207 | Time 2.9753(2.9382) | Loss 4.479034(4.491381) | NFE Forward 38(38.0) | NFE Backward 216(212.1) Iter 1208 | Time 2.9250(2.9373) | Loss 4.489626(4.491258) | NFE Forward 38(38.0) | NFE Backward 210(212.0) Iter 1209 | Time 2.9183(2.9360) | Loss 4.511865(4.492700) | NFE Forward 38(38.0) | NFE Backward 210(211.9) Iter 1210 | Time 2.9729(2.9386) | Loss 4.522509(4.494787) | NFE Forward 38(38.0) | NFE Backward 216(212.1) Iter 1211 | Time 2.9204(2.9373) | Loss 4.470545(4.493090) | NFE Forward 38(38.0) | NFE Backward 210(212.0) Iter 1212 | Time 2.9760(2.9400) | Loss 4.490049(4.492877) | NFE Forward 38(38.0) | NFE Backward 216(212.3) Iter 1213 | Time 2.9731(2.9423) | Loss 4.466419(4.491025) | NFE Forward 38(38.0) | NFE Backward 216(212.5) Iter 1214 | Time 2.9184(2.9406) | Loss 4.484635(4.490578) | NFE Forward 38(38.0) | NFE Backward 210(212.4) Iter 1215 | Time 2.9164(2.9389) | Loss 4.454579(4.488058) | NFE Forward 38(38.0) | NFE Backward 210(212.2) Iter 1216 | Time 2.9150(2.9373) | Loss 4.504141(4.489184) | NFE Forward 38(38.0) | NFE Backward 210(212.0) Iter 1217 | Time 2.9169(2.9358) | Loss 4.515510(4.491027) | NFE Forward 38(38.0) | NFE Backward 210(211.9) Iter 1218 | Time 2.9144(2.9343) | Loss 4.469304(4.489506) | NFE Forward 38(38.0) | NFE Backward 210(211.8) Iter 1219 | Time 2.9155(2.9330) | Loss 4.516204(4.491375) | NFE Forward 38(38.0) | NFE Backward 210(211.6) Iter 1220 | Time 2.9149(2.9318) | Loss 4.480187(4.490592) | NFE Forward 38(38.0) | NFE Backward 210(211.5) Iter 1221 | Time 2.8632(2.9270) | Loss 4.471244(4.489237) | NFE Forward 38(38.0) | NFE Backward 204(211.0) Iter 1222 | Time 2.9826(2.9308) | Loss 4.489479(4.489254) | NFE Forward 38(38.0) | NFE Backward 216(211.3) Iter 1223 | Time 2.9291(2.9307) | Loss 4.504163(4.490298) | NFE Forward 38(38.0) | NFE Backward 210(211.3) Iter 1224 | Time 2.8628(2.9260) | Loss 4.496797(4.490753) | NFE Forward 38(38.0) | NFE Backward 204(210.7) Iter 1225 | Time 2.9862(2.9302) | Loss 4.472862(4.489500) | NFE Forward 38(38.0) | NFE Backward 216(211.1) Iter 1226 | Time 2.9217(2.9296) | Loss 4.503783(4.490500) | NFE Forward 38(38.0) | NFE Backward 210(211.0) Iter 1227 | Time 2.9889(2.9337) | Loss 4.481196(4.489849) | NFE Forward 38(38.0) | NFE Backward 216(211.4) Iter 1228 | Time 2.9906(2.9377) | Loss 4.515381(4.491636) | NFE Forward 38(38.0) | NFE Backward 216(211.7) Iter 1229 | Time 2.9283(2.9371) | Loss 4.438027(4.487884) | NFE Forward 38(38.0) | NFE Backward 210(211.6) Iter 1230 | Time 2.9284(2.9365) | Loss 4.472986(4.486841) | NFE Forward 38(38.0) | NFE Backward 210(211.5) Iter 1231 | Time 2.9906(2.9402) | Loss 4.461880(4.485093) | NFE Forward 38(38.0) | NFE Backward 216(211.8) Iter 1232 | Time 2.9356(2.9399) | Loss 4.520016(4.487538) | NFE Forward 38(38.0) | NFE Backward 210(211.7) Iter 1233 | Time 2.9265(2.9390) | Loss 4.481116(4.487088) | NFE Forward 38(38.0) | NFE Backward 210(211.6) Iter 1234 | Time 2.9251(2.9380) | Loss 4.515835(4.489101) | NFE Forward 38(38.0) | NFE Backward 210(211.4) Iter 1235 | Time 2.9295(2.9374) | Loss 4.477420(4.488283) | NFE Forward 38(38.0) | NFE Backward 210(211.3) Iter 1236 | Time 2.9290(2.9368) | Loss 4.483198(4.487927) | NFE Forward 38(38.0) | NFE Backward 210(211.2) Iter 1237 | Time 2.8109(2.9280) | Loss 4.495748(4.488475) | NFE Forward 38(38.0) | NFE Backward 198(210.3) Iter 1238 | Time 2.9251(2.9278) | Loss 4.490994(4.488651) | NFE Forward 38(38.0) | NFE Backward 210(210.3) Iter 1239 | Time 2.9274(2.9278) | Loss 4.468628(4.487249) | NFE Forward 38(38.0) | NFE Backward 210(210.3) Iter 1240 | Time 2.9228(2.9274) | Loss 4.495276(4.487811) | NFE Forward 38(38.0) | NFE Backward 210(210.3) Iter 1241 | Time 2.9307(2.9277) | Loss 4.503221(4.488890) | NFE Forward 38(38.0) | NFE Backward 210(210.2) Iter 1242 | Time 2.9283(2.9277) | Loss 4.483331(4.488501) | NFE Forward 38(38.0) | NFE Backward 210(210.2) Iter 1243 | Time 2.9348(2.9282) | Loss 4.494545(4.488924) | NFE Forward 38(38.0) | NFE Backward 210(210.2) Iter 1244 | Time 2.9278(2.9282) | Loss 4.468465(4.487492) | NFE Forward 38(38.0) | NFE Backward 210(210.2) Iter 1245 | Time 2.9308(2.9284) | Loss 4.478000(4.486827) | NFE Forward 38(38.0) | NFE Backward 210(210.2) Iter 1246 | Time 2.8721(2.9244) | Loss 4.481883(4.486481) | NFE Forward 38(38.0) | NFE Backward 204(209.7) Iter 1247 | Time 2.9899(2.9290) | Loss 4.453144(4.484148) | NFE Forward 38(38.0) | NFE Backward 216(210.2) Density Loss 0.46015477180480957 Density Loss 0.44232621788978577 Density Loss 0.44348201155662537 Density Loss 0.44776564836502075 Density Loss 0.4340665936470032 Density Loss 0.451528400182724 Density Loss 0.4519622027873993 Density Loss 0.4475078880786896 Density Loss 0.4496905207633972 Density Loss 0.4461803734302521 Density Loss 0.441533625125885 Density Loss 0.4471896290779114 Density Loss 0.44965827465057373 Density Loss 0.45233210921287537 Density Loss 0.4547010660171509 Density Loss 0.44179245829582214 Density Loss 0.43569138646125793 Density Loss 0.44701552391052246 Density Loss 0.4398345649242401 Density Loss 0.44752317667007446 Density Loss 0.45035478472709656 Density Loss 0.4366382658481598 Density Loss 0.45760729908943176 Density Loss 0.44098615646362305 Density Loss 0.45887717604637146 Density Loss 0.44740644097328186 Density Loss 0.4498477280139923 Density Loss 0.4512098431587219 Density Loss 0.4480835795402527 Density Loss 0.4478171765804291 Density Loss 0.45940446853637695 Density Loss 0.4639582931995392 Density Loss 0.44738075137138367 Density Loss 0.44913655519485474 Density Loss 0.44923871755599976 Density Loss 0.4541596472263336 Density Loss 0.4424651265144348 Density Loss 0.4481566250324249 Density Loss 0.4452304542064667 Density Loss 0.45190662145614624 Density Loss 0.44855305552482605 Density Loss 0.4660190939903259 Density Loss 0.43695321679115295 Density Loss 0.4343281090259552 Density Loss 0.4572775661945343 Density Loss 0.4659873843193054 Density Loss 0.45567524433135986 Density Loss 0.4424057602882385 Density Loss 0.46285194158554077 Density Loss 0.45380881428718567 Density Loss 0.4691677987575531 Density Loss 0.4664192795753479 Density Loss 0.4551825225353241 Density Loss 0.47071462869644165 Density Loss 0.45745736360549927 Density Loss 0.4474879205226898 Density Loss 0.4468206465244293 Density Loss 0.46131062507629395 Density Loss 0.45333877205848694 Density Loss 0.4532736837863922 Density Loss 0.45616865158081055 Density Loss 0.4581071138381958 Density Loss 0.45298856496810913 Density Loss 0.45565080642700195 Density Loss 0.4457511603832245 Density Loss 0.45185619592666626 Density Loss 0.4473596513271332 Density Loss 0.4586593210697174 Density Loss 0.46549755334854126 Density Loss 0.44597917795181274 Density Loss 0.444701611995697 Density Loss 0.45117998123168945 Density Loss 0.45682889223098755 Density Loss 0.46026262640953064 Density Loss 0.442953884601593 Density Loss 0.46854734420776367 Density Loss 0.44896015524864197 Density Loss 0.44113415479660034 Density Loss 0.455584853887558 Density Loss 0.465671569108963 Density Loss 0.4530225992202759 Density Loss 0.4542332887649536 Density Loss 0.4579223096370697 Density Loss 0.46907493472099304 Density Loss 0.4466351866722107 Density Loss 0.45594725012779236 Density Loss 0.4525284171104431 Density Loss 0.4549460709095001 Density Loss 0.45811209082603455 Density Loss 0.438205361366272 Density Loss 0.458648145198822 Density Loss 0.4542621970176697 Density Loss 0.45608383417129517 Density Loss 0.4516734182834625 Density Loss 0.45619526505470276 Density Loss 0.45480331778526306 Density Loss 0.45217087864875793 Density Loss 0.440947949886322 Density Loss 0.4636439383029938 Density Loss 0.445761501789093 Density Loss 0.4514850378036499 Density Loss 0.45258083939552307 Density Loss 0.4498516619205475 Density Loss 0.45805037021636963 Density Loss 0.45284590125083923 Density Loss 0.4627065360546112 Density Loss 0.4458850026130676 Density Loss 0.4502679705619812 Density Loss 0.43647146224975586 Density Loss 0.46615856885910034 Density Loss 0.45876386761665344 Density Loss 0.45286282896995544 Density Loss 0.45497703552246094 Density Loss 0.4488116204738617 Density Loss 0.45810437202453613 Density Loss 0.4680001735687256 Density Loss 0.4606494903564453 Density Loss 0.465459942817688 Density Loss 0.4519605338573456 Density Loss 0.44641590118408203 Density Loss 0.461203932762146 Density Loss 0.4563419222831726 Density Loss 0.47097548842430115 Density Loss 0.449558824300766 Density Loss 0.45067185163497925 Density Loss 0.43831443786621094 Density Loss 0.46319687366485596 Density Loss 0.4666162133216858 Density Loss 0.46785643696784973 Density Loss 0.4632683992385864 Density Loss 0.4630257189273834 Density Loss 0.4690992534160614 Density Loss 0.45657816529273987 Density Loss 0.46337518095970154 Density Loss 0.447054386138916 Density Loss 0.4458266496658325 Density Loss 0.4424855709075928 Density Loss 0.4562230408191681 Density Loss 0.45470497012138367 Density Loss 0.4577927589416504 Density Loss 0.4683234691619873 Density Loss 0.44862380623817444 Density Loss 0.4601425528526306 Density Loss 0.4709632694721222 Density Loss 0.45257124304771423 Density Loss 0.4578922688961029 Density Loss 0.45562103390693665 Density Loss 0.4502369165420532 Density Loss 0.4583859145641327 Density Loss 0.46615877747535706 Density Loss 0.45873916149139404 Density Loss 0.4514245092868805 Density Loss 0.4464234709739685 Density Loss 0.46876463294029236 Density Loss 0.45773035287857056 Density Loss 0.4618434011936188 Density Loss 0.4781123995780945 Density Loss 0.44384726881980896 Density Loss 0.4589352309703827 Density Loss 0.4809008240699768 Density Loss 0.4560389518737793 Density Loss 0.4565548598766327 Density Loss 0.45967358350753784 Density Loss 0.45239919424057007 Density Loss 0.46581006050109863 Density Loss 0.4588262438774109 Density Loss 0.4503544867038727 Density Loss 0.4753522276878357 Density Loss 0.4801037013530731 Density Loss 0.45317474007606506 Density Loss 0.4571645259857178 Density Loss 0.4640164077281952 Density Loss 0.47616374492645264 Density Loss 0.4584495425224304 Density Loss 0.46152597665786743 Density Loss 0.46784070134162903 Density Loss 0.48004719614982605 Density Loss 0.4542059600353241 Density Loss 0.470443457365036 Density Loss 0.46768566966056824 Density Loss 0.4635125994682312 Density Loss 0.47061893343925476 Density Loss 0.47051218152046204 Density Loss 0.4660477936267853 Density Loss 0.4707704484462738 Density Loss 0.4724105894565582 Density Loss 0.467624306678772 Density Loss 0.46307480335235596 Density Loss 0.4649730324745178 Density Loss 0.4492763578891754 Density Loss 0.4809269309043884 Density Loss 0.47042468190193176 Density Loss 0.45478206872940063 Density Loss 0.45697206258773804 Density Loss 0.4766062796115875 Density Loss 0.48100125789642334 Density Loss 0.4694303572177887 Density Loss 0.45505017042160034 Density Loss 0.46154558658599854 Density Loss 0.4672011733055115 Density Loss 0.4641329348087311 Density Loss 0.4707210958003998 Density Loss 0.4525505602359772 Density Loss 0.465046763420105 Density Loss 0.4646407961845398 Density Loss 0.46028709411621094 Density Loss 0.4607378840446472 Density Loss 0.45177996158599854 Density Loss 0.4527258574962616 Density Loss 0.45301082730293274 Density Loss 0.4769868552684784 Density Loss 0.4632914960384369 Density Loss 0.45593127608299255 Density Loss 0.47437068819999695 Density Loss 0.4718831777572632 Density Loss 0.45466601848602295 Density Loss 0.4650188386440277 Density Loss 0.4808008670806885 Density Loss 0.4558565318584442 Density Loss 0.4501287043094635 Density Loss 0.46221327781677246 Density Loss 0.49004319310188293 Density Loss 0.468171089887619 Density Loss 0.46035024523735046 Density Loss 0.48067790269851685 Density Loss 0.46076005697250366 Density Loss 0.46002835035324097 Density Loss 0.4944051206111908 Density Loss 0.4569196105003357 Density Loss 0.4732125997543335 Density Loss 0.4620920717716217 Density Loss 0.4701964259147644 Density Loss 0.4823564291000366 Density Loss 0.45996424555778503 Density Loss 0.4533501863479614 Density Loss 0.4656471610069275 Density Loss 0.4684572219848633 Density Loss 0.4682452976703644 Density Loss 0.47410017251968384 Density Loss 0.47932615876197815 Density Loss 0.46274158358573914 Density Loss 0.4749678671360016 Density Loss 0.4629645049571991 Density Loss 0.4659246504306793 Density Loss 0.47851869463920593 Density Loss 0.47515320777893066 Density Loss 0.45901745557785034 Density Loss 0.4544401168823242 Density Loss 0.4703439474105835 Density Loss 0.4586043357849121 Density Loss 0.46316131949424744 Density Loss 0.45963361859321594 Density Loss 0.4788045883178711 Iter 1248 | Time 3.0633(2.9384) | Loss 4.449745(4.481739) | NFE Forward 38(38.0) | NFE Backward 222(211.0) Iter 1249 | Time 2.9323(2.9380) | Loss 4.476859(4.481398) | NFE Forward 38(38.0) | NFE Backward 210(210.9) Iter 1250 | Time 3.0130(2.9432) | Loss 4.461760(4.480023) | NFE Forward 38(38.0) | NFE Backward 210(210.9) Iter 1251 | Time 2.9213(2.9417) | Loss 4.480258(4.480040) | NFE Forward 38(38.0) | NFE Backward 210(210.8) Iter 1252 | Time 2.9216(2.9403) | Loss 4.508622(4.482040) | NFE Forward 38(38.0) | NFE Backward 210(210.8) Iter 1253 | Time 2.9170(2.9387) | Loss 4.506281(4.483737) | NFE Forward 38(38.0) | NFE Backward 210(210.7) Iter 1254 | Time 2.9190(2.9373) | Loss 4.462302(4.482237) | NFE Forward 38(38.0) | NFE Backward 210(210.7) Iter 1255 | Time 2.9229(2.9363) | Loss 4.471877(4.481511) | NFE Forward 38(38.0) | NFE Backward 210(210.6) Iter 1256 | Time 2.9219(2.9353) | Loss 4.474898(4.481049) | NFE Forward 38(38.0) | NFE Backward 210(210.6) Iter 1257 | Time 2.8659(2.9304) | Loss 4.484911(4.481319) | NFE Forward 38(38.0) | NFE Backward 204(210.1) Iter 1258 | Time 2.9383(2.9310) | Loss 4.482083(4.481372) | NFE Forward 38(38.0) | NFE Backward 210(210.1) Iter 1259 | Time 2.9308(2.9310) | Loss 4.482358(4.481441) | NFE Forward 38(38.0) | NFE Backward 210(210.1) Iter 1260 | Time 2.9214(2.9303) | Loss 4.492086(4.482187) | NFE Forward 38(38.0) | NFE Backward 210(210.1) Iter 1261 | Time 2.8621(2.9255) | Loss 4.426350(4.478278) | NFE Forward 38(38.0) | NFE Backward 204(209.7) Iter 1262 | Time 3.0458(2.9339) | Loss 4.465076(4.477354) | NFE Forward 38(38.0) | NFE Backward 222(210.5) Iter 1263 | Time 2.8773(2.9300) | Loss 4.434904(4.474382) | NFE Forward 38(38.0) | NFE Backward 204(210.1) Iter 1264 | Time 2.8768(2.9262) | Loss 4.480757(4.474829) | NFE Forward 38(38.0) | NFE Backward 204(209.6) Iter 1265 | Time 3.0416(2.9343) | Loss 4.479337(4.475144) | NFE Forward 38(38.0) | NFE Backward 222(210.5) Iter 1266 | Time 2.9035(2.9322) | Loss 4.464176(4.474376) | NFE Forward 38(38.0) | NFE Backward 210(210.5) Iter 1267 | Time 2.8420(2.9258) | Loss 4.501740(4.476292) | NFE Forward 38(38.0) | NFE Backward 204(210.0) Iter 1268 | Time 2.9004(2.9241) | Loss 4.457848(4.475001) | NFE Forward 38(38.0) | NFE Backward 210(210.0) Iter 1269 | Time 2.9993(2.9293) | Loss 4.450008(4.473251) | NFE Forward 38(38.0) | NFE Backward 210(210.0) Iter 1270 | Time 2.8982(2.9272) | Loss 4.460540(4.472361) | NFE Forward 38(38.0) | NFE Backward 210(210.0) Iter 1271 | Time 2.8879(2.9244) | Loss 4.418441(4.468587) | NFE Forward 38(38.0) | NFE Backward 210(210.0) Iter 1272 | Time 2.8892(2.9219) | Loss 4.502398(4.470954) | NFE Forward 38(38.0) | NFE Backward 210(210.0) Iter 1273 | Time 2.8335(2.9158) | Loss 4.481099(4.471664) | NFE Forward 38(38.0) | NFE Backward 204(209.6) Iter 1274 | Time 2.9471(2.9179) | Loss 4.452395(4.470315) | NFE Forward 38(38.0) | NFE Backward 216(210.0) Iter 1275 | Time 2.8874(2.9158) | Loss 4.463961(4.469870) | NFE Forward 38(38.0) | NFE Backward 210(210.0) Iter 1276 | Time 2.8887(2.9139) | Loss 4.466672(4.469647) | NFE Forward 38(38.0) | NFE Backward 210(210.0) Iter 1277 | Time 2.9472(2.9162) | Loss 4.484108(4.470659) | NFE Forward 38(38.0) | NFE Backward 216(210.5) Iter 1278 | Time 2.8894(2.9144) | Loss 4.496181(4.472445) | NFE Forward 38(38.0) | NFE Backward 210(210.4) Iter 1279 | Time 2.8253(2.9081) | Loss 4.458930(4.471499) | NFE Forward 38(38.0) | NFE Backward 204(210.0) Iter 1280 | Time 2.8291(2.9026) | Loss 4.488318(4.472677) | NFE Forward 38(38.0) | NFE Backward 204(209.6) Iter 1281 | Time 2.8794(2.9010) | Loss 4.487321(4.473702) | NFE Forward 38(38.0) | NFE Backward 210(209.6) Iter 1282 | Time 2.8795(2.8995) | Loss 4.497070(4.475338) | NFE Forward 38(38.0) | NFE Backward 210(209.6) Iter 1283 | Time 2.8799(2.8981) | Loss 4.443521(4.473110) | NFE Forward 38(38.0) | NFE Backward 210(209.6) Iter 1284 | Time 2.8345(2.8936) | Loss 4.495647(4.474688) | NFE Forward 38(38.0) | NFE Backward 204(209.2) Iter 1285 | Time 2.8209(2.8886) | Loss 4.453813(4.473227) | NFE Forward 38(38.0) | NFE Backward 204(208.9) Iter 1286 | Time 2.8817(2.8881) | Loss 4.454801(4.471937) | NFE Forward 38(38.0) | NFE Backward 210(209.0) Iter 1287 | Time 2.9410(2.8918) | Loss 4.456660(4.470868) | NFE Forward 38(38.0) | NFE Backward 216(209.5) Iter 1288 | Time 2.9949(2.8990) | Loss 4.444063(4.468991) | NFE Forward 38(38.0) | NFE Backward 222(210.3) Iter 1289 | Time 2.8774(2.8975) | Loss 4.442203(4.467116) | NFE Forward 38(38.0) | NFE Backward 210(210.3) Iter 1290 | Time 2.8798(2.8962) | Loss 4.501544(4.469526) | NFE Forward 38(38.0) | NFE Backward 210(210.3) Iter 1291 | Time 2.8760(2.8948) | Loss 4.424911(4.466403) | NFE Forward 38(38.0) | NFE Backward 210(210.3) Iter 1292 | Time 2.8776(2.8936) | Loss 4.434506(4.464170) | NFE Forward 38(38.0) | NFE Backward 210(210.2) Iter 1293 | Time 2.8236(2.8887) | Loss 4.421692(4.461197) | NFE Forward 38(38.0) | NFE Backward 204(209.8) Iter 1294 | Time 2.9439(2.8926) | Loss 4.448484(4.460307) | NFE Forward 38(38.0) | NFE Backward 216(210.2) Iter 1295 | Time 2.9992(2.9001) | Loss 4.482090(4.461832) | NFE Forward 38(38.0) | NFE Backward 222(211.1) Iter 1296 | Time 3.0159(2.9082) | Loss 4.467342(4.462217) | NFE Forward 38(38.0) | NFE Backward 222(211.8) Iter 1297 | Time 2.9001(2.9076) | Loss 4.467847(4.462612) | NFE Forward 38(38.0) | NFE Backward 210(211.7) Iter 1298 | Time 2.8960(2.9068) | Loss 4.456287(4.462169) | NFE Forward 38(38.0) | NFE Backward 210(211.6) Iter 1299 | Time 2.8565(2.9033) | Loss 4.509336(4.465471) | NFE Forward 38(38.0) | NFE Backward 204(211.1) Iter 1300 | Time 2.9742(2.9082) | Loss 4.470792(4.465843) | NFE Forward 38(38.0) | NFE Backward 216(211.4) [TEST] Iter 1300 | Test Loss 4.477488 | NFE 38 Skipping vis as data dimension is >2 Iter 1301 | Time 2.9057(2.9081) | Loss 4.495718(4.467934) | NFE Forward 38(38.0) | NFE Backward 210(211.3) Iter 1302 | Time 2.8497(2.9040) | Loss 4.442355(4.466144) | NFE Forward 38(38.0) | NFE Backward 204(210.8) Iter 1303 | Time 2.7362(2.8922) | Loss 4.475305(4.466785) | NFE Forward 38(38.0) | NFE Backward 192(209.5) Iter 1304 | Time 2.9121(2.8936) | Loss 4.511681(4.469928) | NFE Forward 38(38.0) | NFE Backward 210(209.5) Iter 1305 | Time 3.0503(2.9046) | Loss 4.457727(4.469074) | NFE Forward 38(38.0) | NFE Backward 222(210.4) Iter 1306 | Time 2.8947(2.9039) | Loss 4.506340(4.471682) | NFE Forward 38(38.0) | NFE Backward 210(210.4) Iter 1307 | Time 2.8910(2.9030) | Loss 4.520888(4.475127) | NFE Forward 38(38.0) | NFE Backward 210(210.3) Iter 1308 | Time 2.8373(2.8984) | Loss 4.483200(4.475692) | NFE Forward 38(38.0) | NFE Backward 204(209.9) Iter 1309 | Time 3.0176(2.9067) | Loss 4.459282(4.474543) | NFE Forward 38(38.0) | NFE Backward 222(210.7) Iter 1310 | Time 2.9342(2.9087) | Loss 4.413487(4.470269) | NFE Forward 38(38.0) | NFE Backward 210(210.7) Iter 1311 | Time 2.9560(2.9120) | Loss 4.449947(4.468847) | NFE Forward 38(38.0) | NFE Backward 216(211.1) Iter 1312 | Time 3.0103(2.9189) | Loss 4.475496(4.469312) | NFE Forward 38(38.0) | NFE Backward 222(211.8) Iter 1313 | Time 3.0096(2.9252) | Loss 4.452622(4.468144) | NFE Forward 38(38.0) | NFE Backward 222(212.5) Iter 1314 | Time 2.8368(2.9190) | Loss 4.452756(4.467067) | NFE Forward 38(38.0) | NFE Backward 204(211.9) Iter 1315 | Time 2.8292(2.9127) | Loss 4.467664(4.467108) | NFE Forward 38(38.0) | NFE Backward 204(211.4) Iter 1316 | Time 2.8307(2.9070) | Loss 4.477771(4.467855) | NFE Forward 38(38.0) | NFE Backward 204(210.9) Iter 1317 | Time 2.9476(2.9098) | Loss 4.427277(4.465014) | NFE Forward 38(38.0) | NFE Backward 216(211.2) Iter 1318 | Time 2.8871(2.9082) | Loss 4.478088(4.465930) | NFE Forward 38(38.0) | NFE Backward 210(211.1) Iter 1319 | Time 2.8485(2.9041) | Loss 4.443499(4.464359) | NFE Forward 38(38.0) | NFE Backward 204(210.6) Iter 1320 | Time 2.8356(2.8993) | Loss 4.446223(4.463090) | NFE Forward 38(38.0) | NFE Backward 204(210.2) Iter 1321 | Time 2.8899(2.8986) | Loss 4.506846(4.466153) | NFE Forward 38(38.0) | NFE Backward 210(210.2) Iter 1322 | Time 2.8952(2.8984) | Loss 4.527739(4.470464) | NFE Forward 38(38.0) | NFE Backward 210(210.2) Iter 1323 | Time 2.7760(2.8898) | Loss 4.513270(4.473460) | NFE Forward 38(38.0) | NFE Backward 198(209.3) Iter 1324 | Time 2.8640(2.8880) | Loss 4.476154(4.473649) | NFE Forward 38(38.0) | NFE Backward 204(208.9) Iter 1325 | Time 2.8397(2.8846) | Loss 4.436385(4.471040) | NFE Forward 38(38.0) | NFE Backward 204(208.6) Iter 1326 | Time 3.0273(2.8946) | Loss 4.457425(4.470087) | NFE Forward 38(38.0) | NFE Backward 222(209.5) Iter 1327 | Time 2.8942(2.8946) | Loss 4.400849(4.465241) | NFE Forward 38(38.0) | NFE Backward 210(209.6) Iter 1328 | Time 2.9018(2.8951) | Loss 4.470025(4.465575) | NFE Forward 38(38.0) | NFE Backward 210(209.6) Iter 1329 | Time 2.8957(2.8951) | Loss 4.418407(4.462274) | NFE Forward 38(38.0) | NFE Backward 210(209.6) Iter 1330 | Time 2.9607(2.8997) | Loss 4.495905(4.464628) | NFE Forward 38(38.0) | NFE Backward 216(210.1) Iter 1331 | Time 2.9041(2.9000) | Loss 4.448763(4.463517) | NFE Forward 38(38.0) | NFE Backward 210(210.1) Iter 1332 | Time 2.8346(2.8954) | Loss 4.471204(4.464055) | NFE Forward 38(38.0) | NFE Backward 204(209.6) Iter 1333 | Time 2.8947(2.8954) | Loss 4.435155(4.462032) | NFE Forward 38(38.0) | NFE Backward 210(209.7) Iter 1334 | Time 2.9182(2.8970) | Loss 4.476615(4.463053) | NFE Forward 38(38.0) | NFE Backward 210(209.7) Iter 1335 | Time 2.8510(2.8938) | Loss 4.486708(4.464709) | NFE Forward 38(38.0) | NFE Backward 204(209.3) Iter 1336 | Time 2.8656(2.8918) | Loss 4.450863(4.463740) | NFE Forward 38(38.0) | NFE Backward 204(208.9) Iter 1337 | Time 2.9075(2.8929) | Loss 4.446476(4.462531) | NFE Forward 38(38.0) | NFE Backward 204(208.6) Iter 1338 | Time 2.8780(2.8918) | Loss 4.441224(4.461040) | NFE Forward 38(38.0) | NFE Backward 204(208.3) Iter 1339 | Time 2.8626(2.8898) | Loss 4.465088(4.461323) | NFE Forward 38(38.0) | NFE Backward 204(208.0) Iter 1340 | Time 2.9177(2.8918) | Loss 4.428475(4.459024) | NFE Forward 38(38.0) | NFE Backward 210(208.1) Iter 1341 | Time 2.8564(2.8893) | Loss 4.481590(4.460603) | NFE Forward 38(38.0) | NFE Backward 204(207.8) Iter 1342 | Time 2.9702(2.8949) | Loss 4.426234(4.458198) | NFE Forward 38(38.0) | NFE Backward 216(208.4) Iter 1343 | Time 2.9044(2.8956) | Loss 4.474553(4.459342) | NFE Forward 38(38.0) | NFE Backward 210(208.5) Iter 1344 | Time 2.8473(2.8922) | Loss 4.424784(4.456923) | NFE Forward 38(38.0) | NFE Backward 204(208.2) Iter 1345 | Time 2.9067(2.8932) | Loss 4.471336(4.457932) | NFE Forward 38(38.0) | NFE Backward 210(208.3) Iter 1346 | Time 2.9213(2.8952) | Loss 4.508677(4.461484) | NFE Forward 38(38.0) | NFE Backward 210(208.4) Iter 1347 | Time 2.9105(2.8963) | Loss 4.447476(4.460504) | NFE Forward 38(38.0) | NFE Backward 210(208.5) Iter 1348 | Time 2.8557(2.8934) | Loss 4.485986(4.462288) | NFE Forward 38(38.0) | NFE Backward 204(208.2) Iter 1349 | Time 2.8458(2.8901) | Loss 4.458731(4.462039) | NFE Forward 38(38.0) | NFE Backward 204(207.9) Iter 1350 | Time 2.9667(2.8954) | Loss 4.439444(4.460457) | NFE Forward 38(38.0) | NFE Backward 216(208.5) Iter 1351 | Time 2.8593(2.8929) | Loss 4.448012(4.459586) | NFE Forward 38(38.0) | NFE Backward 204(208.2) Iter 1352 | Time 2.9236(2.8951) | Loss 4.462276(4.459774) | NFE Forward 38(38.0) | NFE Backward 210(208.3) Iter 1353 | Time 2.9820(2.9012) | Loss 4.467628(4.460324) | NFE Forward 38(38.0) | NFE Backward 216(208.8) Iter 1354 | Time 2.8596(2.8982) | Loss 4.482254(4.461859) | NFE Forward 38(38.0) | NFE Backward 204(208.5) Iter 1355 | Time 2.8715(2.8964) | Loss 4.473534(4.462676) | NFE Forward 38(38.0) | NFE Backward 204(208.2) Iter 1356 | Time 2.8628(2.8940) | Loss 4.423923(4.459963) | NFE Forward 38(38.0) | NFE Backward 204(207.9) Iter 1357 | Time 2.8640(2.8919) | Loss 4.454086(4.459552) | NFE Forward 38(38.0) | NFE Backward 204(207.6) Iter 1358 | Time 2.9726(2.8976) | Loss 4.442932(4.458389) | NFE Forward 38(38.0) | NFE Backward 216(208.2) Iter 1359 | Time 2.8602(2.8950) | Loss 4.471407(4.459300) | NFE Forward 38(38.0) | NFE Backward 204(207.9) Iter 1360 | Time 2.8592(2.8924) | Loss 4.464260(4.459647) | NFE Forward 38(38.0) | NFE Backward 204(207.6) Iter 1361 | Time 2.8436(2.8890) | Loss 4.461417(4.459771) | NFE Forward 38(38.0) | NFE Backward 204(207.4) Iter 1362 | Time 2.8420(2.8857) | Loss 4.514575(4.463607) | NFE Forward 38(38.0) | NFE Backward 204(207.1) Iter 1363 | Time 2.8396(2.8825) | Loss 4.431157(4.461336) | NFE Forward 38(38.0) | NFE Backward 204(206.9) Iter 1364 | Time 2.8949(2.8834) | Loss 4.455803(4.460948) | NFE Forward 38(38.0) | NFE Backward 210(207.1) Iter 1365 | Time 2.8337(2.8799) | Loss 4.475525(4.461969) | NFE Forward 38(38.0) | NFE Backward 204(206.9) Iter 1366 | Time 2.8386(2.8770) | Loss 4.415426(4.458711) | NFE Forward 38(38.0) | NFE Backward 204(206.7) Iter 1367 | Time 2.9553(2.8825) | Loss 4.445687(4.457799) | NFE Forward 38(38.0) | NFE Backward 216(207.4) Iter 1368 | Time 2.8394(2.8795) | Loss 4.436498(4.456308) | NFE Forward 38(38.0) | NFE Backward 204(207.1) Iter 1369 | Time 2.9110(2.8817) | Loss 4.424818(4.454104) | NFE Forward 38(38.0) | NFE Backward 210(207.3) Iter 1370 | Time 2.9853(2.8889) | Loss 4.431105(4.452494) | NFE Forward 38(38.0) | NFE Backward 216(207.9) Iter 1371 | Time 2.8498(2.8862) | Loss 4.415018(4.449871) | NFE Forward 38(38.0) | NFE Backward 204(207.7) Iter 1372 | Time 3.0296(2.8962) | Loss 4.501308(4.453471) | NFE Forward 38(38.0) | NFE Backward 222(208.7) Iter 1373 | Time 2.9651(2.9011) | Loss 4.430676(4.451876) | NFE Forward 38(38.0) | NFE Backward 216(209.2) Iter 1374 | Time 2.8410(2.8969) | Loss 4.435099(4.450701) | NFE Forward 38(38.0) | NFE Backward 204(208.8) Iter 1375 | Time 2.8333(2.8924) | Loss 4.467327(4.451865) | NFE Forward 38(38.0) | NFE Backward 204(208.5) Iter 1376 | Time 3.0061(2.9004) | Loss 4.456584(4.452195) | NFE Forward 38(38.0) | NFE Backward 222(209.4) Iter 1377 | Time 2.8284(2.8953) | Loss 4.409074(4.449177) | NFE Forward 38(38.0) | NFE Backward 204(209.0) Iter 1378 | Time 2.8290(2.8907) | Loss 4.432984(4.448043) | NFE Forward 38(38.0) | NFE Backward 204(208.7) Iter 1379 | Time 2.8278(2.8863) | Loss 4.470223(4.449596) | NFE Forward 38(38.0) | NFE Backward 204(208.4) Iter 1380 | Time 2.8263(2.8821) | Loss 4.484114(4.452012) | NFE Forward 38(38.0) | NFE Backward 204(208.1) Iter 1381 | Time 2.9442(2.8864) | Loss 4.436295(4.450912) | NFE Forward 38(38.0) | NFE Backward 216(208.6) Iter 1382 | Time 2.9669(2.8921) | Loss 4.501871(4.454479) | NFE Forward 38(38.0) | NFE Backward 216(209.1) Iter 1383 | Time 2.8956(2.8923) | Loss 4.485952(4.456682) | NFE Forward 38(38.0) | NFE Backward 210(209.2) Iter 1384 | Time 2.8934(2.8924) | Loss 4.480691(4.458363) | NFE Forward 38(38.0) | NFE Backward 210(209.2) Iter 1385 | Time 2.8379(2.8886) | Loss 4.445169(4.457439) | NFE Forward 38(38.0) | NFE Backward 204(208.9) Iter 1386 | Time 2.8373(2.8850) | Loss 4.455575(4.457309) | NFE Forward 38(38.0) | NFE Backward 204(208.5) Iter 1387 | Time 2.9591(2.8902) | Loss 4.429711(4.455377) | NFE Forward 38(38.0) | NFE Backward 216(209.1) Iter 1388 | Time 3.0196(2.8992) | Loss 4.455823(4.455408) | NFE Forward 38(38.0) | NFE Backward 222(210.0) Iter 1389 | Time 2.9269(2.9012) | Loss 4.451945(4.455166) | NFE Forward 38(38.0) | NFE Backward 216(210.4) Iter 1390 | Time 2.8409(2.8969) | Loss 4.426617(4.453167) | NFE Forward 38(38.0) | NFE Backward 204(209.9) Iter 1391 | Time 2.8406(2.8930) | Loss 4.440930(4.452311) | NFE Forward 38(38.0) | NFE Backward 204(209.5) Iter 1392 | Time 2.9604(2.8977) | Loss 4.487106(4.454746) | NFE Forward 38(38.0) | NFE Backward 216(210.0) Iter 1393 | Time 2.9832(2.9037) | Loss 4.456696(4.454883) | NFE Forward 38(38.0) | NFE Backward 216(210.4) Iter 1394 | Time 3.0190(2.9118) | Loss 4.429554(4.453110) | NFE Forward 38(38.0) | NFE Backward 222(211.2) Iter 1395 | Time 2.8494(2.9074) | Loss 4.488279(4.455572) | NFE Forward 38(38.0) | NFE Backward 204(210.7) Iter 1396 | Time 2.8925(2.9064) | Loss 4.414426(4.452691) | NFE Forward 38(38.0) | NFE Backward 204(210.2) Iter 1397 | Time 2.9622(2.9103) | Loss 4.460518(4.453239) | NFE Forward 38(38.0) | NFE Backward 216(210.6) Iter 1398 | Time 2.9611(2.9138) | Loss 4.447765(4.452856) | NFE Forward 38(38.0) | NFE Backward 216(211.0) Iter 1399 | Time 2.9618(2.9172) | Loss 4.452477(4.452830) | NFE Forward 38(38.0) | NFE Backward 216(211.4) Iter 1400 | Time 2.8487(2.9124) | Loss 4.421512(4.450637) | NFE Forward 38(38.0) | NFE Backward 204(210.9) [TEST] Iter 1400 | Test Loss 4.514085 | NFE 38 Skipping vis as data dimension is >2 Iter 1401 | Time 2.8997(2.9115) | Loss 4.417106(4.448290) | NFE Forward 38(38.0) | NFE Backward 210(210.8) Iter 1402 | Time 2.8431(2.9067) | Loss 4.462907(4.449313) | NFE Forward 38(38.0) | NFE Backward 204(210.3) Iter 1403 | Time 2.9972(2.9131) | Loss 4.464347(4.450366) | NFE Forward 38(38.0) | NFE Backward 216(210.7) Iter 1404 | Time 2.9865(2.9182) | Loss 4.469440(4.451701) | NFE Forward 38(38.0) | NFE Backward 216(211.1) Iter 1405 | Time 2.8492(2.9134) | Loss 4.499845(4.455071) | NFE Forward 38(38.0) | NFE Backward 204(210.6) Iter 1406 | Time 2.8439(2.9085) | Loss 4.453993(4.454996) | NFE Forward 38(38.0) | NFE Backward 204(210.1) Iter 1407 | Time 2.8468(2.9042) | Loss 4.481725(4.456867) | NFE Forward 38(38.0) | NFE Backward 204(209.7) Iter 1408 | Time 2.8519(2.9005) | Loss 4.442870(4.455887) | NFE Forward 38(38.0) | NFE Backward 204(209.3) Iter 1409 | Time 2.9604(2.9047) | Loss 4.466650(4.456640) | NFE Forward 38(38.0) | NFE Backward 216(209.8) Iter 1410 | Time 2.9648(2.9089) | Loss 4.458744(4.456787) | NFE Forward 38(38.0) | NFE Backward 216(210.2) Iter 1411 | Time 2.8422(2.9043) | Loss 4.435250(4.455280) | NFE Forward 38(38.0) | NFE Backward 204(209.8) Iter 1412 | Time 2.8437(2.9000) | Loss 4.439481(4.454174) | NFE Forward 38(38.0) | NFE Backward 204(209.4) Iter 1413 | Time 2.8578(2.8971) | Loss 4.450525(4.453919) | NFE Forward 38(38.0) | NFE Backward 204(209.0) Iter 1414 | Time 2.9649(2.9018) | Loss 4.484144(4.456034) | NFE Forward 38(38.0) | NFE Backward 216(209.5) Iter 1415 | Time 2.9626(2.9061) | Loss 4.453807(4.455878) | NFE Forward 38(38.0) | NFE Backward 216(209.9) Iter 1416 | Time 2.8391(2.9014) | Loss 4.414805(4.453003) | NFE Forward 38(38.0) | NFE Backward 204(209.5) Iter 1417 | Time 2.8432(2.8973) | Loss 4.469234(4.454139) | NFE Forward 38(38.0) | NFE Backward 204(209.1) Iter 1418 | Time 2.8385(2.8932) | Loss 4.457281(4.454359) | NFE Forward 38(38.0) | NFE Backward 204(208.8) Iter 1419 | Time 2.9587(2.8978) | Loss 4.437971(4.453212) | NFE Forward 38(38.0) | NFE Backward 216(209.3) Iter 1420 | Time 2.9607(2.9022) | Loss 4.427257(4.451395) | NFE Forward 38(38.0) | NFE Backward 216(209.8) Iter 1421 | Time 2.9621(2.9064) | Loss 4.440657(4.450644) | NFE Forward 38(38.0) | NFE Backward 216(210.2) Iter 1422 | Time 2.8977(2.9058) | Loss 4.418939(4.448424) | NFE Forward 38(38.0) | NFE Backward 210(210.2) Iter 1423 | Time 2.9790(2.9109) | Loss 4.492350(4.451499) | NFE Forward 38(38.0) | NFE Backward 216(210.6) Iter 1424 | Time 2.8394(2.9059) | Loss 4.478988(4.453423) | NFE Forward 38(38.0) | NFE Backward 204(210.1) Iter 1425 | Time 2.8395(2.9012) | Loss 4.452015(4.453325) | NFE Forward 38(38.0) | NFE Backward 204(209.7) Iter 1426 | Time 2.8153(2.8952) | Loss 4.409343(4.450246) | NFE Forward 38(38.0) | NFE Backward 204(209.3) Iter 1427 | Time 2.8412(2.8915) | Loss 4.430313(4.448851) | NFE Forward 38(38.0) | NFE Backward 204(208.9) Iter 1428 | Time 2.8429(2.8881) | Loss 4.465786(4.450036) | NFE Forward 38(38.0) | NFE Backward 204(208.6) Iter 1429 | Time 2.8638(2.8864) | Loss 4.464273(4.451033) | NFE Forward 38(38.0) | NFE Backward 204(208.3) Iter 1430 | Time 2.8546(2.8841) | Loss 4.425768(4.449264) | NFE Forward 38(38.0) | NFE Backward 204(208.0) Iter 1431 | Time 2.8541(2.8820) | Loss 4.430360(4.447941) | NFE Forward 38(38.0) | NFE Backward 204(207.7) Iter 1432 | Time 2.8495(2.8798) | Loss 4.475595(4.449877) | NFE Forward 38(38.0) | NFE Backward 204(207.4) Iter 1433 | Time 2.8670(2.8789) | Loss 4.458801(4.450501) | NFE Forward 38(38.0) | NFE Backward 204(207.2) Iter 1434 | Time 2.8760(2.8787) | Loss 4.389606(4.446239) | NFE Forward 38(38.0) | NFE Backward 204(207.0) Iter 1435 | Time 2.8695(2.8780) | Loss 4.469299(4.447853) | NFE Forward 38(38.0) | NFE Backward 204(206.8) Iter 1436 | Time 2.8654(2.8771) | Loss 4.463798(4.448969) | NFE Forward 38(38.0) | NFE Backward 204(206.6) Iter 1437 | Time 2.8646(2.8763) | Loss 4.462757(4.449934) | NFE Forward 38(38.0) | NFE Backward 204(206.4) Iter 1438 | Time 2.8667(2.8756) | Loss 4.466302(4.451080) | NFE Forward 38(38.0) | NFE Backward 204(206.2) Iter 1439 | Time 2.9840(2.8832) | Loss 4.460502(4.451739) | NFE Forward 38(38.0) | NFE Backward 216(206.9) Iter 1440 | Time 2.8667(2.8820) | Loss 4.467012(4.452809) | NFE Forward 38(38.0) | NFE Backward 204(206.7) Iter 1441 | Time 2.8641(2.8808) | Loss 4.426937(4.450998) | NFE Forward 38(38.0) | NFE Backward 204(206.5) Iter 1442 | Time 2.8665(2.8798) | Loss 4.476233(4.452764) | NFE Forward 38(38.0) | NFE Backward 204(206.3) Iter 1443 | Time 2.8638(2.8787) | Loss 4.466300(4.453712) | NFE Forward 38(38.0) | NFE Backward 204(206.2) Iter 1444 | Time 2.8694(2.8780) | Loss 4.470178(4.454864) | NFE Forward 38(38.0) | NFE Backward 204(206.0) Iter 1445 | Time 2.9879(2.8857) | Loss 4.452192(4.454677) | NFE Forward 38(38.0) | NFE Backward 216(206.7) Iter 1446 | Time 2.8637(2.8842) | Loss 4.409022(4.451481) | NFE Forward 38(38.0) | NFE Backward 204(206.5) Iter 1447 | Time 2.8619(2.8826) | Loss 4.435738(4.450379) | NFE Forward 38(38.0) | NFE Backward 204(206.4) Iter 1448 | Time 2.8635(2.8813) | Loss 4.399740(4.446834) | NFE Forward 38(38.0) | NFE Backward 204(206.2) Iter 1449 | Time 2.8693(2.8804) | Loss 4.430775(4.445710) | NFE Forward 38(38.0) | NFE Backward 204(206.0) Iter 1450 | Time 2.8719(2.8798) | Loss 4.431605(4.444723) | NFE Forward 38(38.0) | NFE Backward 204(205.9) Iter 1451 | Time 2.8708(2.8792) | Loss 4.457736(4.445634) | NFE Forward 38(38.0) | NFE Backward 204(205.8) Iter 1452 | Time 2.8684(2.8784) | Loss 4.467620(4.447173) | NFE Forward 38(38.0) | NFE Backward 204(205.6) Iter 1453 | Time 2.8648(2.8775) | Loss 4.497410(4.450689) | NFE Forward 38(38.0) | NFE Backward 204(205.5) Iter 1454 | Time 2.9892(2.8853) | Loss 4.448784(4.450556) | NFE Forward 38(38.0) | NFE Backward 216(206.3) Iter 1455 | Time 2.8786(2.8848) | Loss 4.463421(4.451457) | NFE Forward 38(38.0) | NFE Backward 204(206.1) Iter 1456 | Time 2.9885(2.8921) | Loss 4.413508(4.448800) | NFE Forward 38(38.0) | NFE Backward 216(206.8) Iter 1457 | Time 2.8674(2.8904) | Loss 4.448862(4.448804) | NFE Forward 38(38.0) | NFE Backward 204(206.6) Iter 1458 | Time 2.9843(2.8969) | Loss 4.459159(4.449529) | NFE Forward 38(38.0) | NFE Backward 216(207.3) Iter 1459 | Time 2.8677(2.8949) | Loss 4.446946(4.449348) | NFE Forward 38(38.0) | NFE Backward 204(207.0) Iter 1460 | Time 2.8645(2.8928) | Loss 4.423105(4.447511) | NFE Forward 38(38.0) | NFE Backward 204(206.8) Iter 1461 | Time 2.8644(2.8908) | Loss 4.462525(4.448562) | NFE Forward 38(38.0) | NFE Backward 204(206.6) Iter 1462 | Time 2.8717(2.8894) | Loss 4.436616(4.447726) | NFE Forward 38(38.0) | NFE Backward 204(206.4) Iter 1463 | Time 2.8628(2.8876) | Loss 4.438848(4.447105) | NFE Forward 38(38.0) | NFE Backward 204(206.3) Iter 1464 | Time 2.8725(2.8865) | Loss 4.466290(4.448448) | NFE Forward 38(38.0) | NFE Backward 204(206.1) Iter 1465 | Time 2.8759(2.8858) | Loss 4.466688(4.449724) | NFE Forward 38(38.0) | NFE Backward 204(206.0) Iter 1466 | Time 2.9954(2.8935) | Loss 4.459718(4.450424) | NFE Forward 38(38.0) | NFE Backward 216(206.7) Iter 1467 | Time 2.8591(2.8911) | Loss 4.430160(4.449005) | NFE Forward 38(38.0) | NFE Backward 204(206.5) Iter 1468 | Time 2.8565(2.8886) | Loss 4.420272(4.446994) | NFE Forward 38(38.0) | NFE Backward 204(206.3) Iter 1469 | Time 2.8579(2.8865) | Loss 4.468308(4.448486) | NFE Forward 38(38.0) | NFE Backward 204(206.1) Iter 1470 | Time 2.9767(2.8928) | Loss 4.477847(4.450541) | NFE Forward 38(38.0) | NFE Backward 216(206.8) Iter 1471 | Time 2.8565(2.8903) | Loss 4.447074(4.450299) | NFE Forward 38(38.0) | NFE Backward 204(206.6) Iter 1472 | Time 2.8547(2.8878) | Loss 4.441833(4.449706) | NFE Forward 38(38.0) | NFE Backward 204(206.4) Iter 1473 | Time 2.9383(2.8913) | Loss 4.518209(4.454501) | NFE Forward 38(38.0) | NFE Backward 210(206.7) Iter 1474 | Time 2.8513(2.8885) | Loss 4.469804(4.455573) | NFE Forward 38(38.0) | NFE Backward 204(206.5) Iter 1475 | Time 2.8394(2.8851) | Loss 4.439536(4.454450) | NFE Forward 38(38.0) | NFE Backward 204(206.3) Iter 1476 | Time 2.9697(2.8910) | Loss 4.444077(4.453724) | NFE Forward 38(38.0) | NFE Backward 216(207.0) Iter 1477 | Time 2.8410(2.8875) | Loss 4.415271(4.451032) | NFE Forward 38(38.0) | NFE Backward 204(206.8) Iter 1478 | Time 2.8482(2.8847) | Loss 4.449872(4.450951) | NFE Forward 38(38.0) | NFE Backward 204(206.6) Iter 1479 | Time 2.7864(2.8779) | Loss 4.403006(4.447595) | NFE Forward 38(38.0) | NFE Backward 198(206.0) Iter 1480 | Time 2.9560(2.8833) | Loss 4.455084(4.448119) | NFE Forward 38(38.0) | NFE Backward 216(206.7) Iter 1481 | Time 2.8337(2.8799) | Loss 4.411935(4.445586) | NFE Forward 38(38.0) | NFE Backward 204(206.5) Iter 1482 | Time 2.8482(2.8776) | Loss 4.439410(4.445154) | NFE Forward 38(38.0) | NFE Backward 204(206.3) Iter 1483 | Time 2.9684(2.8840) | Loss 4.433575(4.444343) | NFE Forward 38(38.0) | NFE Backward 216(207.0) Iter 1484 | Time 2.8437(2.8812) | Loss 4.414821(4.442277) | NFE Forward 38(38.0) | NFE Backward 204(206.8) Iter 1485 | Time 2.8715(2.8805) | Loss 4.431736(4.441539) | NFE Forward 38(38.0) | NFE Backward 204(206.6) Iter 1486 | Time 2.9903(2.8882) | Loss 4.442305(4.441593) | NFE Forward 38(38.0) | NFE Backward 216(207.3) Iter 1487 | Time 2.8714(2.8870) | Loss 4.407357(4.439196) | NFE Forward 38(38.0) | NFE Backward 204(207.0) Iter 1488 | Time 2.8664(2.8856) | Loss 4.431455(4.438654) | NFE Forward 38(38.0) | NFE Backward 204(206.8) Iter 1489 | Time 2.8713(2.8846) | Loss 4.414284(4.436948) | NFE Forward 38(38.0) | NFE Backward 204(206.6) Iter 1490 | Time 2.8708(2.8836) | Loss 4.452957(4.438069) | NFE Forward 38(38.0) | NFE Backward 204(206.4) Iter 1491 | Time 2.9924(2.8912) | Loss 4.414430(4.436414) | NFE Forward 38(38.0) | NFE Backward 216(207.1) Iter 1492 | Time 2.9889(2.8981) | Loss 4.386981(4.432954) | NFE Forward 38(38.0) | NFE Backward 216(207.7) Iter 1493 | Time 2.8504(2.8947) | Loss 4.420849(4.432106) | NFE Forward 38(38.0) | NFE Backward 204(207.5) Iter 1494 | Time 2.8422(2.8910) | Loss 4.448108(4.433226) | NFE Forward 38(38.0) | NFE Backward 204(207.2) Iter 1495 | Time 2.8421(2.8876) | Loss 4.438719(4.433611) | NFE Forward 38(38.0) | NFE Backward 204(207.0) Iter 1496 | Time 2.8571(2.8855) | Loss 4.400085(4.431264) | NFE Forward 38(38.0) | NFE Backward 204(206.8) Iter 1497 | Time 2.9096(2.8872) | Loss 4.443277(4.432105) | NFE Forward 38(38.0) | NFE Backward 210(207.0) Iter 1498 | Time 2.8399(2.8839) | Loss 4.467776(4.434602) | NFE Forward 38(38.0) | NFE Backward 204(206.8) Density Loss 0.47105681896209717 Density Loss 0.4652237296104431 Density Loss 0.46690449118614197 Density Loss 0.4775720536708832 Density Loss 0.4667896330356598 Density Loss 0.46735700964927673 Density Loss 0.48338332772254944 Density Loss 0.47832489013671875 Density Loss 0.472444623708725 Density Loss 0.46874183416366577 Density Loss 0.46667689085006714 Density Loss 0.462316632270813 Density Loss 0.4870871901512146 Density Loss 0.4481401741504669 Density Loss 0.4793243110179901 Density Loss 0.4830549657344818 Density Loss 0.47337058186531067 Density Loss 0.47296127676963806 Density Loss 0.4780517816543579 Density Loss 0.4843795895576477 Density Loss 0.4835936427116394 Density Loss 0.46794822812080383 Density Loss 0.4653604328632355 Density Loss 0.463851660490036 Density Loss 0.45709335803985596 Density Loss 0.4844958782196045 Density Loss 0.4597161114215851 Density Loss 0.4737776815891266 Density Loss 0.4531027674674988 Density Loss 0.4630637466907501 Density Loss 0.47804656624794006 Density Loss 0.46338963508605957 Density Loss 0.4664900302886963 Density Loss 0.4716891348361969 Density Loss 0.463875412940979 Density Loss 0.48140937089920044 Density Loss 0.4688482880592346 Density Loss 0.47525355219841003 Density Loss 0.4726364016532898 Density Loss 0.4588051736354828 Density Loss 0.47785258293151855 Density Loss 0.4648193120956421 Density Loss 0.48455730080604553 Density Loss 0.487295001745224 Density Loss 0.47836825251579285 Density Loss 0.47560614347457886 Density Loss 0.4721468687057495 Density Loss 0.46924036741256714 Density Loss 0.4614878296852112 Density Loss 0.4708843529224396 Density Loss 0.472723126411438 Density Loss 0.4838908612728119 Density Loss 0.4698618948459625 Density Loss 0.46511542797088623 Density Loss 0.4707304537296295 Density Loss 0.45784276723861694 Density Loss 0.47759169340133667 Density Loss 0.4845808446407318 Density Loss 0.46241337060928345 Density Loss 0.4834180474281311 Density Loss 0.48505038022994995 Density Loss 0.4647437036037445 Density Loss 0.4726004898548126 Density Loss 0.4761083722114563 Density Loss 0.4551854431629181 Density Loss 0.4638514518737793 Density Loss 0.46439236402511597 Density Loss 0.4613115191459656 Density Loss 0.4665128290653229 Density Loss 0.46715623140335083 Density Loss 0.47116711735725403 Density Loss 0.4913933575153351 Density Loss 0.46633490920066833 Density Loss 0.48273059725761414 Density Loss 0.4709407091140747 Density Loss 0.48649588227272034 Density Loss 0.4840378165245056 Density Loss 0.47838592529296875 Density Loss 0.4630143344402313 Density Loss 0.4748290479183197 Density Loss 0.4704382121562958 Density Loss 0.48750418424606323 Density Loss 0.48908787965774536 Density Loss 0.48561447858810425 Density Loss 0.4715576171875 Density Loss 0.48014020919799805 Density Loss 0.48705148696899414 Density Loss 0.4646832048892975 Density Loss 0.4628850519657135 Density Loss 0.46927163004875183 Density Loss 0.4847419857978821 Density Loss 0.46554285287857056 Density Loss 0.46872812509536743 Density Loss 0.49652737379074097 Density Loss 0.4683610200881958 Density Loss 0.48206424713134766 Density Loss 0.4892667829990387 Density Loss 0.46981149911880493 Density Loss 0.471244215965271 Density Loss 0.4868960678577423 Density Loss 0.47011756896972656 Density Loss 0.4768964648246765 Density Loss 0.4658571183681488 Density Loss 0.471183717250824 Density Loss 0.47388654947280884 Density Loss 0.470709890127182 Density Loss 0.45954519510269165 Density Loss 0.47834140062332153 Density Loss 0.4667600393295288 Density Loss 0.4945855438709259 Density Loss 0.46247294545173645 Density Loss 0.4553978443145752 Density Loss 0.45140761137008667 Density Loss 0.46456101536750793 Density Loss 0.4812720715999603 Density Loss 0.4838506281375885 Density Loss 0.47239500284194946 Density Loss 0.4761625826358795 Density Loss 0.477516233921051 Density Loss 0.4618105888366699 Density Loss 0.47365325689315796 Density Loss 0.4807560443878174 Density Loss 0.47785115242004395 Density Loss 0.480119913816452 Density Loss 0.49142733216285706 Density Loss 0.4652777314186096 Density Loss 0.4865970015525818 Density Loss 0.4504205882549286 Density Loss 0.4842342734336853 Density Loss 0.47841814160346985 Density Loss 0.4946427047252655 Density Loss 0.48412302136421204 Density Loss 0.48842889070510864 Density Loss 0.46852225065231323 Density Loss 0.4666716754436493 Density Loss 0.4720368981361389 Density Loss 0.4693606197834015 Density Loss 0.45960700511932373 Density Loss 0.4760323166847229 Density Loss 0.4760885238647461 Density Loss 0.4589863121509552 Density Loss 0.4739832282066345 Density Loss 0.4859234392642975 Density Loss 0.4677797257900238 Density Loss 0.47519442439079285 Density Loss 0.48611193895339966 Density Loss 0.47324138879776 Density Loss 0.47154518961906433 Density Loss 0.5049951076507568 Density Loss 0.47578978538513184 Density Loss 0.4949634075164795 Density Loss 0.48450779914855957 Density Loss 0.48500531911849976 Density Loss 0.46748679876327515 Density Loss 0.4848233163356781 Density Loss 0.47969284653663635 Density Loss 0.4888790547847748 Density Loss 0.47578710317611694 Density Loss 0.47654569149017334 Density Loss 0.47710517048835754 Density Loss 0.49013689160346985 Density Loss 0.4799199104309082 Density Loss 0.4793211817741394 Density Loss 0.48343679308891296 Density Loss 0.4658697247505188 Density Loss 0.47513318061828613 Density Loss 0.4666541814804077 Density Loss 0.47886794805526733 Density Loss 0.49857908487319946 Density Loss 0.47083789110183716 Density Loss 0.4716235101222992 Density Loss 0.46815061569213867 Density Loss 0.4716040790081024 Density Loss 0.4798519015312195 Density Loss 0.4783514142036438 Density Loss 0.49119389057159424 Density Loss 0.48546913266181946 Density Loss 0.4904024302959442 Density Loss 0.4905872046947479 Density Loss 0.47951000928878784 Density Loss 0.4830824136734009 Density Loss 0.46474775671958923 Density Loss 0.4674343764781952 Density Loss 0.4813029170036316 Density Loss 0.46456483006477356 Density Loss 0.47318539023399353 Density Loss 0.4764916002750397 Density Loss 0.4834735095500946 Density Loss 0.47202005982398987 Density Loss 0.47505390644073486 Density Loss 0.48520785570144653 Density Loss 0.4807536005973816 Density Loss 0.48336413502693176 Density Loss 0.4703138470649719 Density Loss 0.48058944940567017 Density Loss 0.46975642442703247 Density Loss 0.48195555806159973 Density Loss 0.4832395315170288 Density Loss 0.46883484721183777 Density Loss 0.48691296577453613 Density Loss 0.4934527277946472 Density Loss 0.4821171164512634 Density Loss 0.4791378676891327 Density Loss 0.4829532206058502 Density Loss 0.48387619853019714 Density Loss 0.48863327503204346 Density Loss 0.4700872302055359 Density Loss 0.48563700914382935 Density Loss 0.4782537817955017 Density Loss 0.48382097482681274 Density Loss 0.4776213765144348 Density Loss 0.4775595963001251 Density Loss 0.4850422143936157 Density Loss 0.4711836874485016 Density Loss 0.47125595808029175 Density Loss 0.48273876309394836 Density Loss 0.48350849747657776 Density Loss 0.4803708493709564 Density Loss 0.477222204208374 Density Loss 0.47145789861679077 Density Loss 0.5097503662109375 Density Loss 0.4858149290084839 Density Loss 0.4866052269935608 Density Loss 0.4847666919231415 Density Loss 0.4746406078338623 Density Loss 0.4742826521396637 Density Loss 0.48624539375305176 Density Loss 0.47740232944488525 Density Loss 0.48378095030784607 Density Loss 0.47848957777023315 Density Loss 0.48382946848869324 Density Loss 0.4708655774593353 Density Loss 0.45614051818847656 Density Loss 0.49412763118743896 Density Loss 0.47195595502853394 Density Loss 0.48282137513160706 Density Loss 0.4716576039791107 Density Loss 0.48046454787254333 Density Loss 0.4804708957672119 Density Loss 0.48005056381225586 Density Loss 0.4814135730266571 Density Loss 0.48329177498817444 Density Loss 0.48611965775489807 Density Loss 0.4851026237010956 Density Loss 0.4934585690498352 Density Loss 0.46289682388305664 Density Loss 0.49115321040153503 Density Loss 0.49850618839263916 Density Loss 0.5028679370880127 Density Loss 0.48708245158195496 Density Loss 0.4908502399921417 Density Loss 0.4801620543003082 Density Loss 0.49972298741340637 Iter 1499 | Time 2.8425(2.8810) | Loss 4.435004(4.434630) | NFE Forward 38(38.0) | NFE Backward 204(206.6) Iter 1500 | Time 2.9024(2.8825) | Loss 4.429532(4.434273) | NFE Forward 38(38.0) | NFE Backward 210(206.8) [TEST] Iter 1500 | Test Loss 4.449313 | NFE 38 Skipping vis as data dimension is >2 Iter 1501 | Time 2.7807(2.8753) | Loss 4.484761(4.437807) | NFE Forward 38(38.0) | NFE Backward 198(206.2) Iter 1502 | Time 2.8421(2.8730) | Loss 4.429320(4.437213) | NFE Forward 38(38.0) | NFE Backward 204(206.1) Iter 1503 | Time 2.9038(2.8752) | Loss 4.431491(4.436813) | NFE Forward 38(38.0) | NFE Backward 210(206.3) Iter 1504 | Time 2.9021(2.8771) | Loss 4.453099(4.437953) | NFE Forward 38(38.0) | NFE Backward 210(206.6) Iter 1505 | Time 2.8401(2.8745) | Loss 4.458953(4.439423) | NFE Forward 38(38.0) | NFE Backward 204(206.4) Iter 1506 | Time 2.9664(2.8809) | Loss 4.426262(4.438502) | NFE Forward 38(38.0) | NFE Backward 216(207.1) Iter 1507 | Time 2.8467(2.8785) | Loss 4.412311(4.436668) | NFE Forward 38(38.0) | NFE Backward 204(206.9) Iter 1508 | Time 2.8434(2.8761) | Loss 4.420503(4.435537) | NFE Forward 38(38.0) | NFE Backward 204(206.7) Iter 1509 | Time 2.9600(2.8819) | Loss 4.423814(4.434716) | NFE Forward 38(38.0) | NFE Backward 216(207.3) Iter 1510 | Time 2.8419(2.8791) | Loss 4.461096(4.436563) | NFE Forward 38(38.0) | NFE Backward 204(207.1) Iter 1511 | Time 2.8482(2.8770) | Loss 4.403235(4.434230) | NFE Forward 38(38.0) | NFE Backward 204(206.9) Iter 1512 | Time 2.8573(2.8756) | Loss 4.449172(4.435276) | NFE Forward 38(38.0) | NFE Backward 204(206.7) Iter 1513 | Time 2.8420(2.8732) | Loss 4.443044(4.435819) | NFE Forward 38(38.0) | NFE Backward 204(206.5) Iter 1514 | Time 2.8597(2.8723) | Loss 4.404656(4.433638) | NFE Forward 38(38.0) | NFE Backward 204(206.3) Iter 1515 | Time 2.9164(2.8754) | Loss 4.399328(4.431236) | NFE Forward 38(38.0) | NFE Backward 210(206.6) Iter 1516 | Time 2.8542(2.8739) | Loss 4.421424(4.430549) | NFE Forward 38(38.0) | NFE Backward 204(206.4) Iter 1517 | Time 2.9323(2.8780) | Loss 4.441412(4.431310) | NFE Forward 38(38.0) | NFE Backward 210(206.6) Iter 1518 | Time 2.9277(2.8815) | Loss 4.415034(4.430171) | NFE Forward 38(38.0) | NFE Backward 210(206.9) Iter 1519 | Time 2.9286(2.8848) | Loss 4.448340(4.431442) | NFE Forward 38(38.0) | NFE Backward 210(207.1) Iter 1520 | Time 2.9300(2.8879) | Loss 4.471436(4.434242) | NFE Forward 38(38.0) | NFE Backward 210(207.3) Iter 1521 | Time 2.9317(2.8910) | Loss 4.420775(4.433299) | NFE Forward 38(38.0) | NFE Backward 210(207.5) Iter 1522 | Time 2.9258(2.8934) | Loss 4.408154(4.431539) | NFE Forward 38(38.0) | NFE Backward 210(207.7) Iter 1523 | Time 2.9835(2.8997) | Loss 4.437277(4.431941) | NFE Forward 38(38.0) | NFE Backward 216(208.2) Iter 1524 | Time 2.9265(2.9016) | Loss 4.452131(4.433354) | NFE Forward 38(38.0) | NFE Backward 210(208.4) Iter 1525 | Time 2.9278(2.9034) | Loss 4.435479(4.433503) | NFE Forward 38(38.0) | NFE Backward 210(208.5) Iter 1526 | Time 2.9298(2.9053) | Loss 4.458641(4.435262) | NFE Forward 38(38.0) | NFE Backward 210(208.6) Iter 1527 | Time 2.9319(2.9071) | Loss 4.465658(4.437390) | NFE Forward 38(38.0) | NFE Backward 210(208.7) Iter 1528 | Time 2.9369(2.9092) | Loss 4.386899(4.433856) | NFE Forward 38(38.0) | NFE Backward 210(208.8) Iter 1529 | Time 2.8709(2.9065) | Loss 4.449911(4.434980) | NFE Forward 38(38.0) | NFE Backward 204(208.4) Density Loss 0.4930087625980377 Density Loss 0.47550711035728455 Density Loss 0.47090306878089905 Density Loss 0.47726795077323914 Density Loss 0.48117825388908386 Density Loss 0.4699195325374603 Density Loss 0.482453316450119 Density Loss 0.4911080300807953 Density Loss 0.4705395996570587 Density Loss 0.4743882715702057 Density Loss 0.4944862127304077 Density Loss 0.4748810827732086 Density Loss 0.47647958993911743 Density Loss 0.4918229281902313 Density Loss 0.472005158662796 Density Loss 0.4888821244239807 Density Loss 0.49159446358680725 Density Loss 0.48889070749282837 Density Loss 0.4992266595363617 Density Loss 0.47909021377563477 Density Loss 0.47893425822257996 Density Loss 0.47865957021713257 Density Loss 0.49052298069000244 Density Loss 0.4865018427371979 Density Loss 0.47552165389060974 Density Loss 0.5045297145843506 Density Loss 0.4802846610546112 Density Loss 0.4665418863296509 Density Loss 0.4865705966949463 Density Loss 0.48320749402046204 Density Loss 0.4734364151954651 Density Loss 0.47404101490974426 Traceback (most recent call last): File "/data/bioinfo/TrajectoryNet/TrajectoryNet-master/TrajectoryNet/main.py", line 519, in <module> main(args) File "/data/bioinfo/TrajectoryNet/TrajectoryNet-master/TrajectoryNet/main.py", line 502, in main train( File "/data/bioinfo/TrajectoryNet/TrajectoryNet-master/TrajectoryNet/main.py", line 254, in train loss = compute_loss(device, args, model, growth_model, logger, full_data) File "/data/bioinfo/TrajectoryNet/TrajectoryNet-master/TrajectoryNet/main.py", line 197, in compute_loss int_x = model(x, integration_times=int_t) File "/opt/mamba/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "/opt/mamba/lib/python3.10/site-packages/TrajectoryNet/lib/layers/container.py", line 21, in forward x = self.chain[i](x, reverse=reverse) File "/opt/mamba/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "/opt/mamba/lib/python3.10/site-packages/TrajectoryNet/lib/layers/cnf.py", line 53, in forward state_t = odeint( File "/opt/mamba/lib/python3.10/site-packages/torchdiffeq/_impl/adjoint.py", line 129, in odeint_adjoint ys = OdeintAdjointMethod.apply(*y0, func, t, flat_params, rtol, atol, method, options) File "/opt/mamba/lib/python3.10/site-packages/torch/autograd/function.py", line 506, in apply return super().apply(*args, **kwargs) # type: ignore[misc] File "/opt/mamba/lib/python3.10/site-packages/torchdiffeq/_impl/adjoint.py", line 18, in forward ans = odeint(func, y0, t, rtol=rtol, atol=atol, method=method, options=options) File "/opt/mamba/lib/python3.10/site-packages/torchdiffeq/_impl/odeint.py", line 76, in odeint solution = solver.integrate(t) File "/opt/mamba/lib/python3.10/site-packages/torchdiffeq/_impl/solvers.py", line 31, in integrate y = self.advance(t[i]) File "/opt/mamba/lib/python3.10/site-packages/torchdiffeq/_impl/dopri5.py", line 90, in advance self.rk_state = self._adaptive_dopri5_step(self.rk_state) KeyboardInterrupt Error while terminating subprocess (pid=1550):
Warning: Clipping dimensionality to 5 integrating backwards
4.2 导入轨迹推断的结果
(100, 3332, 5)
TrajectoryNet的输出 zs
是一个张量,对应于100个时间点,最后一个时间点有3332个细胞,每个细胞有5个主成分。为了使这些轨迹对应到PHATE嵌入,我们提取了存储在 phate_operator.graph.data_nu
中的主成分,并使用 StandardScaler
计算均值和标准差。然后,我们将 zs
缩放到前5个主成分的尺度和均值,并绘制从第24天到第27天的一些样本细胞的轨迹。
StandardScaler()In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
StandardScaler()
(16821, 100)

(100, 3332, 17845)
Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.43 seconds. Calculating affinities... Calculated affinities in 0.44 seconds. Calculated graph and diffusion operator in 0.88 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.90 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.42 seconds. Calculating affinities... Calculated affinities in 0.43 seconds. Calculated graph and diffusion operator in 0.85 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.87 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.43 seconds. Calculating affinities... Calculated affinities in 0.44 seconds. Calculated graph and diffusion operator in 0.87 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.89 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.42 seconds. Calculating affinities... Calculated affinities in 0.43 seconds. Calculated graph and diffusion operator in 0.86 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.88 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.44 seconds. Calculating affinities... Calculated affinities in 0.44 seconds. Calculated graph and diffusion operator in 0.89 seconds. Calculating imputation... Calculated imputation in 0.01 seconds. Calculated MAGIC in 0.91 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.42 seconds. Calculating affinities... Calculated affinities in 0.43 seconds. Calculated graph and diffusion operator in 0.86 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.89 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.42 seconds. Calculating affinities... Calculated affinities in 0.43 seconds. Calculated graph and diffusion operator in 0.87 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.89 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.42 seconds. Calculating affinities... Calculated affinities in 0.43 seconds. Calculated graph and diffusion operator in 0.86 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.88 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.42 seconds. Calculating affinities... Calculated affinities in 0.43 seconds. Calculated graph and diffusion operator in 0.86 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.88 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.42 seconds. Calculating affinities... Calculated affinities in 0.43 seconds. Calculated graph and diffusion operator in 0.86 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.88 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.43 seconds. Calculating affinities... Calculated affinities in 0.43 seconds. Calculated graph and diffusion operator in 0.87 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.89 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.42 seconds. Calculating affinities... Calculated affinities in 0.43 seconds. Calculated graph and diffusion operator in 0.86 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.88 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.43 seconds. Calculating affinities... Calculated affinities in 0.44 seconds. Calculated graph and diffusion operator in 0.88 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.90 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.42 seconds. Calculating affinities... Calculated affinities in 0.43 seconds. Calculated graph and diffusion operator in 0.86 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.88 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.42 seconds. Calculating affinities... Calculated affinities in 0.43 seconds. Calculated graph and diffusion operator in 0.86 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.88 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.42 seconds. Calculating affinities... Calculated affinities in 0.44 seconds. Calculated graph and diffusion operator in 0.86 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.89 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.42 seconds. Calculating affinities... Calculated affinities in 0.43 seconds. Calculated graph and diffusion operator in 0.86 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.88 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.42 seconds. Calculating affinities... Calculated affinities in 0.43 seconds. Calculated graph and diffusion operator in 0.86 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.88 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.43 seconds. Calculating affinities... Calculated affinities in 0.44 seconds. Calculated graph and diffusion operator in 0.88 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.90 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.43 seconds. Calculating affinities... Calculated affinities in 0.44 seconds. Calculated graph and diffusion operator in 0.87 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.89 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.43 seconds. Calculating affinities... Calculated affinities in 0.44 seconds. Calculated graph and diffusion operator in 0.88 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.90 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.43 seconds. Calculating affinities... Calculated affinities in 0.43 seconds. Calculated graph and diffusion operator in 0.87 seconds. Calculating imputation... Calculated imputation in 0.01 seconds. Calculated MAGIC in 0.89 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.43 seconds. Calculating affinities... Calculated affinities in 0.44 seconds. Calculated graph and diffusion operator in 0.87 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.90 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.43 seconds. Calculating affinities... Calculated affinities in 0.44 seconds. Calculated graph and diffusion operator in 0.87 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.89 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.44 seconds. Calculating affinities... Calculated affinities in 0.44 seconds. Calculated graph and diffusion operator in 0.89 seconds. Calculating imputation... Calculated imputation in 0.01 seconds. Calculated MAGIC in 0.91 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.43 seconds. Calculating affinities... Calculated affinities in 0.44 seconds. Calculated graph and diffusion operator in 0.87 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.89 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.44 seconds. Calculating affinities... Calculated affinities in 0.45 seconds. Calculated graph and diffusion operator in 0.89 seconds. Calculating imputation... Calculated imputation in 0.01 seconds. Calculated MAGIC in 0.91 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.43 seconds. Calculating affinities... Calculated affinities in 0.44 seconds. Calculated graph and diffusion operator in 0.88 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.90 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.43 seconds. Calculating affinities... Calculated affinities in 0.44 seconds. Calculated graph and diffusion operator in 0.88 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.90 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.43 seconds. Calculating affinities... Calculated affinities in 0.44 seconds. Calculated graph and diffusion operator in 0.88 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.90 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.43 seconds. Calculating affinities... Calculated affinities in 0.44 seconds. Calculated graph and diffusion operator in 0.88 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.90 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.43 seconds. Calculating affinities... Calculated affinities in 0.44 seconds. Calculated graph and diffusion operator in 0.87 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.90 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.43 seconds. Calculating affinities... Calculated affinities in 0.44 seconds. Calculated graph and diffusion operator in 0.88 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.90 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.43 seconds. Calculating affinities... Calculated affinities in 0.44 seconds. Calculated graph and diffusion operator in 0.88 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.90 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.44 seconds. Calculating affinities... Calculated affinities in 0.45 seconds. Calculated graph and diffusion operator in 0.89 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.91 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.44 seconds. Calculating affinities... Calculated affinities in 0.45 seconds. Calculated graph and diffusion operator in 0.89 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.91 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.44 seconds. Calculating affinities... Calculated affinities in 0.44 seconds. Calculated graph and diffusion operator in 0.89 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.91 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.43 seconds. Calculating affinities... Calculated affinities in 0.44 seconds. Calculated graph and diffusion operator in 0.87 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.89 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.44 seconds. Calculating affinities... Calculated affinities in 0.44 seconds. Calculated graph and diffusion operator in 0.88 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.90 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.43 seconds. Calculating affinities... Calculated affinities in 0.44 seconds. Calculated graph and diffusion operator in 0.88 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.90 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.44 seconds. Calculating affinities... Calculated affinities in 0.45 seconds. Calculated graph and diffusion operator in 0.89 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.91 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.43 seconds. Calculating affinities... Calculated affinities in 0.44 seconds. Calculated graph and diffusion operator in 0.88 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.90 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.43 seconds. Calculating affinities... Calculated affinities in 0.44 seconds. Calculated graph and diffusion operator in 0.88 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.90 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.43 seconds. Calculating affinities... Calculated affinities in 0.44 seconds. Calculated graph and diffusion operator in 0.87 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.89 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.43 seconds. Calculating affinities... Calculated affinities in 0.44 seconds. Calculated graph and diffusion operator in 0.88 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.90 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.43 seconds. Calculating affinities... Calculated affinities in 0.44 seconds. Calculated graph and diffusion operator in 0.87 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.89 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.43 seconds. Calculating affinities... Calculated affinities in 0.44 seconds. Calculated graph and diffusion operator in 0.89 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.91 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.43 seconds. Calculating affinities... Calculated affinities in 0.44 seconds. Calculated graph and diffusion operator in 0.88 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.90 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.43 seconds. Calculating affinities... Calculated affinities in 0.44 seconds. Calculated graph and diffusion operator in 0.88 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.90 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.43 seconds. Calculating affinities... Calculated affinities in 0.43 seconds. Calculated graph and diffusion operator in 0.87 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.89 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.43 seconds. Calculating affinities... Calculated affinities in 0.44 seconds. Calculated graph and diffusion operator in 0.88 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.90 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.44 seconds. Calculating affinities... Calculated affinities in 0.45 seconds. Calculated graph and diffusion operator in 0.89 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.92 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.44 seconds. Calculating affinities... Calculated affinities in 0.44 seconds. Calculated graph and diffusion operator in 0.89 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.91 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.44 seconds. Calculating affinities... Calculated affinities in 0.44 seconds. Calculated graph and diffusion operator in 0.88 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.90 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.44 seconds. Calculating affinities... Calculated affinities in 0.45 seconds. Calculated graph and diffusion operator in 0.89 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.91 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.44 seconds. Calculating affinities... Calculated affinities in 0.45 seconds. Calculated graph and diffusion operator in 0.90 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.92 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.44 seconds. Calculating affinities... Calculated affinities in 0.45 seconds. Calculated graph and diffusion operator in 0.90 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.92 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.44 seconds. Calculating affinities... Calculated affinities in 0.45 seconds. Calculated graph and diffusion operator in 0.90 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.92 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.45 seconds. Calculating affinities... Calculated affinities in 0.45 seconds. Calculated graph and diffusion operator in 0.91 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.93 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.45 seconds. Calculating affinities... Calculated affinities in 0.45 seconds. Calculated graph and diffusion operator in 0.91 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.93 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.45 seconds. Calculating affinities... Calculated affinities in 0.46 seconds. Calculated graph and diffusion operator in 0.91 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.93 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.45 seconds. Calculating affinities... Calculated affinities in 0.46 seconds. Calculated graph and diffusion operator in 0.92 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.94 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.45 seconds. Calculating affinities... Calculated affinities in 0.46 seconds. Calculated graph and diffusion operator in 0.92 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.94 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.45 seconds. Calculating affinities... Calculated affinities in 0.45 seconds. Calculated graph and diffusion operator in 0.91 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.93 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.45 seconds. Calculating affinities... Calculated affinities in 0.45 seconds. Calculated graph and diffusion operator in 0.91 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.93 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.45 seconds. Calculating affinities... Calculated affinities in 0.46 seconds. Calculated graph and diffusion operator in 0.92 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.94 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.45 seconds. Calculating affinities... Calculated affinities in 0.46 seconds. Calculated graph and diffusion operator in 0.92 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.94 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.45 seconds. Calculating affinities... Calculated affinities in 0.46 seconds. Calculated graph and diffusion operator in 0.92 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.94 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.46 seconds. Calculating affinities... Calculated affinities in 0.47 seconds. Calculated graph and diffusion operator in 0.93 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.95 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.46 seconds. Calculating affinities... Calculated affinities in 0.46 seconds. Calculated graph and diffusion operator in 0.93 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.95 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.46 seconds. Calculating affinities... Calculated affinities in 0.47 seconds. Calculated graph and diffusion operator in 0.94 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.96 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.47 seconds. Calculating affinities... Calculated affinities in 0.48 seconds. Calculated graph and diffusion operator in 0.96 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.98 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.48 seconds. Calculating affinities... Calculated affinities in 0.50 seconds. Calculated graph and diffusion operator in 0.99 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 1.01 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.47 seconds. Calculating affinities... Calculated affinities in 0.48 seconds. Calculated graph and diffusion operator in 0.96 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.98 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.48 seconds. Calculating affinities... Calculated affinities in 0.48 seconds. Calculated graph and diffusion operator in 0.97 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.99 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.47 seconds. Calculating affinities... Calculated affinities in 0.48 seconds. Calculated graph and diffusion operator in 0.96 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.98 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.49 seconds. Calculating affinities... Calculated affinities in 0.49 seconds. Calculated graph and diffusion operator in 0.99 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 1.01 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.48 seconds. Calculating affinities... Calculated affinities in 0.49 seconds. Calculated graph and diffusion operator in 0.97 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 0.99 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.48 seconds. Calculating affinities... Calculated affinities in 0.49 seconds. Calculated graph and diffusion operator in 0.98 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 1.00 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.49 seconds. Calculating affinities... Calculated affinities in 0.49 seconds. Calculated graph and diffusion operator in 0.99 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 1.01 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.50 seconds. Calculating affinities... Calculated affinities in 0.51 seconds. Calculated graph and diffusion operator in 1.01 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 1.03 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.50 seconds. Calculating affinities... Calculated affinities in 0.51 seconds. Calculated graph and diffusion operator in 1.01 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 1.03 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.51 seconds. Calculating affinities... Calculated affinities in 0.51 seconds. Calculated graph and diffusion operator in 1.03 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 1.05 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.51 seconds. Calculating affinities... Calculated affinities in 0.52 seconds. Calculated graph and diffusion operator in 1.04 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 1.06 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.52 seconds. Calculating affinities... Calculated affinities in 0.53 seconds. Calculated graph and diffusion operator in 1.05 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 1.07 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.51 seconds. Calculating affinities... Calculated affinities in 0.52 seconds. Calculated graph and diffusion operator in 1.04 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 1.06 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.53 seconds. Calculating affinities... Calculated affinities in 0.54 seconds. Calculated graph and diffusion operator in 1.07 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 1.09 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.53 seconds. Calculating affinities... Calculated affinities in 0.54 seconds. Calculated graph and diffusion operator in 1.08 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 1.11 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.53 seconds. Calculating affinities... Calculated affinities in 0.54 seconds. Calculated graph and diffusion operator in 1.08 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 1.10 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.55 seconds. Calculating affinities... Calculated affinities in 0.56 seconds. Calculated graph and diffusion operator in 1.12 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 1.14 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.56 seconds. Calculating affinities... Calculated affinities in 0.57 seconds. Calculated graph and diffusion operator in 1.13 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 1.15 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.56 seconds. Calculating affinities... Calculated affinities in 0.57 seconds. Calculated graph and diffusion operator in 1.14 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 1.17 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.57 seconds. Calculating affinities... Calculated affinities in 0.58 seconds. Calculated graph and diffusion operator in 1.16 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 1.18 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.58 seconds. Calculating affinities... Calculated affinities in 0.58 seconds. Calculated graph and diffusion operator in 1.17 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 1.19 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.58 seconds. Calculating affinities... Calculated affinities in 0.59 seconds. Calculated graph and diffusion operator in 1.19 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 1.21 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.59 seconds. Calculating affinities... Calculated affinities in 0.60 seconds. Calculated graph and diffusion operator in 1.20 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 1.22 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.60 seconds. Calculating affinities... Calculated affinities in 0.61 seconds. Calculated graph and diffusion operator in 1.22 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 1.24 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.60 seconds. Calculating affinities... Calculated affinities in 0.61 seconds. Calculated graph and diffusion operator in 1.22 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 1.24 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.61 seconds. Calculating affinities... Calculated affinities in 0.62 seconds. Calculated graph and diffusion operator in 1.24 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 1.26 seconds. Calculating MAGIC... Running MAGIC on 3332 cells and 68 genes. Calculating graph and diffusion operator... Calculating KNN search... Calculated KNN search in 0.61 seconds. Calculating affinities... Calculated affinities in 0.63 seconds. Calculated graph and diffusion operator in 1.25 seconds. Calculating imputation... Calculated imputation in 0.02 seconds. Calculated MAGIC in 1.27 seconds.
4.3 针对感兴趣的基因进行探究
然后,我们在这个数据集中定义了代表4种不同细胞群体的“终点基因”。我们对这些基因的原始计数运行MAGIC,并可视化了估算的表达。
/root/.local/lib/python3.10/site-packages/magic/magic.py:541: UserWarning: Running MAGIC.transform on different data to that which was used for MAGIC.fit may not produce sensible output, unless it comes from the same manifold. warnings.warn( Calculating imputation... Calculated imputation in 0.03 seconds.

4.4 绘制细胞轨迹
我们将 EBT_5
定义为来自最后一个时间点的计数,然后选择在这个最后时间点中表达最高的每个终点基因的前9个细胞。然后,我们绘制这些细胞的轨迹。
PDGFRA [1896 2642 488 748 664 697 1419 300 432] HAND1 [1959 3195 936 2133 375 668 501 2484 1476] SOX17 [2053 2595 2664 1988 2353 1129 2823 738 532] ONECUT2 [2920 555 2283 718 1275 2125 1838 2129 2277]


4.5 绘制基因表达动态模式
对于这36个细胞(每个终点选择9个基因),我们绘制在100个时间点上的基因表达变化,并确定不同的细胞子集如何显示不同的动态行为。








