Bohrium
robot
新建

空间站广场

论文
Notebooks
比赛
课程
Apps
我的主页
我的Notebooks
我的论文库
我的足迹

我的工作空间

任务
节点
文件
数据集
镜像
项目
数据库
公开
透视细胞命运的迷雾：TrajectoryNet 绘制人类胚胎干细胞发育轨迹
bioinformatics
生物信息学
Machine Learning
单细胞分析
从算法原理到代码实现
scRNA-seq
bioinformatics生物信息学Machine Learning单细胞分析从算法原理到代码实现scRNA-seq
liyongge
发布于 2023-11-10
推荐镜像 :Basic Image:ubuntu:22.04-py3.10-pytorch2.0
推荐机型 :c24_m92_2 * NVIDIA GPU B
赞 6
3
HEB_MoonK(v1)

透视细胞命运的迷雾：TrajectoryNet 绘制人类胚胎干细胞发育轨迹

“It is not our abilities that show what we truly are, it is our choices. ——Albus Percival Wulfric Brian Dumbledore”

在每个命运的抉择点，细胞是如何选择的呢？

在这个教程中，你将学习到 TrajectoryNet[1] 的基本原理，以及如何使用 TrajectoryNet 和 PHATE [2] （Potential of Heat-diffusion for Affinity-based Transition Embedding）的数据集来分析一个包含31,000个细胞的、为期27天的人类胚胎干细胞上（EB）分化时间序列。

本教程参考了 TrajectoryNet 原作者的讲解PPT，实际演示部分搬运自原作者提供的Notebook.

我们将遵循以下步骤:

0. TrajectoryNet的基本介绍
1. 加载10X数据
2. 预处理：过滤、标准化和转换
3. 使用PHATE嵌入数据
4. 使用TrajectoryNet建模细胞动态转换

参考文献：

  1. Tong, A., Huang, J., Wolf, G., van Dijk, D. & Krishnaswamy, S. TrajectoryNet: A Dynamic Optimal Transport Network for Modeling Cellular Dynamics. in Proceedings of the 37th International Conference on Machine Learning (2020). url
  2. Moon, K. R. et al. Visualizing structure and transitions in high-dimensional biological data. Nature Biotechnoly 37, 1482–1492 (2019). url

📖 上手指南
本文档可在 Bohrium Notebook 上直接运行。如果是在案例广场，你可以直接运行；如果是保存到自己的空间中，你可以点击界面上方按钮 开始连接，选择基础镜像 ubuntu:22.04-py3.10-pytorch2.0 及 c24_m92_2 * NVIDIA GPU B 节点配置，稍等片刻即可运行。

代码
文本

0. TrajectoryNet的基本介绍

代码
文本

0.0 TrajectoryNet模型介绍

代码
文本

TrajectoryNet 是一个用于模拟细胞动态的动态最优传输网络，而 PHATE 是一种基于亲和性转换嵌入的热扩散潜力方法，用于可视化高维生物数据中的结构和转换。这些工具在理解细胞如何随时间发展和分化方面非常有用，特别是在大规模细胞数据集中。
下面，我们简单介绍 TrajectoryNet。

  • 问题定义

在针对单细胞组学数据的分析中，一个重要的分析是找到各个细胞在分化或是演化上的先后顺序，也就是得到单细胞的拟时序（pseudotime）。拟时序（pseudotime）是一种生物学概念，用于研究单细胞生物在发育过程中所经历的时间轨迹。单细胞拟时序分析可以揭示细胞在不同阶段的功能特性和转录水平，为我们提供了深入了解细胞发育过程的机会。通过对单细胞数据的拟时序分析，我们可以描绘出细胞类型和状态的演变过程，从而为研究生物学过程，如组织发育、疾病进程和信号转导等提供重要信息。此外，拟时序分析还可以帮助我们发现潜在的细胞命运决定因素，为进一步实验研究和临床应用奠定基础。

普遍使用的单细胞的拟时序方法，通常是用已知的或是一些差异分析方法获得的标志物来判断拟时序。这样的方法存在bias，文献中也有失败案例，此外，这样的方法没有考虑真正的实验时间。

而 TrajectoryNet 利用了真实实验时间，对于每个时间点的单细胞数据，将其视为一个分布的采样结果，从而建模分布之间演化的模型。因此，TrajectoryNet 将单细胞的拟时序问题定义为一个 unbalanced dynamic transport 问题。

换而言之，TrajectoryNet 希望定义一种高效的、有意义的转换方程，建立两个分布之间的转换关系。

  • 连续标准化流

通常，针对分布之间的变换，我们会想到连续标准化流（Continuous Normalizing Flows，CNF）。CNF 通过连续变换将任意复杂分布映射到简单的基础分布（如高斯分布），从而实现有效的采样和概率密度计算。通常应用于图像生成、异常检测、强化学习、自然语言处理等领域。而将最优传输作为优化目标（视为一种正则化，如下右图），将诱导出一种分布之间的最优传输路径。我们可以直观地定义最优传输中的能量最小作为正则化，而定义了能量最小作为正则化的CNF，其实就是本文介绍的 TrajectoryNet 的主体。

  • 最优传输

从另一个角度来看，如果我们希望两个分布之间转换能实现能量最小化，那么我们可以要求 Wasserstein distance 最小。也就是实现最优传输（如图），其中两个分布 μ 和 ν 分别是连个边界条件。

这样两个边界条件的方程并不容易求解，如果我们将边界条件 ν 改为软性的限制加入loss中，则得到正则化的CNF（如图）。

  • TrajectoryNet

再将 RNAseq 数据中的生物学先验纳入考虑，包括密度、速度和生长，TrajectoryNet 就构建了一种使用动态最优传输，从而能够模拟系统中实体的连续动态和非线性路径的方法。

最终，TrajectoryNet 将损失函数定义为动态最优传输与生物学先验的结合。

其中，密度主要考虑到 KNN 密度估计，也就是细胞必须通过状态空间中允许的部分进行转换，比如，图中展示了一个不合理的路径，并提供了密度惩罚的公式。

速度主要考虑到RNA变化的方向，也就是细胞状态空间的速度向量，这主要参考了La Manno等人在2018年的Velocyto和Volker等人的ScVelo。

生长主要考虑到在状态转换之间，可能存在一些细胞状态的消失，所以 TrajectoryNet 允许不平衡的运输，从而允许细胞“死亡”而不是将它们移动到不合理的位置。

代码
文本

0.1 本教程数据集说明：时间序列的人类胚胎干细胞

研究人员使用特定的培养条件和技术来培养和分化人类胚胎干细胞（hESCs）。这些细胞在特定条件下可以分化为不同类型的细胞，模拟早期人类胚胎的发育过程。研究团队收集了不同时间点的细胞样本，以研究细胞分化的动态过程。此外，他们通过单细胞测序技术（10x Genomics）来分析这些细胞，这是一种强大的技术，可以提供对每个单独细胞的深入了解。

人类胚胎体分化的时间过程如下：低通道H1人胚胎干细胞（hESCs）在用Matrigel涂层的培养皿中维持，使用DMEM/F12-N2B27培养基和FGF2补充。为了形成胚胎体，细胞用Dispase处理，分离成小团块，并在加有20%胎牛血清的培养基中进行非粘附培养，这种血清经过筛选，适用于胚胎体分化。在为期27天的分化时间序列中，每隔3天收集样本。同时还包括一个未分化的hESC样本（图S7D）。通过qPCR验证了这些胚胎体培养中主要胚层标记物的诱导（数据未显示）。对于单细胞分析，胚胎体培养物被分离，通过流式细胞术（FACS）排序以去除双细胞和死细胞，并在10x基因组仪器上处理，生成cDNA文库，随后进行测序。小规模测序确定我们已经成功收集了大约31,000个细胞的数据，这些细胞在整个时间序列中均匀分布。

代码
文本

0.2 相关包的安装

代码
文本
[1]
! pip install TrajectoryNet
! pip install python-magic
! apt-get update
! apt-get install -y libmagic1
! pip install phate
! pip install scprep
! pip install --user magic-impute
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Requirement already satisfied: TrajectoryNet in /opt/mamba/lib/python3.10/site-packages (0.2.4)
Requirement already satisfied: POT>=0.7.0 in /opt/mamba/lib/python3.10/site-packages (from TrajectoryNet) (0.9.1)
Requirement already satisfied: matplotlib>=3.2.1 in /opt/mamba/lib/python3.10/site-packages (from TrajectoryNet) (3.8.1)
Requirement already satisfied: scanpy in /opt/mamba/lib/python3.10/site-packages (from TrajectoryNet) (1.9.6)
Requirement already satisfied: numpy>=1.18.4 in /opt/mamba/lib/python3.10/site-packages (from TrajectoryNet) (1.24.2)
Requirement already satisfied: scikit-learn>=0.23.1 in /opt/mamba/lib/python3.10/site-packages (from TrajectoryNet) (1.3.2)
Requirement already satisfied: torch>=1.5.0 in /opt/mamba/lib/python3.10/site-packages (from TrajectoryNet) (2.0.0+cu118)
Requirement already satisfied: torchdiffeq==0.0.1 in /opt/mamba/lib/python3.10/site-packages (from TrajectoryNet) (0.0.1)
Requirement already satisfied: scipy>=1.4.1 in /opt/mamba/lib/python3.10/site-packages (from TrajectoryNet) (1.10.1)
Collecting argparse
  Using cached https://pypi.tuna.tsinghua.edu.cn/packages/f2/94/3af39d34be01a24a6e65433d19e107099374224905f1e0cc6bbe1fd22a2f/argparse-1.4.0-py2.py3-none-any.whl (23 kB)
Requirement already satisfied: kiwisolver>=1.3.1 in /opt/mamba/lib/python3.10/site-packages (from matplotlib>=3.2.1->TrajectoryNet) (1.4.5)
Requirement already satisfied: fonttools>=4.22.0 in /opt/mamba/lib/python3.10/site-packages (from matplotlib>=3.2.1->TrajectoryNet) (4.44.0)
Requirement already satisfied: python-dateutil>=2.7 in /opt/mamba/lib/python3.10/site-packages (from matplotlib>=3.2.1->TrajectoryNet) (2.8.2)
Requirement already satisfied: pyparsing>=2.3.1 in /opt/mamba/lib/python3.10/site-packages (from matplotlib>=3.2.1->TrajectoryNet) (3.1.1)
Requirement already satisfied: pillow>=8 in /opt/mamba/lib/python3.10/site-packages (from matplotlib>=3.2.1->TrajectoryNet) (10.1.0)
Requirement already satisfied: cycler>=0.10 in /opt/mamba/lib/python3.10/site-packages (from matplotlib>=3.2.1->TrajectoryNet) (0.12.1)
Requirement already satisfied: packaging>=20.0 in /opt/mamba/lib/python3.10/site-packages (from matplotlib>=3.2.1->TrajectoryNet) (23.0)
Requirement already satisfied: contourpy>=1.0.1 in /opt/mamba/lib/python3.10/site-packages (from matplotlib>=3.2.1->TrajectoryNet) (1.2.0)
Requirement already satisfied: threadpoolctl>=2.0.0 in /opt/mamba/lib/python3.10/site-packages (from scikit-learn>=0.23.1->TrajectoryNet) (3.2.0)
Requirement already satisfied: joblib>=1.1.1 in /opt/mamba/lib/python3.10/site-packages (from scikit-learn>=0.23.1->TrajectoryNet) (1.3.2)
Requirement already satisfied: typing-extensions in /opt/mamba/lib/python3.10/site-packages (from torch>=1.5.0->TrajectoryNet) (4.5.0)
Requirement already satisfied: jinja2 in /opt/mamba/lib/python3.10/site-packages (from torch>=1.5.0->TrajectoryNet) (3.1.2)
Requirement already satisfied: networkx in /opt/mamba/lib/python3.10/site-packages (from torch>=1.5.0->TrajectoryNet) (3.0)
Requirement already satisfied: filelock in /opt/mamba/lib/python3.10/site-packages (from torch>=1.5.0->TrajectoryNet) (3.10.0)
Requirement already satisfied: triton==2.0.0 in /opt/mamba/lib/python3.10/site-packages (from torch>=1.5.0->TrajectoryNet) (2.0.0)
Requirement already satisfied: sympy in /opt/mamba/lib/python3.10/site-packages (from torch>=1.5.0->TrajectoryNet) (1.11.1)
Requirement already satisfied: cmake in /opt/mamba/lib/python3.10/site-packages (from triton==2.0.0->torch>=1.5.0->TrajectoryNet) (3.26.0)
Requirement already satisfied: lit in /opt/mamba/lib/python3.10/site-packages (from triton==2.0.0->torch>=1.5.0->TrajectoryNet) (15.0.7)
Requirement already satisfied: patsy in /opt/mamba/lib/python3.10/site-packages (from scanpy->TrajectoryNet) (0.5.3)
Requirement already satisfied: umap-learn>=0.3.10 in /opt/mamba/lib/python3.10/site-packages (from scanpy->TrajectoryNet) (0.5.4)
Requirement already satisfied: numba>=0.41.0 in /opt/mamba/lib/python3.10/site-packages (from scanpy->TrajectoryNet) (0.58.1)
Requirement already satisfied: session-info in /opt/mamba/lib/python3.10/site-packages (from scanpy->TrajectoryNet) (1.0.0)
Requirement already satisfied: statsmodels>=0.10.0rc2 in /opt/mamba/lib/python3.10/site-packages (from scanpy->TrajectoryNet) (0.14.0)
Requirement already satisfied: seaborn!=0.13.0 in /opt/mamba/lib/python3.10/site-packages (from scanpy->TrajectoryNet) (0.12.2)
Requirement already satisfied: pandas!=2.1.2,>=1.1.1 in /opt/mamba/lib/python3.10/site-packages (from scanpy->TrajectoryNet) (1.5.3)
Requirement already satisfied: h5py>=3 in /opt/mamba/lib/python3.10/site-packages (from scanpy->TrajectoryNet) (3.8.0)
Requirement already satisfied: tqdm in /opt/mamba/lib/python3.10/site-packages (from scanpy->TrajectoryNet) (4.64.1)
Requirement already satisfied: anndata>=0.7.4 in /opt/mamba/lib/python3.10/site-packages (from scanpy->TrajectoryNet) (0.10.3)
Requirement already satisfied: natsort in /opt/mamba/lib/python3.10/site-packages (from scanpy->TrajectoryNet) (8.4.0)
Requirement already satisfied: exceptiongroup in /opt/mamba/lib/python3.10/site-packages (from anndata>=0.7.4->scanpy->TrajectoryNet) (1.1.3)
Requirement already satisfied: array-api-compat in /opt/mamba/lib/python3.10/site-packages (from anndata>=0.7.4->scanpy->TrajectoryNet) (1.4)
Requirement already satisfied: llvmlite<0.42,>=0.41.0dev0 in /opt/mamba/lib/python3.10/site-packages (from numba>=0.41.0->scanpy->TrajectoryNet) (0.41.1)
Requirement already satisfied: pytz>=2020.1 in /opt/mamba/lib/python3.10/site-packages (from pandas!=2.1.2,>=1.1.1->scanpy->TrajectoryNet) (2022.7.1)
Requirement already satisfied: six>=1.5 in /opt/mamba/lib/python3.10/site-packages (from python-dateutil>=2.7->matplotlib>=3.2.1->TrajectoryNet) (1.16.0)
Requirement already satisfied: tbb>=2019.0 in /opt/mamba/lib/python3.10/site-packages (from umap-learn>=0.3.10->scanpy->TrajectoryNet) (2021.10.0)
Requirement already satisfied: pynndescent>=0.5 in /opt/mamba/lib/python3.10/site-packages (from umap-learn>=0.3.10->scanpy->TrajectoryNet) (0.5.10)
Requirement already satisfied: MarkupSafe>=2.0 in /opt/mamba/lib/python3.10/site-packages (from jinja2->torch>=1.5.0->TrajectoryNet) (2.1.2)
Requirement already satisfied: stdlib-list in /opt/mamba/lib/python3.10/site-packages (from session-info->scanpy->TrajectoryNet) (0.9.0)
Requirement already satisfied: mpmath>=0.19 in /opt/mamba/lib/python3.10/site-packages (from sympy->torch>=1.5.0->TrajectoryNet) (1.3.0)
Installing collected packages: argparse
Successfully installed argparse-1.4.0
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Requirement already satisfied: python-magic in /opt/mamba/lib/python3.10/site-packages (0.4.27)
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB]      
Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease                         
Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB]
Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB]
Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1466 kB]
Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1274 kB]
Fetched 3078 kB in 8s (403 kB/s)                                               
Reading package lists... Done
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
libmagic1 is already the newest version (1:5.41-3ubuntu0.1).
0 upgraded, 0 newly installed, 0 to remove and 119 not upgraded.
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Requirement already satisfied: phate in /opt/mamba/lib/python3.10/site-packages (1.0.11)
Requirement already satisfied: tasklogger>=1.2 in /opt/mamba/lib/python3.10/site-packages (from phate) (1.2.0)
Requirement already satisfied: numpy>=1.16.0 in /opt/mamba/lib/python3.10/site-packages (from phate) (1.24.2)
Requirement already satisfied: scprep>=0.11.1 in /opt/mamba/lib/python3.10/site-packages (from phate) (1.2.3)
Requirement already satisfied: s-gd2>=1.8.1 in /opt/mamba/lib/python3.10/site-packages (from phate) (1.8.1)
Requirement already satisfied: scikit-learn>=0.20.0 in /opt/mamba/lib/python3.10/site-packages (from phate) (1.3.2)
Requirement already satisfied: Deprecated in /opt/mamba/lib/python3.10/site-packages (from phate) (1.2.14)
Requirement already satisfied: graphtools>=1.5.3 in /opt/mamba/lib/python3.10/site-packages (from phate) (1.5.3)
Requirement already satisfied: future in /opt/mamba/lib/python3.10/site-packages (from phate) (0.18.3)
Requirement already satisfied: scipy>=1.1.0 in /opt/mamba/lib/python3.10/site-packages (from phate) (1.10.1)
Requirement already satisfied: matplotlib>=3.0 in /opt/mamba/lib/python3.10/site-packages (from phate) (3.8.1)
Requirement already satisfied: pygsp>=0.5.1 in /opt/mamba/lib/python3.10/site-packages (from graphtools>=1.5.3->phate) (0.5.1)
Requirement already satisfied: contourpy>=1.0.1 in /opt/mamba/lib/python3.10/site-packages (from matplotlib>=3.0->phate) (1.2.0)
Requirement already satisfied: kiwisolver>=1.3.1 in /opt/mamba/lib/python3.10/site-packages (from matplotlib>=3.0->phate) (1.4.5)
Requirement already satisfied: cycler>=0.10 in /opt/mamba/lib/python3.10/site-packages (from matplotlib>=3.0->phate) (0.12.1)
Requirement already satisfied: pyparsing>=2.3.1 in /opt/mamba/lib/python3.10/site-packages (from matplotlib>=3.0->phate) (3.1.1)
Requirement already satisfied: pillow>=8 in /opt/mamba/lib/python3.10/site-packages (from matplotlib>=3.0->phate) (10.1.0)
Requirement already satisfied: python-dateutil>=2.7 in /opt/mamba/lib/python3.10/site-packages (from matplotlib>=3.0->phate) (2.8.2)
Requirement already satisfied: packaging>=20.0 in /opt/mamba/lib/python3.10/site-packages (from matplotlib>=3.0->phate) (23.0)
Requirement already satisfied: fonttools>=4.22.0 in /opt/mamba/lib/python3.10/site-packages (from matplotlib>=3.0->phate) (4.44.0)
Requirement already satisfied: joblib>=1.1.1 in /opt/mamba/lib/python3.10/site-packages (from scikit-learn>=0.20.0->phate) (1.3.2)
Requirement already satisfied: threadpoolctl>=2.0.0 in /opt/mamba/lib/python3.10/site-packages (from scikit-learn>=0.20.0->phate) (3.2.0)
Requirement already satisfied: decorator>=4.3.0 in /opt/mamba/lib/python3.10/site-packages (from scprep>=0.11.1->phate) (5.1.1)
Requirement already satisfied: pandas<2.1,>=0.25 in /opt/mamba/lib/python3.10/site-packages (from scprep>=0.11.1->phate) (1.5.3)
Requirement already satisfied: wrapt<2,>=1.10 in /opt/mamba/lib/python3.10/site-packages (from Deprecated->phate) (1.16.0)
Requirement already satisfied: pytz>=2020.1 in /opt/mamba/lib/python3.10/site-packages (from pandas<2.1,>=0.25->scprep>=0.11.1->phate) (2022.7.1)
Requirement already satisfied: six>=1.5 in /opt/mamba/lib/python3.10/site-packages (from python-dateutil>=2.7->matplotlib>=3.0->phate) (1.16.0)
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Requirement already satisfied: scprep in /opt/mamba/lib/python3.10/site-packages (1.2.3)
Requirement already satisfied: pandas<2.1,>=0.25 in /opt/mamba/lib/python3.10/site-packages (from scprep) (1.5.3)
Requirement already satisfied: scipy>=0.18.1 in /opt/mamba/lib/python3.10/site-packages (from scprep) (1.10.1)
Requirement already satisfied: scikit-learn>=0.19.1 in /opt/mamba/lib/python3.10/site-packages (from scprep) (1.3.2)
Requirement already satisfied: numpy>=1.12.0 in /opt/mamba/lib/python3.10/site-packages (from scprep) (1.24.2)
Requirement already satisfied: packaging in /opt/mamba/lib/python3.10/site-packages (from scprep) (23.0)
Requirement already satisfied: decorator>=4.3.0 in /opt/mamba/lib/python3.10/site-packages (from scprep) (5.1.1)
Requirement already satisfied: python-dateutil>=2.8.1 in /opt/mamba/lib/python3.10/site-packages (from pandas<2.1,>=0.25->scprep) (2.8.2)
Requirement already satisfied: pytz>=2020.1 in /opt/mamba/lib/python3.10/site-packages (from pandas<2.1,>=0.25->scprep) (2022.7.1)
Requirement already satisfied: joblib>=1.1.1 in /opt/mamba/lib/python3.10/site-packages (from scikit-learn>=0.19.1->scprep) (1.3.2)
Requirement already satisfied: threadpoolctl>=2.0.0 in /opt/mamba/lib/python3.10/site-packages (from scikit-learn>=0.19.1->scprep) (3.2.0)
Requirement already satisfied: six>=1.5 in /opt/mamba/lib/python3.10/site-packages (from python-dateutil>=2.8.1->pandas<2.1,>=0.25->scprep) (1.16.0)
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Requirement already satisfied: magic-impute in /root/.local/lib/python3.10/site-packages (3.0.0)
Requirement already satisfied: tasklogger>=1.0.0 in /opt/mamba/lib/python3.10/site-packages (from magic-impute) (1.2.0)
Requirement already satisfied: pandas>=0.25 in /opt/mamba/lib/python3.10/site-packages (from magic-impute) (1.5.3)
Requirement already satisfied: graphtools>=1.4.0 in /opt/mamba/lib/python3.10/site-packages (from magic-impute) (1.5.3)
Requirement already satisfied: future in /opt/mamba/lib/python3.10/site-packages (from magic-impute) (0.18.3)
Requirement already satisfied: scipy>=1.1.0 in /opt/mamba/lib/python3.10/site-packages (from magic-impute) (1.10.1)
Requirement already satisfied: numpy>=1.14.0 in /opt/mamba/lib/python3.10/site-packages (from magic-impute) (1.24.2)
Requirement already satisfied: scprep>=1.0 in /opt/mamba/lib/python3.10/site-packages (from magic-impute) (1.2.3)
Requirement already satisfied: scikit-learn>=0.19.1 in /opt/mamba/lib/python3.10/site-packages (from magic-impute) (1.3.2)
Requirement already satisfied: matplotlib in /opt/mamba/lib/python3.10/site-packages (from magic-impute) (3.8.1)
Requirement already satisfied: pygsp>=0.5.1 in /opt/mamba/lib/python3.10/site-packages (from graphtools>=1.4.0->magic-impute) (0.5.1)
Requirement already satisfied: Deprecated in /opt/mamba/lib/python3.10/site-packages (from graphtools>=1.4.0->magic-impute) (1.2.14)
Requirement already satisfied: python-dateutil>=2.8.1 in /opt/mamba/lib/python3.10/site-packages (from pandas>=0.25->magic-impute) (2.8.2)
Requirement already satisfied: pytz>=2020.1 in /opt/mamba/lib/python3.10/site-packages (from pandas>=0.25->magic-impute) (2022.7.1)
Requirement already satisfied: threadpoolctl>=2.0.0 in /opt/mamba/lib/python3.10/site-packages (from scikit-learn>=0.19.1->magic-impute) (3.2.0)
Requirement already satisfied: joblib>=1.1.1 in /opt/mamba/lib/python3.10/site-packages (from scikit-learn>=0.19.1->magic-impute) (1.3.2)
Requirement already satisfied: packaging in /opt/mamba/lib/python3.10/site-packages (from scprep>=1.0->magic-impute) (23.0)
Requirement already satisfied: decorator>=4.3.0 in /opt/mamba/lib/python3.10/site-packages (from scprep>=1.0->magic-impute) (5.1.1)
Requirement already satisfied: fonttools>=4.22.0 in /opt/mamba/lib/python3.10/site-packages (from matplotlib->magic-impute) (4.44.0)
Requirement already satisfied: kiwisolver>=1.3.1 in /opt/mamba/lib/python3.10/site-packages (from matplotlib->magic-impute) (1.4.5)
Requirement already satisfied: pillow>=8 in /opt/mamba/lib/python3.10/site-packages (from matplotlib->magic-impute) (10.1.0)
Requirement already satisfied: cycler>=0.10 in /opt/mamba/lib/python3.10/site-packages (from matplotlib->magic-impute) (0.12.1)
Requirement already satisfied: pyparsing>=2.3.1 in /opt/mamba/lib/python3.10/site-packages (from matplotlib->magic-impute) (3.1.1)
Requirement already satisfied: contourpy>=1.0.1 in /opt/mamba/lib/python3.10/site-packages (from matplotlib->magic-impute) (1.2.0)
Requirement already satisfied: six>=1.5 in /opt/mamba/lib/python3.10/site-packages (from python-dateutil>=2.8.1->pandas>=0.25->magic-impute) (1.16.0)
Requirement already satisfied: wrapt<2,>=1.10 in /opt/mamba/lib/python3.10/site-packages (from Deprecated->graphtools>=1.4.0->magic-impute) (1.16.0)
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
代码
文本
[ ]
! wget https://codeload.github.com/KrishnaswamyLab/TrajectoryNet/zip/refs/heads/master
! unzip master
--2023-11-10 18:19:06--  https://codeload.github.com/KrishnaswamyLab/TrajectoryNet/zip/refs/heads/master
Resolving ga.dp.tech (ga.dp.tech)... 10.255.254.7, 10.255.254.18, 10.255.254.37
Connecting to ga.dp.tech (ga.dp.tech)|10.255.254.7|:8118... connected.
Proxy request sent, awaiting response... 200 OK
Length: unspecified [application/zip]
Saving to: ‘master.1’

master.1                [               <=>  ]  35.99M  3.96MB/s    in 11s     

2023-11-10 18:19:18 (3.32 MB/s) - ‘master.1’ saved [37736431]

Archive:  master
162e6c77728135f27ad04f1c83d78a319e79dff4
replace TrajectoryNet-master/.flake8? [y]es, [n]o, [A]ll, [N]one, [r]ename:
代码
文本

接下来，进入实际演示部分。

代码
文本

1. 加载10X数据

代码
文本

从 Mendeley Datasets 导入数据

这个胚胎体数据集来源于公开数据Mendeley数据集https://data.mendeley.com/datasets/v6n743h5ng/，本教程已经为您下载并以挂载数据集的方式挂载好了，在路径 /bohr/bio-notebook-trajectorynet-uo5o/v1 中。

这些文件是单细胞RNA测序数据的标准格式，通常由CellRanger软件生成。了解更多关于CellRanger如何生成这些文件的信息，请查看 Gene-Barcode Matrices Documentation。

以下是目录结构:

download_path
└── scRNAseq
    ├── scRNAseq.zip
    ├── T0_1A
    │   ├── barcodes.tsv
    │   ├── genes.tsv
    │   └── matrix.mtx
    ├── T2_3B
    │   ├── barcodes.tsv
    │   ├── genes.tsv
    │   └── matrix.mtx
    ├── T4_5C
    │   ├── barcodes.tsv
    │   ├── genes.tsv
    │   └── matrix.mtx
    ├── T6_7D
    │   ├── barcodes.tsv
    │   ├── genes.tsv
    │   └── matrix.mtx
    └── T8_9E
        ├── barcodes.tsv
        ├── genes.tsv
        └── matrix.mtx
代码
文本
[1]
import os
import scprep
# download_path = os.path.expanduser("~")
download_path = "/bohr/bio-notebook-trajectorynet-uo5o/v1"
print(download_path)
/bohr/bio-notebook-trajectorynet-uo5o/v1
代码
文本

使用scprep导入数据到Pandas DataFrame

接下来，使用一个名为scprep的工具包来加载和操作单细胞数据。函数load_10X可以自动将10X单细胞RNA测序数据集（以及其他数据集）加载到Pandas DataFrame中。

让我们加载数据，并创建一个单一矩阵，我们可以用它来进行预处理、可视化和分析。

代码
文本

1. 导入数据

代码
文本
[2]
import pandas as pd
import numpy as np
import phate
import scprep
import magic
import matplotlib.pyplot as plt
import sklearn.preprocessing

# matplotlib settings for Jupyter notebooks only
%matplotlib inline
代码
文本

2. 使用 scprep.io.load_10X 导入每个样本的三个矩阵到一个DataFrame中（读取数据会花费约5分钟的时间）

注意：默认情况下，scprep.io.load_10X 使用 Pandas 的 SparseDataFrame (see Pandas docs) 来最大化内存效率。然而，这会比加载密集矩阵慢一些。要加载密集矩阵，请将 sparse=False 参数传递给 load_10X。我们使用 gene_labels = 'both' 以便在保留基因ID的唯一性的同时看到基因符号。

代码
文本
[3]
sparse=True
T1 = scprep.io.load_10X(os.path.join(download_path, "scRNAseq", "T0_1A"), sparse=sparse, gene_labels='both')
T2 = scprep.io.load_10X(os.path.join(download_path, "scRNAseq", "T2_3B"), sparse=sparse, gene_labels='both')
T3 = scprep.io.load_10X(os.path.join(download_path, "scRNAseq", "T4_5C"), sparse=sparse, gene_labels='both')
T4 = scprep.io.load_10X(os.path.join(download_path, "scRNAseq", "T6_7D"), sparse=sparse, gene_labels='both')
T5 = scprep.io.load_10X(os.path.join(download_path, "scRNAseq", "T8_9E"), sparse=sparse, gene_labels='both')
T1.head()
RP11-34P13.3 (ENSG00000243485) FAM138A (ENSG00000237613) OR4F5 (ENSG00000186092) RP11-34P13.7 (ENSG00000238009) RP11-34P13.8 (ENSG00000239945) RP11-34P13.14 (ENSG00000239906) RP11-34P13.9 (ENSG00000241599) FO538757.3 (ENSG00000279928) FO538757.2 (ENSG00000279457) AP006222.2 (ENSG00000228463) ... AC007325.2 (ENSG00000277196) BX072566.1 (ENSG00000277630) AL354822.1 (ENSG00000278384) AC023491.2 (ENSG00000278633) AC004556.1 (ENSG00000276345) AC233755.2 (ENSG00000277856) AC233755.1 (ENSG00000275063) AC240274.1 (ENSG00000271254) AC213203.1 (ENSG00000277475) FAM231B (ENSG00000268674)
0
AAACATACCAGAGG-1 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 ... 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
AAACATTGAAAGCA-1 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 ... 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
AAACATTGAAGTGA-1 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 ... 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
AAACATTGGAGGTG-1 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 ... 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
AAACATTGGTTTCT-1 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 ... 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0

5 rows × 33694 columns

代码
文本

3. Library 的大小筛选

我们筛选掉具有非常大或非常小库大小的细胞。对于这个数据集，库大小与样本有一定的相关性，因此我们基于每个样本进行筛选。在这种情况下，我们排除了每个样本中前20%和后20%的细胞。采用更简单、不太保守的筛选方法也能获得类似的结果。

代码
文本
[4]
scprep.plot.plot_library_size(T1, percentile=20)
代码
文本
[5]
filtered_batches = []
for batch in [T1, T2, T3, T4, T5]:
batch = scprep.filter.filter_library_size(batch, percentile=20, keep_cells='above')
batch = scprep.filter.filter_library_size(batch, percentile=75, keep_cells='below')
filtered_batches.append(batch)
del T1, T2, T3, T4, T5 # removes objects from memory
代码
文本

4.合并所有的数据集，并建立一个表示每个样本的时间序列向量

代码
文本
[6]
EBT_counts, sample_labels = scprep.utils.combine_batches(
filtered_batches,
["Day 00-03", "Day 06-09", "Day 12-15", "Day 18-21", "Day 24-27"],
append_to_cell_names=True
)
del filtered_batches # removes objects from memory
EBT_counts.head()
A1BG (ENSG00000121410) A1BG-AS1 (ENSG00000268895) A1CF (ENSG00000148584) A2M (ENSG00000175899) A2M-AS1 (ENSG00000245105) A2ML1 (ENSG00000166535) A2ML1-AS1 (ENSG00000256661) A2ML1-AS2 (ENSG00000256904) A3GALT2 (ENSG00000184389) A4GALT (ENSG00000128274) ... ZXDC (ENSG00000070476) ZYG11A (ENSG00000203995) ZYG11B (ENSG00000162378) ZYX (ENSG00000159840) ZZEF1 (ENSG00000074755) ZZZ3 (ENSG00000036549) bP-21264C1.2 (ENSG00000278932) bP-2171C21.3 (ENSG00000279501) bP-2189O9.3 (ENSG00000279579) hsa-mir-1253 (ENSG00000272920)
AAACATTGAAAGCA-1_Day 00-03 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 ... 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
AAACCGTGCAGAAA-1_Day 00-03 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 ... 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
AAACCGTGGAAGGC-1_Day 00-03 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 ... 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
AAACGCACCGGTAT-1_Day 00-03 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 ... 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
AAACGCACCTATTC-1_Day 00-03 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 ... 0.0 0.0 0.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0

5 rows × 33694 columns

代码
文本

2. 数据预处理：过滤、归一化和转换

过滤

我们通过以下方式对数据进行过滤：

1.根据库大小进行筛选（如果我们在合并批次之前没有进行此操作）。
2.剔除在相对较少的细胞中表达的基因。
3.剔除死细胞。

需要注意的是，在库大小归一化之后筛选死细胞，因为库大小不一定与细胞状态相关。

代码
文本

过滤 I：库大小筛选

我们之前已经进行了库大小筛选，因为库大小与我们的样本之间存在强相关性。然而，如果你想要进行更简单的筛选，你可以在这里运行以下操作：

EBT_counts, sample_labels = scprep.filter.library_size_filter(EBT_counts, sample_labels, cutoff=2000)

代码
文本

过滤 II：将检测率较小的基因过滤

我们过滤了在少于等于10个细胞中表达的基因。

代码
文本
[7]
EBT_counts = scprep.filter.filter_rare_genes(EBT_counts, min_cells=10)
代码
文本

归一化

为了校正不同库大小之间的差异，我们将每个细胞的表达量除以其库大小，然后通过中位数库大小进行重新缩放。

在Python中，可以使用预处理方法 library_size_normalize() 来执行这个操作。

代码
文本
[8]
EBT_counts = scprep.normalize.library_size_normalize(EBT_counts)
代码
文本

过滤 III：去除死细胞

死细胞很可能具有比活细胞更高的线粒体RNA表达水平。因此，我们通过消除平均具有最高线粒体RNA表达水平的细胞来去除疑似死细胞。

首先，让我们看一下线粒体基因的分布情况。

代码
文本
[10]
mito_genes = scprep.select.get_gene_set(EBT_counts, starts_with="MT-") # Get all mitochondrial genes. There are 14, FYI.
scprep.plot.plot_gene_set_expression(EBT_counts, genes=mito_genes, percentile=90)
代码
文本

我们可以看到，在0.9分位数以上的地方，线粒体RNA的表达出现了急剧增加。因此，我们过滤掉这一部分细胞。

代码
文本
[11]
EBT_counts, sample_labels = scprep.filter.filter_gene_set_expression(
EBT_counts, sample_labels, genes=mito_genes,
percentile=90, keep_cells='below')
代码
文本

转换

在单细胞RNA测序分析中，数据通常会进行 log 转换。通常需要添加一些小值以避免取 log(0)。本教程中，我们完全避免了这个问题，而是采用了平方根变换。平方根函数具有与 log 函数类似的形式，而且在0处更加稳定。

代码
文本
[12]
EBT_counts = scprep.transform.sqrt(EBT_counts)
代码
文本

3. 使用PHATE嵌入数据

代码
文本

3.1 实例化PHATE估计器

PHATE模型的API与Scikit Learn的API相似。首先，您需要使用适用于给定数据集的参数来实例化一个PHATE估计器对象。然后，您可以使用 fit 和 fit_transform 函数来生成嵌入。要获取更多信息，请查看 PHATE的readthedocs页面。

现在我们只使用默认参数，但以下参数可以进行调整（请阅读文档 phate.readthedocs.io 以了解更多信息）：

  • knn：最近邻居的数量（默认值：5）。如果您的PHATE嵌入看起来非常不连通，可以增加此值（例如，设为20）。如果您的数据集非常大（例如，>100k个细胞），还应考虑增加 knn。
  • decay：Alpha衰减（默认值：15）。减小 decay 会增加图中的连接性，增加 decay 会减少连接性。这很少需要调整。将其设置为 None 以获得k最近邻居核。
  • t：操作符幂次数（默认值：'auto'）。这等于对数据进行的平滑处理次数。默认情况下自动选择，但如果您的嵌入缺乏结构，可以增加它，或者如果结构看起来太紧凑，可以减小它。
  • gamma：信息距离常数（默认值：1）。gamma=1 给出PHATE对数潜力，但其他信息距离也可能很有趣。如果大部分点似乎集中在绘图的一个部分，可以尝试 gamma=0。
代码
文本

由于我们正在寻找细节的结构，而且我们预计一些轨迹可能会稀疏，因此我们可能希望将 knn 从默认值5减小，并将 t 从自动值21（如上面的输出中所示）降低。对于单细胞RNA测序，如果您正在寻找微妙的结构，可以将 knn 设为3或4，如果您有数十万个细胞，也可以将其设为30或40。我们还会将 alpha 降低到15，部分抵消由于减小 knn 而导致的连接性减少。

代码
文本
[13]
phate_operator = phate.PHATE(n_jobs=-2, random_state=42)
Y_phate = phate_operator.fit_transform(EBT_counts)
Calculating PHATE...
  Running PHATE on 16821 observations and 17845 variables.
  Calculating graph and diffusion operator...
    Calculating PCA...
    Calculated PCA in 43.01 seconds.
    Calculating KNN search...
    Calculated KNN search in 9.08 seconds.
    Calculating affinities...
    Calculated affinities in 0.90 seconds.
  Calculated graph and diffusion operator in 55.26 seconds.
  Calculating landmark operator...
    Calculating SVD...
    Calculated SVD in 2.47 seconds.
    Calculating KMeans...
    Calculated KMeans in 2.98 seconds.
  Calculated landmark operator in 6.91 seconds.
  Calculating optimal t...
    Automatically selected t = 19
  Calculated optimal t in 1.31 seconds.
  Calculating diffusion potential...
  Calculated diffusion potential in 0.20 seconds.
  Calculating metric MDS...
  Calculated metric MDS in 5.78 seconds.
Calculated PHATE in 69.49 seconds.
代码
文本
[14]
scprep.plot.scatter2d(Y_phate, c=sample_labels, figsize=(12,8), cmap="Spectral",
ticks=False, label_prefix="PHATE")
代码
文本

4. 使用TrajectoryNet建模细胞动态转换

本节内容将它展示了如何在基因空间中使用TrajectoryNet计算细胞转换轨迹。

本节计算的backward_trajectories.npy 是一个形状为 [timepoints, cells, pcs] 的数组，与论文中使用的相同，使用以下命令生成：

python main.py --save [SAVE_DIR] --dataset EB-PCA --top_k_reg 0.1 --training_noise 0.0 --max_dim 5

这基本上是默认设置，带有少量密度正则化（称为 top_k_reg）。这将计算模型并保存一些检查点，最终的权重存储在 checkpt.pt 中。在论文中进行实验的主要参数是 --top_k_reg（密度正则化）和 --vecint（速度正则化）。

然后我们运行：

python eval.py --save [SAVE_DIR] --dataset EB-PCA --top_k_reg 0.1 --training_noise 0.0 --max_dim 5

这将从保存的模型中在 [SAVE_DIR] 目录中创建 backwards_trajectories.npy。

完整的训练运行输出保存在 results/fig8_results/ 中。

这些轨迹将在最终时间点中的点向后积分到起始时间点，共计100个均匀分布的时间点。也就是说，基于一些已有的隐空间中的嵌入（也就是降维后的数据），如PCA的结果，TrajectoryNet构建了这个嵌入数据的基因空间中的动态轨迹。

代码
文本

4.1 运行 TrajectoryNet，进行轨迹推断

代码
文本

📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢
注意！4.1 小节运行时间可能超过 1h ，运行结果已经存在了，因此你可以选择不运行本节！
📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢📢

代码
文本
[18]
%%bash
cd ./TrajectoryNet-master/TrajectoryNet/
python main.py --save ../results/fig8_results/ --dataset EB-PCA --top_k_reg 0.1 --training_noise 0.0 --max_dim 5

mkdir: cannot create directory ‘notebook_results’: File exists
/data/bioinfo/TrajectoryNet/TrajectoryNet-master/TrajectoryNet/main.py
""" main.py

Learns ODE from scrna data

"""
import os
import matplotlib
import matplotlib.pyplot as plt
import numpy as np
import time

import torch
import torch.nn.functional as F
import torch.optim as optim

from TrajectoryNet.lib.growth_net import GrowthNet
from TrajectoryNet.lib import utils
from TrajectoryNet.lib.visualize_flow import visualize_transform
from TrajectoryNet.lib.viz_scrna import (
    save_trajectory,
    trajectory_to_video,
    save_vectors,
)
from TrajectoryNet.lib.viz_scrna import save_trajectory_density


# from train_misc import standard_normal_logprob
from TrajectoryNet.train_misc import (
    set_cnf_options,
    count_nfe,
    count_parameters,
    count_total_time,
    add_spectral_norm,
    spectral_norm_power_iteration,
    create_regularization_fns,
    get_regularization,
    append_regularization_to_log,
    build_model_tabular,
)

from TrajectoryNet import dataset
from TrajectoryNet.parse import parser

matplotlib.use("Agg")


def get_transforms(device, args, model, integration_times):
    """
    Given a list of integration points,
    returns a function giving integration times
    """

    def sample_fn(z, logpz=None):
        int_list = [
            torch.tensor([it - args.time_scale, it]).type(torch.float32).to(device)
            for it in integration_times
        ]
        if logpz is not None:
            # TODO this works right?
            for it in int_list:
                z, logpz = model(z, logpz, integration_times=it, reverse=True)
            return z, logpz
        else:
            for it in int_list:
                z = model(z, integration_times=it, reverse=True)
            return z

    def density_fn(x, logpx=None):
        int_list = [
            torch.tensor([it - args.time_scale, it]).type(torch.float32).to(device)
            for it in integration_times[::-1]
        ]
        if logpx is not None:
            for it in int_list:
                x, logpx = model(x, logpx, integration_times=it, reverse=False)
            return x, logpx
        else:
            for it in int_list:
                x = model(x, integration_times=it, reverse=False)
            return x

    return sample_fn, density_fn


def compute_loss(device, args, model, growth_model, logger, full_data):
    """
    Compute loss by integrating backwards from the last time step
    At each time step integrate back one time step, and concatenate that
    to samples of the empirical distribution at that previous timestep
    repeating over and over to calculate the likelihood of samples in
    later timepoints iteratively, making sure that the ODE is evaluated
    at every time step to calculate those later points.

    The growth model is a single model of time independent cell growth /
    death rate defined as a variation from uniform.
    """

    # Backward pass accumulating losses, previous state and deltas
    deltas = []
    zs = []
    z = None
    interp_loss = 0.0
    for i, (itp, tp) in enumerate(zip(args.int_tps[::-1], args.timepoints[::-1])):
        # tp counts down from last
        integration_times = torch.tensor([itp - args.time_scale, itp])
        integration_times = integration_times.type(torch.float32).to(device)
        # integration_times.requires_grad = True

        # load data and add noise
        idx = args.data.sample_index(args.batch_size, tp)
        x = args.data.get_data()[idx]
        if args.training_noise > 0.0:
            x += np.random.randn(*x.shape) * args.training_noise
        x = torch.from_numpy(x).type(torch.float32).to(device)

        if i > 0:
            x = torch.cat((z, x))
            zs.append(z)
        zero = torch.zeros(x.shape[0], 1).to(x)

        # transform to previous timepoint
        z, delta_logp = model(x, zero, integration_times=integration_times)
        deltas.append(delta_logp)

        # Straightline regularization
        # Integrate to random point at time t and assert close to (1 - t) * end + t * start
        if args.interp_reg:
            t = np.random.rand()
            int_t = torch.tensor([itp - t * args.time_scale, itp])
            int_t = int_t.type(torch.float32).to(device)
            int_x = model(x, integration_times=int_t)
            int_x = int_x.detach()
            actual_int_x = x * (1 - t) + z * t
            interp_loss += F.mse_loss(int_x, actual_int_x)
    if args.interp_reg:
        print("interp_loss", interp_loss)

    logpz = args.data.base_density()(z)

    # build growth rates
    if args.use_growth:
        growthrates = [torch.ones_like(logpz)]
        for z_state, tp in zip(zs[::-1], args.timepoints[:-1]):
            # Full state includes time parameter to growth_model
            time_state = tp * torch.ones(z_state.shape[0], 1).to(z_state)
            full_state = torch.cat([z_state, time_state], 1)
            growthrates.append(growth_model(full_state))

    # Accumulate losses
    losses = []
    logps = [logpz]
    for i, delta_logp in enumerate(deltas[::-1]):
        logpx = logps[-1] - delta_logp
        if args.use_growth:
            logpx += torch.log(torch.clamp(growthrates[i], 1e-4, 1e4))
        logps.append(logpx[: -args.batch_size])
        losses.append(-torch.mean(logpx[-args.batch_size :]))
    losses = torch.stack(losses)
    weights = torch.ones_like(losses).to(logpx)
    if args.leaveout_timepoint >= 0:
        weights[args.leaveout_timepoint] = 0
    losses = torch.mean(losses * weights)

    # Direction regularization
    if args.vecint:
        similarity_loss = 0
        for i, (itp, tp) in enumerate(zip(args.int_tps, args.timepoints)):
            itp = torch.tensor(itp).type(torch.float32).to(device)
            idx = args.data.sample_index(args.batch_size, tp)
            x = args.data.get_data()[idx]
            v = args.data.get_velocity()[idx]
            x = torch.from_numpy(x).type(torch.float32).to(device)
            v = torch.from_numpy(v).type(torch.float32).to(device)
            x += torch.randn_like(x) * 0.1
            # Only penalizes at the time / place of visible samples
            direction = -model.chain[0].odefunc.odefunc.diffeq(itp, x)
            if args.use_magnitude:
                similarity_loss += torch.mean(F.mse_loss(direction, v))
            else:
                similarity_loss -= torch.mean(F.cosine_similarity(direction, v))
        logger.info(similarity_loss)
        losses += similarity_loss * args.vecint

    # Density regularization
    if args.top_k_reg > 0:
        density_loss = 0
        tp_z_map = dict(zip(args.timepoints[:-1], zs[::-1]))
        if args.leaveout_timepoint not in tp_z_map:
            idx = args.data.sample_index(args.batch_size, tp)
            x = args.data.get_data()[idx]
            if args.training_noise > 0.0:
                x += np.random.randn(*x.shape) * args.training_noise
            x = torch.from_numpy(x).type(torch.float32).to(device)
            t = np.random.rand()
            int_t = torch.tensor([itp - t * args.time_scale, itp])
            int_t = int_t.type(torch.float32).to(device)
            int_x = model(x, integration_times=int_t)
            samples_05 = int_x
        else:
            # If we are leaving out a timepoint the regularize there
            samples_05 = tp_z_map[args.leaveout_timepoint]

        # Calculate distance to 5 closest neighbors
        # WARNING: This currently fails in the backward pass with cuda on pytorch < 1.4.0
        #          works on CPU. Fixed in pytorch 1.5.0
        # RuntimeError: CUDA error: invalid configuration argument
        # The workaround is to run on cpu on pytorch <= 1.4.0 or upgrade
        cdist = torch.cdist(samples_05, full_data)
        values, _ = torch.topk(cdist, 5, dim=1, largest=False, sorted=False)
        # Hinge loss
        hinge_value = 0.1
        values -= hinge_value
        values[values < 0] = 0
        density_loss = torch.mean(values)
        print("Density Loss", density_loss.item())
        losses += density_loss * args.top_k_reg
    losses += interp_loss
    return losses


def train(
    device, args, model, growth_model, regularization_coeffs, regularization_fns, logger
):
    optimizer = optim.Adam(
        model.parameters(), lr=args.lr, weight_decay=args.weight_decay
    )

    time_meter = utils.RunningAverageMeter(0.93)
    loss_meter = utils.RunningAverageMeter(0.93)
    nfef_meter = utils.RunningAverageMeter(0.93)
    nfeb_meter = utils.RunningAverageMeter(0.93)
    tt_meter = utils.RunningAverageMeter(0.93)

    full_data = (
        torch.from_numpy(
            args.data.get_data()[args.data.get_times() != args.leaveout_timepoint]
        )
        .type(torch.float32)
        .to(device)
    )

    best_loss = float("inf")
    if args.use_growth:
        growth_model.eval()
    end = time.time()
    for itr in range(1, args.niters + 1):
        model.train()
        optimizer.zero_grad()

        # Train
        if args.spectral_norm:
            spectral_norm_power_iteration(model, 1)

        loss = compute_loss(device, args, model, growth_model, logger, full_data)
        loss_meter.update(loss.item())

        if len(regularization_coeffs) > 0:
            # Only regularize on the last timepoint
            reg_states = get_regularization(model, regularization_coeffs)
            reg_loss = sum(
                reg_state * coeff
                for reg_state, coeff in zip(reg_states, regularization_coeffs)
                if coeff != 0
            )
            loss = loss + reg_loss
        total_time = count_total_time(model)
        nfe_forward = count_nfe(model)

        loss.backward()
        optimizer.step()

        # Eval
        nfe_total = count_nfe(model)
        nfe_backward = nfe_total - nfe_forward
        nfef_meter.update(nfe_forward)
        nfeb_meter.update(nfe_backward)
        time_meter.update(time.time() - end)
        tt_meter.update(total_time)

        log_message = (
            "Iter {:04d} | Time {:.4f}({:.4f}) | Loss {:.6f}({:.6f}) |"
            " NFE Forward {:.0f}({:.1f})"
            " | NFE Backward {:.0f}({:.1f})".format(
                itr,
                time_meter.val,
                time_meter.avg,
                loss_meter.val,
                loss_meter.avg,
                nfef_meter.val,
                nfef_meter.avg,
                nfeb_meter.val,
                nfeb_meter.avg,
            )
        )
        if len(regularization_coeffs) > 0:
            log_message = append_regularization_to_log(
                log_message, regularization_fns, reg_states
            )
        logger.info(log_message)

        if itr % args.val_freq == 0 or itr == args.niters:
            with torch.no_grad():
                train_eval(
                    device, args, model, growth_model, itr, best_loss, logger, full_data
                )

        if itr % args.viz_freq == 0:
            if args.data.get_shape()[0] > 2:
                logger.warning("Skipping vis as data dimension is >2")
            else:
                with torch.no_grad():
                    visualize(device, args, model, itr)
        if itr % args.save_freq == 0:
            chkpt = {
                "state_dict": model.state_dict(),
            }
            if args.use_growth:
                chkpt.update({"growth_state_dict": growth_model.state_dict()})
            utils.save_checkpoint(
                chkpt,
                args.save,
                epoch=itr,
            )
        end = time.time()
    logger.info("Training has finished.")


def train_eval(device, args, model, growth_model, itr, best_loss, logger, full_data):
    model.eval()
    test_loss = compute_loss(device, args, model, growth_model, logger, full_data)
    test_nfe = count_nfe(model)
    log_message = "[TEST] Iter {:04d} | Test Loss {:.6f} |" " NFE {:.0f}".format(
        itr, test_loss, test_nfe
    )
    logger.info(log_message)
    utils.makedirs(args.save)
    with open(os.path.join(args.save, "train_eval.csv"), "a") as f:
        import csv

        writer = csv.writer(f)
        writer.writerow((itr, test_loss))

    if test_loss.item() < best_loss:
        best_loss = test_loss.item()
        chkpt = {
            "state_dict": model.state_dict(),
        }
        if args.use_growth:
            chkpt.update({"growth_state_dict": growth_model.state_dict()})
        torch.save(
            chkpt,
            os.path.join(args.save, "checkpt.pth"),
        )


def visualize(device, args, model, itr):
    model.eval()
    for i, tp in enumerate(args.timepoints):
        idx = args.data.sample_index(args.viz_batch_size, tp)
        p_samples = args.data.get_data()[idx]
        sample_fn, density_fn = get_transforms(
            device, args, model, args.int_tps[: i + 1]
        )
        plt.figure(figsize=(9, 3))
        visualize_transform(
            p_samples,
            args.data.base_sample(),
            args.data.base_density(),
            transform=sample_fn,
            inverse_transform=density_fn,
            samples=True,
            npts=100,
            device=device,
        )
        fig_filename = os.path.join(
            args.save, "figs", "{:04d}_{:01d}.jpg".format(itr, i)
        )
        utils.makedirs(os.path.dirname(fig_filename))
        plt.savefig(fig_filename)
        plt.close()


def plot_output(device, args, model):
    save_traj_dir = os.path.join(args.save, "trajectory")
    # logger.info('Plotting trajectory to {}'.format(save_traj_dir))
    data_samples = args.data.get_data()[args.data.sample_index(2000, 0)]
    np.random.seed(42)
    start_points = args.data.base_sample()(1000, 2)
    # idx = args.data.sample_index(50, 0)
    # start_points = args.data.get_data()[idx]
    # start_points = torch.from_numpy(start_points).type(torch.float32)
    save_vectors(
        args.data.base_density(),
        model,
        start_points,
        args.data.get_data(),
        args.data.get_times(),
        args.save,
        skip_first=(not args.data.known_base_density()),
        device=device,
        end_times=args.int_tps,
        ntimes=100,
    )
    
    save_trajectory(
        args.data.base_density(),
        args.data.base_sample(),
        model,
        data_samples,
        save_traj_dir,
        device=device,
        end_times=args.int_tps,
        ntimes=25,
    )
    
    density_dir = os.path.join(args.save, "density2")
    save_trajectory_density(
        args.data.base_density(),
        model,
        data_samples,
        density_dir,
        device=device,
        end_times=args.int_tps,
        ntimes=25,
        memory=0.1,
    )
    
    if args.save_movie:
        trajectory_to_video(save_traj_dir)
        trajectory_to_video(density_dir)


def main(args):
    # logger
    print(args.no_display_loss)
    utils.makedirs(args.save)
    logger = utils.get_logger(
        logpath=os.path.join(args.save, "logs"),
        filepath=os.path.abspath(__file__),
        displaying=~args.no_display_loss,
    )

    if args.layer_type == "blend":
        logger.info("!! Setting time_scale from None to 1.0 for Blend layers.")
        args.time_scale = 1.0

    logger.info(args)

    device = torch.device(
        "cuda:" + str(args.gpu) if torch.cuda.is_available() else "cpu"
    )
    if args.use_cpu:
        device = torch.device("cpu")

    args.data = dataset.SCData.factory(args.dataset, args)

    args.timepoints = args.data.get_unique_times()
    # Use maximum timepoint to establish integration_times
    # as some timepoints may be left out for validation etc.
    args.int_tps = (np.arange(max(args.timepoints) + 1) + 1.0) * args.time_scale

    regularization_fns, regularization_coeffs = create_regularization_fns(args)
    model = build_model_tabular(args, args.data.get_shape()[0], regularization_fns).to(
        device
    )
    growth_model = None
    if args.use_growth:
        if args.leaveout_timepoint == -1:
            growth_model_path = "../data/externel/growth_model_v2.ckpt"
        elif args.leaveout_timepoint in [1, 2, 3]:
            assert args.max_dim == 5
            growth_model_path = "../data/growth/model_%d" % args.leaveout_timepoint
        else:
            print("WARNING: Cannot use growth with this timepoint")

        growth_model = torch.load(growth_model_path, map_location=device)
    if args.spectral_norm:
        add_spectral_norm(model)
    set_cnf_options(args, model)

    if args.test:
        state_dict = torch.load(args.save + "/checkpt.pth", map_location=device)
        model.load_state_dict(state_dict["state_dict"])
        # if "growth_state_dict" not in state_dict:
        #    print("error growth model note in save")
        #    growth_model = None
        # else:
        #    checkpt = torch.load(args.save + "/checkpt.pth", map_location=device)
        #    growth_model.load_state_dict(checkpt["growth_state_dict"])
        # TODO can we load the arguments from the save?
        # eval_utils.generate_samples(
        #    device, args, model, growth_model, timepoint=args.leaveout_timepoint
        # )
        # with torch.no_grad():
        #    evaluate(device, args, model, growth_model)
    #    exit()
    else:
        logger.info(model)
        n_param = count_parameters(model)
        logger.info("Number of trainable parameters: {}".format(n_param))

        train(
            device,
            args,
            model,
            growth_model,
            regularization_coeffs,
            regularization_fns,
            logger,
        )

    if args.data.data.shape[1] == 2:
        plot_output(device, args, model)


if __name__ == "__main__":

    args = parser.parse_args()
    main(args)

Namespace(test=False, dataset='EB-PCA', use_growth=False, use_density=False, leaveout_timepoint=-1, layer_type='concatsquash', max_dim=5, dims='64-64-64', num_blocks=1, time_scale=0.5, train_T=True, divergence_fn='brute_force', nonlinearity='tanh', stochastic=False, alpha=0.0, solver='dopri5', atol=1e-05, rtol=1e-05, step_size=None, test_solver=None, test_atol=None, test_rtol=None, residual=False, rademacher=False, spectral_norm=False, batch_norm=False, bn_lag=0, niters=10000, num_workers=8, batch_size=1000, test_batch_size=1000, viz_batch_size=2000, lr=0.001, weight_decay=1e-05, l1int=None, l2int=None, sl2int=None, dl2int=None, dtl2int=None, JFrobint=None, JdiagFrobint=None, JoffdiagFrobint=None, vecint=None, use_magnitude=False, interp_reg=None, save='../results/fig8_results/', save_freq=1000, viz_freq=100, viz_freq_growth=100, val_freq=100, log_freq=10, gpu=0, use_cpu=False, no_display_loss=True, top_k_reg=0.1, training_noise=0.0, embedding_name='pca', whiten=False)
SequentialFlow(
  (chain): ModuleList(
    (0): CNF(
      (odefunc): RegularizedODEfunc(
        (odefunc): ODEfunc(
          (diffeq): ODEnet(
            (layers): ModuleList(
              (0): ConcatSquashLinear(
                (_layer): Linear(in_features=5, out_features=64, bias=True)
                (_hyper_bias): Linear(in_features=1, out_features=64, bias=False)
                (_hyper_gate): Linear(in_features=1, out_features=64, bias=True)
              )
              (1-2): 2 x ConcatSquashLinear(
                (_layer): Linear(in_features=64, out_features=64, bias=True)
                (_hyper_bias): Linear(in_features=1, out_features=64, bias=False)
                (_hyper_gate): Linear(in_features=1, out_features=64, bias=True)
              )
              (3): ConcatSquashLinear(
                (_layer): Linear(in_features=64, out_features=5, bias=True)
                (_hyper_bias): Linear(in_features=1, out_features=5, bias=False)
                (_hyper_gate): Linear(in_features=1, out_features=5, bias=True)
              )
            )
            (activation_fns): ModuleList(
              (0-2): 3 x Tanh()
            )
          )
        )
      )
    )
  )
)
Number of trainable parameters: 9621
Iter 0001 | Time 2.5157(2.5157) | Loss 10.659542(10.659542) | NFE Forward 14(14.0) | NFE Backward 138(138.0)
Iter 0002 | Time 1.7738(2.4638) | Loss 10.414104(10.642361) | NFE Forward 14(14.0) | NFE Backward 138(138.0)
Iter 0003 | Time 1.7819(2.4161) | Loss 10.223143(10.613016) | NFE Forward 14(14.0) | NFE Backward 138(138.0)
Iter 0004 | Time 1.7488(2.3694) | Loss 10.002523(10.570282) | NFE Forward 14(14.0) | NFE Backward 138(138.0)
Iter 0005 | Time 1.7800(2.3281) | Loss 9.876067(10.521687) | NFE Forward 14(14.0) | NFE Backward 138(138.0)
Iter 0006 | Time 1.7440(2.2872) | Loss 9.736125(10.466697) | NFE Forward 14(14.0) | NFE Backward 138(138.0)
Iter 0007 | Time 1.7443(2.2492) | Loss 9.479450(10.397590) | NFE Forward 14(14.0) | NFE Backward 138(138.0)
Iter 0008 | Time 1.7452(2.2139) | Loss 9.319873(10.322150) | NFE Forward 14(14.0) | NFE Backward 138(138.0)
Iter 0009 | Time 1.7454(2.1811) | Loss 9.203041(10.243812) | NFE Forward 14(14.0) | NFE Backward 138(138.0)
Iter 0010 | Time 1.7762(2.1528) | Loss 9.020422(10.158175) | NFE Forward 14(14.0) | NFE Backward 138(138.0)
Iter 0011 | Time 1.7768(2.1265) | Loss 8.881227(10.068789) | NFE Forward 14(14.0) | NFE Backward 138(138.0)
Iter 0012 | Time 1.7458(2.0998) | Loss 8.806569(9.980433) | NFE Forward 14(14.0) | NFE Backward 138(138.0)
Iter 0013 | Time 1.7445(2.0749) | Loss 8.714346(9.891807) | NFE Forward 14(14.0) | NFE Backward 138(138.0)
Iter 0014 | Time 1.7518(2.0523) | Loss 8.550598(9.797922) | NFE Forward 14(14.0) | NFE Backward 138(138.0)
Iter 0015 | Time 1.7746(2.0329) | Loss 8.479905(9.705661) | NFE Forward 14(14.0) | NFE Backward 138(138.0)
Iter 0016 | Time 1.7731(2.0147) | Loss 8.392025(9.613707) | NFE Forward 14(14.0) | NFE Backward 138(138.0)
Iter 0017 | Time 1.7746(1.9979) | Loss 8.293889(9.521319) | NFE Forward 14(14.0) | NFE Backward 138(138.0)
Iter 0018 | Time 1.7722(1.9821) | Loss 8.228414(9.430816) | NFE Forward 14(14.0) | NFE Backward 138(138.0)
Iter 0019 | Time 1.7472(1.9657) | Loss 8.193113(9.344177) | NFE Forward 14(14.0) | NFE Backward 138(138.0)
Iter 0020 | Time 1.7801(1.9527) | Loss 8.169664(9.261961) | NFE Forward 14(14.0) | NFE Backward 138(138.0)
Iter 0021 | Time 1.7516(1.9386) | Loss 7.971884(9.171656) | NFE Forward 14(14.0) | NFE Backward 138(138.0)
Iter 0022 | Time 1.7812(1.9276) | Loss 7.917172(9.083842) | NFE Forward 14(14.0) | NFE Backward 138(138.0)
Iter 0023 | Time 1.7546(1.9155) | Loss 7.947235(9.004279) | NFE Forward 14(14.0) | NFE Backward 138(138.0)
Iter 0024 | Time 1.7852(1.9063) | Loss 7.843535(8.923027) | NFE Forward 14(14.0) | NFE Backward 138(138.0)
Iter 0025 | Time 1.7558(1.8958) | Loss 7.834055(8.846799) | NFE Forward 14(14.0) | NFE Backward 138(138.0)
Iter 0026 | Time 1.8131(1.8900) | Loss 7.764037(8.771006) | NFE Forward 14(14.0) | NFE Backward 144(138.4)
Iter 0027 | Time 1.8149(1.8848) | Loss 7.755235(8.699902) | NFE Forward 14(14.0) | NFE Backward 144(138.8)
Iter 0028 | Time 1.8182(1.8801) | Loss 7.645949(8.626125) | NFE Forward 14(14.0) | NFE Backward 144(139.2)
Iter 0029 | Time 1.8317(1.8767) | Loss 7.637852(8.556946) | NFE Forward 14(14.0) | NFE Backward 144(139.5)
Iter 0030 | Time 1.8405(1.8742) | Loss 7.651845(8.493589) | NFE Forward 14(14.0) | NFE Backward 144(139.8)
Iter 0031 | Time 1.8105(1.8697) | Loss 7.644297(8.434138) | NFE Forward 14(14.0) | NFE Backward 144(140.1)
Iter 0032 | Time 1.8151(1.8659) | Loss 7.538164(8.371420) | NFE Forward 14(14.0) | NFE Backward 144(140.4)
Iter 0033 | Time 1.6958(1.8540) | Loss 7.546865(8.313701) | NFE Forward 14(14.0) | NFE Backward 132(139.8)
Iter 0034 | Time 1.6848(1.8421) | Loss 7.485286(8.255712) | NFE Forward 14(14.0) | NFE Backward 132(139.3)
Iter 0035 | Time 1.6602(1.8294) | Loss 7.427310(8.197724) | NFE Forward 14(14.0) | NFE Backward 132(138.7)
Iter 0036 | Time 1.7769(1.8257) | Loss 7.418245(8.143161) | NFE Forward 14(14.0) | NFE Backward 144(139.1)
Iter 0037 | Time 1.5710(1.8079) | Loss 7.405585(8.091530) | NFE Forward 14(14.0) | NFE Backward 120(137.8)
Iter 0038 | Time 1.6831(1.7992) | Loss 7.388819(8.042341) | NFE Forward 14(14.0) | NFE Backward 132(137.4)
Iter 0039 | Time 1.7688(1.7970) | Loss 7.337600(7.993009) | NFE Forward 14(14.0) | NFE Backward 138(137.4)
Iter 0040 | Time 1.6253(1.7850) | Loss 7.359450(7.948660) | NFE Forward 14(14.0) | NFE Backward 126(136.6)
Iter 0041 | Time 1.5989(1.7720) | Loss 7.334590(7.905675) | NFE Forward 14(14.0) | NFE Backward 126(135.9)
Iter 0042 | Time 1.6254(1.7617) | Loss 7.355865(7.867188) | NFE Forward 14(14.0) | NFE Backward 126(135.2)
Iter 0043 | Time 1.6261(1.7522) | Loss 7.295446(7.827166) | NFE Forward 14(14.0) | NFE Backward 126(134.5)
Iter 0044 | Time 1.4825(1.7333) | Loss 7.303817(7.790532) | NFE Forward 14(14.0) | NFE Backward 114(133.1)
Iter 0045 | Time 1.6520(1.7277) | Loss 7.322201(7.757749) | NFE Forward 14(14.0) | NFE Backward 126(132.6)
Iter 0046 | Time 1.5095(1.7124) | Loss 7.202320(7.718869) | NFE Forward 14(14.0) | NFE Backward 114(131.3)
Iter 0047 | Time 1.5357(1.7000) | Loss 7.252171(7.686200) | NFE Forward 14(14.0) | NFE Backward 114(130.1)
Iter 0048 | Time 1.4807(1.6847) | Loss 7.246337(7.655409) | NFE Forward 14(14.0) | NFE Backward 114(129.0)
Iter 0049 | Time 1.6015(1.6788) | Loss 7.211323(7.624323) | NFE Forward 14(14.0) | NFE Backward 126(128.8)
Iter 0050 | Time 1.6546(1.6771) | Loss 7.217341(7.595835) | NFE Forward 14(14.0) | NFE Backward 126(128.6)
Iter 0051 | Time 1.5112(1.6655) | Loss 7.198859(7.568046) | NFE Forward 14(14.0) | NFE Backward 114(127.5)
Iter 0052 | Time 1.5392(1.6567) | Loss 7.126257(7.537121) | NFE Forward 14(14.0) | NFE Backward 114(126.6)
Iter 0053 | Time 1.5390(1.6485) | Loss 7.172043(7.511566) | NFE Forward 14(14.0) | NFE Backward 114(125.7)
Iter 0054 | Time 1.3957(1.6308) | Loss 7.161476(7.487059) | NFE Forward 14(14.0) | NFE Backward 102(124.1)
Iter 0055 | Time 1.5380(1.6243) | Loss 7.147567(7.463295) | NFE Forward 14(14.0) | NFE Backward 114(123.4)
Iter 0056 | Time 1.5612(1.6199) | Loss 7.127177(7.439767) | NFE Forward 14(14.0) | NFE Backward 114(122.7)
Iter 0057 | Time 1.3913(1.6039) | Loss 7.104535(7.416300) | NFE Forward 14(14.0) | NFE Backward 102(121.2)
Iter 0058 | Time 1.4186(1.5909) | Loss 7.100783(7.394214) | NFE Forward 14(14.0) | NFE Backward 102(119.9)
Iter 0059 | Time 1.5641(1.5890) | Loss 7.092834(7.373117) | NFE Forward 14(14.0) | NFE Backward 114(119.5)
Iter 0060 | Time 1.5917(1.5892) | Loss 7.097281(7.353809) | NFE Forward 14(14.0) | NFE Backward 114(119.1)
Iter 0061 | Time 1.5664(1.5876) | Loss 7.080300(7.334663) | NFE Forward 14(14.0) | NFE Backward 114(118.7)
Iter 0062 | Time 1.4728(1.5796) | Loss 7.106249(7.318674) | NFE Forward 14(14.0) | NFE Backward 102(117.6)
Iter 0063 | Time 1.4443(1.5701) | Loss 7.080820(7.302025) | NFE Forward 14(14.0) | NFE Backward 102(116.5)
Iter 0064 | Time 1.4704(1.5631) | Loss 7.086061(7.286907) | NFE Forward 14(14.0) | NFE Backward 102(115.5)
Iter 0065 | Time 1.4465(1.5550) | Loss 7.067868(7.271574) | NFE Forward 14(14.0) | NFE Backward 102(114.5)
Iter 0066 | Time 1.4762(1.5494) | Loss 7.039664(7.255341) | NFE Forward 14(14.0) | NFE Backward 102(113.7)
Iter 0067 | Time 1.5869(1.5521) | Loss 7.052956(7.241174) | NFE Forward 14(14.0) | NFE Backward 114(113.7)
Iter 0068 | Time 1.5327(1.5507) | Loss 7.045708(7.227491) | NFE Forward 14(14.0) | NFE Backward 114(113.7)
Iter 0069 | Time 1.5615(1.5515) | Loss 7.059662(7.215743) | NFE Forward 14(14.0) | NFE Backward 114(113.7)
Iter 0070 | Time 1.4462(1.5441) | Loss 7.026733(7.202512) | NFE Forward 14(14.0) | NFE Backward 102(112.9)
Iter 0071 | Time 1.5962(1.5477) | Loss 7.005133(7.188696) | NFE Forward 14(14.0) | NFE Backward 114(113.0)
Iter 0072 | Time 1.4739(1.5426) | Loss 7.010466(7.176220) | NFE Forward 20(14.4) | NFE Backward 102(112.2)
Iter 0073 | Time 1.4455(1.5358) | Loss 7.043934(7.166960) | NFE Forward 14(14.4) | NFE Backward 102(111.5)
Iter 0074 | Time 1.5594(1.5374) | Loss 6.982516(7.154049) | NFE Forward 14(14.4) | NFE Backward 114(111.7)
Iter 0075 | Time 1.5588(1.5389) | Loss 6.981989(7.142004) | NFE Forward 14(14.3) | NFE Backward 114(111.8)
Iter 0076 | Time 1.4472(1.5325) | Loss 6.969278(7.129914) | NFE Forward 14(14.3) | NFE Backward 102(111.1)
Iter 0077 | Time 1.4452(1.5264) | Loss 6.988391(7.120007) | NFE Forward 14(14.3) | NFE Backward 102(110.5)
Iter 0078 | Time 1.4445(1.5207) | Loss 6.954866(7.108447) | NFE Forward 14(14.3) | NFE Backward 102(109.9)
Iter 0079 | Time 1.5321(1.5215) | Loss 6.987312(7.099968) | NFE Forward 14(14.3) | NFE Backward 108(109.8)
Iter 0080 | Time 1.4721(1.5180) | Loss 6.936859(7.088550) | NFE Forward 14(14.2) | NFE Backward 102(109.2)
Iter 0081 | Time 1.4672(1.5145) | Loss 6.946655(7.078617) | NFE Forward 20(14.6) | NFE Backward 102(108.7)
Iter 0082 | Time 1.4958(1.5131) | Loss 6.945632(7.069308) | NFE Forward 20(15.0) | NFE Backward 102(108.3)
Iter 0083 | Time 1.4430(1.5082) | Loss 6.920443(7.058888) | NFE Forward 14(14.9) | NFE Backward 102(107.8)
Iter 0084 | Time 1.5584(1.5117) | Loss 6.952826(7.051464) | NFE Forward 14(14.9) | NFE Backward 114(108.2)
Iter 0085 | Time 1.5861(1.5170) | Loss 6.860269(7.038080) | NFE Forward 20(15.2) | NFE Backward 114(108.7)
Iter 0086 | Time 1.5576(1.5198) | Loss 6.911075(7.029190) | NFE Forward 14(15.1) | NFE Backward 114(109.0)
Iter 0087 | Time 1.5304(1.5205) | Loss 6.870991(7.018116) | NFE Forward 14(15.1) | NFE Backward 114(109.4)
Iter 0088 | Time 1.5583(1.5232) | Loss 6.872494(7.007922) | NFE Forward 14(15.0) | NFE Backward 114(109.7)
Iter 0089 | Time 1.5638(1.5260) | Loss 6.879437(6.998928) | NFE Forward 14(14.9) | NFE Backward 114(110.0)
Iter 0090 | Time 1.5899(1.5305) | Loss 6.847491(6.988328) | NFE Forward 14(14.9) | NFE Backward 114(110.3)
Iter 0091 | Time 1.4454(1.5245) | Loss 6.846432(6.978395) | NFE Forward 14(14.8) | NFE Backward 102(109.7)
Iter 0092 | Time 1.4714(1.5208) | Loss 6.838494(6.968602) | NFE Forward 14(14.7) | NFE Backward 102(109.2)
Iter 0093 | Time 1.6102(1.5271) | Loss 6.859857(6.960990) | NFE Forward 20(15.1) | NFE Backward 114(109.5)
Iter 0094 | Time 1.4421(1.5211) | Loss 6.817008(6.950911) | NFE Forward 14(15.0) | NFE Backward 102(109.0)
Iter 0095 | Time 1.5582(1.5237) | Loss 6.762992(6.937757) | NFE Forward 14(15.0) | NFE Backward 114(109.3)
Iter 0096 | Time 1.4429(1.5181) | Loss 6.799840(6.928103) | NFE Forward 14(14.9) | NFE Backward 102(108.8)
Iter 0097 | Time 1.5573(1.5208) | Loss 6.759663(6.916312) | NFE Forward 14(14.8) | NFE Backward 114(109.2)
Iter 0098 | Time 1.4709(1.5173) | Loss 6.774218(6.906365) | NFE Forward 14(14.8) | NFE Backward 102(108.7)
Iter 0099 | Time 1.5874(1.5222) | Loss 6.751725(6.895540) | NFE Forward 14(14.7) | NFE Backward 114(109.0)
Iter 0100 | Time 1.5927(1.5272) | Loss 6.744266(6.884951) | NFE Forward 14(14.7) | NFE Backward 114(109.4)
[TEST] Iter 0100 | Test Loss 6.757403 | NFE 14
Skipping vis as data dimension is >2
Iter 0101 | Time 1.4416(1.5212) | Loss 6.730072(6.874110) | NFE Forward 14(14.6) | NFE Backward 102(108.9)
Iter 0102 | Time 1.5608(1.5239) | Loss 6.781401(6.867620) | NFE Forward 14(14.6) | NFE Backward 114(109.2)
Iter 0103 | Time 1.4436(1.5183) | Loss 6.690221(6.855202) | NFE Forward 14(14.5) | NFE Backward 102(108.7)
Iter 0104 | Time 1.5919(1.5235) | Loss 6.717668(6.845575) | NFE Forward 14(14.5) | NFE Backward 114(109.1)
Iter 0105 | Time 1.5634(1.5263) | Loss 6.661384(6.832681) | NFE Forward 14(14.5) | NFE Backward 114(109.4)
Iter 0106 | Time 1.4485(1.5208) | Loss 6.663709(6.820853) | NFE Forward 14(14.4) | NFE Backward 102(108.9)
Iter 0107 | Time 1.5609(1.5236) | Loss 6.664721(6.809924) | NFE Forward 14(14.4) | NFE Backward 114(109.3)
Iter 0108 | Time 1.5605(1.5262) | Loss 6.666580(6.799890) | NFE Forward 14(14.4) | NFE Backward 114(109.6)
Iter 0109 | Time 1.5631(1.5288) | Loss 6.653235(6.789624) | NFE Forward 14(14.3) | NFE Backward 114(109.9)
Iter 0110 | Time 1.5643(1.5313) | Loss 6.611124(6.777129) | NFE Forward 14(14.3) | NFE Backward 114(110.2)
Iter 0111 | Time 1.4752(1.5274) | Loss 6.665914(6.769344) | NFE Forward 14(14.3) | NFE Backward 102(109.6)
Iter 0112 | Time 1.5609(1.5297) | Loss 6.637329(6.760103) | NFE Forward 14(14.3) | NFE Backward 114(109.9)
Iter 0113 | Time 1.4736(1.5258) | Loss 6.561581(6.746206) | NFE Forward 14(14.3) | NFE Backward 102(109.4)
Iter 0114 | Time 1.5631(1.5284) | Loss 6.595440(6.735653) | NFE Forward 14(14.2) | NFE Backward 114(109.7)
Iter 0115 | Time 1.6160(1.5345) | Loss 6.575023(6.724409) | NFE Forward 20(14.6) | NFE Backward 114(110.0)
Iter 0116 | Time 1.6231(1.5407) | Loss 6.557133(6.712699) | NFE Forward 14(14.6) | NFE Backward 120(110.7)
Iter 0117 | Time 1.6167(1.5460) | Loss 6.610967(6.705578) | NFE Forward 14(14.6) | NFE Backward 114(110.9)
Iter 0118 | Time 1.6735(1.5550) | Loss 6.556471(6.695140) | NFE Forward 14(14.5) | NFE Backward 120(111.6)
Iter 0119 | Time 1.6722(1.5632) | Loss 6.557287(6.685491) | NFE Forward 20(14.9) | NFE Backward 120(112.2)
Iter 0120 | Time 1.6719(1.5708) | Loss 6.572733(6.677598) | NFE Forward 20(15.3) | NFE Backward 120(112.7)
Iter 0121 | Time 1.6721(1.5779) | Loss 6.509355(6.665821) | NFE Forward 20(15.6) | NFE Backward 120(113.2)
Iter 0122 | Time 1.6983(1.5863) | Loss 6.570173(6.659125) | NFE Forward 20(15.9) | NFE Backward 120(113.7)
Iter 0123 | Time 1.7015(1.5944) | Loss 6.515835(6.649095) | NFE Forward 20(16.2) | NFE Backward 120(114.1)
Iter 0124 | Time 1.6976(1.6016) | Loss 6.561468(6.642961) | NFE Forward 20(16.5) | NFE Backward 120(114.5)
Iter 0125 | Time 1.7547(1.6123) | Loss 6.531626(6.635168) | NFE Forward 20(16.7) | NFE Backward 126(115.3)
Iter 0126 | Time 1.7585(1.6225) | Loss 6.546728(6.628977) | NFE Forward 20(16.9) | NFE Backward 126(116.1)
Iter 0127 | Time 1.7593(1.6321) | Loss 6.519599(6.621320) | NFE Forward 20(17.1) | NFE Backward 126(116.8)
Iter 0128 | Time 1.8165(1.6450) | Loss 6.551576(6.616438) | NFE Forward 20(17.3) | NFE Backward 132(117.9)
Iter 0129 | Time 1.9370(1.6655) | Loss 6.485641(6.607282) | NFE Forward 20(17.5) | NFE Backward 144(119.7)
Iter 0130 | Time 1.7560(1.6718) | Loss 6.509239(6.600419) | NFE Forward 20(17.7) | NFE Backward 126(120.1)
Iter 0131 | Time 1.8161(1.6819) | Loss 6.515427(6.594470) | NFE Forward 20(17.9) | NFE Backward 132(121.0)
Iter 0132 | Time 1.7034(1.6834) | Loss 6.493918(6.587431) | NFE Forward 20(18.0) | NFE Backward 120(120.9)
Iter 0133 | Time 1.8742(1.6968) | Loss 6.502629(6.581495) | NFE Forward 20(18.2) | NFE Backward 138(122.1)
Iter 0134 | Time 1.8169(1.7052) | Loss 6.506875(6.576272) | NFE Forward 20(18.3) | NFE Backward 132(122.8)
Iter 0135 | Time 1.7559(1.7087) | Loss 6.521713(6.572453) | NFE Forward 20(18.4) | NFE Backward 126(123.0)
Iter 0136 | Time 1.7603(1.7123) | Loss 6.471766(6.565405) | NFE Forward 20(18.5) | NFE Backward 126(123.2)
Iter 0137 | Time 1.8170(1.7197) | Loss 6.508933(6.561452) | NFE Forward 20(18.6) | NFE Backward 132(123.8)
Iter 0138 | Time 1.8198(1.7267) | Loss 6.466777(6.554824) | NFE Forward 20(18.7) | NFE Backward 132(124.4)
Iter 0139 | Time 1.8176(1.7330) | Loss 6.476082(6.549312) | NFE Forward 20(18.8) | NFE Backward 132(124.9)
Iter 0140 | Time 1.8197(1.7391) | Loss 6.466837(6.543539) | NFE Forward 20(18.9) | NFE Backward 132(125.4)
Iter 0141 | Time 1.8222(1.7449) | Loss 6.481664(6.539208) | NFE Forward 20(19.0) | NFE Backward 132(125.9)
Iter 0142 | Time 1.7637(1.7462) | Loss 6.454792(6.533299) | NFE Forward 20(19.0) | NFE Backward 126(125.9)
Iter 0143 | Time 1.7670(1.7477) | Loss 6.506814(6.531445) | NFE Forward 20(19.1) | NFE Backward 126(125.9)
Iter 0144 | Time 1.7658(1.7490) | Loss 6.445365(6.525419) | NFE Forward 20(19.2) | NFE Backward 126(125.9)
Iter 0145 | Time 1.8328(1.7548) | Loss 6.456510(6.520596) | NFE Forward 20(19.2) | NFE Backward 132(126.3)
Iter 0146 | Time 1.7657(1.7556) | Loss 6.398702(6.512063) | NFE Forward 20(19.3) | NFE Backward 126(126.3)
Iter 0147 | Time 1.8253(1.7605) | Loss 6.427059(6.506113) | NFE Forward 20(19.3) | NFE Backward 132(126.7)
Iter 0148 | Time 1.7663(1.7609) | Loss 6.395658(6.498381) | NFE Forward 20(19.4) | NFE Backward 126(126.7)
Iter 0149 | Time 1.8318(1.7658) | Loss 6.411268(6.492283) | NFE Forward 20(19.4) | NFE Backward 132(127.0)
Iter 0150 | Time 1.7668(1.7659) | Loss 6.440076(6.488628) | NFE Forward 20(19.5) | NFE Backward 126(127.0)
Iter 0151 | Time 1.8275(1.7702) | Loss 6.432609(6.484707) | NFE Forward 20(19.5) | NFE Backward 132(127.3)
Iter 0152 | Time 1.7731(1.7704) | Loss 6.419179(6.480120) | NFE Forward 20(19.5) | NFE Backward 126(127.2)
Iter 0153 | Time 1.9503(1.7830) | Loss 6.403602(6.474764) | NFE Forward 20(19.6) | NFE Backward 144(128.4)
Iter 0154 | Time 1.7751(1.7825) | Loss 6.399434(6.469491) | NFE Forward 20(19.6) | NFE Backward 126(128.2)
Iter 0155 | Time 1.8320(1.7859) | Loss 6.413655(6.465582) | NFE Forward 20(19.6) | NFE Backward 132(128.5)
Iter 0156 | Time 1.8393(1.7897) | Loss 6.348957(6.457418) | NFE Forward 20(19.7) | NFE Backward 132(128.7)
Iter 0157 | Time 1.7762(1.7887) | Loss 6.378783(6.451914) | NFE Forward 20(19.7) | NFE Backward 126(128.5)
Iter 0158 | Time 1.8374(1.7921) | Loss 6.354018(6.445061) | NFE Forward 20(19.7) | NFE Backward 132(128.8)
Iter 0159 | Time 1.9540(1.8035) | Loss 6.381140(6.440587) | NFE Forward 20(19.7) | NFE Backward 144(129.9)
Iter 0160 | Time 1.8391(1.8060) | Loss 6.399700(6.437725) | NFE Forward 20(19.7) | NFE Backward 132(130.0)
Iter 0161 | Time 1.8392(1.8083) | Loss 6.396057(6.434808) | NFE Forward 20(19.8) | NFE Backward 132(130.1)
Iter 0162 | Time 1.8376(1.8103) | Loss 6.297676(6.425209) | NFE Forward 20(19.8) | NFE Backward 132(130.3)
Iter 0163 | Time 1.9557(1.8205) | Loss 6.326290(6.418284) | NFE Forward 20(19.8) | NFE Backward 144(131.2)
Iter 0164 | Time 1.9579(1.8301) | Loss 6.346894(6.413287) | NFE Forward 20(19.8) | NFE Backward 144(132.1)
Iter 0165 | Time 1.9573(1.8390) | Loss 6.322408(6.406925) | NFE Forward 20(19.8) | NFE Backward 144(133.0)
Iter 0166 | Time 1.8687(1.8411) | Loss 6.353167(6.403162) | NFE Forward 20(19.8) | NFE Backward 132(132.9)
Iter 0167 | Time 1.8431(1.8412) | Loss 6.319182(6.397284) | NFE Forward 20(19.8) | NFE Backward 132(132.8)
Iter 0168 | Time 1.9626(1.8497) | Loss 6.289198(6.389718) | NFE Forward 20(19.9) | NFE Backward 144(133.6)
Iter 0169 | Time 1.9596(1.8574) | Loss 6.308218(6.384013) | NFE Forward 20(19.9) | NFE Backward 144(134.3)
Iter 0170 | Time 1.9545(1.8642) | Loss 6.292892(6.377634) | NFE Forward 20(19.9) | NFE Backward 144(135.0)
Iter 0171 | Time 1.9534(1.8705) | Loss 6.281129(6.370879) | NFE Forward 20(19.9) | NFE Backward 144(135.6)
Iter 0172 | Time 1.8378(1.8682) | Loss 6.247387(6.362235) | NFE Forward 20(19.9) | NFE Backward 132(135.4)
Iter 0173 | Time 1.9560(1.8743) | Loss 6.250247(6.354395) | NFE Forward 20(19.9) | NFE Backward 144(136.0)
Iter 0174 | Time 1.9545(1.8799) | Loss 6.277206(6.348992) | NFE Forward 20(19.9) | NFE Backward 144(136.6)
Iter 0175 | Time 1.9538(1.8851) | Loss 6.247478(6.341886) | NFE Forward 20(19.9) | NFE Backward 144(137.1)
Iter 0176 | Time 1.9549(1.8900) | Loss 6.230546(6.334092) | NFE Forward 20(19.9) | NFE Backward 144(137.6)
Iter 0177 | Time 1.9588(1.8948) | Loss 6.250772(6.328260) | NFE Forward 20(19.9) | NFE Backward 144(138.0)
Iter 0178 | Time 1.9560(1.8991) | Loss 6.219993(6.320681) | NFE Forward 20(19.9) | NFE Backward 144(138.4)
Iter 0179 | Time 1.9548(1.9030) | Loss 6.240479(6.315067) | NFE Forward 20(19.9) | NFE Backward 144(138.8)
Iter 0180 | Time 1.9583(1.9069) | Loss 6.208460(6.307605) | NFE Forward 20(19.9) | NFE Backward 144(139.2)
Iter 0181 | Time 1.9002(1.9064) | Loss 6.203256(6.300300) | NFE Forward 20(19.9) | NFE Backward 138(139.1)
Iter 0182 | Time 1.8970(1.9057) | Loss 6.201762(6.293402) | NFE Forward 20(19.9) | NFE Backward 138(139.0)
Iter 0183 | Time 1.8998(1.9053) | Loss 6.200816(6.286921) | NFE Forward 20(20.0) | NFE Backward 138(139.0)
Iter 0184 | Time 1.9565(1.9089) | Loss 6.171173(6.278819) | NFE Forward 20(20.0) | NFE Backward 144(139.3)
Iter 0185 | Time 1.8959(1.9080) | Loss 6.163555(6.270751) | NFE Forward 20(20.0) | NFE Backward 138(139.2)
Iter 0186 | Time 1.8976(1.9073) | Loss 6.132141(6.261048) | NFE Forward 20(20.0) | NFE Backward 138(139.1)
Iter 0187 | Time 1.8965(1.9065) | Loss 6.175517(6.255061) | NFE Forward 20(20.0) | NFE Backward 138(139.0)
Iter 0188 | Time 1.8996(1.9060) | Loss 6.093469(6.243749) | NFE Forward 20(20.0) | NFE Backward 138(139.0)
Iter 0189 | Time 1.8994(1.9056) | Loss 6.119079(6.235022) | NFE Forward 20(20.0) | NFE Backward 138(138.9)
Iter 0190 | Time 1.8971(1.9050) | Loss 6.100731(6.225622) | NFE Forward 20(20.0) | NFE Backward 138(138.8)
Iter 0191 | Time 1.9566(1.9086) | Loss 6.093403(6.216367) | NFE Forward 20(20.0) | NFE Backward 144(139.2)
Iter 0192 | Time 1.9395(1.9107) | Loss 6.109213(6.208866) | NFE Forward 20(20.0) | NFE Backward 144(139.5)
Iter 0193 | Time 1.8802(1.9086) | Loss 6.052131(6.197894) | NFE Forward 20(20.0) | NFE Backward 138(139.4)
Iter 0194 | Time 1.9375(1.9106) | Loss 6.091786(6.190467) | NFE Forward 20(20.0) | NFE Backward 144(139.8)
Iter 0195 | Time 1.8814(1.9086) | Loss 6.029783(6.179219) | NFE Forward 20(20.0) | NFE Backward 138(139.6)
Iter 0196 | Time 1.8846(1.9069) | Loss 6.024668(6.168400) | NFE Forward 20(20.0) | NFE Backward 138(139.5)
Iter 0197 | Time 1.8810(1.9051) | Loss 6.025043(6.158365) | NFE Forward 20(20.0) | NFE Backward 138(139.4)
Iter 0198 | Time 1.8797(1.9033) | Loss 6.004670(6.147607) | NFE Forward 20(20.0) | NFE Backward 138(139.3)
Iter 0199 | Time 1.8818(1.9018) | Loss 5.993629(6.136828) | NFE Forward 20(20.0) | NFE Backward 138(139.2)
Iter 0200 | Time 1.9407(1.9045) | Loss 5.984060(6.126134) | NFE Forward 20(20.0) | NFE Backward 144(139.6)
[TEST] Iter 0200 | Test Loss 5.955402 | NFE 20
Skipping vis as data dimension is >2
Iter 0201 | Time 1.9349(1.9067) | Loss 6.008430(6.117895) | NFE Forward 20(20.0) | NFE Backward 144(139.9)
Iter 0202 | Time 1.8842(1.9051) | Loss 5.943787(6.105708) | NFE Forward 20(20.0) | NFE Backward 138(139.7)
Iter 0203 | Time 1.8906(1.9041) | Loss 5.956751(6.095281) | NFE Forward 20(20.0) | NFE Backward 138(139.6)
Iter 0204 | Time 1.9488(1.9072) | Loss 5.912519(6.082487) | NFE Forward 20(20.0) | NFE Backward 144(139.9)
Iter 0205 | Time 1.9472(1.9100) | Loss 5.932334(6.071977) | NFE Forward 20(20.0) | NFE Backward 144(140.2)
Iter 0206 | Time 1.9476(1.9126) | Loss 5.899208(6.059883) | NFE Forward 20(20.0) | NFE Backward 144(140.5)
Iter 0207 | Time 1.9642(1.9162) | Loss 5.890640(6.048036) | NFE Forward 20(20.0) | NFE Backward 144(140.7)
Iter 0208 | Time 1.9782(1.9206) | Loss 5.911300(6.038464) | NFE Forward 20(20.0) | NFE Backward 144(140.9)
Iter 0209 | Time 1.9636(1.9236) | Loss 5.906520(6.029228) | NFE Forward 20(20.0) | NFE Backward 144(141.2)
Iter 0210 | Time 1.9655(1.9265) | Loss 5.888303(6.019363) | NFE Forward 20(20.0) | NFE Backward 144(141.4)
Iter 0211 | Time 1.9630(1.9291) | Loss 5.848775(6.007422) | NFE Forward 20(20.0) | NFE Backward 144(141.5)
Iter 0212 | Time 1.9642(1.9315) | Loss 5.877208(5.998307) | NFE Forward 20(20.0) | NFE Backward 144(141.7)
Iter 0213 | Time 1.9635(1.9338) | Loss 5.826449(5.986277) | NFE Forward 20(20.0) | NFE Backward 144(141.9)
Iter 0214 | Time 1.9640(1.9359) | Loss 5.836379(5.975784) | NFE Forward 20(20.0) | NFE Backward 144(142.0)
Iter 0215 | Time 1.9673(1.9381) | Loss 5.836134(5.966009) | NFE Forward 20(20.0) | NFE Backward 144(142.2)
Iter 0216 | Time 1.9620(1.9398) | Loss 5.846865(5.957669) | NFE Forward 20(20.0) | NFE Backward 144(142.3)
Iter 0217 | Time 1.9638(1.9414) | Loss 5.827922(5.948586) | NFE Forward 20(20.0) | NFE Backward 144(142.4)
Iter 0218 | Time 1.9660(1.9432) | Loss 5.787573(5.937316) | NFE Forward 20(20.0) | NFE Backward 144(142.5)
Iter 0219 | Time 1.9643(1.9446) | Loss 5.789978(5.927002) | NFE Forward 20(20.0) | NFE Backward 144(142.6)
Iter 0220 | Time 1.9633(1.9460) | Loss 5.791696(5.917530) | NFE Forward 20(20.0) | NFE Backward 144(142.7)
Iter 0221 | Time 1.9639(1.9472) | Loss 5.794932(5.908949) | NFE Forward 20(20.0) | NFE Backward 144(142.8)
Iter 0222 | Time 1.9739(1.9491) | Loss 5.804194(5.901616) | NFE Forward 20(20.0) | NFE Backward 144(142.9)
Iter 0223 | Time 1.9696(1.9505) | Loss 5.789808(5.893789) | NFE Forward 20(20.0) | NFE Backward 144(143.0)
Iter 0224 | Time 1.9660(1.9516) | Loss 5.799236(5.887171) | NFE Forward 20(20.0) | NFE Backward 144(143.0)
Iter 0225 | Time 1.9623(1.9523) | Loss 5.788774(5.880283) | NFE Forward 20(20.0) | NFE Backward 144(143.1)
Iter 0226 | Time 1.9626(1.9531) | Loss 5.791587(5.874074) | NFE Forward 20(20.0) | NFE Backward 144(143.2)
Iter 0227 | Time 1.9664(1.9540) | Loss 5.742929(5.864894) | NFE Forward 20(20.0) | NFE Backward 144(143.2)
Iter 0228 | Time 1.9638(1.9547) | Loss 5.746096(5.856578) | NFE Forward 20(20.0) | NFE Backward 144(143.3)
Iter 0229 | Time 1.9645(1.9554) | Loss 5.756027(5.849540) | NFE Forward 20(20.0) | NFE Backward 144(143.3)
Iter 0230 | Time 1.9658(1.9561) | Loss 5.734987(5.841521) | NFE Forward 20(20.0) | NFE Backward 144(143.4)
Iter 0231 | Time 1.9671(1.9569) | Loss 5.755117(5.835473) | NFE Forward 20(20.0) | NFE Backward 144(143.4)
Iter 0232 | Time 1.9652(1.9575) | Loss 5.743825(5.829057) | NFE Forward 20(20.0) | NFE Backward 144(143.5)
Iter 0233 | Time 1.9660(1.9580) | Loss 5.760581(5.824264) | NFE Forward 20(20.0) | NFE Backward 144(143.5)
Iter 0234 | Time 1.9656(1.9586) | Loss 5.743564(5.818615) | NFE Forward 20(20.0) | NFE Backward 144(143.5)
Iter 0235 | Time 1.9522(1.9581) | Loss 5.721539(5.811820) | NFE Forward 20(20.0) | NFE Backward 144(143.6)
Iter 0236 | Time 1.9471(1.9574) | Loss 5.738442(5.806683) | NFE Forward 20(20.0) | NFE Backward 144(143.6)
Iter 0237 | Time 1.9497(1.9568) | Loss 5.760239(5.803432) | NFE Forward 20(20.0) | NFE Backward 144(143.6)
Iter 0238 | Time 1.9498(1.9563) | Loss 5.730726(5.798343) | NFE Forward 20(20.0) | NFE Backward 144(143.7)
Iter 0239 | Time 1.9529(1.9561) | Loss 5.695800(5.791165) | NFE Forward 20(20.0) | NFE Backward 144(143.7)
Iter 0240 | Time 1.9475(1.9555) | Loss 5.739498(5.787548) | NFE Forward 20(20.0) | NFE Backward 144(143.7)
Iter 0241 | Time 1.9484(1.9550) | Loss 5.741899(5.784353) | NFE Forward 20(20.0) | NFE Backward 144(143.7)
Iter 0242 | Time 1.9485(1.9545) | Loss 5.690208(5.777762) | NFE Forward 20(20.0) | NFE Backward 144(143.7)
Iter 0243 | Time 1.9489(1.9542) | Loss 5.714485(5.773333) | NFE Forward 20(20.0) | NFE Backward 144(143.8)
Iter 0244 | Time 1.9480(1.9537) | Loss 5.695996(5.767919) | NFE Forward 20(20.0) | NFE Backward 144(143.8)
Iter 0245 | Time 2.0067(1.9574) | Loss 5.687671(5.762302) | NFE Forward 20(20.0) | NFE Backward 150(144.2)
Iter 0246 | Time 1.9457(1.9566) | Loss 5.723979(5.759619) | NFE Forward 20(20.0) | NFE Backward 144(144.2)
Iter 0247 | Time 1.9424(1.9556) | Loss 5.665385(5.753023) | NFE Forward 20(20.0) | NFE Backward 144(144.2)
True
Warning: Clipping dimensionality to 5
Density Loss 0.13210906088352203
Density Loss 0.13177283108234406
Density Loss 0.13476777076721191
Density Loss 0.1338060349225998
Density Loss 0.1291569620370865
Density Loss 0.13306009769439697
Density Loss 0.12396083772182465
Density Loss 0.1287621706724167
Density Loss 0.12588725984096527
Density Loss 0.12763330340385437
Density Loss 0.1281084269285202
Density Loss 0.1258411407470703
Density Loss 0.12026677280664444
Density Loss 0.12425815314054489
Density Loss 0.12457025796175003
Density Loss 0.13219855725765228
Density Loss 0.12235518544912338
Density Loss 0.12184897065162659
Density Loss 0.12086571007966995
Density Loss 0.1250736117362976
Density Loss 0.12421943247318268
Density Loss 0.12107440829277039
Density Loss 0.12442752718925476
Density Loss 0.12812486290931702
Density Loss 0.12955373525619507
Density Loss 0.1260351836681366
Density Loss 0.12576425075531006
Density Loss 0.13068576157093048
Density Loss 0.12661434710025787
Density Loss 0.12533815205097198
Density Loss 0.12406262010335922
Density Loss 0.12458495795726776
Density Loss 0.1241413876414299
Density Loss 0.12588103115558624
Density Loss 0.12132642418146133
Density Loss 0.12045449018478394
Density Loss 0.12681932747364044
Density Loss 0.12720371782779694
Density Loss 0.12255080044269562
Density Loss 0.12313717603683472
Density Loss 0.12283317744731903
Density Loss 0.12400723248720169
Density Loss 0.12161631882190704
Density Loss 0.12905146181583405
Density Loss 0.1224924623966217
Density Loss 0.126572847366333
Density Loss 0.12356250733137131
Density Loss 0.12627045810222626
Density Loss 0.1229015439748764
Density Loss 0.12180852890014648
Density Loss 0.12114052474498749
Density Loss 0.12267707288265228
Density Loss 0.1217079684138298
Density Loss 0.12335262447595596
Density Loss 0.12521761655807495
Density Loss 0.12082555890083313
Density Loss 0.12364137917757034
Density Loss 0.12202716618776321
Density Loss 0.12441632896661758
Density Loss 0.12129644304513931
Density Loss 0.12355612218379974
Density Loss 0.1280321478843689
Density Loss 0.11998212337493896
Density Loss 0.12590830028057098
Density Loss 0.12624408304691315
Density Loss 0.1243106871843338
Density Loss 0.12483764439821243
Density Loss 0.1251819133758545
Density Loss 0.12884408235549927
Density Loss 0.13178078830242157
Density Loss 0.1364227682352066
Density Loss 0.13489042222499847
Density Loss 0.13656190037727356
Density Loss 0.13873964548110962
Density Loss 0.14349383115768433
Density Loss 0.14285632967948914
Density Loss 0.1452075093984604
Density Loss 0.1434512585401535
Density Loss 0.14673298597335815
Density Loss 0.14892059564590454
Density Loss 0.14648188650608063
Density Loss 0.15790502727031708
Density Loss 0.15567292273044586
Density Loss 0.15841728448867798
Density Loss 0.1585211604833603
Density Loss 0.16040600836277008
Density Loss 0.16300846636295319
Density Loss 0.16010990738868713
Density Loss 0.15948082506656647
Density Loss 0.16416704654693604
Density Loss 0.16687709093093872
Density Loss 0.1634892225265503
Density Loss 0.17030315101146698
Density Loss 0.16662199795246124
Density Loss 0.17173396050930023
Density Loss 0.173207625746727
Density Loss 0.1700994372367859
Density Loss 0.17332561314105988
Density Loss 0.17419323325157166
Density Loss 0.17386016249656677
Density Loss 0.17397728562355042
Density Loss 0.1766740381717682
Density Loss 0.1725229024887085
Density Loss 0.17776483297348022
Density Loss 0.17332381010055542
Density Loss 0.17938381433486938
Density Loss 0.1727907955646515
Density Loss 0.17741598188877106
Density Loss 0.17660531401634216
Density Loss 0.18063461780548096
Density Loss 0.17581675946712494
Density Loss 0.1740906983613968
Density Loss 0.18104977905750275
Density Loss 0.18201279640197754
Density Loss 0.18528343737125397
Density Loss 0.17664803564548492
Density Loss 0.1780485361814499
Density Loss 0.17820623517036438
Density Loss 0.18353348970413208
Density Loss 0.18185938894748688
Density Loss 0.18259193003177643
Density Loss 0.18024155497550964
Density Loss 0.18166832625865936
Density Loss 0.18775421380996704
Density Loss 0.18606041371822357
Density Loss 0.18969933688640594
Density Loss 0.1903911828994751
Density Loss 0.18941891193389893
Density Loss 0.18416056036949158
Density Loss 0.18604899942874908
Density Loss 0.19093191623687744
Density Loss 0.19077396392822266
Density Loss 0.1912817656993866
Density Loss 0.1871187686920166
Density Loss 0.19259744882583618
Density Loss 0.19321554899215698
Density Loss 0.19609031081199646
Density Loss 0.1949804276227951
Density Loss 0.19519491493701935
Density Loss 0.1946553885936737
Density Loss 0.19345279037952423
Density Loss 0.19717340171337128
Density Loss 0.19932135939598083
Density Loss 0.19662316143512726
Density Loss 0.2010144740343094
Density Loss 0.19336120784282684
Density Loss 0.19601237773895264
Density Loss 0.19427387416362762
Density Loss 0.20014826953411102
Density Loss 0.19784684479236603
Density Loss 0.204414501786232
Density Loss 0.20006339251995087
Density Loss 0.20250999927520752
Density Loss 0.2033420354127884
Density Loss 0.2017551213502884
Density Loss 0.201350599527359
Density Loss 0.20471803843975067
Density Loss 0.20120105147361755
Density Loss 0.20251737534999847
Density Loss 0.20760761201381683
Density Loss 0.20572875440120697
Density Loss 0.2055394947528839
Density Loss 0.2048395276069641
Density Loss 0.20618973672389984
Density Loss 0.2063029259443283
Density Loss 0.20209474861621857
Density Loss 0.2062987983226776
Density Loss 0.20615936815738678
Density Loss 0.20386581122875214
Density Loss 0.20877547562122345
Density Loss 0.20289288461208344
Density Loss 0.20340687036514282
Density Loss 0.20798222720623016
Density Loss 0.20993642508983612
Density Loss 0.20885932445526123
Density Loss 0.2052297741174698
Density Loss 0.21205377578735352
Density Loss 0.20586071908473969
Density Loss 0.21368622779846191
Density Loss 0.2105390578508377
Density Loss 0.21313060820102692
Density Loss 0.2034730613231659
Density Loss 0.2153327763080597
Density Loss 0.20490145683288574
Density Loss 0.21675819158554077
Density Loss 0.21777768433094025
Density Loss 0.2153950184583664
Density Loss 0.20924828946590424
Density Loss 0.20823007822036743
Density Loss 0.21218529343605042
Density Loss 0.21140624582767487
Density Loss 0.21272489428520203
Density Loss 0.21404828131198883
Density Loss 0.21711650490760803
Density Loss 0.22505559027194977
Density Loss 0.22212326526641846
Density Loss 0.21326109766960144
Density Loss 0.22446562349796295
Density Loss 0.22760140895843506
Density Loss 0.22760163247585297
Density Loss 0.23005948960781097
Density Loss 0.2304525524377823
Density Loss 0.2244047373533249
Density Loss 0.22177568078041077
Density Loss 0.2244587391614914
Density Loss 0.23909872770309448
Density Loss 0.23314803838729858
Density Loss 0.23573081195354462
Density Loss 0.23463402688503265
Density Loss 0.23549355566501617
Density Loss 0.24589335918426514
Density Loss 0.24826495349407196
Density Loss 0.24288344383239746
Density Loss 0.254038006067276
Density Loss 0.2622949779033661
Density Loss 0.26235926151275635
Density Loss 0.2532086968421936
Density Loss 0.266194224357605
Density Loss 0.27509447932243347
Density Loss 0.2720476984977722
Density Loss 0.2767185568809509
Density Loss 0.281005859375
Density Loss 0.28180545568466187
Density Loss 0.29104095697402954
Density Loss 0.2907642126083374
Density Loss 0.2859564423561096
Density Loss 0.2988934814929962
Density Loss 0.29294484853744507
Density Loss 0.29926684498786926
Density Loss 0.30478692054748535
Density Loss 0.3089229166507721
Density Loss 0.3091104328632355
Density Loss 0.3119639456272125
Density Loss 0.31412985920906067
Density Loss 0.3204321563243866
Density Loss 0.31585025787353516
Density Loss 0.3111954629421234
Density Loss 0.32244062423706055
Density Loss 0.3177626430988312
Density Loss 0.328848272562027
Density Loss 0.3272128701210022
Density Loss 0.33174842596054077
Density Loss 0.3313232362270355
Density Loss 0.32638847827911377
Density Loss 0.33731961250305176
Density Loss 0.3345792293548584
Density Loss 0.3313305377960205
Density Loss 0.33830657601356506
Density Loss 0.34077557921409607
Iter 0248 | Time 2.0939(1.9653) | Loss 5.685529(5.748298) | NFE Forward 20(20.0) | NFE Backward 144(144.2)
Iter 0249 | Time 2.0001(1.9677) | Loss 5.669011(5.742748) | NFE Forward 20(20.0) | NFE Backward 150(144.6)
Iter 0250 | Time 1.9442(1.9661) | Loss 5.708893(5.740378) | NFE Forward 20(20.0) | NFE Backward 144(144.5)
Iter 0251 | Time 1.9678(1.9662) | Loss 5.664450(5.735063) | NFE Forward 26(20.4) | NFE Backward 144(144.5)
Iter 0252 | Time 1.9696(1.9664) | Loss 5.680000(5.731209) | NFE Forward 26(20.8) | NFE Backward 144(144.5)
Iter 0253 | Time 2.0308(1.9710) | Loss 5.628992(5.724054) | NFE Forward 26(21.2) | NFE Backward 150(144.9)
Iter 0254 | Time 1.9703(1.9709) | Loss 5.667716(5.720110) | NFE Forward 26(21.5) | NFE Backward 144(144.8)
Iter 0255 | Time 2.0270(1.9748) | Loss 5.654430(5.715513) | NFE Forward 26(21.8) | NFE Backward 150(145.2)
Iter 0256 | Time 2.0277(1.9785) | Loss 5.657428(5.711447) | NFE Forward 26(22.1) | NFE Backward 150(145.5)
Iter 0257 | Time 1.9678(1.9778) | Loss 5.689026(5.709877) | NFE Forward 26(22.4) | NFE Backward 144(145.4)
Iter 0258 | Time 1.9458(1.9755) | Loss 5.649311(5.705638) | NFE Forward 20(22.2) | NFE Backward 144(145.3)
Iter 0259 | Time 2.0038(1.9775) | Loss 5.663151(5.702663) | NFE Forward 20(22.1) | NFE Backward 150(145.6)
Iter 0260 | Time 1.9716(1.9771) | Loss 5.682167(5.701229) | NFE Forward 26(22.3) | NFE Backward 144(145.5)
Iter 0261 | Time 1.9738(1.9769) | Loss 5.656106(5.698070) | NFE Forward 26(22.6) | NFE Backward 144(145.4)
Iter 0262 | Time 2.0285(1.9805) | Loss 5.623555(5.692854) | NFE Forward 26(22.8) | NFE Backward 150(145.7)
Iter 0263 | Time 2.0034(1.9821) | Loss 5.659452(5.690516) | NFE Forward 20(22.6) | NFE Backward 150(146.0)
Iter 0264 | Time 2.0280(1.9853) | Loss 5.690794(5.690535) | NFE Forward 26(22.9) | NFE Backward 150(146.3)
Iter 0265 | Time 2.0264(1.9882) | Loss 5.660279(5.688417) | NFE Forward 26(23.1) | NFE Backward 150(146.6)
Iter 0266 | Time 2.0274(1.9909) | Loss 5.653003(5.685938) | NFE Forward 26(23.3) | NFE Backward 150(146.8)
Iter 0267 | Time 2.0275(1.9935) | Loss 5.621610(5.681435) | NFE Forward 26(23.5) | NFE Backward 150(147.0)
Iter 0268 | Time 2.0288(1.9960) | Loss 5.629742(5.677817) | NFE Forward 26(23.7) | NFE Backward 150(147.2)
Iter 0269 | Time 2.0058(1.9966) | Loss 5.605756(5.672773) | NFE Forward 20(23.4) | NFE Backward 150(147.4)
Iter 0270 | Time 2.0373(1.9995) | Loss 5.623728(5.669340) | NFE Forward 26(23.6) | NFE Backward 150(147.6)
Iter 0271 | Time 2.0367(2.0021) | Loss 5.607391(5.665003) | NFE Forward 26(23.8) | NFE Backward 150(147.8)
Iter 0272 | Time 2.0354(2.0044) | Loss 5.644848(5.663592) | NFE Forward 26(23.9) | NFE Backward 150(147.9)
Iter 0273 | Time 2.0358(2.0066) | Loss 5.625104(5.660898) | NFE Forward 26(24.1) | NFE Backward 150(148.1)
Iter 0274 | Time 2.0372(2.0088) | Loss 5.636712(5.659205) | NFE Forward 26(24.2) | NFE Backward 150(148.2)
Iter 0275 | Time 2.0129(2.0090) | Loss 5.608507(5.655656) | NFE Forward 20(23.9) | NFE Backward 150(148.3)
Iter 0276 | Time 2.0396(2.0112) | Loss 5.608768(5.652374) | NFE Forward 26(24.0) | NFE Backward 150(148.5)
Iter 0277 | Time 2.0145(2.0114) | Loss 5.602769(5.648902) | NFE Forward 20(23.8) | NFE Backward 150(148.6)
Iter 0278 | Time 2.0104(2.0113) | Loss 5.594454(5.645090) | NFE Forward 20(23.5) | NFE Backward 150(148.7)
Iter 0279 | Time 1.9983(2.0104) | Loss 5.612619(5.642817) | NFE Forward 20(23.3) | NFE Backward 150(148.8)
Iter 0280 | Time 1.9915(2.0091) | Loss 5.583425(5.638660) | NFE Forward 20(23.0) | NFE Backward 150(148.8)
Iter 0281 | Time 2.0160(2.0096) | Loss 5.543221(5.631979) | NFE Forward 26(23.2) | NFE Backward 150(148.9)
Iter 0282 | Time 1.9954(2.0086) | Loss 5.597652(5.629576) | NFE Forward 20(23.0) | NFE Backward 150(149.0)
Iter 0283 | Time 1.9991(2.0079) | Loss 5.576221(5.625841) | NFE Forward 20(22.8) | NFE Backward 150(149.1)
Iter 0284 | Time 1.9939(2.0070) | Loss 5.597573(5.623863) | NFE Forward 20(22.6) | NFE Backward 150(149.1)
Iter 0285 | Time 2.0055(2.0068) | Loss 5.576615(5.620555) | NFE Forward 20(22.4) | NFE Backward 150(149.2)
Iter 0286 | Time 2.0104(2.0071) | Loss 5.561300(5.616407) | NFE Forward 20(22.3) | NFE Backward 150(149.3)
Iter 0287 | Time 2.0005(2.0066) | Loss 5.548493(5.611654) | NFE Forward 20(22.1) | NFE Backward 150(149.3)
Iter 0288 | Time 2.0019(2.0063) | Loss 5.587155(5.609939) | NFE Forward 20(21.9) | NFE Backward 150(149.4)
Iter 0289 | Time 2.0008(2.0059) | Loss 5.573034(5.607355) | NFE Forward 20(21.8) | NFE Backward 150(149.4)
Iter 0290 | Time 2.0585(2.0096) | Loss 5.557947(5.603897) | NFE Forward 20(21.7) | NFE Backward 156(149.9)
Iter 0291 | Time 2.0611(2.0132) | Loss 5.567557(5.601353) | NFE Forward 20(21.6) | NFE Backward 156(150.3)
Iter 0292 | Time 2.0574(2.0163) | Loss 5.583779(5.600123) | NFE Forward 20(21.5) | NFE Backward 156(150.7)
Iter 0293 | Time 2.0584(2.0193) | Loss 5.564404(5.597622) | NFE Forward 20(21.4) | NFE Backward 156(151.1)
Iter 0294 | Time 2.0602(2.0221) | Loss 5.560371(5.595015) | NFE Forward 20(21.3) | NFE Backward 156(151.4)
Iter 0295 | Time 2.0693(2.0254) | Loss 5.589104(5.594601) | NFE Forward 20(21.2) | NFE Backward 156(151.7)
Iter 0296 | Time 2.0739(2.0288) | Loss 5.598058(5.594843) | NFE Forward 20(21.1) | NFE Backward 156(152.0)
Iter 0297 | Time 2.0722(2.0319) | Loss 5.538700(5.590913) | NFE Forward 20(21.0) | NFE Backward 156(152.3)
Iter 0298 | Time 2.0822(2.0354) | Loss 5.573984(5.589728) | NFE Forward 20(20.9) | NFE Backward 156(152.6)
Iter 0299 | Time 2.0762(2.0382) | Loss 5.562727(5.587838) | NFE Forward 20(20.9) | NFE Backward 156(152.8)
Iter 0300 | Time 2.0751(2.0408) | Loss 5.560860(5.585950) | NFE Forward 20(20.8) | NFE Backward 156(153.0)
[TEST] Iter 0300 | Test Loss 5.537739 | NFE 20
Skipping vis as data dimension is >2
Iter 0301 | Time 2.0719(2.0430) | Loss 5.561346(5.584227) | NFE Forward 20(20.8) | NFE Backward 156(153.2)
Iter 0302 | Time 2.0737(2.0451) | Loss 5.537640(5.580966) | NFE Forward 20(20.7) | NFE Backward 156(153.4)
Iter 0303 | Time 2.0761(2.0473) | Loss 5.544763(5.578432) | NFE Forward 20(20.7) | NFE Backward 156(153.6)
Iter 0304 | Time 2.0569(2.0480) | Loss 5.597000(5.579732) | NFE Forward 20(20.6) | NFE Backward 156(153.8)
Iter 0305 | Time 2.0645(2.0491) | Loss 5.557058(5.578145) | NFE Forward 20(20.6) | NFE Backward 156(153.9)
Iter 0306 | Time 2.0574(2.0497) | Loss 5.539016(5.575406) | NFE Forward 20(20.5) | NFE Backward 156(154.1)
Iter 0307 | Time 2.0574(2.0503) | Loss 5.552461(5.573799) | NFE Forward 20(20.5) | NFE Backward 156(154.2)
Iter 0308 | Time 2.0596(2.0509) | Loss 5.541531(5.571541) | NFE Forward 20(20.5) | NFE Backward 156(154.3)
Iter 0309 | Time 2.0584(2.0514) | Loss 5.556806(5.570509) | NFE Forward 20(20.4) | NFE Backward 156(154.5)
Iter 0310 | Time 2.0564(2.0518) | Loss 5.534858(5.568014) | NFE Forward 20(20.4) | NFE Backward 156(154.6)
Iter 0311 | Time 2.0591(2.0523) | Loss 5.579520(5.568819) | NFE Forward 20(20.4) | NFE Backward 156(154.7)
Iter 0312 | Time 2.0573(2.0526) | Loss 5.509792(5.564687) | NFE Forward 20(20.3) | NFE Backward 156(154.8)
Iter 0313 | Time 2.0573(2.0530) | Loss 5.535476(5.562642) | NFE Forward 20(20.3) | NFE Backward 156(154.8)
Iter 0314 | Time 2.0565(2.0532) | Loss 5.550681(5.561805) | NFE Forward 20(20.3) | NFE Backward 156(154.9)
Iter 0315 | Time 2.0573(2.0535) | Loss 5.540405(5.560307) | NFE Forward 20(20.3) | NFE Backward 156(155.0)
Iter 0316 | Time 2.0802(2.0554) | Loss 5.529691(5.558164) | NFE Forward 26(20.7) | NFE Backward 156(155.1)
Iter 0317 | Time 2.0567(2.0555) | Loss 5.529434(5.556153) | NFE Forward 20(20.6) | NFE Backward 156(155.1)
Iter 0318 | Time 2.0560(2.0555) | Loss 5.513088(5.553138) | NFE Forward 20(20.6) | NFE Backward 156(155.2)
Iter 0319 | Time 2.0569(2.0556) | Loss 5.543017(5.552430) | NFE Forward 20(20.5) | NFE Backward 156(155.3)
Iter 0320 | Time 2.0582(2.0558) | Loss 5.511023(5.549531) | NFE Forward 20(20.5) | NFE Backward 156(155.3)
Iter 0321 | Time 2.0557(2.0558) | Loss 5.530949(5.548231) | NFE Forward 20(20.5) | NFE Backward 156(155.4)
Iter 0322 | Time 2.0580(2.0559) | Loss 5.526533(5.546712) | NFE Forward 20(20.4) | NFE Backward 156(155.4)
Iter 0323 | Time 2.0568(2.0560) | Loss 5.555581(5.547333) | NFE Forward 20(20.4) | NFE Backward 156(155.4)
Iter 0324 | Time 2.0583(2.0561) | Loss 5.513851(5.544989) | NFE Forward 20(20.4) | NFE Backward 156(155.5)
Iter 0325 | Time 2.0574(2.0562) | Loss 5.509975(5.542538) | NFE Forward 20(20.4) | NFE Backward 156(155.5)
Iter 0326 | Time 2.0827(2.0581) | Loss 5.463199(5.536984) | NFE Forward 26(20.7) | NFE Backward 156(155.5)
Iter 0327 | Time 2.0863(2.0601) | Loss 5.502748(5.534588) | NFE Forward 26(21.1) | NFE Backward 156(155.6)
Iter 0328 | Time 2.0591(2.0600) | Loss 5.489775(5.531451) | NFE Forward 20(21.0) | NFE Backward 156(155.6)
Iter 0329 | Time 2.0570(2.0598) | Loss 5.536750(5.531822) | NFE Forward 20(21.0) | NFE Backward 156(155.6)
Iter 0330 | Time 2.0583(2.0597) | Loss 5.474711(5.527824) | NFE Forward 20(20.9) | NFE Backward 156(155.7)
Iter 0331 | Time 2.0850(2.0615) | Loss 5.520307(5.527298) | NFE Forward 26(21.3) | NFE Backward 156(155.7)
Iter 0332 | Time 2.0841(2.0630) | Loss 5.533433(5.527727) | NFE Forward 26(21.6) | NFE Backward 156(155.7)
Iter 0333 | Time 2.0571(2.0626) | Loss 5.483296(5.524617) | NFE Forward 20(21.5) | NFE Backward 156(155.7)
Iter 0334 | Time 2.0996(2.0652) | Loss 5.480460(5.521526) | NFE Forward 26(21.8) | NFE Backward 156(155.7)
Iter 0335 | Time 2.0824(2.0664) | Loss 5.514124(5.521008) | NFE Forward 26(22.1) | NFE Backward 156(155.8)
Iter 0336 | Time 2.0598(2.0660) | Loss 5.481862(5.518268) | NFE Forward 20(21.9) | NFE Backward 156(155.8)
Iter 0337 | Time 2.0939(2.0679) | Loss 5.505810(5.517396) | NFE Forward 26(22.2) | NFE Backward 156(155.8)
Iter 0338 | Time 2.0625(2.0675) | Loss 5.448216(5.512553) | NFE Forward 20(22.1) | NFE Backward 156(155.8)
Iter 0339 | Time 2.0612(2.0671) | Loss 5.504679(5.512002) | NFE Forward 20(21.9) | NFE Backward 156(155.8)
Iter 0340 | Time 2.1153(2.0705) | Loss 5.447509(5.507487) | NFE Forward 26(22.2) | NFE Backward 156(155.8)
Iter 0341 | Time 2.0670(2.0702) | Loss 5.465243(5.504530) | NFE Forward 20(22.1) | NFE Backward 156(155.8)
Iter 0342 | Time 2.0937(2.0719) | Loss 5.485644(5.503208) | NFE Forward 20(21.9) | NFE Backward 156(155.9)
Iter 0343 | Time 2.0913(2.0732) | Loss 5.484897(5.501926) | NFE Forward 20(21.8) | NFE Backward 156(155.9)
Iter 0344 | Time 2.0910(2.0745) | Loss 5.460620(5.499035) | NFE Forward 20(21.7) | NFE Backward 156(155.9)
Iter 0345 | Time 2.0911(2.0756) | Loss 5.434469(5.494515) | NFE Forward 20(21.5) | NFE Backward 156(155.9)
Iter 0346 | Time 2.0901(2.0767) | Loss 5.520014(5.496300) | NFE Forward 20(21.4) | NFE Backward 156(155.9)
Iter 0347 | Time 2.0907(2.0776) | Loss 5.431917(5.491794) | NFE Forward 20(21.3) | NFE Backward 156(155.9)
Iter 0348 | Time 2.0948(2.0788) | Loss 5.472907(5.490471) | NFE Forward 20(21.2) | NFE Backward 156(155.9)
Iter 0349 | Time 2.0959(2.0800) | Loss 5.504851(5.491478) | NFE Forward 20(21.1) | NFE Backward 156(155.9)
Iter 0350 | Time 2.0951(2.0811) | Loss 5.436507(5.487630) | NFE Forward 20(21.1) | NFE Backward 156(155.9)
Iter 0351 | Time 2.1285(2.0844) | Loss 5.462539(5.485874) | NFE Forward 26(21.4) | NFE Backward 156(155.9)
Iter 0352 | Time 2.2285(2.0945) | Loss 5.459304(5.484014) | NFE Forward 26(21.7) | NFE Backward 156(155.9)
Iter 0353 | Time 2.1303(2.0970) | Loss 5.448079(5.481498) | NFE Forward 26(22.0) | NFE Backward 156(155.9)
Iter 0354 | Time 2.1025(2.0974) | Loss 5.453594(5.479545) | NFE Forward 20(21.9) | NFE Backward 156(155.9)
Iter 0355 | Time 2.1069(2.0981) | Loss 5.416075(5.475102) | NFE Forward 20(21.8) | NFE Backward 156(155.9)
Iter 0356 | Time 2.1081(2.0988) | Loss 5.431089(5.472021) | NFE Forward 20(21.6) | NFE Backward 156(155.9)
Iter 0357 | Time 2.1077(2.0994) | Loss 5.410117(5.467688) | NFE Forward 20(21.5) | NFE Backward 156(156.0)
Iter 0358 | Time 2.1294(2.1015) | Loss 5.413502(5.463895) | NFE Forward 26(21.8) | NFE Backward 156(156.0)
Iter 0359 | Time 2.1031(2.1016) | Loss 5.444116(5.462510) | NFE Forward 20(21.7) | NFE Backward 156(156.0)
Iter 0360 | Time 2.1328(2.1038) | Loss 5.406072(5.458560) | NFE Forward 26(22.0) | NFE Backward 156(156.0)
Iter 0361 | Time 2.1053(2.1039) | Loss 5.425965(5.456278) | NFE Forward 20(21.9) | NFE Backward 156(156.0)
Iter 0362 | Time 2.1661(2.1082) | Loss 5.412292(5.453199) | NFE Forward 20(21.7) | NFE Backward 162(156.4)
Iter 0363 | Time 2.1051(2.1080) | Loss 5.399038(5.449408) | NFE Forward 20(21.6) | NFE Backward 156(156.4)
Iter 0364 | Time 2.1891(2.1137) | Loss 5.385902(5.444962) | NFE Forward 26(21.9) | NFE Backward 162(156.8)
Iter 0365 | Time 2.1340(2.1151) | Loss 5.395072(5.441470) | NFE Forward 26(22.2) | NFE Backward 156(156.7)
Iter 0366 | Time 2.1910(2.1204) | Loss 5.408018(5.439128) | NFE Forward 26(22.5) | NFE Backward 162(157.1)
Iter 0367 | Time 2.1646(2.1235) | Loss 5.402430(5.436560) | NFE Forward 20(22.3) | NFE Backward 162(157.4)
Iter 0368 | Time 2.1648(2.1264) | Loss 5.418814(5.435317) | NFE Forward 20(22.1) | NFE Backward 162(157.7)
Iter 0369 | Time 2.1900(2.1309) | Loss 5.403148(5.433065) | NFE Forward 26(22.4) | NFE Backward 162(158.0)
Iter 0370 | Time 2.1601(2.1329) | Loss 5.399229(5.430697) | NFE Forward 20(22.2) | NFE Backward 162(158.3)
Iter 0371 | Time 2.1832(2.1364) | Loss 5.387803(5.427694) | NFE Forward 26(22.5) | NFE Backward 162(158.6)
Iter 0372 | Time 2.1562(2.1378) | Loss 5.347780(5.422100) | NFE Forward 20(22.3) | NFE Backward 162(158.8)
Iter 0373 | Time 2.1832(2.1410) | Loss 5.386788(5.419628) | NFE Forward 26(22.6) | NFE Backward 162(159.0)
Iter 0374 | Time 2.1556(2.1420) | Loss 5.405633(5.418649) | NFE Forward 20(22.4) | NFE Backward 162(159.2)
Iter 0375 | Time 2.1567(2.1430) | Loss 5.363051(5.414757) | NFE Forward 20(22.2) | NFE Backward 162(159.4)
Iter 0376 | Time 2.1612(2.1443) | Loss 5.354948(5.410570) | NFE Forward 20(22.1) | NFE Backward 162(159.6)
Iter 0377 | Time 2.1528(2.1449) | Loss 5.372182(5.407883) | NFE Forward 20(21.9) | NFE Backward 162(159.8)
Iter 0378 | Time 2.1528(2.1455) | Loss 5.396619(5.407095) | NFE Forward 20(21.8) | NFE Backward 162(159.9)
Iter 0379 | Time 2.1760(2.1476) | Loss 5.327524(5.401525) | NFE Forward 26(22.1) | NFE Backward 162(160.1)
Iter 0380 | Time 2.1526(2.1479) | Loss 5.350325(5.397941) | NFE Forward 20(21.9) | NFE Backward 162(160.2)
Iter 0381 | Time 2.1549(2.1484) | Loss 5.365336(5.395658) | NFE Forward 20(21.8) | NFE Backward 162(160.3)
Iter 0382 | Time 2.2755(2.1573) | Loss 5.347785(5.392307) | NFE Forward 20(21.7) | NFE Backward 174(161.3)
Iter 0383 | Time 2.2861(2.1663) | Loss 5.389565(5.392115) | NFE Forward 26(22.0) | NFE Backward 168(161.8)
Iter 0384 | Time 2.2810(2.1744) | Loss 5.339841(5.388456) | NFE Forward 20(21.8) | NFE Backward 174(162.6)
Iter 0385 | Time 2.2768(2.1815) | Loss 5.352577(5.385945) | NFE Forward 20(21.7) | NFE Backward 174(163.4)
Iter 0386 | Time 2.2778(2.1883) | Loss 5.336533(5.382486) | NFE Forward 20(21.6) | NFE Backward 174(164.2)
Iter 0387 | Time 2.2766(2.1945) | Loss 5.350140(5.380222) | NFE Forward 20(21.5) | NFE Backward 174(164.8)
Iter 0388 | Time 2.2782(2.2003) | Loss 5.372801(5.379702) | NFE Forward 20(21.4) | NFE Backward 174(165.5)
Iter 0389 | Time 2.3020(2.2074) | Loss 5.287089(5.373219) | NFE Forward 26(21.7) | NFE Backward 174(166.1)
Iter 0390 | Time 2.2771(2.2123) | Loss 5.361524(5.372400) | NFE Forward 20(21.6) | NFE Backward 174(166.6)
Iter 0391 | Time 2.2768(2.2168) | Loss 5.320208(5.368747) | NFE Forward 20(21.5) | NFE Backward 174(167.2)
Iter 0392 | Time 2.2819(2.2214) | Loss 5.317879(5.365186) | NFE Forward 20(21.4) | NFE Backward 174(167.6)
Iter 0393 | Time 2.3045(2.2272) | Loss 5.277724(5.359064) | NFE Forward 26(21.7) | NFE Backward 174(168.1)
Iter 0394 | Time 2.3048(2.2326) | Loss 5.295471(5.354612) | NFE Forward 26(22.0) | NFE Backward 174(168.5)
Iter 0395 | Time 2.3047(2.2377) | Loss 5.325189(5.352553) | NFE Forward 26(22.3) | NFE Backward 174(168.9)
Iter 0396 | Time 2.3098(2.2427) | Loss 5.298845(5.348793) | NFE Forward 26(22.5) | NFE Backward 174(169.2)
Iter 0397 | Time 2.2785(2.2452) | Loss 5.272722(5.343468) | NFE Forward 20(22.4) | NFE Backward 174(169.6)
Iter 0398 | Time 2.2787(2.2476) | Loss 5.337760(5.343069) | NFE Forward 20(22.2) | NFE Backward 174(169.9)
Iter 0399 | Time 2.2770(2.2496) | Loss 5.311019(5.340825) | NFE Forward 20(22.0) | NFE Backward 174(170.2)
Iter 0400 | Time 2.2794(2.2517) | Loss 5.319466(5.339330) | NFE Forward 20(21.9) | NFE Backward 174(170.4)
[TEST] Iter 0400 | Test Loss 5.249241 | NFE 26
Skipping vis as data dimension is >2
Iter 0401 | Time 2.3036(2.2553) | Loss 5.289839(5.335866) | NFE Forward 26(22.2) | NFE Backward 174(170.7)
Iter 0402 | Time 2.3325(2.2607) | Loss 5.259589(5.330526) | NFE Forward 26(22.5) | NFE Backward 174(170.9)
Iter 0403 | Time 2.2809(2.2622) | Loss 5.247334(5.324703) | NFE Forward 20(22.3) | NFE Backward 174(171.1)
Iter 0404 | Time 2.3053(2.2652) | Loss 5.278488(5.321468) | NFE Forward 20(22.1) | NFE Backward 174(171.3)
Iter 0405 | Time 2.3064(2.2681) | Loss 5.271868(5.317996) | NFE Forward 20(22.0) | NFE Backward 174(171.5)
Iter 0406 | Time 2.3063(2.2707) | Loss 5.283293(5.315567) | NFE Forward 26(22.3) | NFE Backward 174(171.7)
Iter 0407 | Time 2.3261(2.2746) | Loss 5.308787(5.315092) | NFE Forward 26(22.5) | NFE Backward 174(171.9)
Iter 0408 | Time 2.2990(2.2763) | Loss 5.268854(5.311855) | NFE Forward 20(22.3) | NFE Backward 174(172.0)
Iter 0409 | Time 2.3237(2.2796) | Loss 5.219579(5.305396) | NFE Forward 20(22.2) | NFE Backward 174(172.1)
Iter 0410 | Time 2.3299(2.2832) | Loss 5.213999(5.298998) | NFE Forward 20(22.0) | NFE Backward 180(172.7)
Iter 0411 | Time 2.2701(2.2822) | Loss 5.248075(5.295434) | NFE Forward 20(21.9) | NFE Backward 174(172.8)
Iter 0412 | Time 2.3560(2.2874) | Loss 5.216464(5.289906) | NFE Forward 20(21.8) | NFE Backward 180(173.3)
Iter 0413 | Time 2.3816(2.2940) | Loss 5.216811(5.284789) | NFE Forward 26(22.0) | NFE Backward 180(173.8)
Iter 0414 | Time 2.3837(2.3003) | Loss 5.274034(5.284036) | NFE Forward 26(22.3) | NFE Backward 180(174.2)
Iter 0415 | Time 2.3819(2.3060) | Loss 5.271420(5.283153) | NFE Forward 26(22.6) | NFE Backward 180(174.6)
Iter 0416 | Time 2.3759(2.3109) | Loss 5.249748(5.280815) | NFE Forward 26(22.8) | NFE Backward 180(175.0)
Iter 0417 | Time 2.3789(2.3157) | Loss 5.198930(5.275083) | NFE Forward 26(23.0) | NFE Backward 180(175.3)
Iter 0418 | Time 2.3882(2.3207) | Loss 5.238145(5.272497) | NFE Forward 26(23.3) | NFE Backward 180(175.7)
Iter 0419 | Time 2.3766(2.3246) | Loss 5.194855(5.267062) | NFE Forward 26(23.4) | NFE Backward 180(176.0)
Iter 0420 | Time 2.3749(2.3282) | Loss 5.213845(5.263337) | NFE Forward 26(23.6) | NFE Backward 180(176.2)
Iter 0421 | Time 2.3767(2.3316) | Loss 5.234905(5.261347) | NFE Forward 26(23.8) | NFE Backward 180(176.5)
Iter 0422 | Time 2.3734(2.3345) | Loss 5.215339(5.258126) | NFE Forward 26(23.9) | NFE Backward 180(176.8)
Iter 0423 | Time 2.3729(2.3372) | Loss 5.222168(5.255609) | NFE Forward 26(24.1) | NFE Backward 180(177.0)
Iter 0424 | Time 2.3720(2.3396) | Loss 5.219851(5.253106) | NFE Forward 26(24.2) | NFE Backward 180(177.2)
Iter 0425 | Time 2.3714(2.3418) | Loss 5.158823(5.246506) | NFE Forward 26(24.3) | NFE Backward 180(177.4)
Iter 0426 | Time 2.3714(2.3439) | Loss 5.196484(5.243005) | NFE Forward 26(24.5) | NFE Backward 180(177.6)
Iter 0427 | Time 2.3970(2.3476) | Loss 5.210701(5.240743) | NFE Forward 26(24.6) | NFE Backward 180(177.7)
Iter 0428 | Time 2.3993(2.3512) | Loss 5.210165(5.238603) | NFE Forward 26(24.7) | NFE Backward 180(177.9)
Iter 0429 | Time 2.3992(2.3546) | Loss 5.164100(5.233388) | NFE Forward 26(24.8) | NFE Backward 180(178.0)
Iter 0430 | Time 2.3939(2.3574) | Loss 5.217740(5.232292) | NFE Forward 26(24.8) | NFE Backward 180(178.2)
Iter 0431 | Time 2.3994(2.3603) | Loss 5.190958(5.229399) | NFE Forward 26(24.9) | NFE Backward 180(178.3)
Iter 0432 | Time 2.3981(2.3629) | Loss 5.164476(5.224854) | NFE Forward 26(25.0) | NFE Backward 180(178.4)
Iter 0433 | Time 2.3955(2.3652) | Loss 5.149021(5.219546) | NFE Forward 26(25.1) | NFE Backward 180(178.5)
Iter 0434 | Time 2.4026(2.3678) | Loss 5.152526(5.214855) | NFE Forward 26(25.1) | NFE Backward 180(178.6)
Iter 0435 | Time 2.4042(2.3704) | Loss 5.171482(5.211819) | NFE Forward 26(25.2) | NFE Backward 180(178.7)
Iter 0436 | Time 2.3984(2.3724) | Loss 5.219259(5.212339) | NFE Forward 26(25.3) | NFE Backward 180(178.8)
Iter 0437 | Time 2.3973(2.3741) | Loss 5.144687(5.207604) | NFE Forward 26(25.3) | NFE Backward 180(178.9)
Iter 0438 | Time 2.4005(2.3759) | Loss 5.136404(5.202620) | NFE Forward 26(25.4) | NFE Backward 180(179.0)
Iter 0439 | Time 2.3970(2.3774) | Loss 5.146794(5.198712) | NFE Forward 26(25.4) | NFE Backward 180(179.1)
Iter 0440 | Time 2.4564(2.3830) | Loss 5.166458(5.196454) | NFE Forward 26(25.4) | NFE Backward 186(179.5)
Iter 0441 | Time 2.4177(2.3854) | Loss 5.127325(5.191615) | NFE Forward 26(25.5) | NFE Backward 180(179.6)
Iter 0442 | Time 2.4366(2.3890) | Loss 5.102553(5.185381) | NFE Forward 32(25.9) | NFE Backward 180(179.6)
Iter 0443 | Time 2.5688(2.4016) | Loss 5.139082(5.182140) | NFE Forward 32(26.4) | NFE Backward 186(180.1)
Iter 0444 | Time 2.4321(2.4037) | Loss 5.115035(5.177443) | NFE Forward 32(26.8) | NFE Backward 180(180.0)
Iter 0445 | Time 2.4281(2.4054) | Loss 5.112051(5.172865) | NFE Forward 32(27.1) | NFE Backward 180(180.0)
Iter 0446 | Time 2.4276(2.4070) | Loss 5.119226(5.169110) | NFE Forward 26(27.0) | NFE Backward 180(180.0)
Iter 0447 | Time 2.4074(2.4070) | Loss 5.119718(5.165653) | NFE Forward 26(27.0) | NFE Backward 180(180.0)
Iter 0448 | Time 2.4773(2.4119) | Loss 5.116934(5.162243) | NFE Forward 32(27.3) | NFE Backward 186(180.5)
Iter 0449 | Time 2.3927(2.4106) | Loss 5.075800(5.156192) | NFE Forward 26(27.2) | NFE Backward 180(180.4)
Iter 0450 | Time 2.3923(2.4093) | Loss 5.088017(5.151419) | NFE Forward 26(27.1) | NFE Backward 180(180.4)
Iter 0451 | Time 2.4177(2.4099) | Loss 5.137006(5.150410) | NFE Forward 32(27.5) | NFE Backward 180(180.4)
Iter 0452 | Time 2.4499(2.4127) | Loss 5.106774(5.147356) | NFE Forward 26(27.4) | NFE Backward 186(180.8)
Iter 0453 | Time 2.4520(2.4154) | Loss 5.083057(5.142855) | NFE Forward 26(27.3) | NFE Backward 186(181.1)
Iter 0454 | Time 2.4748(2.4196) | Loss 5.117283(5.141065) | NFE Forward 32(27.6) | NFE Backward 186(181.5)
Iter 0455 | Time 2.4771(2.4236) | Loss 5.100429(5.138220) | NFE Forward 32(27.9) | NFE Backward 186(181.8)
Iter 0456 | Time 2.5012(2.4290) | Loss 5.117259(5.136753) | NFE Forward 32(28.2) | NFE Backward 186(182.1)
Iter 0457 | Time 2.4182(2.4283) | Loss 5.082797(5.132976) | NFE Forward 32(28.5) | NFE Backward 180(181.9)
Iter 0458 | Time 2.4578(2.4303) | Loss 5.067832(5.128416) | NFE Forward 32(28.7) | NFE Backward 180(181.8)
Iter 0459 | Time 2.4762(2.4336) | Loss 5.052147(5.123077) | NFE Forward 32(28.9) | NFE Backward 186(182.1)
Iter 0460 | Time 2.4698(2.4361) | Loss 5.058591(5.118563) | NFE Forward 32(29.2) | NFE Backward 186(182.4)
Iter 0461 | Time 2.4670(2.4383) | Loss 5.085937(5.116279) | NFE Forward 32(29.4) | NFE Backward 186(182.6)
Iter 0462 | Time 2.4646(2.4401) | Loss 5.068943(5.112966) | NFE Forward 32(29.5) | NFE Backward 186(182.9)
Iter 0463 | Time 2.4690(2.4421) | Loss 5.067272(5.109767) | NFE Forward 32(29.7) | NFE Backward 186(183.1)
Iter 0464 | Time 2.4319(2.4414) | Loss 5.080828(5.107741) | NFE Forward 26(29.5) | NFE Backward 186(183.3)
Iter 0465 | Time 2.3978(2.4384) | Loss 5.077271(5.105609) | NFE Forward 32(29.6) | NFE Backward 180(183.1)
Iter 0466 | Time 2.4291(2.4377) | Loss 5.071556(5.103225) | NFE Forward 26(29.4) | NFE Backward 186(183.3)
Iter 0467 | Time 2.3999(2.4351) | Loss 5.082212(5.101754) | NFE Forward 32(29.6) | NFE Backward 180(183.0)
Iter 0468 | Time 2.4326(2.4349) | Loss 5.054872(5.098472) | NFE Forward 26(29.3) | NFE Backward 186(183.2)
Iter 0469 | Time 2.4815(2.4381) | Loss 5.051657(5.095195) | NFE Forward 32(29.5) | NFE Backward 186(183.4)
Iter 0470 | Time 2.4026(2.4357) | Loss 5.087147(5.094632) | NFE Forward 32(29.7) | NFE Backward 180(183.2)
Iter 0471 | Time 2.4019(2.4333) | Loss 5.075002(5.093258) | NFE Forward 32(29.8) | NFE Backward 180(183.0)
Iter 0472 | Time 2.4007(2.4310) | Loss 5.010445(5.087461) | NFE Forward 32(30.0) | NFE Backward 180(182.8)
Iter 0473 | Time 2.4016(2.4290) | Loss 5.041224(5.084224) | NFE Forward 32(30.1) | NFE Backward 180(182.6)
Iter 0474 | Time 2.4538(2.4307) | Loss 5.073129(5.083447) | NFE Forward 32(30.3) | NFE Backward 180(182.4)
Iter 0475 | Time 2.3953(2.4282) | Loss 5.046418(5.080855) | NFE Forward 32(30.4) | NFE Backward 174(181.8)
Iter 0476 | Time 2.4511(2.4298) | Loss 5.025108(5.076953) | NFE Forward 32(30.5) | NFE Backward 180(181.7)
Iter 0477 | Time 2.4543(2.4315) | Loss 5.024568(5.073286) | NFE Forward 32(30.6) | NFE Backward 180(181.6)
Iter 0478 | Time 2.4490(2.4328) | Loss 4.991920(5.067590) | NFE Forward 32(30.7) | NFE Backward 180(181.4)
Iter 0479 | Time 2.5080(2.4380) | Loss 5.027802(5.064805) | NFE Forward 32(30.8) | NFE Backward 186(181.8)
Iter 0480 | Time 2.4521(2.4390) | Loss 5.046018(5.063490) | NFE Forward 32(30.9) | NFE Backward 180(181.6)
Iter 0481 | Time 2.4528(2.4400) | Loss 5.034821(5.061483) | NFE Forward 32(31.0) | NFE Backward 180(181.5)
Iter 0482 | Time 2.3911(2.4365) | Loss 5.039774(5.059964) | NFE Forward 32(31.0) | NFE Backward 174(181.0)
Iter 0483 | Time 2.4752(2.4393) | Loss 5.032221(5.058022) | NFE Forward 32(31.1) | NFE Backward 180(180.9)
Iter 0484 | Time 2.4737(2.4417) | Loss 5.023918(5.055634) | NFE Forward 32(31.2) | NFE Backward 180(180.9)
Iter 0485 | Time 2.3993(2.4387) | Loss 4.976318(5.050082) | NFE Forward 32(31.2) | NFE Backward 174(180.4)
Iter 0486 | Time 2.4574(2.4400) | Loss 4.958096(5.043643) | NFE Forward 32(31.3) | NFE Backward 180(180.4)
Iter 0487 | Time 2.4571(2.4412) | Loss 5.026014(5.042409) | NFE Forward 32(31.3) | NFE Backward 180(180.3)
Iter 0488 | Time 2.4019(2.4385) | Loss 4.989119(5.038679) | NFE Forward 32(31.4) | NFE Backward 174(179.9)
Iter 0489 | Time 2.4591(2.4399) | Loss 5.003662(5.036228) | NFE Forward 32(31.4) | NFE Backward 180(179.9)
Iter 0490 | Time 2.4604(2.4413) | Loss 5.045918(5.036906) | NFE Forward 32(31.5) | NFE Backward 180(179.9)
Iter 0491 | Time 2.3998(2.4384) | Loss 4.963475(5.031766) | NFE Forward 32(31.5) | NFE Backward 174(179.5)
Iter 0492 | Time 2.4321(2.4380) | Loss 4.987168(5.028644) | NFE Forward 26(31.1) | NFE Backward 180(179.5)
Iter 0493 | Time 2.4005(2.4354) | Loss 5.006474(5.027092) | NFE Forward 32(31.2) | NFE Backward 174(179.1)
Iter 0494 | Time 2.4579(2.4369) | Loss 5.021466(5.026698) | NFE Forward 32(31.2) | NFE Backward 180(179.2)
Iter 0495 | Time 2.4408(2.4372) | Loss 4.994740(5.024461) | NFE Forward 32(31.3) | NFE Backward 180(179.3)
Iter 0496 | Time 2.4646(2.4391) | Loss 5.004718(5.023079) | NFE Forward 32(31.3) | NFE Backward 180(179.3)
Iter 0497 | Time 2.4513(2.4400) | Loss 5.003932(5.021739) | NFE Forward 32(31.4) | NFE Backward 180(179.4)
Iter 0498 | Time 2.3900(2.4365) | Loss 4.985498(5.019202) | NFE Forward 32(31.4) | NFE Backward 174(179.0)
Density Loss 0.33056142926216125
Density Loss 0.3342806100845337
Density Loss 0.3430637717247009
Density Loss 0.33398711681365967
Density Loss 0.3495252728462219
Density Loss 0.3357544243335724
Density Loss 0.33347177505493164
Density Loss 0.3377465307712555
Density Loss 0.3271194398403168
Density Loss 0.3394697904586792
Density Loss 0.33881619572639465
Density Loss 0.34066474437713623
Density Loss 0.3440743386745453
Density Loss 0.33804449439048767
Density Loss 0.3410717248916626
Density Loss 0.3297233283519745
Density Loss 0.3447830080986023
Density Loss 0.3413858413696289
Density Loss 0.34782448410987854
Density Loss 0.33606964349746704
Density Loss 0.34323203563690186
Density Loss 0.34040552377700806
Density Loss 0.3409649431705475
Density Loss 0.34052175283432007
Density Loss 0.343374639749527
Density Loss 0.349914014339447
Density Loss 0.34011417627334595
Density Loss 0.3434253931045532
Density Loss 0.345670610666275
Density Loss 0.34449100494384766
Density Loss 0.34808796644210815
Density Loss 0.3481149673461914
Density Loss 0.3517858684062958
Density Loss 0.34114372730255127
Density Loss 0.3494010865688324
Density Loss 0.34092068672180176
Density Loss 0.34063073992729187
Density Loss 0.34465163946151733
Density Loss 0.348513662815094
Density Loss 0.3527269959449768
Density Loss 0.3363023102283478
Density Loss 0.3404964804649353
Density Loss 0.3464694619178772
Density Loss 0.34371256828308105
Density Loss 0.3421710729598999
Density Loss 0.34055474400520325
Density Loss 0.35042595863342285
Density Loss 0.34820976853370667
Density Loss 0.35125747323036194
Density Loss 0.339542955160141
Density Loss 0.3426421284675598
Density Loss 0.3492335379123688
Density Loss 0.3490980267524719
Density Loss 0.34223783016204834
Density Loss 0.3403252363204956
Density Loss 0.3411365747451782
Density Loss 0.34951797127723694
Density Loss 0.3399015963077545
Density Loss 0.3450671434402466
Density Loss 0.34834346175193787
Density Loss 0.35028520226478577
Density Loss 0.3432237207889557
Density Loss 0.3441218137741089
Density Loss 0.3481502830982208
Density Loss 0.34155479073524475
Density Loss 0.3395906686782837
Density Loss 0.3453766405582428
Density Loss 0.34051379561424255
Density Loss 0.3582109212875366
Density Loss 0.3450428247451782
Density Loss 0.34704363346099854
Density Loss 0.3462802767753601
Density Loss 0.3416333496570587
Density Loss 0.3474321663379669
Density Loss 0.35549792647361755
Density Loss 0.33799442648887634
Density Loss 0.34329086542129517
Density Loss 0.34385809302330017
Density Loss 0.3587783873081207
Density Loss 0.34163880348205566
Density Loss 0.34252607822418213
Density Loss 0.34546738862991333
Density Loss 0.3559201657772064
Density Loss 0.33887410163879395
Density Loss 0.3558359146118164
Density Loss 0.3502699136734009
Density Loss 0.34439677000045776
Density Loss 0.3490707576274872
Density Loss 0.35888466238975525
Density Loss 0.355656236410141
Density Loss 0.34885141253471375
Density Loss 0.3611695468425751
Density Loss 0.3594261705875397
Density Loss 0.35415124893188477
Density Loss 0.35377755761146545
Density Loss 0.35215020179748535
Density Loss 0.35463884472846985
Density Loss 0.34492743015289307
Density Loss 0.3556059002876282
Density Loss 0.3481394052505493
Density Loss 0.3484189212322235
Density Loss 0.35319674015045166
Density Loss 0.3588373064994812
Density Loss 0.3529534637928009
Density Loss 0.3542386591434479
Density Loss 0.34747040271759033
Density Loss 0.3376157283782959
Density Loss 0.3513616621494293
Density Loss 0.3511822521686554
Density Loss 0.351575642824173
Density Loss 0.3529316186904907
Density Loss 0.36379939317703247
Density Loss 0.35334882140159607
Density Loss 0.3474750816822052
Density Loss 0.35721129179000854
Density Loss 0.35947367548942566
Density Loss 0.3560446798801422
Density Loss 0.3501538932323456
Density Loss 0.35141536593437195
Density Loss 0.35794568061828613
Density Loss 0.3557520806789398
Density Loss 0.3508467972278595
Density Loss 0.3579273819923401
Density Loss 0.35399046540260315
Density Loss 0.3537355959415436
Density Loss 0.3457679748535156
Density Loss 0.3521207869052887
Density Loss 0.35782045125961304
Density Loss 0.34854599833488464
Density Loss 0.35187259316444397
Density Loss 0.3485312759876251
Density Loss 0.36326682567596436
Density Loss 0.34403926134109497
Density Loss 0.35601258277893066
Density Loss 0.35870328545570374
Density Loss 0.34673506021499634
Density Loss 0.35169899463653564
Density Loss 0.3626113533973694
Density Loss 0.3599590063095093
Density Loss 0.35173550248146057
Density Loss 0.35515037178993225
Density Loss 0.3594522476196289
Density Loss 0.35986843705177307
Density Loss 0.36258333921432495
Density Loss 0.3595811724662781
Density Loss 0.36772629618644714
Density Loss 0.3511419892311096
Density Loss 0.36051180958747864
Density Loss 0.3594870865345001
Density Loss 0.3597182631492615
Density Loss 0.3635668158531189
Density Loss 0.3573795258998871
Density Loss 0.35763269662857056
Density Loss 0.36254334449768066
Density Loss 0.3552294671535492
Density Loss 0.36158016324043274
Density Loss 0.3602466583251953
Density Loss 0.35587796568870544
Density Loss 0.3751535713672638
Density Loss 0.3676300644874573
Density Loss 0.3609134554862976
Density Loss 0.3680764436721802
Density Loss 0.3628658056259155
Density Loss 0.3575982451438904
Density Loss 0.36263930797576904
Density Loss 0.36303040385246277
Density Loss 0.3601185083389282
Density Loss 0.37070536613464355
Density Loss 0.3620213568210602
Density Loss 0.37116187810897827
Density Loss 0.3601396977901459
Density Loss 0.3683118522167206
Density Loss 0.3645257353782654
Density Loss 0.367088258266449
Density Loss 0.3621387183666229
Density Loss 0.36209696531295776
Density Loss 0.3727242648601532
Density Loss 0.3630576729774475
Density Loss 0.3664869964122772
Density Loss 0.37067481875419617
Density Loss 0.3696717917919159
Density Loss 0.36787810921669006
Density Loss 0.36432307958602905
Density Loss 0.3721081614494324
Density Loss 0.3742915987968445
Density Loss 0.3643566966056824
Density Loss 0.3645118772983551
Density Loss 0.36820006370544434
Density Loss 0.3728235065937042
Density Loss 0.3739452064037323
Density Loss 0.3749653697013855
Density Loss 0.3749805986881256
Density Loss 0.37143099308013916
Density Loss 0.37281206250190735
Density Loss 0.3797115981578827
Density Loss 0.37909817695617676
Density Loss 0.3687250316143036
Density Loss 0.37015148997306824
Density Loss 0.36698657274246216
Density Loss 0.3698808252811432
Density Loss 0.37351271510124207
Density Loss 0.3840036988258362
Density Loss 0.3775433301925659
Density Loss 0.3661201298236847
Density Loss 0.37474799156188965
Density Loss 0.3780530095100403
Density Loss 0.37426459789276123
Density Loss 0.374433308839798
Density Loss 0.36904382705688477
Density Loss 0.37929341197013855
Density Loss 0.37272509932518005
Density Loss 0.3724250793457031
Density Loss 0.3745363652706146
Density Loss 0.3695428967475891
Density Loss 0.3746533989906311
Density Loss 0.38222503662109375
Density Loss 0.367110937833786
Density Loss 0.3701258599758148
Density Loss 0.3803837299346924
Density Loss 0.37517890334129333
Density Loss 0.38463863730430603
Density Loss 0.37302133440971375
Density Loss 0.3767459988594055
Density Loss 0.36433789134025574
Density Loss 0.37666791677474976
Density Loss 0.38324350118637085
Density Loss 0.3736036419868469
Density Loss 0.386203408241272
Density Loss 0.3822644054889679
Density Loss 0.38550013303756714
Density Loss 0.36720573902130127
Density Loss 0.3739098906517029
Density Loss 0.37568968534469604
Density Loss 0.37876617908477783
Density Loss 0.3743564486503601
Density Loss 0.3861066401004791
Density Loss 0.3802948594093323
Density Loss 0.38332873582839966
Density Loss 0.3840266764163971
Density Loss 0.37605932354927063
Density Loss 0.3716777265071869
Density Loss 0.3752495050430298
Density Loss 0.37542229890823364
Density Loss 0.3817208409309387
Density Loss 0.3741617202758789
Density Loss 0.3799871504306793
Density Loss 0.3738763630390167
Density Loss 0.3850192427635193
Density Loss 0.3855670690536499
Density Loss 0.3762757182121277
Density Loss 0.3773749768733978
Density Loss 0.37870341539382935
Density Loss 0.3862733840942383
Iter 0499 | Time 2.3922(2.4334) | Loss 5.000800(5.017914) | NFE Forward 32(31.5) | NFE Backward 174(178.6)
Iter 0500 | Time 2.4018(2.4312) | Loss 5.039250(5.019407) | NFE Forward 32(31.5) | NFE Backward 174(178.3)
[TEST] Iter 0500 | Test Loss 4.980343 | NFE 32
Skipping vis as data dimension is >2
Iter 0501 | Time 2.4453(2.4322) | Loss 4.970253(5.015967) | NFE Forward 32(31.5) | NFE Backward 180(178.4)
Iter 0502 | Time 2.4504(2.4334) | Loss 5.018943(5.016175) | NFE Forward 32(31.6) | NFE Backward 180(178.5)
Iter 0503 | Time 2.4455(2.4343) | Loss 4.959968(5.012240) | NFE Forward 32(31.6) | NFE Backward 180(178.6)
Iter 0504 | Time 2.3832(2.4307) | Loss 4.955378(5.008260) | NFE Forward 32(31.6) | NFE Backward 174(178.3)
Iter 0505 | Time 2.3834(2.4274) | Loss 4.997303(5.007493) | NFE Forward 32(31.7) | NFE Backward 174(178.0)
Iter 0506 | Time 2.3822(2.4242) | Loss 4.950933(5.003534) | NFE Forward 32(31.7) | NFE Backward 174(177.7)
Iter 0507 | Time 2.4411(2.4254) | Loss 4.949750(4.999769) | NFE Forward 32(31.7) | NFE Backward 180(177.9)
Iter 0508 | Time 2.4512(2.4272) | Loss 4.982060(4.998529) | NFE Forward 32(31.7) | NFE Backward 180(178.0)
Iter 0509 | Time 2.3812(2.4240) | Loss 4.990338(4.997956) | NFE Forward 32(31.7) | NFE Backward 174(177.8)
Iter 0510 | Time 2.3542(2.4191) | Loss 4.962291(4.995460) | NFE Forward 26(31.3) | NFE Backward 174(177.5)
Iter 0511 | Time 2.3779(2.4162) | Loss 4.922564(4.990357) | NFE Forward 32(31.4) | NFE Backward 174(177.2)
Iter 0512 | Time 2.4372(2.4177) | Loss 4.946499(4.987287) | NFE Forward 32(31.4) | NFE Backward 180(177.4)
Iter 0513 | Time 2.3801(2.4151) | Loss 4.903219(4.981402) | NFE Forward 32(31.5) | NFE Backward 174(177.2)
Iter 0514 | Time 2.4352(2.4165) | Loss 4.931636(4.977918) | NFE Forward 32(31.5) | NFE Backward 180(177.4)
Iter 0515 | Time 2.4349(2.4178) | Loss 4.941983(4.975403) | NFE Forward 32(31.5) | NFE Backward 180(177.6)
Iter 0516 | Time 2.3777(2.4150) | Loss 4.923591(4.971776) | NFE Forward 32(31.6) | NFE Backward 174(177.3)
Iter 0517 | Time 2.4417(2.4168) | Loss 4.934100(4.969139) | NFE Forward 32(31.6) | NFE Backward 180(177.5)
Iter 0518 | Time 2.3836(2.4145) | Loss 4.949921(4.967793) | NFE Forward 32(31.6) | NFE Backward 174(177.3)
Iter 0519 | Time 2.4382(2.4162) | Loss 5.002295(4.970209) | NFE Forward 32(31.7) | NFE Backward 180(177.5)
Iter 0520 | Time 2.3533(2.4118) | Loss 4.951230(4.968880) | NFE Forward 26(31.3) | NFE Backward 174(177.2)
Iter 0521 | Time 2.3806(2.4096) | Loss 4.927615(4.965991) | NFE Forward 32(31.3) | NFE Backward 174(177.0)
Iter 0522 | Time 2.3794(2.4075) | Loss 4.983617(4.967225) | NFE Forward 32(31.4) | NFE Backward 174(176.8)
Iter 0523 | Time 2.4383(2.4096) | Loss 4.952325(4.966182) | NFE Forward 32(31.4) | NFE Backward 180(177.0)
Iter 0524 | Time 2.4100(2.4097) | Loss 4.936164(4.964081) | NFE Forward 26(31.0) | NFE Backward 180(177.2)
Iter 0525 | Time 2.3877(2.4081) | Loss 4.897400(4.959413) | NFE Forward 26(30.7) | NFE Backward 180(177.4)
Iter 0526 | Time 2.4365(2.4101) | Loss 4.943783(4.958319) | NFE Forward 32(30.8) | NFE Backward 180(177.6)
Iter 0527 | Time 2.3822(2.4081) | Loss 4.920797(4.955693) | NFE Forward 32(30.9) | NFE Backward 174(177.3)
Iter 0528 | Time 2.3815(2.4063) | Loss 4.930736(4.953946) | NFE Forward 32(30.9) | NFE Backward 174(177.1)
Iter 0529 | Time 2.3580(2.4029) | Loss 4.922330(4.951733) | NFE Forward 26(30.6) | NFE Backward 174(176.9)
Iter 0530 | Time 2.3829(2.4015) | Loss 4.904763(4.948445) | NFE Forward 32(30.7) | NFE Backward 174(176.7)
Iter 0531 | Time 2.3864(2.4004) | Loss 4.932128(4.947303) | NFE Forward 32(30.8) | NFE Backward 174(176.5)
Iter 0532 | Time 2.4443(2.4035) | Loss 4.936165(4.946523) | NFE Forward 32(30.9) | NFE Backward 180(176.7)
Iter 0533 | Time 2.4201(2.4047) | Loss 4.903244(4.943493) | NFE Forward 32(30.9) | NFE Backward 174(176.6)
Iter 0534 | Time 2.3619(2.4017) | Loss 4.929209(4.942494) | NFE Forward 26(30.6) | NFE Backward 174(176.4)
Iter 0535 | Time 2.3613(2.3989) | Loss 4.906195(4.939953) | NFE Forward 26(30.3) | NFE Backward 174(176.2)
Iter 0536 | Time 2.4139(2.3999) | Loss 4.928374(4.939142) | NFE Forward 26(30.0) | NFE Backward 180(176.5)
Iter 0537 | Time 2.3598(2.3971) | Loss 4.924852(4.938142) | NFE Forward 26(29.7) | NFE Backward 174(176.3)
Iter 0538 | Time 2.3658(2.3949) | Loss 4.911867(4.936303) | NFE Forward 26(29.4) | NFE Backward 174(176.1)
Iter 0539 | Time 2.3575(2.3923) | Loss 4.911384(4.934558) | NFE Forward 26(29.2) | NFE Backward 174(176.0)
Iter 0540 | Time 2.3589(2.3900) | Loss 4.935035(4.934592) | NFE Forward 26(29.0) | NFE Backward 174(175.9)
Iter 0541 | Time 2.3847(2.3896) | Loss 4.939582(4.934941) | NFE Forward 32(29.2) | NFE Backward 174(175.7)
Iter 0542 | Time 2.3636(2.3878) | Loss 4.949921(4.935990) | NFE Forward 26(29.0) | NFE Backward 174(175.6)
Iter 0543 | Time 2.3584(2.3857) | Loss 4.930307(4.935592) | NFE Forward 26(28.8) | NFE Backward 174(175.5)
Iter 0544 | Time 2.3578(2.3838) | Loss 4.948982(4.936529) | NFE Forward 26(28.6) | NFE Backward 174(175.4)
Iter 0545 | Time 2.3589(2.3820) | Loss 4.904546(4.934290) | NFE Forward 26(28.4) | NFE Backward 174(175.3)
Iter 0546 | Time 2.3533(2.3800) | Loss 4.896293(4.931630) | NFE Forward 26(28.2) | NFE Backward 174(175.2)
Iter 0547 | Time 2.4387(2.3841) | Loss 4.935405(4.931895) | NFE Forward 32(28.5) | NFE Backward 180(175.5)
Iter 0548 | Time 2.3787(2.3837) | Loss 4.856141(4.926592) | NFE Forward 32(28.7) | NFE Backward 174(175.4)
Iter 0549 | Time 2.3769(2.3833) | Loss 4.902629(4.924915) | NFE Forward 32(29.0) | NFE Backward 174(175.3)
Iter 0550 | Time 2.3536(2.3812) | Loss 4.901504(4.923276) | NFE Forward 26(28.8) | NFE Backward 174(175.2)
Iter 0551 | Time 2.3550(2.3794) | Loss 4.905555(4.922035) | NFE Forward 26(28.6) | NFE Backward 174(175.1)
Iter 0552 | Time 2.3607(2.3780) | Loss 4.894563(4.920112) | NFE Forward 26(28.4) | NFE Backward 174(175.1)
Iter 0553 | Time 2.3535(2.3763) | Loss 4.924101(4.920391) | NFE Forward 26(28.2) | NFE Backward 174(175.0)
Iter 0554 | Time 2.3556(2.3749) | Loss 4.900963(4.919031) | NFE Forward 26(28.1) | NFE Backward 174(174.9)
Iter 0555 | Time 2.3543(2.3734) | Loss 4.890364(4.917025) | NFE Forward 26(27.9) | NFE Backward 174(174.9)
Iter 0556 | Time 2.3548(2.3721) | Loss 4.890397(4.915161) | NFE Forward 26(27.8) | NFE Backward 174(174.8)
Iter 0557 | Time 2.3539(2.3709) | Loss 4.924588(4.915821) | NFE Forward 26(27.7) | NFE Backward 174(174.7)
Iter 0558 | Time 2.3578(2.3699) | Loss 4.907970(4.915271) | NFE Forward 26(27.5) | NFE Backward 174(174.7)
Iter 0559 | Time 2.3608(2.3693) | Loss 4.903455(4.914444) | NFE Forward 26(27.4) | NFE Backward 174(174.6)
Iter 0560 | Time 2.3529(2.3682) | Loss 4.900703(4.913482) | NFE Forward 26(27.3) | NFE Backward 174(174.6)
Iter 0561 | Time 2.3539(2.3672) | Loss 4.886419(4.911588) | NFE Forward 26(27.2) | NFE Backward 174(174.6)
Iter 0562 | Time 2.3542(2.3662) | Loss 4.896128(4.910506) | NFE Forward 26(27.2) | NFE Backward 174(174.5)
Iter 0563 | Time 2.3550(2.3655) | Loss 4.873224(4.907896) | NFE Forward 26(27.1) | NFE Backward 174(174.5)
Iter 0564 | Time 2.3533(2.3646) | Loss 4.868642(4.905148) | NFE Forward 26(27.0) | NFE Backward 174(174.4)
Iter 0565 | Time 2.3536(2.3638) | Loss 4.907229(4.905294) | NFE Forward 26(26.9) | NFE Backward 174(174.4)
Iter 0566 | Time 2.3546(2.3632) | Loss 4.868525(4.902720) | NFE Forward 26(26.9) | NFE Backward 174(174.4)
Iter 0567 | Time 2.3569(2.3628) | Loss 4.901663(4.902646) | NFE Forward 26(26.8) | NFE Backward 174(174.4)
Iter 0568 | Time 2.4139(2.3663) | Loss 4.895747(4.902163) | NFE Forward 26(26.7) | NFE Backward 180(174.8)
Iter 0569 | Time 2.4226(2.3703) | Loss 4.888276(4.901191) | NFE Forward 26(26.7) | NFE Backward 180(175.1)
Iter 0570 | Time 2.4409(2.3752) | Loss 4.863374(4.898544) | NFE Forward 32(27.1) | NFE Backward 180(175.5)
Iter 0571 | Time 2.4156(2.3781) | Loss 4.883993(4.897525) | NFE Forward 26(27.0) | NFE Backward 180(175.8)
Iter 0572 | Time 2.4126(2.3805) | Loss 4.896073(4.897423) | NFE Forward 26(26.9) | NFE Backward 180(176.1)
Iter 0573 | Time 2.4138(2.3828) | Loss 4.879975(4.896202) | NFE Forward 26(26.9) | NFE Backward 180(176.4)
Iter 0574 | Time 2.4139(2.3850) | Loss 4.879798(4.895054) | NFE Forward 26(26.8) | NFE Backward 180(176.6)
Iter 0575 | Time 2.4135(2.3870) | Loss 4.864902(4.892943) | NFE Forward 26(26.7) | NFE Backward 180(176.8)
Iter 0576 | Time 2.4157(2.3890) | Loss 4.861309(4.890729) | NFE Forward 26(26.7) | NFE Backward 180(177.1)
Iter 0577 | Time 2.4152(2.3908) | Loss 4.855332(4.888251) | NFE Forward 26(26.6) | NFE Backward 180(177.3)
Iter 0578 | Time 2.4130(2.3924) | Loss 4.870275(4.886993) | NFE Forward 26(26.6) | NFE Backward 180(177.5)
Iter 0579 | Time 2.4136(2.3939) | Loss 4.868814(4.885720) | NFE Forward 26(26.6) | NFE Backward 180(177.6)
Iter 0580 | Time 2.4170(2.3955) | Loss 4.848928(4.883145) | NFE Forward 26(26.5) | NFE Backward 180(177.8)
Iter 0581 | Time 2.4134(2.3967) | Loss 4.904493(4.884639) | NFE Forward 26(26.5) | NFE Backward 180(178.0)
Iter 0582 | Time 2.4168(2.3981) | Loss 4.898924(4.885639) | NFE Forward 26(26.4) | NFE Backward 180(178.1)
Iter 0583 | Time 2.4177(2.3995) | Loss 4.877103(4.885042) | NFE Forward 26(26.4) | NFE Backward 180(178.2)
Iter 0584 | Time 2.4195(2.4009) | Loss 4.833281(4.881418) | NFE Forward 26(26.4) | NFE Backward 180(178.4)
Iter 0585 | Time 2.4131(2.4018) | Loss 4.836526(4.878276) | NFE Forward 26(26.4) | NFE Backward 180(178.5)
Iter 0586 | Time 2.4403(2.4045) | Loss 4.916377(4.880943) | NFE Forward 32(26.8) | NFE Backward 180(178.6)
Iter 0587 | Time 2.4138(2.4051) | Loss 4.866986(4.879966) | NFE Forward 26(26.7) | NFE Backward 180(178.7)
Iter 0588 | Time 2.4168(2.4059) | Loss 4.820110(4.875776) | NFE Forward 26(26.7) | NFE Backward 180(178.8)
Iter 0589 | Time 2.4144(2.4065) | Loss 4.856689(4.874440) | NFE Forward 26(26.6) | NFE Backward 180(178.9)
Iter 0590 | Time 2.4154(2.4071) | Loss 4.877529(4.874656) | NFE Forward 26(26.6) | NFE Backward 180(178.9)
Iter 0591 | Time 2.4144(2.4077) | Loss 4.889664(4.875707) | NFE Forward 26(26.5) | NFE Backward 180(179.0)
Iter 0592 | Time 2.4168(2.4083) | Loss 4.846986(4.873696) | NFE Forward 26(26.5) | NFE Backward 180(179.1)
Iter 0593 | Time 2.4169(2.4089) | Loss 4.835933(4.871053) | NFE Forward 26(26.5) | NFE Backward 180(179.1)
Iter 0594 | Time 2.4180(2.4095) | Loss 4.845952(4.869296) | NFE Forward 26(26.4) | NFE Backward 180(179.2)
Iter 0595 | Time 2.4266(2.4107) | Loss 4.876643(4.869810) | NFE Forward 26(26.4) | NFE Backward 180(179.3)
Iter 0596 | Time 2.4430(2.4130) | Loss 4.847040(4.868216) | NFE Forward 32(26.8) | NFE Backward 180(179.3)
Iter 0597 | Time 2.4134(2.4130) | Loss 4.842511(4.866417) | NFE Forward 26(26.7) | NFE Backward 180(179.4)
Iter 0598 | Time 2.4154(2.4132) | Loss 4.828727(4.863779) | NFE Forward 26(26.7) | NFE Backward 180(179.4)
Iter 0599 | Time 2.4400(2.4151) | Loss 4.812532(4.860191) | NFE Forward 32(27.1) | NFE Backward 180(179.4)
Iter 0600 | Time 2.4412(2.4169) | Loss 4.816317(4.857120) | NFE Forward 26(27.0) | NFE Backward 180(179.5)
[TEST] Iter 0600 | Test Loss 4.838513 | NFE 26
Skipping vis as data dimension is >2
Iter 0601 | Time 2.4408(2.4186) | Loss 4.839795(4.855907) | NFE Forward 26(26.9) | NFE Backward 180(179.5)
Iter 0602 | Time 2.4444(2.4204) | Loss 4.884872(4.857935) | NFE Forward 26(26.8) | NFE Backward 180(179.6)
Iter 0603 | Time 2.4255(2.4207) | Loss 4.825031(4.855632) | NFE Forward 26(26.8) | NFE Backward 180(179.6)
Iter 0604 | Time 2.4786(2.4248) | Loss 4.823422(4.853377) | NFE Forward 32(27.2) | NFE Backward 180(179.6)
Iter 0605 | Time 2.4532(2.4268) | Loss 4.834447(4.852052) | NFE Forward 26(27.1) | NFE Backward 180(179.6)
Iter 0606 | Time 2.4781(2.4304) | Loss 4.795221(4.848074) | NFE Forward 32(27.4) | NFE Backward 180(179.7)
Iter 0607 | Time 2.4786(2.4337) | Loss 4.867915(4.849463) | NFE Forward 32(27.7) | NFE Backward 180(179.7)
Iter 0608 | Time 2.4936(2.4379) | Loss 4.807065(4.846495) | NFE Forward 32(28.0) | NFE Backward 180(179.7)
Iter 0609 | Time 2.4768(2.4407) | Loss 4.812575(4.844120) | NFE Forward 32(28.3) | NFE Backward 180(179.7)
Iter 0610 | Time 2.4799(2.4434) | Loss 4.827189(4.842935) | NFE Forward 32(28.6) | NFE Backward 180(179.8)
Iter 0611 | Time 2.4641(2.4449) | Loss 4.816154(4.841061) | NFE Forward 32(28.8) | NFE Backward 180(179.8)
Iter 0612 | Time 2.4691(2.4466) | Loss 4.807623(4.838720) | NFE Forward 32(29.0) | NFE Backward 180(179.8)
Iter 0613 | Time 2.5243(2.4520) | Loss 4.791735(4.835431) | NFE Forward 32(29.2) | NFE Backward 186(180.2)
Iter 0614 | Time 2.4989(2.4553) | Loss 4.779813(4.831538) | NFE Forward 26(29.0) | NFE Backward 186(180.6)
Iter 0615 | Time 2.5242(2.4601) | Loss 4.783416(4.828169) | NFE Forward 32(29.2) | NFE Backward 186(181.0)
Iter 0616 | Time 2.5250(2.4646) | Loss 4.858166(4.830269) | NFE Forward 32(29.4) | NFE Backward 186(181.4)
Iter 0617 | Time 2.5241(2.4688) | Loss 4.813563(4.829100) | NFE Forward 32(29.6) | NFE Backward 186(181.7)
Iter 0618 | Time 2.5258(2.4728) | Loss 4.831250(4.829250) | NFE Forward 32(29.8) | NFE Backward 186(182.0)
Iter 0619 | Time 2.5110(2.4755) | Loss 4.804504(4.827518) | NFE Forward 32(29.9) | NFE Backward 186(182.3)
Iter 0620 | Time 2.5398(2.4800) | Loss 4.828302(4.827573) | NFE Forward 32(30.1) | NFE Backward 186(182.5)
Iter 0621 | Time 2.5381(2.4840) | Loss 4.788766(4.824856) | NFE Forward 32(30.2) | NFE Backward 186(182.8)
Iter 0622 | Time 2.5385(2.4878) | Loss 4.825687(4.824915) | NFE Forward 32(30.3) | NFE Backward 186(183.0)
Iter 0623 | Time 2.5468(2.4920) | Loss 4.795219(4.822836) | NFE Forward 32(30.4) | NFE Backward 186(183.2)
Iter 0624 | Time 2.5522(2.4962) | Loss 4.860134(4.825447) | NFE Forward 32(30.6) | NFE Backward 186(183.4)
Iter 0625 | Time 2.5623(2.5008) | Loss 4.836402(4.826214) | NFE Forward 32(30.7) | NFE Backward 186(183.6)
Iter 0626 | Time 2.5641(2.5053) | Loss 4.798772(4.824293) | NFE Forward 32(30.8) | NFE Backward 186(183.7)
Iter 0627 | Time 2.5602(2.5091) | Loss 4.813421(4.823532) | NFE Forward 32(30.8) | NFE Backward 186(183.9)
Iter 0628 | Time 2.5535(2.5122) | Loss 4.809001(4.822514) | NFE Forward 32(30.9) | NFE Backward 186(184.1)
Iter 0629 | Time 2.5497(2.5148) | Loss 4.812720(4.821829) | NFE Forward 32(31.0) | NFE Backward 186(184.2)
Iter 0630 | Time 2.5507(2.5173) | Loss 4.804223(4.820596) | NFE Forward 32(31.1) | NFE Backward 186(184.3)
Iter 0631 | Time 2.6449(2.5263) | Loss 4.765191(4.816718) | NFE Forward 32(31.1) | NFE Backward 186(184.4)
Iter 0632 | Time 2.5538(2.5282) | Loss 4.790802(4.814904) | NFE Forward 32(31.2) | NFE Backward 186(184.5)
Iter 0633 | Time 2.5481(2.5296) | Loss 4.844099(4.816948) | NFE Forward 32(31.2) | NFE Backward 186(184.6)
Iter 0634 | Time 2.5498(2.5310) | Loss 4.829321(4.817814) | NFE Forward 32(31.3) | NFE Backward 186(184.7)
Iter 0635 | Time 2.5547(2.5327) | Loss 4.779773(4.815151) | NFE Forward 32(31.3) | NFE Backward 186(184.8)
Iter 0636 | Time 2.5534(2.5341) | Loss 4.796067(4.813815) | NFE Forward 32(31.4) | NFE Backward 186(184.9)
Iter 0637 | Time 2.5499(2.5352) | Loss 4.832262(4.815106) | NFE Forward 32(31.4) | NFE Backward 186(185.0)
Iter 0638 | Time 2.5513(2.5363) | Loss 4.806302(4.814490) | NFE Forward 32(31.5) | NFE Backward 186(185.1)
Iter 0639 | Time 2.5498(2.5373) | Loss 4.796994(4.813265) | NFE Forward 32(31.5) | NFE Backward 186(185.1)
Iter 0640 | Time 2.5544(2.5385) | Loss 4.808084(4.812902) | NFE Forward 32(31.5) | NFE Backward 186(185.2)
Iter 0641 | Time 2.5580(2.5399) | Loss 4.806823(4.812477) | NFE Forward 32(31.6) | NFE Backward 186(185.2)
Iter 0642 | Time 2.5706(2.5420) | Loss 4.802147(4.811754) | NFE Forward 32(31.6) | NFE Backward 186(185.3)
Iter 0643 | Time 2.5533(2.5428) | Loss 4.767859(4.808681) | NFE Forward 32(31.6) | NFE Backward 186(185.3)
Iter 0644 | Time 2.5565(2.5438) | Loss 4.774682(4.806301) | NFE Forward 32(31.7) | NFE Backward 186(185.4)
Iter 0645 | Time 2.5531(2.5444) | Loss 4.747845(4.802209) | NFE Forward 32(31.7) | NFE Backward 186(185.4)
Iter 0646 | Time 2.5562(2.5452) | Loss 4.810699(4.802804) | NFE Forward 32(31.7) | NFE Backward 186(185.5)
Iter 0647 | Time 2.5589(2.5462) | Loss 4.807237(4.803114) | NFE Forward 32(31.7) | NFE Backward 186(185.5)
Iter 0648 | Time 2.5598(2.5471) | Loss 4.748694(4.799305) | NFE Forward 32(31.7) | NFE Backward 186(185.5)
Iter 0649 | Time 2.5624(2.5482) | Loss 4.795764(4.799057) | NFE Forward 32(31.8) | NFE Backward 186(185.6)
Iter 0650 | Time 2.5549(2.5487) | Loss 4.763824(4.796590) | NFE Forward 32(31.8) | NFE Backward 186(185.6)
Iter 0651 | Time 2.5559(2.5492) | Loss 4.758220(4.793904) | NFE Forward 32(31.8) | NFE Backward 186(185.6)
Iter 0652 | Time 2.5570(2.5497) | Loss 4.729554(4.789400) | NFE Forward 32(31.8) | NFE Backward 186(185.7)
Iter 0653 | Time 2.5534(2.5500) | Loss 4.782605(4.788924) | NFE Forward 32(31.8) | NFE Backward 186(185.7)
Iter 0654 | Time 2.5543(2.5503) | Loss 4.788897(4.788922) | NFE Forward 32(31.8) | NFE Backward 186(185.7)
Iter 0655 | Time 2.5635(2.5512) | Loss 4.790551(4.789036) | NFE Forward 32(31.8) | NFE Backward 186(185.7)
Iter 0656 | Time 2.5363(2.5502) | Loss 4.837354(4.792419) | NFE Forward 32(31.9) | NFE Backward 186(185.7)
Iter 0657 | Time 2.5420(2.5496) | Loss 4.736359(4.788494) | NFE Forward 32(31.9) | NFE Backward 186(185.8)
Iter 0658 | Time 2.5408(2.5490) | Loss 4.821864(4.790830) | NFE Forward 32(31.9) | NFE Backward 186(185.8)
Iter 0659 | Time 2.5425(2.5485) | Loss 4.773426(4.789612) | NFE Forward 32(31.9) | NFE Backward 186(185.8)
Iter 0660 | Time 2.5384(2.5478) | Loss 4.783272(4.789168) | NFE Forward 32(31.9) | NFE Backward 186(185.8)
Iter 0661 | Time 2.5398(2.5473) | Loss 4.809199(4.790570) | NFE Forward 32(31.9) | NFE Backward 186(185.8)
Iter 0662 | Time 2.5382(2.5466) | Loss 4.776723(4.789601) | NFE Forward 32(31.9) | NFE Backward 186(185.8)
Iter 0663 | Time 2.5414(2.5463) | Loss 4.733506(4.785674) | NFE Forward 32(31.9) | NFE Backward 186(185.8)
Iter 0664 | Time 2.5442(2.5461) | Loss 4.755322(4.783550) | NFE Forward 32(31.9) | NFE Backward 186(185.9)
Iter 0665 | Time 2.5400(2.5457) | Loss 4.732025(4.779943) | NFE Forward 32(31.9) | NFE Backward 186(185.9)
Iter 0666 | Time 2.5445(2.5456) | Loss 4.817101(4.782544) | NFE Forward 32(31.9) | NFE Backward 186(185.9)
Iter 0667 | Time 2.5406(2.5453) | Loss 4.736100(4.779293) | NFE Forward 32(31.9) | NFE Backward 186(185.9)
Iter 0668 | Time 2.5403(2.5449) | Loss 4.765929(4.778357) | NFE Forward 32(31.9) | NFE Backward 186(185.9)
Iter 0669 | Time 2.5422(2.5447) | Loss 4.736880(4.775454) | NFE Forward 32(31.9) | NFE Backward 186(185.9)
Iter 0670 | Time 2.5433(2.5446) | Loss 4.784191(4.776066) | NFE Forward 32(31.9) | NFE Backward 186(185.9)
Iter 0671 | Time 2.5536(2.5452) | Loss 4.766878(4.775422) | NFE Forward 32(32.0) | NFE Backward 186(185.9)
Iter 0672 | Time 2.5479(2.5454) | Loss 4.765064(4.774697) | NFE Forward 32(32.0) | NFE Backward 186(185.9)
Iter 0673 | Time 2.5438(2.5453) | Loss 4.741936(4.772404) | NFE Forward 32(32.0) | NFE Backward 186(185.9)
Iter 0674 | Time 2.5396(2.5449) | Loss 4.757749(4.771378) | NFE Forward 32(32.0) | NFE Backward 186(185.9)
Iter 0675 | Time 2.5534(2.5455) | Loss 4.729947(4.768478) | NFE Forward 32(32.0) | NFE Backward 186(185.9)
Iter 0676 | Time 2.5512(2.5459) | Loss 4.760894(4.767947) | NFE Forward 32(32.0) | NFE Backward 186(185.9)
Iter 0677 | Time 2.5498(2.5462) | Loss 4.810967(4.770959) | NFE Forward 32(32.0) | NFE Backward 186(185.9)
Iter 0678 | Time 2.5540(2.5467) | Loss 4.730913(4.768155) | NFE Forward 32(32.0) | NFE Backward 186(185.9)
Iter 0679 | Time 2.5647(2.5480) | Loss 4.739574(4.766155) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0680 | Time 2.5520(2.5483) | Loss 4.785549(4.767512) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0681 | Time 2.5515(2.5485) | Loss 4.787257(4.768894) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0682 | Time 2.5500(2.5486) | Loss 4.723143(4.765692) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0683 | Time 2.5527(2.5489) | Loss 4.760483(4.765327) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0684 | Time 2.5545(2.5493) | Loss 4.728657(4.762760) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0685 | Time 2.5613(2.5501) | Loss 4.745853(4.761577) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0686 | Time 2.5552(2.5505) | Loss 4.738367(4.759952) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0687 | Time 2.5559(2.5509) | Loss 4.782442(4.761526) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0688 | Time 2.5572(2.5513) | Loss 4.734272(4.759619) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0689 | Time 2.5606(2.5520) | Loss 4.730934(4.757611) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0690 | Time 2.5595(2.5525) | Loss 4.765209(4.758142) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0691 | Time 2.5559(2.5527) | Loss 4.732094(4.756319) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0692 | Time 2.5792(2.5546) | Loss 4.794508(4.758992) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0693 | Time 2.5524(2.5544) | Loss 4.774494(4.760077) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0694 | Time 2.5524(2.5543) | Loss 4.742290(4.758832) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0695 | Time 2.5566(2.5544) | Loss 4.769546(4.759582) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0696 | Time 2.5557(2.5545) | Loss 4.725620(4.757205) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0697 | Time 2.5535(2.5545) | Loss 4.758493(4.757295) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0698 | Time 2.5572(2.5546) | Loss 4.743593(4.756336) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0699 | Time 2.5598(2.5550) | Loss 4.754664(4.756219) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0700 | Time 2.5540(2.5549) | Loss 4.720950(4.753750) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
[TEST] Iter 0700 | Test Loss 4.730682 | NFE 32
Skipping vis as data dimension is >2
Iter 0701 | Time 2.5526(2.5548) | Loss 4.704379(4.750294) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0702 | Time 2.5884(2.5571) | Loss 4.744310(4.749875) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0703 | Time 2.5626(2.5575) | Loss 4.732353(4.748649) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0704 | Time 2.5658(2.5581) | Loss 4.703542(4.745491) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0705 | Time 2.5592(2.5582) | Loss 4.745003(4.745457) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0706 | Time 2.5615(2.5584) | Loss 4.727102(4.744172) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0707 | Time 2.5581(2.5584) | Loss 4.746023(4.744302) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0708 | Time 2.5520(2.5579) | Loss 4.764412(4.745709) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0709 | Time 2.5387(2.5566) | Loss 4.763384(4.746947) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0710 | Time 2.5643(2.5571) | Loss 4.761900(4.747993) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0711 | Time 2.5349(2.5556) | Loss 4.699147(4.744574) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0712 | Time 2.5388(2.5544) | Loss 4.768498(4.746249) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0713 | Time 2.5635(2.5550) | Loss 4.706963(4.743499) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0714 | Time 2.5622(2.5555) | Loss 4.738999(4.743184) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0715 | Time 2.5584(2.5557) | Loss 4.737166(4.742763) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0716 | Time 2.5610(2.5561) | Loss 4.717165(4.740971) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0717 | Time 2.5359(2.5547) | Loss 4.758938(4.742228) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0718 | Time 2.5648(2.5554) | Loss 4.663561(4.736722) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0719 | Time 2.5405(2.5544) | Loss 4.757828(4.738199) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0720 | Time 2.5344(2.5530) | Loss 4.718338(4.736809) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0721 | Time 2.5334(2.5516) | Loss 4.714887(4.735274) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0722 | Time 2.5367(2.5505) | Loss 4.681723(4.731526) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0723 | Time 2.5364(2.5496) | Loss 4.733985(4.731698) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0724 | Time 2.5317(2.5483) | Loss 4.735361(4.731954) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0725 | Time 2.5652(2.5495) | Loss 4.712944(4.730624) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0726 | Time 2.5645(2.5505) | Loss 4.646559(4.724739) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0727 | Time 2.5616(2.5513) | Loss 4.730014(4.725108) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0728 | Time 2.5590(2.5519) | Loss 4.729544(4.725419) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0729 | Time 2.5351(2.5507) | Loss 4.698377(4.723526) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0730 | Time 2.5637(2.5516) | Loss 4.713428(4.722819) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0731 | Time 2.5600(2.5522) | Loss 4.675150(4.719482) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0732 | Time 2.5591(2.5527) | Loss 4.663092(4.715535) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0733 | Time 2.5393(2.5517) | Loss 4.666181(4.712080) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0734 | Time 2.5593(2.5523) | Loss 4.694521(4.710851) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0735 | Time 2.5361(2.5511) | Loss 4.715218(4.711157) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0736 | Time 2.5641(2.5520) | Loss 4.729486(4.712440) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0737 | Time 2.5589(2.5525) | Loss 4.714523(4.712586) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0738 | Time 2.5609(2.5531) | Loss 4.748360(4.715090) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0739 | Time 2.5615(2.5537) | Loss 4.693156(4.713554) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0740 | Time 2.5600(2.5541) | Loss 4.739811(4.715392) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0741 | Time 2.5616(2.5546) | Loss 4.736269(4.716854) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0742 | Time 2.5652(2.5554) | Loss 4.720840(4.717133) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0743 | Time 2.5755(2.5568) | Loss 4.689800(4.715220) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0744 | Time 2.5621(2.5572) | Loss 4.710569(4.714894) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0745 | Time 2.5675(2.5579) | Loss 4.706969(4.714339) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0746 | Time 2.5369(2.5564) | Loss 4.684198(4.712229) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0747 | Time 2.5639(2.5569) | Loss 4.731931(4.713608) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Density Loss 0.37270602583885193
Density Loss 0.382041871547699
Density Loss 0.3784896433353424
Density Loss 0.3771039843559265
Density Loss 0.3778480589389801
Density Loss 0.36793166399002075
Density Loss 0.37873387336730957
Density Loss 0.381743848323822
Density Loss 0.37768763303756714
Density Loss 0.3780282735824585
Density Loss 0.3907093107700348
Density Loss 0.38167810440063477
Density Loss 0.37672197818756104
Density Loss 0.37867632508277893
Density Loss 0.3879319131374359
Density Loss 0.3764607608318329
Density Loss 0.37841683626174927
Density Loss 0.3831029236316681
Density Loss 0.38086625933647156
Density Loss 0.37563756108283997
Density Loss 0.37994205951690674
Density Loss 0.38277387619018555
Density Loss 0.3746994435787201
Density Loss 0.3903951644897461
Density Loss 0.3763657510280609
Density Loss 0.38743215799331665
Density Loss 0.3871108889579773
Density Loss 0.3701678216457367
Density Loss 0.3815653622150421
Density Loss 0.3830493688583374
Density Loss 0.38136810064315796
Density Loss 0.38561248779296875
Density Loss 0.3845670223236084
Density Loss 0.38334035873413086
Density Loss 0.37422990798950195
Density Loss 0.38085371255874634
Density Loss 0.3765237331390381
Density Loss 0.37756428122520447
Density Loss 0.37515825033187866
Density Loss 0.39383643865585327
Density Loss 0.3819665312767029
Density Loss 0.3866230249404907
Density Loss 0.37774690985679626
Density Loss 0.3854832351207733
Density Loss 0.39171892404556274
Density Loss 0.387381374835968
Density Loss 0.37656542658805847
Density Loss 0.3834843039512634
Density Loss 0.38816770911216736
Density Loss 0.39327141642570496
Density Loss 0.38678106665611267
Density Loss 0.3783244490623474
Density Loss 0.37589311599731445
Density Loss 0.37880000472068787
Density Loss 0.388197124004364
Density Loss 0.37247616052627563
Density Loss 0.37668576836586
Density Loss 0.3922196328639984
Density Loss 0.37630781531333923
Density Loss 0.3890778124332428
Density Loss 0.3911206126213074
Density Loss 0.38614946603775024
Density Loss 0.38878774642944336
Density Loss 0.37506628036499023
Density Loss 0.38830801844596863
Density Loss 0.38170161843299866
Density Loss 0.37730830907821655
Density Loss 0.3945411145687103
Density Loss 0.38653796911239624
Density Loss 0.39135798811912537
Density Loss 0.3829972743988037
Density Loss 0.3820481598377228
Density Loss 0.37870994210243225
Density Loss 0.3834321200847626
Density Loss 0.3913589417934418
Density Loss 0.4008595943450928
Density Loss 0.37869101762771606
Density Loss 0.37284141778945923
Density Loss 0.3777552843093872
Density Loss 0.39824846386909485
Density Loss 0.3732772469520569
Density Loss 0.38041502237319946
Density Loss 0.39160484075546265
Density Loss 0.37406301498413086
Density Loss 0.3960438668727875
Density Loss 0.3923437297344208
Density Loss 0.38828179240226746
Density Loss 0.39043354988098145
Density Loss 0.38743534684181213
Density Loss 0.3797842264175415
Density Loss 0.39056894183158875
Density Loss 0.3822542130947113
Density Loss 0.38130345940589905
Density Loss 0.3885250389575958
Density Loss 0.3937722444534302
Density Loss 0.39123499393463135
Density Loss 0.3866942822933197
Density Loss 0.38498276472091675
Density Loss 0.40154340863227844
Density Loss 0.38702312111854553
Density Loss 0.3987887501716614
Density Loss 0.38826948404312134
Density Loss 0.38025984168052673
Density Loss 0.38376763463020325
Density Loss 0.38494300842285156
Density Loss 0.39026880264282227
Density Loss 0.3951303958892822
Density Loss 0.3929232954978943
Density Loss 0.391252338886261
Density Loss 0.37309515476226807
Density Loss 0.39374271035194397
Density Loss 0.3886706531047821
Density Loss 0.387592077255249
Density Loss 0.3841540813446045
Density Loss 0.3957439959049225
Density Loss 0.3877934515476227
Density Loss 0.39452269673347473
Density Loss 0.38598501682281494
Density Loss 0.40001603960990906
Density Loss 0.3952161967754364
Density Loss 0.3888600468635559
Density Loss 0.38154953718185425
Density Loss 0.38497722148895264
Density Loss 0.40454238653182983
Density Loss 0.39404886960983276
Density Loss 0.3853103518486023
Density Loss 0.39114394783973694
Density Loss 0.413472980260849
Density Loss 0.3983972668647766
Density Loss 0.38415372371673584
Density Loss 0.38002312183380127
Density Loss 0.396415650844574
Density Loss 0.39004388451576233
Density Loss 0.3875497877597809
Density Loss 0.39958831667900085
Density Loss 0.3850298821926117
Density Loss 0.37815216183662415
Density Loss 0.39261987805366516
Density Loss 0.3924159109592438
Density Loss 0.38726845383644104
Density Loss 0.3973895311355591
Density Loss 0.39389970898628235
Density Loss 0.4009617865085602
Density Loss 0.38897088170051575
Density Loss 0.39072659611701965
Density Loss 0.4019591212272644
Density Loss 0.3975820243358612
Density Loss 0.38013315200805664
Density Loss 0.39968276023864746
Density Loss 0.39483702182769775
Density Loss 0.3973986804485321
Density Loss 0.40376850962638855
Density Loss 0.40029144287109375
Density Loss 0.4043067395687103
Density Loss 0.39705780148506165
Density Loss 0.4024614095687866
Density Loss 0.39903098344802856
Density Loss 0.395405113697052
Density Loss 0.41097375750541687
Density Loss 0.3980000615119934
Density Loss 0.3961201012134552
Density Loss 0.3958694338798523
Density Loss 0.40116941928863525
Density Loss 0.39196622371673584
Density Loss 0.4039151668548584
Density Loss 0.4026358425617218
Density Loss 0.38537895679473877
Density Loss 0.3985097408294678
Density Loss 0.4016866385936737
Density Loss 0.3933990001678467
Density Loss 0.40080371499061584
Density Loss 0.39735522866249084
Density Loss 0.3909198045730591
Density Loss 0.39363303780555725
Density Loss 0.4043467342853546
Density Loss 0.396941214799881
Density Loss 0.38899511098861694
Density Loss 0.4035729467868805
Density Loss 0.39829298853874207
Density Loss 0.39218154549598694
Density Loss 0.4039570093154907
Density Loss 0.3963412046432495
Density Loss 0.3942188024520874
Density Loss 0.4011414051055908
Density Loss 0.40322351455688477
Density Loss 0.40041255950927734
Density Loss 0.40146347880363464
Density Loss 0.3954765200614929
Density Loss 0.3947635591030121
Density Loss 0.39568808674812317
Density Loss 0.4078460931777954
Density Loss 0.4087505638599396
Density Loss 0.3906497061252594
Density Loss 0.40244266390800476
Density Loss 0.3962973654270172
Density Loss 0.39431536197662354
Density Loss 0.4069017171859741
Density Loss 0.3952160179615021
Density Loss 0.4080410599708557
Density Loss 0.4054029881954193
Density Loss 0.3982178270816803
Density Loss 0.40149441361427307
Density Loss 0.4105270504951477
Density Loss 0.40216824412345886
Density Loss 0.4043779671192169
Density Loss 0.39710789918899536
Density Loss 0.39841949939727783
Density Loss 0.4036193788051605
Density Loss 0.4161200225353241
Density Loss 0.41772380471229553
Density Loss 0.4079228341579437
Density Loss 0.4161437749862671
Density Loss 0.40805715322494507
Density Loss 0.41513681411743164
Density Loss 0.41232073307037354
Density Loss 0.40419113636016846
Density Loss 0.402901828289032
Density Loss 0.40438830852508545
Density Loss 0.41130882501602173
Density Loss 0.4114045202732086
Density Loss 0.40964236855506897
Density Loss 0.3952566087245941
Density Loss 0.3988342583179474
Density Loss 0.40400463342666626
Density Loss 0.3979824483394623
Density Loss 0.41482284665107727
Density Loss 0.4121607840061188
Density Loss 0.4112860858440399
Density Loss 0.41429686546325684
Density Loss 0.40316224098205566
Density Loss 0.4115401804447174
Density Loss 0.4079354405403137
Density Loss 0.40172553062438965
Density Loss 0.41647058725357056
Density Loss 0.4007073640823364
Density Loss 0.41965723037719727
Density Loss 0.4149686396121979
Density Loss 0.42077893018722534
Density Loss 0.4060744643211365
Density Loss 0.4077649414539337
Density Loss 0.3930310606956482
Density Loss 0.4176250994205475
Density Loss 0.41808021068573
Density Loss 0.3987770080566406
Density Loss 0.4140177071094513
Density Loss 0.4050677716732025
Density Loss 0.41712430119514465
Density Loss 0.40290188789367676
Density Loss 0.41294172406196594
Density Loss 0.4072449207305908
Density Loss 0.40661439299583435
Density Loss 0.4112662971019745
Iter 0748 | Time 2.5610(2.5572) | Loss 4.674620(4.710879) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0749 | Time 2.5716(2.5582) | Loss 4.698936(4.710043) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0750 | Time 2.5635(2.5586) | Loss 4.690918(4.708705) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0751 | Time 2.5686(2.5593) | Loss 4.711659(4.708911) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0752 | Time 2.5603(2.5594) | Loss 4.719291(4.709638) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0753 | Time 2.5617(2.5595) | Loss 4.720787(4.710418) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0754 | Time 2.5590(2.5595) | Loss 4.662562(4.707068) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0755 | Time 2.5580(2.5594) | Loss 4.702570(4.706754) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0756 | Time 2.5618(2.5596) | Loss 4.687392(4.705398) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0757 | Time 2.5651(2.5599) | Loss 4.657485(4.702044) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0758 | Time 2.5568(2.5597) | Loss 4.704589(4.702222) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0759 | Time 2.5585(2.5596) | Loss 4.695499(4.701752) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0760 | Time 2.5599(2.5597) | Loss 4.686641(4.700694) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0761 | Time 2.5584(2.5596) | Loss 4.689493(4.699910) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0762 | Time 2.5567(2.5594) | Loss 4.709836(4.700605) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0763 | Time 2.5579(2.5593) | Loss 4.704689(4.700891) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0764 | Time 2.5605(2.5594) | Loss 4.689427(4.700088) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0765 | Time 2.5596(2.5594) | Loss 4.685608(4.699075) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0766 | Time 2.5628(2.5596) | Loss 4.736187(4.701672) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0767 | Time 2.5601(2.5596) | Loss 4.662461(4.698928) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0768 | Time 2.5642(2.5600) | Loss 4.691298(4.698394) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0769 | Time 2.5632(2.5602) | Loss 4.652005(4.695146) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0770 | Time 2.5589(2.5601) | Loss 4.698010(4.695347) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0771 | Time 2.5556(2.5598) | Loss 4.673377(4.693809) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0772 | Time 2.5619(2.5599) | Loss 4.677528(4.692669) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0773 | Time 2.5572(2.5597) | Loss 4.695305(4.692854) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0774 | Time 2.5594(2.5597) | Loss 4.724692(4.695082) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0775 | Time 2.5568(2.5595) | Loss 4.649195(4.691870) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0776 | Time 2.5593(2.5595) | Loss 4.708619(4.693043) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0777 | Time 2.5600(2.5595) | Loss 4.636453(4.689081) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0778 | Time 2.5766(2.5607) | Loss 4.720212(4.691261) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0779 | Time 2.5750(2.5617) | Loss 4.704694(4.692201) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0780 | Time 2.5768(2.5628) | Loss 4.672777(4.690841) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0781 | Time 2.5761(2.5637) | Loss 4.696205(4.691217) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0782 | Time 2.5733(2.5644) | Loss 4.655893(4.688744) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0783 | Time 2.5784(2.5654) | Loss 4.641618(4.685445) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0784 | Time 2.5760(2.5661) | Loss 4.693915(4.686038) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0785 | Time 2.5747(2.5667) | Loss 4.689657(4.686291) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0786 | Time 2.5725(2.5671) | Loss 4.655389(4.684128) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0787 | Time 2.5740(2.5676) | Loss 4.673023(4.683351) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0788 | Time 2.5741(2.5681) | Loss 4.677020(4.682908) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0789 | Time 2.6044(2.5706) | Loss 4.639319(4.679856) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0790 | Time 2.5751(2.5709) | Loss 4.663329(4.678700) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0791 | Time 2.5786(2.5714) | Loss 4.676671(4.678558) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0792 | Time 2.5757(2.5718) | Loss 4.648180(4.676431) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0793 | Time 2.5751(2.5720) | Loss 4.656558(4.675040) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0794 | Time 2.5740(2.5721) | Loss 4.688497(4.675982) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0795 | Time 2.5760(2.5724) | Loss 4.680855(4.676323) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0796 | Time 2.5803(2.5730) | Loss 4.656240(4.674917) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0797 | Time 2.5714(2.5728) | Loss 4.695768(4.676377) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0798 | Time 2.6007(2.5748) | Loss 4.655221(4.674896) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0799 | Time 2.5728(2.5747) | Loss 4.637170(4.672255) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0800 | Time 2.5747(2.5747) | Loss 4.631171(4.669379) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
[TEST] Iter 0800 | Test Loss 4.676871 | NFE 32
Skipping vis as data dimension is >2
Iter 0801 | Time 2.5736(2.5746) | Loss 4.649311(4.667974) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0802 | Time 2.5854(2.5753) | Loss 4.702480(4.670390) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0803 | Time 2.5796(2.5756) | Loss 4.611154(4.666243) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0804 | Time 2.5763(2.5757) | Loss 4.654765(4.665440) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0805 | Time 2.5733(2.5755) | Loss 4.700031(4.667861) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0806 | Time 2.5905(2.5766) | Loss 4.674522(4.668327) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0807 | Time 2.5843(2.5771) | Loss 4.659611(4.667717) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0808 | Time 2.5854(2.5777) | Loss 4.647048(4.666270) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0809 | Time 2.5841(2.5781) | Loss 4.688427(4.667821) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0810 | Time 2.5835(2.5785) | Loss 4.658952(4.667201) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0811 | Time 2.5846(2.5789) | Loss 4.670353(4.667421) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0812 | Time 2.6068(2.5809) | Loss 4.662439(4.667073) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0813 | Time 2.5783(2.5807) | Loss 4.669324(4.667230) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0814 | Time 2.6004(2.5821) | Loss 4.656133(4.666453) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0815 | Time 2.5690(2.5812) | Loss 4.651401(4.665400) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0816 | Time 2.5933(2.5820) | Loss 4.623647(4.662477) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0817 | Time 2.5661(2.5809) | Loss 4.596869(4.657884) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0818 | Time 2.6016(2.5824) | Loss 4.672848(4.658932) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0819 | Time 2.5735(2.5817) | Loss 4.608129(4.655376) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0820 | Time 2.5914(2.5824) | Loss 4.630888(4.653661) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0821 | Time 2.5922(2.5831) | Loss 4.622427(4.651475) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0822 | Time 2.5668(2.5820) | Loss 4.648979(4.651300) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0823 | Time 2.5673(2.5809) | Loss 4.611723(4.648530) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0824 | Time 2.5923(2.5817) | Loss 4.628806(4.647149) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0825 | Time 2.5930(2.5825) | Loss 4.633686(4.646207) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0826 | Time 2.5942(2.5833) | Loss 4.677064(4.648367) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0827 | Time 2.5676(2.5822) | Loss 4.611080(4.645757) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0828 | Time 2.5929(2.5830) | Loss 4.645463(4.645736) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0829 | Time 2.5863(2.5832) | Loss 4.680958(4.648202) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0830 | Time 2.5706(2.5823) | Loss 4.632587(4.647109) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0831 | Time 2.5666(2.5812) | Loss 4.614697(4.644840) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0832 | Time 2.6539(2.5863) | Loss 4.678977(4.647229) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0833 | Time 2.5644(2.5848) | Loss 4.693970(4.650501) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0834 | Time 2.5669(2.5835) | Loss 4.650959(4.650533) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0835 | Time 2.5973(2.5845) | Loss 4.636183(4.649529) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0836 | Time 2.5699(2.5835) | Loss 4.653123(4.649780) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0837 | Time 2.5666(2.5823) | Loss 4.659140(4.650436) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0838 | Time 2.6075(2.5841) | Loss 4.625899(4.648718) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0839 | Time 2.6044(2.5855) | Loss 4.616862(4.646488) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0840 | Time 2.5939(2.5861) | Loss 4.631101(4.645411) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0841 | Time 2.6005(2.5871) | Loss 4.597785(4.642077) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0842 | Time 2.6068(2.5885) | Loss 4.639114(4.641870) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0843 | Time 2.5750(2.5875) | Loss 4.638449(4.641630) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0844 | Time 2.6004(2.5884) | Loss 4.661570(4.643026) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0845 | Time 2.6003(2.5892) | Loss 4.632027(4.642256) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0846 | Time 2.6002(2.5900) | Loss 4.617195(4.640502) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0847 | Time 2.5995(2.5907) | Loss 4.599194(4.637610) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0848 | Time 2.6109(2.5921) | Loss 4.576862(4.633358) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0849 | Time 2.6106(2.5934) | Loss 4.599902(4.631016) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0850 | Time 2.6122(2.5947) | Loss 4.660584(4.633086) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0851 | Time 2.6056(2.5955) | Loss 4.599568(4.630740) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0852 | Time 2.6080(2.5963) | Loss 4.667662(4.633324) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0853 | Time 2.6033(2.5968) | Loss 4.638175(4.633664) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0854 | Time 2.6064(2.5975) | Loss 4.643799(4.634373) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0855 | Time 2.6012(2.5978) | Loss 4.628683(4.633975) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0856 | Time 2.6042(2.5982) | Loss 4.622497(4.633171) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0857 | Time 2.6047(2.5987) | Loss 4.642835(4.633848) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0858 | Time 2.6010(2.5988) | Loss 4.637456(4.634100) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0859 | Time 2.6049(2.5992) | Loss 4.626681(4.633581) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0860 | Time 2.5950(2.5989) | Loss 4.632084(4.633476) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0861 | Time 2.5960(2.5987) | Loss 4.596617(4.630896) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0862 | Time 2.5983(2.5987) | Loss 4.619281(4.630083) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0863 | Time 2.5950(2.5984) | Loss 4.594022(4.627559) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0864 | Time 2.5995(2.5985) | Loss 4.618296(4.626910) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0865 | Time 2.6112(2.5994) | Loss 4.607152(4.625527) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0866 | Time 2.6005(2.5995) | Loss 4.601938(4.623876) | NFE Forward 32(32.0) | NFE Backward 186(186.0)
Iter 0867 | Time 2.6586(2.6036) | Loss 4.615934(4.623320) | NFE Forward 32(32.0) | NFE Backward 192(186.4)
Iter 0868 | Time 2.5965(2.6031) | Loss 4.641735(4.624609) | NFE Forward 32(32.0) | NFE Backward 186(186.4)
Iter 0869 | Time 2.5990(2.6028) | Loss 4.619409(4.624245) | NFE Forward 32(32.0) | NFE Backward 186(186.4)
Iter 0870 | Time 2.5972(2.6024) | Loss 4.605617(4.622941) | NFE Forward 32(32.0) | NFE Backward 186(186.3)
Iter 0871 | Time 2.5974(2.6021) | Loss 4.600851(4.621395) | NFE Forward 32(32.0) | NFE Backward 186(186.3)
Iter 0872 | Time 2.5998(2.6019) | Loss 4.580026(4.618499) | NFE Forward 32(32.0) | NFE Backward 186(186.3)
Iter 0873 | Time 2.6137(2.6028) | Loss 4.619385(4.618561) | NFE Forward 32(32.0) | NFE Backward 186(186.3)
Iter 0874 | Time 2.6124(2.6034) | Loss 4.631147(4.619442) | NFE Forward 32(32.0) | NFE Backward 186(186.3)
Iter 0875 | Time 2.6205(2.6046) | Loss 4.665291(4.622651) | NFE Forward 32(32.0) | NFE Backward 186(186.2)
Iter 0876 | Time 2.7639(2.6158) | Loss 4.618814(4.622383) | NFE Forward 32(32.0) | NFE Backward 198(187.1)
Iter 0877 | Time 2.6253(2.6164) | Loss 4.635691(4.623314) | NFE Forward 32(32.0) | NFE Backward 186(187.0)
Iter 0878 | Time 2.6228(2.6169) | Loss 4.654740(4.625514) | NFE Forward 32(32.0) | NFE Backward 186(186.9)
Iter 0879 | Time 2.6240(2.6174) | Loss 4.611419(4.624528) | NFE Forward 32(32.0) | NFE Backward 186(186.9)
Iter 0880 | Time 2.6860(2.6222) | Loss 4.610539(4.623548) | NFE Forward 32(32.0) | NFE Backward 192(187.2)
Iter 0881 | Time 2.6847(2.6266) | Loss 4.652838(4.625599) | NFE Forward 32(32.0) | NFE Backward 192(187.5)
Iter 0882 | Time 2.6309(2.6269) | Loss 4.610694(4.624555) | NFE Forward 32(32.0) | NFE Backward 186(187.4)
Iter 0883 | Time 2.6260(2.6268) | Loss 4.592422(4.622306) | NFE Forward 32(32.0) | NFE Backward 186(187.3)
Iter 0884 | Time 2.6855(2.6309) | Loss 4.573494(4.618889) | NFE Forward 32(32.0) | NFE Backward 192(187.7)
Iter 0885 | Time 2.6233(2.6304) | Loss 4.631715(4.619787) | NFE Forward 32(32.0) | NFE Backward 186(187.5)
Iter 0886 | Time 2.6208(2.6297) | Loss 4.632702(4.620691) | NFE Forward 32(32.0) | NFE Backward 186(187.4)
Iter 0887 | Time 2.6278(2.6296) | Loss 4.576525(4.617599) | NFE Forward 32(32.0) | NFE Backward 186(187.3)
Iter 0888 | Time 2.6356(2.6300) | Loss 4.640747(4.619220) | NFE Forward 32(32.0) | NFE Backward 186(187.2)
Iter 0889 | Time 2.6222(2.6295) | Loss 4.604590(4.618196) | NFE Forward 32(32.0) | NFE Backward 186(187.2)
Iter 0890 | Time 2.6322(2.6297) | Loss 4.623794(4.618588) | NFE Forward 32(32.0) | NFE Backward 186(187.1)
Iter 0891 | Time 2.6601(2.6318) | Loss 4.560741(4.614538) | NFE Forward 32(32.0) | NFE Backward 186(187.0)
Iter 0892 | Time 2.6943(2.6362) | Loss 4.565881(4.611132) | NFE Forward 32(32.0) | NFE Backward 192(187.4)
Iter 0893 | Time 2.6184(2.6349) | Loss 4.577891(4.608805) | NFE Forward 32(32.0) | NFE Backward 186(187.3)
Iter 0894 | Time 2.6210(2.6339) | Loss 4.554774(4.605023) | NFE Forward 32(32.0) | NFE Backward 186(187.2)
Iter 0895 | Time 2.7405(2.6414) | Loss 4.632188(4.606925) | NFE Forward 32(32.0) | NFE Backward 198(187.9)
Iter 0896 | Time 2.6810(2.6442) | Loss 4.610398(4.607168) | NFE Forward 32(32.0) | NFE Backward 192(188.2)
Iter 0897 | Time 2.6170(2.6423) | Loss 4.586990(4.605755) | NFE Forward 32(32.0) | NFE Backward 186(188.1)
Iter 0898 | Time 2.6212(2.6408) | Loss 4.569597(4.603224) | NFE Forward 32(32.0) | NFE Backward 186(187.9)
Iter 0899 | Time 2.6333(2.6403) | Loss 4.588427(4.602189) | NFE Forward 32(32.0) | NFE Backward 186(187.8)
Iter 0900 | Time 2.6995(2.6444) | Loss 4.559232(4.599182) | NFE Forward 32(32.0) | NFE Backward 192(188.1)
[TEST] Iter 0900 | Test Loss 4.628522 | NFE 32
Skipping vis as data dimension is >2
Iter 0901 | Time 2.7392(2.6510) | Loss 4.615647(4.600334) | NFE Forward 32(32.0) | NFE Backward 198(188.8)
Iter 0902 | Time 2.7428(2.6575) | Loss 4.641807(4.603237) | NFE Forward 32(32.0) | NFE Backward 198(189.4)
Iter 0903 | Time 2.6245(2.6552) | Loss 4.606071(4.603436) | NFE Forward 32(32.0) | NFE Backward 186(189.2)
Iter 0904 | Time 2.7444(2.6614) | Loss 4.592448(4.602666) | NFE Forward 32(32.0) | NFE Backward 198(189.8)
Iter 0905 | Time 2.7426(2.6671) | Loss 4.566234(4.600116) | NFE Forward 32(32.0) | NFE Backward 198(190.4)
Iter 0906 | Time 2.6856(2.6684) | Loss 4.613245(4.601035) | NFE Forward 32(32.0) | NFE Backward 192(190.5)
Iter 0907 | Time 2.6170(2.6648) | Loss 4.590179(4.600275) | NFE Forward 32(32.0) | NFE Backward 186(190.2)
Iter 0908 | Time 2.5974(2.6601) | Loss 4.600008(4.600257) | NFE Forward 32(32.0) | NFE Backward 186(189.9)
Iter 0909 | Time 2.7156(2.6640) | Loss 4.609648(4.600914) | NFE Forward 32(32.0) | NFE Backward 198(190.4)
Iter 0910 | Time 2.5858(2.6585) | Loss 4.601633(4.600964) | NFE Forward 32(32.0) | NFE Backward 186(190.1)
Iter 0911 | Time 2.6958(2.6611) | Loss 4.637633(4.603531) | NFE Forward 32(32.0) | NFE Backward 198(190.7)
Iter 0912 | Time 2.6396(2.6596) | Loss 4.642970(4.606292) | NFE Forward 32(32.0) | NFE Backward 192(190.8)
Iter 0913 | Time 2.5791(2.6540) | Loss 4.613534(4.606799) | NFE Forward 32(32.0) | NFE Backward 186(190.4)
Iter 0914 | Time 2.5814(2.6489) | Loss 4.609339(4.606977) | NFE Forward 32(32.0) | NFE Backward 186(190.1)
Iter 0915 | Time 2.5804(2.6441) | Loss 4.592804(4.605984) | NFE Forward 32(32.0) | NFE Backward 186(189.8)
Iter 0916 | Time 2.5860(2.6400) | Loss 4.562557(4.602945) | NFE Forward 32(32.0) | NFE Backward 186(189.6)
Iter 0917 | Time 2.7157(2.6453) | Loss 4.597420(4.602558) | NFE Forward 32(32.0) | NFE Backward 198(190.2)
Iter 0918 | Time 2.7142(2.6501) | Loss 4.620861(4.603839) | NFE Forward 32(32.0) | NFE Backward 198(190.7)
Iter 0919 | Time 2.5942(2.6462) | Loss 4.600482(4.603604) | NFE Forward 32(32.0) | NFE Backward 186(190.4)
Iter 0920 | Time 2.6111(2.6438) | Loss 4.598246(4.603229) | NFE Forward 32(32.0) | NFE Backward 186(190.1)
Iter 0921 | Time 2.5960(2.6404) | Loss 4.547814(4.599350) | NFE Forward 32(32.0) | NFE Backward 186(189.8)
Iter 0922 | Time 2.6092(2.6382) | Loss 4.591323(4.598788) | NFE Forward 32(32.0) | NFE Backward 186(189.5)
Iter 0923 | Time 2.7548(2.6464) | Loss 4.586133(4.597902) | NFE Forward 32(32.0) | NFE Backward 198(190.1)
Iter 0924 | Time 2.5926(2.6426) | Loss 4.596945(4.597835) | NFE Forward 32(32.0) | NFE Backward 186(189.8)
Iter 0925 | Time 2.5921(2.6391) | Loss 4.569735(4.595868) | NFE Forward 32(32.0) | NFE Backward 186(189.6)
Iter 0926 | Time 2.7093(2.6440) | Loss 4.624093(4.597844) | NFE Forward 32(32.0) | NFE Backward 198(190.2)
Iter 0927 | Time 2.6069(2.6414) | Loss 4.558831(4.595113) | NFE Forward 32(32.0) | NFE Backward 186(189.9)
Iter 0928 | Time 2.7262(2.6473) | Loss 4.574031(4.593637) | NFE Forward 32(32.0) | NFE Backward 198(190.4)
Iter 0929 | Time 2.7257(2.6528) | Loss 4.594827(4.593721) | NFE Forward 32(32.0) | NFE Backward 198(191.0)
Iter 0930 | Time 2.6056(2.6495) | Loss 4.592668(4.593647) | NFE Forward 32(32.0) | NFE Backward 186(190.6)
Iter 0931 | Time 2.7235(2.6547) | Loss 4.561351(4.591386) | NFE Forward 32(32.0) | NFE Backward 198(191.1)
Iter 0932 | Time 2.6755(2.6562) | Loss 4.629200(4.594033) | NFE Forward 32(32.0) | NFE Backward 192(191.2)
Iter 0933 | Time 2.7280(2.6612) | Loss 4.606547(4.594909) | NFE Forward 32(32.0) | NFE Backward 198(191.7)
Iter 0934 | Time 2.7230(2.6655) | Loss 4.599806(4.595252) | NFE Forward 32(32.0) | NFE Backward 198(192.1)
Iter 0935 | Time 2.7799(2.6735) | Loss 4.581087(4.594260) | NFE Forward 32(32.0) | NFE Backward 204(192.9)
Iter 0936 | Time 2.7368(2.6780) | Loss 4.622571(4.596242) | NFE Forward 32(32.0) | NFE Backward 198(193.3)
Iter 0937 | Time 2.7372(2.6821) | Loss 4.553566(4.593255) | NFE Forward 32(32.0) | NFE Backward 198(193.6)
Iter 0938 | Time 2.6283(2.6783) | Loss 4.609794(4.594413) | NFE Forward 32(32.0) | NFE Backward 186(193.1)
Iter 0939 | Time 2.6071(2.6733) | Loss 4.568819(4.592621) | NFE Forward 32(32.0) | NFE Backward 186(192.6)
Iter 0940 | Time 2.7282(2.6772) | Loss 4.550532(4.589675) | NFE Forward 32(32.0) | NFE Backward 198(193.0)
Iter 0941 | Time 2.7257(2.6806) | Loss 4.564356(4.587902) | NFE Forward 32(32.0) | NFE Backward 198(193.3)
Iter 0942 | Time 2.7261(2.6838) | Loss 4.580209(4.587364) | NFE Forward 32(32.0) | NFE Backward 198(193.7)
Iter 0943 | Time 2.7238(2.6866) | Loss 4.601198(4.588332) | NFE Forward 32(32.0) | NFE Backward 198(194.0)
Iter 0944 | Time 2.7263(2.6893) | Loss 4.620930(4.590614) | NFE Forward 32(32.0) | NFE Backward 198(194.2)
Iter 0945 | Time 2.7266(2.6920) | Loss 4.594364(4.590877) | NFE Forward 32(32.0) | NFE Backward 198(194.5)
Iter 0946 | Time 2.7213(2.6940) | Loss 4.562251(4.588873) | NFE Forward 32(32.0) | NFE Backward 198(194.7)
Iter 0947 | Time 2.7246(2.6961) | Loss 4.588236(4.588828) | NFE Forward 32(32.0) | NFE Backward 198(195.0)
Iter 0948 | Time 2.7260(2.6982) | Loss 4.617544(4.590838) | NFE Forward 32(32.0) | NFE Backward 198(195.2)
Iter 0949 | Time 2.7283(2.7003) | Loss 4.602602(4.591662) | NFE Forward 32(32.0) | NFE Backward 198(195.4)
Iter 0950 | Time 2.7237(2.7020) | Loss 4.558817(4.589363) | NFE Forward 32(32.0) | NFE Backward 198(195.6)
Iter 0951 | Time 2.7313(2.7040) | Loss 4.581663(4.588824) | NFE Forward 32(32.0) | NFE Backward 198(195.7)
Iter 0952 | Time 2.7248(2.7055) | Loss 4.540500(4.585441) | NFE Forward 32(32.0) | NFE Backward 198(195.9)
Iter 0953 | Time 2.7270(2.7070) | Loss 4.570562(4.584400) | NFE Forward 32(32.0) | NFE Backward 198(196.0)
Iter 0954 | Time 2.7269(2.7084) | Loss 4.550517(4.582028) | NFE Forward 32(32.0) | NFE Backward 198(196.2)
Iter 0955 | Time 2.7371(2.7104) | Loss 4.587893(4.582438) | NFE Forward 32(32.0) | NFE Backward 198(196.3)
Iter 0956 | Time 2.7255(2.7115) | Loss 4.598687(4.583576) | NFE Forward 32(32.0) | NFE Backward 198(196.4)
Iter 0957 | Time 2.7238(2.7123) | Loss 4.561402(4.582024) | NFE Forward 32(32.0) | NFE Backward 198(196.5)
Iter 0958 | Time 2.7262(2.7133) | Loss 4.585487(4.582266) | NFE Forward 32(32.0) | NFE Backward 198(196.6)
Iter 0959 | Time 2.7251(2.7141) | Loss 4.573889(4.581680) | NFE Forward 32(32.0) | NFE Backward 198(196.7)
Iter 0960 | Time 2.7810(2.7188) | Loss 4.568540(4.580760) | NFE Forward 32(32.0) | NFE Backward 204(197.2)
Iter 0961 | Time 2.7259(2.7193) | Loss 4.551837(4.578735) | NFE Forward 32(32.0) | NFE Backward 198(197.3)
Iter 0962 | Time 2.7845(2.7239) | Loss 4.565935(4.577839) | NFE Forward 32(32.0) | NFE Backward 204(197.8)
Iter 0963 | Time 2.7262(2.7240) | Loss 4.602060(4.579535) | NFE Forward 32(32.0) | NFE Backward 198(197.8)
Iter 0964 | Time 2.7915(2.7287) | Loss 4.522926(4.575572) | NFE Forward 32(32.0) | NFE Backward 204(198.2)
Iter 0965 | Time 2.7482(2.7301) | Loss 4.509095(4.570919) | NFE Forward 32(32.0) | NFE Backward 198(198.2)
Iter 0966 | Time 2.7300(2.7301) | Loss 4.595165(4.572616) | NFE Forward 32(32.0) | NFE Backward 198(198.2)
Iter 0967 | Time 2.7254(2.7298) | Loss 4.587974(4.573691) | NFE Forward 32(32.0) | NFE Backward 198(198.2)
Iter 0968 | Time 2.7243(2.7294) | Loss 4.572640(4.573617) | NFE Forward 32(32.0) | NFE Backward 198(198.2)
Iter 0969 | Time 2.7287(2.7293) | Loss 4.527419(4.570384) | NFE Forward 32(32.0) | NFE Backward 198(198.2)
Iter 0970 | Time 2.7256(2.7291) | Loss 4.601124(4.572535) | NFE Forward 32(32.0) | NFE Backward 198(198.1)
Iter 0971 | Time 2.7298(2.7291) | Loss 4.563646(4.571913) | NFE Forward 32(32.0) | NFE Backward 198(198.1)
Iter 0972 | Time 2.7830(2.7329) | Loss 4.543092(4.569896) | NFE Forward 32(32.0) | NFE Backward 204(198.5)
Iter 0973 | Time 2.7882(2.7368) | Loss 4.569229(4.569849) | NFE Forward 32(32.0) | NFE Backward 204(198.9)
Iter 0974 | Time 2.7275(2.7361) | Loss 4.593182(4.571482) | NFE Forward 32(32.0) | NFE Backward 198(198.9)
Iter 0975 | Time 2.7290(2.7356) | Loss 4.524783(4.568213) | NFE Forward 32(32.0) | NFE Backward 198(198.8)
Iter 0976 | Time 2.7427(2.7361) | Loss 4.569974(4.568337) | NFE Forward 32(32.0) | NFE Backward 198(198.7)
Iter 0977 | Time 2.7440(2.7367) | Loss 4.549925(4.567048) | NFE Forward 32(32.0) | NFE Backward 198(198.7)
Iter 0978 | Time 2.7888(2.7403) | Loss 4.606359(4.569800) | NFE Forward 32(32.0) | NFE Backward 204(199.1)
Iter 0979 | Time 2.7324(2.7398) | Loss 4.591538(4.571321) | NFE Forward 32(32.0) | NFE Backward 198(199.0)
Iter 0980 | Time 2.7874(2.7431) | Loss 4.625446(4.575110) | NFE Forward 32(32.0) | NFE Backward 204(199.3)
Iter 0981 | Time 2.7857(2.7461) | Loss 4.596976(4.576641) | NFE Forward 32(32.0) | NFE Backward 204(199.7)
Iter 0982 | Time 2.7283(2.7448) | Loss 4.540820(4.574133) | NFE Forward 32(32.0) | NFE Backward 198(199.5)
Iter 0983 | Time 2.7309(2.7439) | Loss 4.563597(4.573396) | NFE Forward 32(32.0) | NFE Backward 198(199.4)
Iter 0984 | Time 2.7344(2.7432) | Loss 4.542668(4.571245) | NFE Forward 32(32.0) | NFE Backward 198(199.3)
Iter 0985 | Time 2.7287(2.7422) | Loss 4.510025(4.566959) | NFE Forward 32(32.0) | NFE Backward 198(199.2)
Iter 0986 | Time 2.7886(2.7454) | Loss 4.513718(4.563232) | NFE Forward 32(32.0) | NFE Backward 204(199.6)
Iter 0987 | Time 2.7265(2.7441) | Loss 4.528303(4.560787) | NFE Forward 32(32.0) | NFE Backward 198(199.5)
Iter 0988 | Time 2.7866(2.7471) | Loss 4.571104(4.561510) | NFE Forward 32(32.0) | NFE Backward 204(199.8)
Iter 0989 | Time 2.7255(2.7456) | Loss 4.572364(4.562269) | NFE Forward 32(32.0) | NFE Backward 198(199.7)
Iter 0990 | Time 2.7417(2.7453) | Loss 4.582394(4.563678) | NFE Forward 32(32.0) | NFE Backward 198(199.5)
Iter 0991 | Time 2.8024(2.7493) | Loss 4.544945(4.562367) | NFE Forward 32(32.0) | NFE Backward 204(199.9)
Iter 0992 | Time 2.7479(2.7492) | Loss 4.591417(4.564400) | NFE Forward 32(32.0) | NFE Backward 198(199.7)
Iter 0993 | Time 2.7433(2.7488) | Loss 4.524448(4.561604) | NFE Forward 32(32.0) | NFE Backward 198(199.6)
Iter 0994 | Time 2.7418(2.7483) | Loss 4.565515(4.561877) | NFE Forward 32(32.0) | NFE Backward 198(199.5)
Iter 0995 | Time 2.8087(2.7525) | Loss 4.561641(4.561861) | NFE Forward 32(32.0) | NFE Backward 204(199.8)
Iter 0996 | Time 2.8009(2.7559) | Loss 4.561960(4.561868) | NFE Forward 32(32.0) | NFE Backward 204(200.1)
Iter 0997 | Time 2.8014(2.7591) | Loss 4.572731(4.562628) | NFE Forward 32(32.0) | NFE Backward 204(200.4)
Density Loss 0.41956639289855957
Density Loss 0.400348424911499
Density Loss 0.41030028462409973
Density Loss 0.39142152667045593
Density Loss 0.41392529010772705
Density Loss 0.4176879823207855
Density Loss 0.41037556529045105
Density Loss 0.4031701982021332
Density Loss 0.41450798511505127
Density Loss 0.41370370984077454
Density Loss 0.40607839822769165
Density Loss 0.40714511275291443
Density Loss 0.41438817977905273
Density Loss 0.39539968967437744
Density Loss 0.42885345220565796
Density Loss 0.40864717960357666
Density Loss 0.40404441952705383
Density Loss 0.41303691267967224
Density Loss 0.4206524193286896
Density Loss 0.4140646457672119
Density Loss 0.4211447536945343
Density Loss 0.4110080897808075
Density Loss 0.4072974622249603
Density Loss 0.42506423592567444
Density Loss 0.40886202454566956
Density Loss 0.4167785048484802
Density Loss 0.41967079043388367
Density Loss 0.4191971719264984
Density Loss 0.4051292836666107
Density Loss 0.39866432547569275
Density Loss 0.41316482424736023
Density Loss 0.41697943210601807
Density Loss 0.404948890209198
Density Loss 0.40889498591423035
Density Loss 0.4222969710826874
Density Loss 0.4091481864452362
Density Loss 0.41323134303092957
Density Loss 0.4139295220375061
Density Loss 0.4259176552295685
Density Loss 0.42233026027679443
Density Loss 0.4097784161567688
Density Loss 0.4337080419063568
Density Loss 0.4222075045108795
Density Loss 0.40391284227371216
Density Loss 0.41103944182395935
Density Loss 0.4220496118068695
Density Loss 0.42235901951789856
Density Loss 0.4203159213066101
Density Loss 0.40522268414497375
Density Loss 0.4184694290161133
Density Loss 0.405798077583313
Density Loss 0.42142337560653687
Density Loss 0.4108923375606537
Density Loss 0.41580936312675476
Density Loss 0.41934365034103394
Density Loss 0.42170071601867676
Density Loss 0.41695961356163025
Density Loss 0.4065488278865814
Density Loss 0.4364114999771118
Density Loss 0.42080605030059814
Density Loss 0.419898122549057
Density Loss 0.42231059074401855
Density Loss 0.4203777313232422
Density Loss 0.4191795885562897
Density Loss 0.4216364622116089
Density Loss 0.40784603357315063
Density Loss 0.42753446102142334
Density Loss 0.42155417799949646
Density Loss 0.4174886643886566
Density Loss 0.40898269414901733
Density Loss 0.4255756139755249
Density Loss 0.4315449595451355
Density Loss 0.4293544888496399
Density Loss 0.42632240056991577
Density Loss 0.40815046429634094
Density Loss 0.4118505120277405
Density Loss 0.43051961064338684
Density Loss 0.42665809392929077
Density Loss 0.43049919605255127
Density Loss 0.41998007893562317
Density Loss 0.42547473311424255
Density Loss 0.43292152881622314
Density Loss 0.4195672273635864
Density Loss 0.4153519570827484
Density Loss 0.41658318042755127
Density Loss 0.41977497935295105
Density Loss 0.42669007182121277
Density Loss 0.42507895827293396
Density Loss 0.4318493902683258
Density Loss 0.4241548776626587
Density Loss 0.42210838198661804
Density Loss 0.4257117211818695
Density Loss 0.4291073679924011
Density Loss 0.42262473702430725
Density Loss 0.4299566149711609
Density Loss 0.41712841391563416
Density Loss 0.4205993115901947
Density Loss 0.4312349557876587
Density Loss 0.4434499740600586
Density Loss 0.4279860258102417
Density Loss 0.4351498782634735
Density Loss 0.41797885298728943
Density Loss 0.42689648270606995
Density Loss 0.441977322101593
Density Loss 0.42824530601501465
Density Loss 0.4273308515548706
Density Loss 0.4254894554615021
Density Loss 0.4421374797821045
Density Loss 0.4168483316898346
Density Loss 0.43339085578918457
Density Loss 0.44268855452537537
Density Loss 0.43354833126068115
Density Loss 0.42810505628585815
Density Loss 0.4283676743507385
Density Loss 0.4326910972595215
Density Loss 0.4270854890346527
Density Loss 0.43844226002693176
Density Loss 0.42532458901405334
Density Loss 0.4211476445198059
Density Loss 0.4286503791809082
Density Loss 0.4146074056625366
Density Loss 0.42286059260368347
Density Loss 0.4364018440246582
Density Loss 0.4324667751789093
Density Loss 0.4239119589328766
Density Loss 0.43678122758865356
Density Loss 0.42705219984054565
Density Loss 0.421652615070343
Density Loss 0.4193687438964844
Density Loss 0.42037007212638855
Density Loss 0.42323675751686096
Density Loss 0.42766883969306946
Density Loss 0.42798933386802673
Density Loss 0.43695423007011414
Density Loss 0.4475032091140747
Density Loss 0.43505075573921204
Density Loss 0.4345524311065674
Density Loss 0.42121511697769165
Density Loss 0.4323328137397766
Density Loss 0.42097577452659607
Density Loss 0.43410995602607727
Density Loss 0.4317871332168579
Density Loss 0.42632338404655457
Density Loss 0.4356110692024231
Density Loss 0.4309076964855194
Density Loss 0.43012815713882446
Density Loss 0.4286089241504669
Density Loss 0.4273931384086609
Density Loss 0.4426097571849823
Density Loss 0.44073328375816345
Density Loss 0.4410654604434967
Density Loss 0.42759785056114197
Density Loss 0.4472585916519165
Density Loss 0.41969504952430725
Density Loss 0.4329858720302582
Density Loss 0.43348410725593567
Density Loss 0.43029090762138367
Density Loss 0.4376709461212158
Density Loss 0.43164560198783875
Density Loss 0.4421582520008087
Density Loss 0.43101438879966736
Density Loss 0.4340737760066986
Density Loss 0.43591785430908203
Density Loss 0.43591395020484924
Density Loss 0.41946449875831604
Density Loss 0.44309574365615845
Density Loss 0.4410938322544098
Density Loss 0.4378035068511963
Density Loss 0.43501025438308716
Density Loss 0.441244512796402
Density Loss 0.4248485267162323
Density Loss 0.43800750374794006
Density Loss 0.4332834482192993
Density Loss 0.44245022535324097
Density Loss 0.4358176290988922
Density Loss 0.4416983425617218
Density Loss 0.43425917625427246
Density Loss 0.4409603774547577
Density Loss 0.4444236755371094
Density Loss 0.4444008767604828
Density Loss 0.4414699971675873
Density Loss 0.43203645944595337
Density Loss 0.44200581312179565
Density Loss 0.4456728994846344
Density Loss 0.43163496255874634
Density Loss 0.43812161684036255
Density Loss 0.44432634115219116
Density Loss 0.44473591446876526
Density Loss 0.44555047154426575
Density Loss 0.4384961724281311
Density Loss 0.4526212215423584
Density Loss 0.4364148676395416
Density Loss 0.4435241222381592
Density Loss 0.437578409910202
Density Loss 0.4231720566749573
Density Loss 0.45250019431114197
Density Loss 0.43990159034729004
Density Loss 0.4265674352645874
Density Loss 0.4410835802555084
Density Loss 0.4441390931606293
Density Loss 0.43451014161109924
Density Loss 0.4451174736022949
Density Loss 0.4502880275249481
Density Loss 0.4428701102733612
Density Loss 0.43126624822616577
Density Loss 0.43725359439849854
Density Loss 0.4313686490058899
Density Loss 0.42290887236595154
Density Loss 0.4492443799972534
Density Loss 0.4325845539569855
Density Loss 0.42008379101753235
Density Loss 0.4407268464565277
Density Loss 0.43706029653549194
Density Loss 0.45039159059524536
Density Loss 0.44708874821662903
Density Loss 0.4393995404243469
Density Loss 0.43571677803993225
Density Loss 0.4358876347541809
Density Loss 0.4401392936706543
Density Loss 0.4484124779701233
Density Loss 0.45450302958488464
Density Loss 0.4455207288265228
Density Loss 0.45966365933418274
Density Loss 0.4460254907608032
Density Loss 0.4491479992866516
Density Loss 0.4413047730922699
Density Loss 0.4409564733505249
Density Loss 0.4411602318286896
Density Loss 0.45452094078063965
Density Loss 0.4551241993904114
Density Loss 0.43189096450805664
Density Loss 0.44770145416259766
Density Loss 0.4266737699508667
Density Loss 0.44610658288002014
Density Loss 0.45802196860313416
Density Loss 0.44263777136802673
Density Loss 0.4540007710456848
Density Loss 0.44975730776786804
Density Loss 0.4472540020942688
Density Loss 0.4624861180782318
Density Loss 0.4472923278808594
Density Loss 0.46404892206192017
Density Loss 0.4498467743396759
Density Loss 0.44867321848869324
Density Loss 0.45152026414871216
Density Loss 0.4462090730667114
Density Loss 0.4408535063266754
Density Loss 0.44302502274513245
Density Loss 0.4461447596549988
Density Loss 0.4370726943016052
Density Loss 0.4332694709300995
Density Loss 0.4535726308822632
Iter 0998 | Time 2.8036(2.7622) | Loss 4.594349(4.564849) | NFE Forward 32(32.0) | NFE Backward 204(200.6)
Iter 0999 | Time 2.7809(2.7635) | Loss 4.537234(4.562916) | NFE Forward 32(32.0) | NFE Backward 204(200.9)
Iter 1000 | Time 2.7239(2.7608) | Loss 4.572667(4.563598) | NFE Forward 32(32.0) | NFE Backward 198(200.7)
[TEST] Iter 1000 | Test Loss 4.565606 | NFE 32
Skipping vis as data dimension is >2
Iter 1001 | Time 2.7770(2.7619) | Loss 4.514576(4.560167) | NFE Forward 32(32.0) | NFE Backward 204(200.9)
Iter 1002 | Time 2.7782(2.7630) | Loss 4.575545(4.561243) | NFE Forward 32(32.0) | NFE Backward 204(201.1)
Iter 1003 | Time 2.7133(2.7595) | Loss 4.597102(4.563753) | NFE Forward 32(32.0) | NFE Backward 198(200.9)
Iter 1004 | Time 2.7686(2.7602) | Loss 4.563985(4.563770) | NFE Forward 32(32.0) | NFE Backward 204(201.1)
Iter 1005 | Time 2.7735(2.7611) | Loss 4.587363(4.565421) | NFE Forward 32(32.0) | NFE Backward 204(201.3)
Iter 1006 | Time 2.7718(2.7619) | Loss 4.565230(4.565408) | NFE Forward 32(32.0) | NFE Backward 204(201.5)
Iter 1007 | Time 2.7727(2.7626) | Loss 4.554226(4.564625) | NFE Forward 32(32.0) | NFE Backward 204(201.7)
Iter 1008 | Time 2.7779(2.7637) | Loss 4.521390(4.561599) | NFE Forward 32(32.0) | NFE Backward 204(201.8)
Iter 1009 | Time 2.7159(2.7604) | Loss 4.517414(4.558506) | NFE Forward 32(32.0) | NFE Backward 198(201.6)
Iter 1010 | Time 2.7152(2.7572) | Loss 4.545860(4.557621) | NFE Forward 32(32.0) | NFE Backward 198(201.3)
Iter 1011 | Time 2.7137(2.7542) | Loss 4.591354(4.559982) | NFE Forward 32(32.0) | NFE Backward 198(201.1)
Iter 1012 | Time 2.7716(2.7554) | Loss 4.540809(4.558640) | NFE Forward 32(32.0) | NFE Backward 204(201.3)
Iter 1013 | Time 2.7746(2.7567) | Loss 4.605666(4.561932) | NFE Forward 32(32.0) | NFE Backward 204(201.5)
Iter 1014 | Time 2.7757(2.7580) | Loss 4.557418(4.561616) | NFE Forward 32(32.0) | NFE Backward 204(201.7)
Iter 1015 | Time 2.7699(2.7589) | Loss 4.564585(4.561823) | NFE Forward 32(32.0) | NFE Backward 204(201.8)
Iter 1016 | Time 2.7227(2.7563) | Loss 4.546104(4.560723) | NFE Forward 32(32.0) | NFE Backward 198(201.6)
Iter 1017 | Time 2.7746(2.7576) | Loss 4.562832(4.560871) | NFE Forward 32(32.0) | NFE Backward 204(201.7)
Iter 1018 | Time 2.7737(2.7587) | Loss 4.507828(4.557158) | NFE Forward 32(32.0) | NFE Backward 204(201.9)
Iter 1019 | Time 2.7202(2.7560) | Loss 4.556908(4.557140) | NFE Forward 32(32.0) | NFE Backward 198(201.6)
Iter 1020 | Time 2.7287(2.7541) | Loss 4.570930(4.558106) | NFE Forward 32(32.0) | NFE Backward 198(201.4)
Iter 1021 | Time 2.7728(2.7554) | Loss 4.546783(4.557313) | NFE Forward 32(32.0) | NFE Backward 204(201.5)
Iter 1022 | Time 2.7795(2.7571) | Loss 4.589585(4.559572) | NFE Forward 32(32.0) | NFE Backward 204(201.7)
Iter 1023 | Time 2.7807(2.7588) | Loss 4.522407(4.556970) | NFE Forward 32(32.0) | NFE Backward 204(201.9)
Iter 1024 | Time 2.7239(2.7563) | Loss 4.570228(4.557898) | NFE Forward 32(32.0) | NFE Backward 198(201.6)
Iter 1025 | Time 2.7229(2.7540) | Loss 4.568040(4.558608) | NFE Forward 32(32.0) | NFE Backward 198(201.4)
Iter 1026 | Time 2.7786(2.7557) | Loss 4.543559(4.557555) | NFE Forward 32(32.0) | NFE Backward 204(201.5)
Iter 1027 | Time 2.7801(2.7574) | Loss 4.525725(4.555327) | NFE Forward 32(32.0) | NFE Backward 204(201.7)
Iter 1028 | Time 2.7302(2.7555) | Loss 4.576235(4.556790) | NFE Forward 32(32.0) | NFE Backward 198(201.5)
Iter 1029 | Time 2.7863(2.7577) | Loss 4.581320(4.558508) | NFE Forward 32(32.0) | NFE Backward 204(201.6)
Iter 1030 | Time 2.7839(2.7595) | Loss 4.589354(4.560667) | NFE Forward 32(32.0) | NFE Backward 204(201.8)
Iter 1031 | Time 2.7730(2.7605) | Loss 4.589346(4.562674) | NFE Forward 32(32.0) | NFE Backward 204(201.9)
Iter 1032 | Time 2.7705(2.7612) | Loss 4.553514(4.562033) | NFE Forward 32(32.0) | NFE Backward 204(202.1)
Iter 1033 | Time 2.7729(2.7620) | Loss 4.494319(4.557293) | NFE Forward 32(32.0) | NFE Backward 204(202.2)
Iter 1034 | Time 2.7769(2.7630) | Loss 4.555759(4.557186) | NFE Forward 32(32.0) | NFE Backward 204(202.4)
Iter 1035 | Time 2.7663(2.7633) | Loss 4.556180(4.557115) | NFE Forward 32(32.0) | NFE Backward 204(202.5)
Iter 1036 | Time 2.7109(2.7596) | Loss 4.522163(4.554669) | NFE Forward 32(32.0) | NFE Backward 198(202.2)
Iter 1037 | Time 2.7645(2.7599) | Loss 4.522335(4.552405) | NFE Forward 32(32.0) | NFE Backward 204(202.3)
Iter 1038 | Time 2.7658(2.7603) | Loss 4.543546(4.551785) | NFE Forward 32(32.0) | NFE Backward 204(202.4)
Iter 1039 | Time 2.7637(2.7606) | Loss 4.509645(4.548835) | NFE Forward 32(32.0) | NFE Backward 204(202.5)
Iter 1040 | Time 2.7634(2.7608) | Loss 4.567635(4.550151) | NFE Forward 32(32.0) | NFE Backward 204(202.6)
Iter 1041 | Time 2.7104(2.7572) | Loss 4.542739(4.549632) | NFE Forward 32(32.0) | NFE Backward 198(202.3)
Iter 1042 | Time 2.7624(2.7576) | Loss 4.544460(4.549270) | NFE Forward 32(32.0) | NFE Backward 204(202.4)
Iter 1043 | Time 2.7602(2.7578) | Loss 4.512158(4.546673) | NFE Forward 32(32.0) | NFE Backward 204(202.5)
Iter 1044 | Time 2.7624(2.7581) | Loss 4.512099(4.544252) | NFE Forward 32(32.0) | NFE Backward 204(202.6)
Iter 1045 | Time 2.7601(2.7583) | Loss 4.536722(4.543725) | NFE Forward 32(32.0) | NFE Backward 204(202.7)
Iter 1046 | Time 2.7596(2.7583) | Loss 4.527181(4.542567) | NFE Forward 32(32.0) | NFE Backward 204(202.8)
Iter 1047 | Time 2.7583(2.7583) | Loss 4.559706(4.543767) | NFE Forward 32(32.0) | NFE Backward 204(202.9)
Iter 1048 | Time 2.7896(2.7605) | Loss 4.632055(4.549947) | NFE Forward 38(32.4) | NFE Backward 204(203.0)
Iter 1049 | Time 2.7045(2.7566) | Loss 4.488336(4.545634) | NFE Forward 32(32.4) | NFE Backward 198(202.6)
Iter 1050 | Time 2.7618(2.7570) | Loss 4.552024(4.546082) | NFE Forward 32(32.4) | NFE Backward 204(202.7)
Iter 1051 | Time 2.7689(2.7578) | Loss 4.570754(4.547809) | NFE Forward 32(32.3) | NFE Backward 204(202.8)
Iter 1052 | Time 2.7708(2.7587) | Loss 4.553167(4.548184) | NFE Forward 32(32.3) | NFE Backward 204(202.9)
Iter 1053 | Time 2.7088(2.7552) | Loss 4.508487(4.545405) | NFE Forward 32(32.3) | NFE Backward 198(202.6)
Iter 1054 | Time 2.7611(2.7556) | Loss 4.557537(4.546254) | NFE Forward 32(32.3) | NFE Backward 204(202.7)
Iter 1055 | Time 2.8479(2.7621) | Loss 4.510732(4.543768) | NFE Forward 38(32.7) | NFE Backward 210(203.2)
Iter 1056 | Time 2.7670(2.7624) | Loss 4.536531(4.543261) | NFE Forward 32(32.6) | NFE Backward 204(203.2)
Iter 1057 | Time 2.7619(2.7624) | Loss 4.555174(4.544095) | NFE Forward 32(32.6) | NFE Backward 204(203.3)
Iter 1058 | Time 2.7878(2.7642) | Loss 4.553329(4.544741) | NFE Forward 38(33.0) | NFE Backward 204(203.3)
Iter 1059 | Time 2.7595(2.7639) | Loss 4.518323(4.542892) | NFE Forward 32(32.9) | NFE Backward 204(203.4)
Iter 1060 | Time 2.7620(2.7637) | Loss 4.523531(4.541537) | NFE Forward 32(32.8) | NFE Backward 204(203.4)
Iter 1061 | Time 2.7868(2.7653) | Loss 4.522912(4.540233) | NFE Forward 38(33.2) | NFE Backward 204(203.5)
Iter 1062 | Time 2.7078(2.7613) | Loss 4.489423(4.536676) | NFE Forward 32(33.1) | NFE Backward 198(203.1)
Iter 1063 | Time 2.7733(2.7621) | Loss 4.543707(4.537169) | NFE Forward 32(33.0) | NFE Backward 204(203.1)
Iter 1064 | Time 2.7595(2.7620) | Loss 4.538571(4.537267) | NFE Forward 32(33.0) | NFE Backward 204(203.2)
Iter 1065 | Time 2.7625(2.7620) | Loss 4.578029(4.540120) | NFE Forward 32(32.9) | NFE Backward 204(203.3)
Iter 1066 | Time 2.8174(2.7659) | Loss 4.496967(4.537099) | NFE Forward 32(32.8) | NFE Backward 210(203.7)
Iter 1067 | Time 2.7026(2.7614) | Loss 4.573649(4.539658) | NFE Forward 32(32.8) | NFE Backward 198(203.3)
Iter 1068 | Time 2.8508(2.7677) | Loss 4.568003(4.541642) | NFE Forward 38(33.1) | NFE Backward 210(203.8)
Iter 1069 | Time 2.8541(2.7737) | Loss 4.511767(4.539551) | NFE Forward 38(33.5) | NFE Backward 210(204.2)
Iter 1070 | Time 2.7347(2.7710) | Loss 4.551649(4.540398) | NFE Forward 38(33.8) | NFE Backward 198(203.8)
Iter 1071 | Time 2.7643(2.7705) | Loss 4.467531(4.535297) | NFE Forward 32(33.7) | NFE Backward 204(203.8)
Iter 1072 | Time 2.7033(2.7658) | Loss 4.534340(4.535230) | NFE Forward 32(33.6) | NFE Backward 198(203.4)
Iter 1073 | Time 2.7649(2.7658) | Loss 4.520100(4.534171) | NFE Forward 32(33.4) | NFE Backward 204(203.4)
Iter 1074 | Time 2.8242(2.7699) | Loss 4.522561(4.533358) | NFE Forward 32(33.3) | NFE Backward 210(203.9)
Iter 1075 | Time 2.7620(2.7693) | Loss 4.507596(4.531555) | NFE Forward 32(33.2) | NFE Backward 204(203.9)
Iter 1076 | Time 2.7617(2.7688) | Loss 4.508928(4.529971) | NFE Forward 32(33.2) | NFE Backward 204(203.9)
Iter 1077 | Time 2.8212(2.7724) | Loss 4.499390(4.527830) | NFE Forward 32(33.1) | NFE Backward 210(204.3)
Iter 1078 | Time 2.8517(2.7780) | Loss 4.552518(4.529558) | NFE Forward 38(33.4) | NFE Backward 210(204.7)
Iter 1079 | Time 2.8505(2.7831) | Loss 4.549869(4.530980) | NFE Forward 38(33.7) | NFE Backward 210(205.1)
Iter 1080 | Time 2.7648(2.7818) | Loss 4.508342(4.529396) | NFE Forward 32(33.6) | NFE Backward 204(205.0)
Iter 1081 | Time 2.7332(2.7784) | Loss 4.563317(4.531770) | NFE Forward 38(33.9) | NFE Backward 198(204.5)
Iter 1082 | Time 2.7884(2.7791) | Loss 4.508441(4.530137) | NFE Forward 38(34.2) | NFE Backward 204(204.5)
Iter 1083 | Time 2.8216(2.7821) | Loss 4.526504(4.529883) | NFE Forward 32(34.1) | NFE Backward 210(204.9)
Iter 1084 | Time 2.8503(2.7868) | Loss 4.542248(4.530748) | NFE Forward 38(34.3) | NFE Backward 210(205.2)
Iter 1085 | Time 2.8875(2.7939) | Loss 4.542603(4.531578) | NFE Forward 38(34.6) | NFE Backward 210(205.6)
Iter 1086 | Time 2.8502(2.7978) | Loss 4.501306(4.529459) | NFE Forward 38(34.8) | NFE Backward 210(205.9)
Iter 1087 | Time 2.7619(2.7953) | Loss 4.466123(4.525025) | NFE Forward 32(34.6) | NFE Backward 204(205.8)
Iter 1088 | Time 2.8511(2.7992) | Loss 4.490609(4.522616) | NFE Forward 38(34.9) | NFE Backward 210(206.1)
Iter 1089 | Time 2.8651(2.8038) | Loss 4.528549(4.523032) | NFE Forward 38(35.1) | NFE Backward 210(206.3)
Iter 1090 | Time 2.8520(2.8072) | Loss 4.532950(4.523726) | NFE Forward 38(35.3) | NFE Backward 210(206.6)
Iter 1091 | Time 2.8153(2.8078) | Loss 4.505532(4.522452) | NFE Forward 38(35.5) | NFE Backward 204(206.4)
Iter 1092 | Time 2.8228(2.8088) | Loss 4.530762(4.523034) | NFE Forward 32(35.2) | NFE Backward 210(206.7)
Iter 1093 | Time 2.8481(2.8116) | Loss 4.487484(4.520546) | NFE Forward 38(35.4) | NFE Backward 210(206.9)
Iter 1094 | Time 2.7662(2.8084) | Loss 4.570267(4.524026) | NFE Forward 32(35.2) | NFE Backward 204(206.7)
Iter 1095 | Time 2.8767(2.8132) | Loss 4.550589(4.525885) | NFE Forward 38(35.4) | NFE Backward 210(206.9)
Iter 1096 | Time 2.8542(2.8161) | Loss 4.519147(4.525414) | NFE Forward 32(35.1) | NFE Backward 210(207.1)
Iter 1097 | Time 2.8495(2.8184) | Loss 4.548004(4.526995) | NFE Forward 38(35.3) | NFE Backward 210(207.3)
Iter 1098 | Time 2.7612(2.8144) | Loss 4.523193(4.526729) | NFE Forward 32(35.1) | NFE Backward 204(207.1)
Iter 1099 | Time 2.8518(2.8170) | Loss 4.523011(4.526469) | NFE Forward 38(35.3) | NFE Backward 210(207.3)
Iter 1100 | Time 2.7584(2.8129) | Loss 4.524315(4.526318) | NFE Forward 38(35.5) | NFE Backward 198(206.7)
[TEST] Iter 1100 | Test Loss 4.528575 | NFE 38
Skipping vis as data dimension is >2
Iter 1101 | Time 2.8754(2.8173) | Loss 4.538145(4.527146) | NFE Forward 38(35.7) | NFE Backward 210(206.9)
Iter 1102 | Time 2.8797(2.8216) | Loss 4.541679(4.528163) | NFE Forward 38(35.8) | NFE Backward 210(207.1)
Iter 1103 | Time 2.8015(2.8202) | Loss 4.524684(4.527920) | NFE Forward 32(35.6) | NFE Backward 204(206.9)
Iter 1104 | Time 2.9168(2.8270) | Loss 4.525517(4.527751) | NFE Forward 38(35.7) | NFE Backward 210(207.1)
Iter 1105 | Time 2.9817(2.8378) | Loss 4.515893(4.526921) | NFE Forward 38(35.9) | NFE Backward 216(207.7)
Iter 1106 | Time 2.9827(2.8480) | Loss 4.515845(4.526146) | NFE Forward 38(36.0) | NFE Backward 216(208.3)
Iter 1107 | Time 2.8639(2.8491) | Loss 4.534083(4.526702) | NFE Forward 32(35.8) | NFE Backward 210(208.4)
Iter 1108 | Time 2.8674(2.8504) | Loss 4.569757(4.529715) | NFE Forward 32(35.5) | NFE Backward 210(208.5)
Iter 1109 | Time 2.8939(2.8534) | Loss 4.519157(4.528976) | NFE Forward 38(35.7) | NFE Backward 210(208.6)
Iter 1110 | Time 2.9758(2.8620) | Loss 4.548557(4.530347) | NFE Forward 38(35.8) | NFE Backward 216(209.2)
Iter 1111 | Time 2.9209(2.8661) | Loss 4.500707(4.528272) | NFE Forward 38(36.0) | NFE Backward 210(209.2)
Iter 1112 | Time 2.9204(2.8699) | Loss 4.520455(4.527725) | NFE Forward 38(36.1) | NFE Backward 210(209.3)
Iter 1113 | Time 2.8908(2.8714) | Loss 4.523555(4.527433) | NFE Forward 38(36.3) | NFE Backward 210(209.3)
Iter 1114 | Time 2.9174(2.8746) | Loss 4.451326(4.522106) | NFE Forward 38(36.4) | NFE Backward 210(209.4)
Iter 1115 | Time 2.8921(2.8758) | Loss 4.542595(4.523540) | NFE Forward 32(36.1) | NFE Backward 210(209.4)
Iter 1116 | Time 2.9228(2.8791) | Loss 4.539339(4.524646) | NFE Forward 38(36.2) | NFE Backward 210(209.5)
Iter 1117 | Time 2.9201(2.8820) | Loss 4.523835(4.524589) | NFE Forward 38(36.3) | NFE Backward 210(209.5)
Iter 1118 | Time 2.9163(2.8844) | Loss 4.522086(4.524414) | NFE Forward 38(36.5) | NFE Backward 210(209.5)
Iter 1119 | Time 2.9182(2.8867) | Loss 4.512846(4.523604) | NFE Forward 38(36.6) | NFE Backward 210(209.6)
Iter 1120 | Time 2.9799(2.8933) | Loss 4.478127(4.520421) | NFE Forward 38(36.7) | NFE Backward 216(210.0)
Iter 1121 | Time 2.9816(2.8994) | Loss 4.529918(4.521085) | NFE Forward 38(36.8) | NFE Backward 216(210.4)
Iter 1122 | Time 2.9150(2.9005) | Loss 4.518353(4.520894) | NFE Forward 38(36.8) | NFE Backward 210(210.4)
Iter 1123 | Time 2.9735(2.9056) | Loss 4.522241(4.520988) | NFE Forward 38(36.9) | NFE Backward 216(210.8)
Iter 1124 | Time 2.9128(2.9061) | Loss 4.514284(4.520519) | NFE Forward 38(37.0) | NFE Backward 210(210.7)
Iter 1125 | Time 2.9150(2.9068) | Loss 4.486502(4.518138) | NFE Forward 38(37.1) | NFE Backward 210(210.7)
Iter 1126 | Time 2.9281(2.9083) | Loss 4.516208(4.518003) | NFE Forward 38(37.1) | NFE Backward 210(210.6)
Iter 1127 | Time 2.9153(2.9087) | Loss 4.520564(4.518182) | NFE Forward 38(37.2) | NFE Backward 210(210.6)
Iter 1128 | Time 2.9169(2.9093) | Loss 4.501484(4.517013) | NFE Forward 38(37.3) | NFE Backward 210(210.6)
Iter 1129 | Time 2.9176(2.9099) | Loss 4.493052(4.515336) | NFE Forward 38(37.3) | NFE Backward 210(210.5)
Iter 1130 | Time 2.9269(2.9111) | Loss 4.543677(4.517320) | NFE Forward 38(37.4) | NFE Backward 210(210.5)
Iter 1131 | Time 2.9228(2.9119) | Loss 4.523383(4.517744) | NFE Forward 38(37.4) | NFE Backward 210(210.4)
Iter 1132 | Time 2.7788(2.9026) | Loss 4.496749(4.516275) | NFE Forward 32(37.0) | NFE Backward 198(209.6)
Iter 1133 | Time 2.9235(2.9040) | Loss 4.518143(4.516405) | NFE Forward 38(37.1) | NFE Backward 210(209.6)
Iter 1134 | Time 2.9747(2.9090) | Loss 4.544599(4.518379) | NFE Forward 38(37.2) | NFE Backward 216(210.0)
Iter 1135 | Time 2.9249(2.9101) | Loss 4.489592(4.516364) | NFE Forward 38(37.2) | NFE Backward 210(210.0)
Iter 1136 | Time 2.9789(2.9149) | Loss 4.504163(4.515510) | NFE Forward 38(37.3) | NFE Backward 216(210.5)
Iter 1137 | Time 2.9196(2.9153) | Loss 4.460423(4.511654) | NFE Forward 38(37.3) | NFE Backward 210(210.4)
Iter 1138 | Time 2.9746(2.9194) | Loss 4.555848(4.514747) | NFE Forward 38(37.4) | NFE Backward 216(210.8)
Iter 1139 | Time 2.9208(2.9195) | Loss 4.542709(4.516705) | NFE Forward 38(37.4) | NFE Backward 210(210.8)
Iter 1140 | Time 2.9205(2.9196) | Loss 4.537355(4.518150) | NFE Forward 38(37.5) | NFE Backward 210(210.7)
Iter 1141 | Time 2.9868(2.9243) | Loss 4.507532(4.517407) | NFE Forward 38(37.5) | NFE Backward 216(211.1)
Iter 1142 | Time 2.9781(2.9281) | Loss 4.492004(4.515629) | NFE Forward 38(37.5) | NFE Backward 216(211.4)
Iter 1143 | Time 2.9210(2.9276) | Loss 4.537669(4.517172) | NFE Forward 38(37.6) | NFE Backward 210(211.3)
Iter 1144 | Time 2.8730(2.9237) | Loss 4.513054(4.516883) | NFE Forward 38(37.6) | NFE Backward 204(210.8)
Iter 1145 | Time 2.9172(2.9233) | Loss 4.512102(4.516549) | NFE Forward 38(37.6) | NFE Backward 210(210.8)
Iter 1146 | Time 2.8004(2.9147) | Loss 4.472503(4.513465) | NFE Forward 38(37.6) | NFE Backward 198(209.9)
Iter 1147 | Time 2.9215(2.9152) | Loss 4.507669(4.513060) | NFE Forward 38(37.7) | NFE Backward 210(209.9)
Iter 1148 | Time 2.9156(2.9152) | Loss 4.515608(4.513238) | NFE Forward 38(37.7) | NFE Backward 210(209.9)
Iter 1149 | Time 2.9188(2.9154) | Loss 4.496051(4.512035) | NFE Forward 38(37.7) | NFE Backward 210(209.9)
Iter 1150 | Time 2.9161(2.9155) | Loss 4.523712(4.512852) | NFE Forward 38(37.7) | NFE Backward 210(209.9)
Iter 1151 | Time 2.9154(2.9155) | Loss 4.474625(4.510176) | NFE Forward 38(37.8) | NFE Backward 210(209.9)
Iter 1152 | Time 2.9151(2.9155) | Loss 4.519102(4.510801) | NFE Forward 38(37.8) | NFE Backward 210(209.9)
Iter 1153 | Time 2.9144(2.9154) | Loss 4.520514(4.511481) | NFE Forward 38(37.8) | NFE Backward 210(209.9)
Iter 1154 | Time 2.9201(2.9157) | Loss 4.454560(4.507497) | NFE Forward 38(37.8) | NFE Backward 210(209.9)
Iter 1155 | Time 2.9167(2.9158) | Loss 4.502367(4.507138) | NFE Forward 38(37.8) | NFE Backward 210(209.9)
Iter 1156 | Time 2.9173(2.9159) | Loss 4.528269(4.508617) | NFE Forward 38(37.8) | NFE Backward 210(209.9)
Iter 1157 | Time 2.9173(2.9160) | Loss 4.462116(4.505362) | NFE Forward 38(37.8) | NFE Backward 210(209.9)
Iter 1158 | Time 2.9156(2.9160) | Loss 4.501666(4.505103) | NFE Forward 38(37.9) | NFE Backward 210(209.9)
Iter 1159 | Time 2.9718(2.9199) | Loss 4.454255(4.501544) | NFE Forward 38(37.9) | NFE Backward 216(210.4)
Iter 1160 | Time 2.9141(2.9195) | Loss 4.519133(4.502775) | NFE Forward 38(37.9) | NFE Backward 210(210.3)
Iter 1161 | Time 2.9785(2.9236) | Loss 4.502234(4.502737) | NFE Forward 38(37.9) | NFE Backward 216(210.7)
Iter 1162 | Time 2.7995(2.9149) | Loss 4.558053(4.506609) | NFE Forward 38(37.9) | NFE Backward 198(209.8)
Iter 1163 | Time 2.9778(2.9193) | Loss 4.479885(4.504738) | NFE Forward 38(37.9) | NFE Backward 216(210.3)
Iter 1164 | Time 2.9175(2.9192) | Loss 4.453631(4.501161) | NFE Forward 38(37.9) | NFE Backward 210(210.3)
Iter 1165 | Time 2.9147(2.9189) | Loss 4.524029(4.502762) | NFE Forward 38(37.9) | NFE Backward 210(210.2)
Iter 1166 | Time 2.9145(2.9186) | Loss 4.531660(4.504785) | NFE Forward 38(37.9) | NFE Backward 210(210.2)
Iter 1167 | Time 2.9208(2.9187) | Loss 4.486846(4.503529) | NFE Forward 38(37.9) | NFE Backward 210(210.2)
Iter 1168 | Time 2.9176(2.9186) | Loss 4.519719(4.504662) | NFE Forward 38(37.9) | NFE Backward 210(210.2)
Iter 1169 | Time 2.9147(2.9184) | Loss 4.499124(4.504274) | NFE Forward 38(37.9) | NFE Backward 210(210.2)
Iter 1170 | Time 2.9156(2.9182) | Loss 4.487756(4.503118) | NFE Forward 38(37.9) | NFE Backward 210(210.2)
Iter 1171 | Time 2.9192(2.9182) | Loss 4.536762(4.505473) | NFE Forward 38(37.9) | NFE Backward 210(210.2)
Iter 1172 | Time 2.9754(2.9222) | Loss 4.491795(4.504516) | NFE Forward 38(37.9) | NFE Backward 216(210.6)
Iter 1173 | Time 2.9127(2.9216) | Loss 4.506409(4.504648) | NFE Forward 38(37.9) | NFE Backward 210(210.5)
Iter 1174 | Time 2.9183(2.9213) | Loss 4.520177(4.505735) | NFE Forward 38(38.0) | NFE Backward 210(210.5)
Iter 1175 | Time 2.9119(2.9207) | Loss 4.478849(4.503853) | NFE Forward 38(38.0) | NFE Backward 210(210.5)
Iter 1176 | Time 2.9158(2.9203) | Loss 4.544164(4.506675) | NFE Forward 38(38.0) | NFE Backward 210(210.4)
Iter 1177 | Time 2.9371(2.9215) | Loss 4.481516(4.504914) | NFE Forward 38(38.0) | NFE Backward 210(210.4)
Iter 1178 | Time 2.9152(2.9211) | Loss 4.469662(4.502446) | NFE Forward 38(38.0) | NFE Backward 210(210.4)
Iter 1179 | Time 2.9130(2.9205) | Loss 4.510532(4.503012) | NFE Forward 38(38.0) | NFE Backward 210(210.3)
Iter 1180 | Time 2.9728(2.9242) | Loss 4.511880(4.503633) | NFE Forward 38(38.0) | NFE Backward 216(210.7)
Iter 1181 | Time 2.9140(2.9235) | Loss 4.512070(4.504224) | NFE Forward 38(38.0) | NFE Backward 210(210.7)
Iter 1182 | Time 2.9801(2.9274) | Loss 4.471908(4.501962) | NFE Forward 38(38.0) | NFE Backward 216(211.1)
Iter 1183 | Time 2.9150(2.9266) | Loss 4.531281(4.504014) | NFE Forward 38(38.0) | NFE Backward 210(211.0)
Iter 1184 | Time 2.9178(2.9259) | Loss 4.512589(4.504614) | NFE Forward 38(38.0) | NFE Backward 210(210.9)
Iter 1185 | Time 2.9726(2.9292) | Loss 4.485399(4.503269) | NFE Forward 38(38.0) | NFE Backward 216(211.3)
Iter 1186 | Time 2.9749(2.9324) | Loss 4.478105(4.501508) | NFE Forward 38(38.0) | NFE Backward 216(211.6)
Iter 1187 | Time 2.9184(2.9314) | Loss 4.493775(4.500966) | NFE Forward 38(38.0) | NFE Backward 210(211.5)
Iter 1188 | Time 2.9774(2.9346) | Loss 4.514885(4.501941) | NFE Forward 38(38.0) | NFE Backward 216(211.8)
Iter 1189 | Time 2.9139(2.9332) | Loss 4.499242(4.501752) | NFE Forward 38(38.0) | NFE Backward 210(211.7)
Iter 1190 | Time 2.9736(2.9360) | Loss 4.512989(4.502538) | NFE Forward 38(38.0) | NFE Backward 216(212.0)
Iter 1191 | Time 2.9166(2.9347) | Loss 4.531407(4.504559) | NFE Forward 38(38.0) | NFE Backward 210(211.8)
Iter 1192 | Time 2.9719(2.9373) | Loss 4.477264(4.502648) | NFE Forward 38(38.0) | NFE Backward 216(212.1)
Iter 1193 | Time 2.9166(2.9358) | Loss 4.553344(4.506197) | NFE Forward 38(38.0) | NFE Backward 210(212.0)
Iter 1194 | Time 2.9835(2.9392) | Loss 4.495901(4.505476) | NFE Forward 38(38.0) | NFE Backward 216(212.3)
Iter 1195 | Time 2.9784(2.9419) | Loss 4.490796(4.504449) | NFE Forward 38(38.0) | NFE Backward 216(212.5)
Iter 1196 | Time 2.9826(2.9448) | Loss 4.466456(4.501789) | NFE Forward 38(38.0) | NFE Backward 216(212.8)
Iter 1197 | Time 2.9806(2.9473) | Loss 4.503176(4.501886) | NFE Forward 38(38.0) | NFE Backward 216(213.0)
Iter 1198 | Time 2.9173(2.9452) | Loss 4.474424(4.499964) | NFE Forward 38(38.0) | NFE Backward 210(212.8)
Iter 1199 | Time 2.9186(2.9433) | Loss 4.505722(4.500367) | NFE Forward 38(38.0) | NFE Backward 210(212.6)
Iter 1200 | Time 2.9167(2.9414) | Loss 4.503331(4.500575) | NFE Forward 38(38.0) | NFE Backward 210(212.4)
[TEST] Iter 1200 | Test Loss 4.462631 | NFE 38
Skipping vis as data dimension is >2
Iter 1201 | Time 2.9724(2.9436) | Loss 4.441529(4.496441) | NFE Forward 38(38.0) | NFE Backward 216(212.7)
Iter 1202 | Time 2.9220(2.9421) | Loss 4.493988(4.496270) | NFE Forward 38(38.0) | NFE Backward 210(212.5)
Iter 1203 | Time 2.9135(2.9401) | Loss 4.476356(4.494876) | NFE Forward 38(38.0) | NFE Backward 210(212.3)
Iter 1204 | Time 2.9152(2.9384) | Loss 4.489758(4.494517) | NFE Forward 38(38.0) | NFE Backward 210(212.1)
Iter 1205 | Time 2.9178(2.9369) | Loss 4.468195(4.492675) | NFE Forward 38(38.0) | NFE Backward 210(212.0)
Iter 1206 | Time 2.9159(2.9354) | Loss 4.487465(4.492310) | NFE Forward 38(38.0) | NFE Backward 210(211.9)
Iter 1207 | Time 2.9753(2.9382) | Loss 4.479034(4.491381) | NFE Forward 38(38.0) | NFE Backward 216(212.1)
Iter 1208 | Time 2.9250(2.9373) | Loss 4.489626(4.491258) | NFE Forward 38(38.0) | NFE Backward 210(212.0)
Iter 1209 | Time 2.9183(2.9360) | Loss 4.511865(4.492700) | NFE Forward 38(38.0) | NFE Backward 210(211.9)
Iter 1210 | Time 2.9729(2.9386) | Loss 4.522509(4.494787) | NFE Forward 38(38.0) | NFE Backward 216(212.1)
Iter 1211 | Time 2.9204(2.9373) | Loss 4.470545(4.493090) | NFE Forward 38(38.0) | NFE Backward 210(212.0)
Iter 1212 | Time 2.9760(2.9400) | Loss 4.490049(4.492877) | NFE Forward 38(38.0) | NFE Backward 216(212.3)
Iter 1213 | Time 2.9731(2.9423) | Loss 4.466419(4.491025) | NFE Forward 38(38.0) | NFE Backward 216(212.5)
Iter 1214 | Time 2.9184(2.9406) | Loss 4.484635(4.490578) | NFE Forward 38(38.0) | NFE Backward 210(212.4)
Iter 1215 | Time 2.9164(2.9389) | Loss 4.454579(4.488058) | NFE Forward 38(38.0) | NFE Backward 210(212.2)
Iter 1216 | Time 2.9150(2.9373) | Loss 4.504141(4.489184) | NFE Forward 38(38.0) | NFE Backward 210(212.0)
Iter 1217 | Time 2.9169(2.9358) | Loss 4.515510(4.491027) | NFE Forward 38(38.0) | NFE Backward 210(211.9)
Iter 1218 | Time 2.9144(2.9343) | Loss 4.469304(4.489506) | NFE Forward 38(38.0) | NFE Backward 210(211.8)
Iter 1219 | Time 2.9155(2.9330) | Loss 4.516204(4.491375) | NFE Forward 38(38.0) | NFE Backward 210(211.6)
Iter 1220 | Time 2.9149(2.9318) | Loss 4.480187(4.490592) | NFE Forward 38(38.0) | NFE Backward 210(211.5)
Iter 1221 | Time 2.8632(2.9270) | Loss 4.471244(4.489237) | NFE Forward 38(38.0) | NFE Backward 204(211.0)
Iter 1222 | Time 2.9826(2.9308) | Loss 4.489479(4.489254) | NFE Forward 38(38.0) | NFE Backward 216(211.3)
Iter 1223 | Time 2.9291(2.9307) | Loss 4.504163(4.490298) | NFE Forward 38(38.0) | NFE Backward 210(211.3)
Iter 1224 | Time 2.8628(2.9260) | Loss 4.496797(4.490753) | NFE Forward 38(38.0) | NFE Backward 204(210.7)
Iter 1225 | Time 2.9862(2.9302) | Loss 4.472862(4.489500) | NFE Forward 38(38.0) | NFE Backward 216(211.1)
Iter 1226 | Time 2.9217(2.9296) | Loss 4.503783(4.490500) | NFE Forward 38(38.0) | NFE Backward 210(211.0)
Iter 1227 | Time 2.9889(2.9337) | Loss 4.481196(4.489849) | NFE Forward 38(38.0) | NFE Backward 216(211.4)
Iter 1228 | Time 2.9906(2.9377) | Loss 4.515381(4.491636) | NFE Forward 38(38.0) | NFE Backward 216(211.7)
Iter 1229 | Time 2.9283(2.9371) | Loss 4.438027(4.487884) | NFE Forward 38(38.0) | NFE Backward 210(211.6)
Iter 1230 | Time 2.9284(2.9365) | Loss 4.472986(4.486841) | NFE Forward 38(38.0) | NFE Backward 210(211.5)
Iter 1231 | Time 2.9906(2.9402) | Loss 4.461880(4.485093) | NFE Forward 38(38.0) | NFE Backward 216(211.8)
Iter 1232 | Time 2.9356(2.9399) | Loss 4.520016(4.487538) | NFE Forward 38(38.0) | NFE Backward 210(211.7)
Iter 1233 | Time 2.9265(2.9390) | Loss 4.481116(4.487088) | NFE Forward 38(38.0) | NFE Backward 210(211.6)
Iter 1234 | Time 2.9251(2.9380) | Loss 4.515835(4.489101) | NFE Forward 38(38.0) | NFE Backward 210(211.4)
Iter 1235 | Time 2.9295(2.9374) | Loss 4.477420(4.488283) | NFE Forward 38(38.0) | NFE Backward 210(211.3)
Iter 1236 | Time 2.9290(2.9368) | Loss 4.483198(4.487927) | NFE Forward 38(38.0) | NFE Backward 210(211.2)
Iter 1237 | Time 2.8109(2.9280) | Loss 4.495748(4.488475) | NFE Forward 38(38.0) | NFE Backward 198(210.3)
Iter 1238 | Time 2.9251(2.9278) | Loss 4.490994(4.488651) | NFE Forward 38(38.0) | NFE Backward 210(210.3)
Iter 1239 | Time 2.9274(2.9278) | Loss 4.468628(4.487249) | NFE Forward 38(38.0) | NFE Backward 210(210.3)
Iter 1240 | Time 2.9228(2.9274) | Loss 4.495276(4.487811) | NFE Forward 38(38.0) | NFE Backward 210(210.3)
Iter 1241 | Time 2.9307(2.9277) | Loss 4.503221(4.488890) | NFE Forward 38(38.0) | NFE Backward 210(210.2)
Iter 1242 | Time 2.9283(2.9277) | Loss 4.483331(4.488501) | NFE Forward 38(38.0) | NFE Backward 210(210.2)
Iter 1243 | Time 2.9348(2.9282) | Loss 4.494545(4.488924) | NFE Forward 38(38.0) | NFE Backward 210(210.2)
Iter 1244 | Time 2.9278(2.9282) | Loss 4.468465(4.487492) | NFE Forward 38(38.0) | NFE Backward 210(210.2)
Iter 1245 | Time 2.9308(2.9284) | Loss 4.478000(4.486827) | NFE Forward 38(38.0) | NFE Backward 210(210.2)
Iter 1246 | Time 2.8721(2.9244) | Loss 4.481883(4.486481) | NFE Forward 38(38.0) | NFE Backward 204(209.7)
Iter 1247 | Time 2.9899(2.9290) | Loss 4.453144(4.484148) | NFE Forward 38(38.0) | NFE Backward 216(210.2)
Density Loss 0.46015477180480957
Density Loss 0.44232621788978577
Density Loss 0.44348201155662537
Density Loss 0.44776564836502075
Density Loss 0.4340665936470032
Density Loss 0.451528400182724
Density Loss 0.4519622027873993
Density Loss 0.4475078880786896
Density Loss 0.4496905207633972
Density Loss 0.4461803734302521
Density Loss 0.441533625125885
Density Loss 0.4471896290779114
Density Loss 0.44965827465057373
Density Loss 0.45233210921287537
Density Loss 0.4547010660171509
Density Loss 0.44179245829582214
Density Loss 0.43569138646125793
Density Loss 0.44701552391052246
Density Loss 0.4398345649242401
Density Loss 0.44752317667007446
Density Loss 0.45035478472709656
Density Loss 0.4366382658481598
Density Loss 0.45760729908943176
Density Loss 0.44098615646362305
Density Loss 0.45887717604637146
Density Loss 0.44740644097328186
Density Loss 0.4498477280139923
Density Loss 0.4512098431587219
Density Loss 0.4480835795402527
Density Loss 0.4478171765804291
Density Loss 0.45940446853637695
Density Loss 0.4639582931995392
Density Loss 0.44738075137138367
Density Loss 0.44913655519485474
Density Loss 0.44923871755599976
Density Loss 0.4541596472263336
Density Loss 0.4424651265144348
Density Loss 0.4481566250324249
Density Loss 0.4452304542064667
Density Loss 0.45190662145614624
Density Loss 0.44855305552482605
Density Loss 0.4660190939903259
Density Loss 0.43695321679115295
Density Loss 0.4343281090259552
Density Loss 0.4572775661945343
Density Loss 0.4659873843193054
Density Loss 0.45567524433135986
Density Loss 0.4424057602882385
Density Loss 0.46285194158554077
Density Loss 0.45380881428718567
Density Loss 0.4691677987575531
Density Loss 0.4664192795753479
Density Loss 0.4551825225353241
Density Loss 0.47071462869644165
Density Loss 0.45745736360549927
Density Loss 0.4474879205226898
Density Loss 0.4468206465244293
Density Loss 0.46131062507629395
Density Loss 0.45333877205848694
Density Loss 0.4532736837863922
Density Loss 0.45616865158081055
Density Loss 0.4581071138381958
Density Loss 0.45298856496810913
Density Loss 0.45565080642700195
Density Loss 0.4457511603832245
Density Loss 0.45185619592666626
Density Loss 0.4473596513271332
Density Loss 0.4586593210697174
Density Loss 0.46549755334854126
Density Loss 0.44597917795181274
Density Loss 0.444701611995697
Density Loss 0.45117998123168945
Density Loss 0.45682889223098755
Density Loss 0.46026262640953064
Density Loss 0.442953884601593
Density Loss 0.46854734420776367
Density Loss 0.44896015524864197
Density Loss 0.44113415479660034
Density Loss 0.455584853887558
Density Loss 0.465671569108963
Density Loss 0.4530225992202759
Density Loss 0.4542332887649536
Density Loss 0.4579223096370697
Density Loss 0.46907493472099304
Density Loss 0.4466351866722107
Density Loss 0.45594725012779236
Density Loss 0.4525284171104431
Density Loss 0.4549460709095001
Density Loss 0.45811209082603455
Density Loss 0.438205361366272
Density Loss 0.458648145198822
Density Loss 0.4542621970176697
Density Loss 0.45608383417129517
Density Loss 0.4516734182834625
Density Loss 0.45619526505470276
Density Loss 0.45480331778526306
Density Loss 0.45217087864875793
Density Loss 0.440947949886322
Density Loss 0.4636439383029938
Density Loss 0.445761501789093
Density Loss 0.4514850378036499
Density Loss 0.45258083939552307
Density Loss 0.4498516619205475
Density Loss 0.45805037021636963
Density Loss 0.45284590125083923
Density Loss 0.4627065360546112
Density Loss 0.4458850026130676
Density Loss 0.4502679705619812
Density Loss 0.43647146224975586
Density Loss 0.46615856885910034
Density Loss 0.45876386761665344
Density Loss 0.45286282896995544
Density Loss 0.45497703552246094
Density Loss 0.4488116204738617
Density Loss 0.45810437202453613
Density Loss 0.4680001735687256
Density Loss 0.4606494903564453
Density Loss 0.465459942817688
Density Loss 0.4519605338573456
Density Loss 0.44641590118408203
Density Loss 0.461203932762146
Density Loss 0.4563419222831726
Density Loss 0.47097548842430115
Density Loss 0.449558824300766
Density Loss 0.45067185163497925
Density Loss 0.43831443786621094
Density Loss 0.46319687366485596
Density Loss 0.4666162133216858
Density Loss 0.46785643696784973
Density Loss 0.4632683992385864
Density Loss 0.4630257189273834
Density Loss 0.4690992534160614
Density Loss 0.45657816529273987
Density Loss 0.46337518095970154
Density Loss 0.447054386138916
Density Loss 0.4458266496658325
Density Loss 0.4424855709075928
Density Loss 0.4562230408191681
Density Loss 0.45470497012138367
Density Loss 0.4577927589416504
Density Loss 0.4683234691619873
Density Loss 0.44862380623817444
Density Loss 0.4601425528526306
Density Loss 0.4709632694721222
Density Loss 0.45257124304771423
Density Loss 0.4578922688961029
Density Loss 0.45562103390693665
Density Loss 0.4502369165420532
Density Loss 0.4583859145641327
Density Loss 0.46615877747535706
Density Loss 0.45873916149139404
Density Loss 0.4514245092868805
Density Loss 0.4464234709739685
Density Loss 0.46876463294029236
Density Loss 0.45773035287857056
Density Loss 0.4618434011936188
Density Loss 0.4781123995780945
Density Loss 0.44384726881980896
Density Loss 0.4589352309703827
Density Loss 0.4809008240699768
Density Loss 0.4560389518737793
Density Loss 0.4565548598766327
Density Loss 0.45967358350753784
Density Loss 0.45239919424057007
Density Loss 0.46581006050109863
Density Loss 0.4588262438774109
Density Loss 0.4503544867038727
Density Loss 0.4753522276878357
Density Loss 0.4801037013530731
Density Loss 0.45317474007606506
Density Loss 0.4571645259857178
Density Loss 0.4640164077281952
Density Loss 0.47616374492645264
Density Loss 0.4584495425224304
Density Loss 0.46152597665786743
Density Loss 0.46784070134162903
Density Loss 0.48004719614982605
Density Loss 0.4542059600353241
Density Loss 0.470443457365036
Density Loss 0.46768566966056824
Density Loss 0.4635125994682312
Density Loss 0.47061893343925476
Density Loss 0.47051218152046204
Density Loss 0.4660477936267853
Density Loss 0.4707704484462738
Density Loss 0.4724105894565582
Density Loss 0.467624306678772
Density Loss 0.46307480335235596
Density Loss 0.4649730324745178
Density Loss 0.4492763578891754
Density Loss 0.4809269309043884
Density Loss 0.47042468190193176
Density Loss 0.45478206872940063
Density Loss 0.45697206258773804
Density Loss 0.4766062796115875
Density Loss 0.48100125789642334
Density Loss 0.4694303572177887
Density Loss 0.45505017042160034
Density Loss 0.46154558658599854
Density Loss 0.4672011733055115
Density Loss 0.4641329348087311
Density Loss 0.4707210958003998
Density Loss 0.4525505602359772
Density Loss 0.465046763420105
Density Loss 0.4646407961845398
Density Loss 0.46028709411621094
Density Loss 0.4607378840446472
Density Loss 0.45177996158599854
Density Loss 0.4527258574962616
Density Loss 0.45301082730293274
Density Loss 0.4769868552684784
Density Loss 0.4632914960384369
Density Loss 0.45593127608299255
Density Loss 0.47437068819999695
Density Loss 0.4718831777572632
Density Loss 0.45466601848602295
Density Loss 0.4650188386440277
Density Loss 0.4808008670806885
Density Loss 0.4558565318584442
Density Loss 0.4501287043094635
Density Loss 0.46221327781677246
Density Loss 0.49004319310188293
Density Loss 0.468171089887619
Density Loss 0.46035024523735046
Density Loss 0.48067790269851685
Density Loss 0.46076005697250366
Density Loss 0.46002835035324097
Density Loss 0.4944051206111908
Density Loss 0.4569196105003357
Density Loss 0.4732125997543335
Density Loss 0.4620920717716217
Density Loss 0.4701964259147644
Density Loss 0.4823564291000366
Density Loss 0.45996424555778503
Density Loss 0.4533501863479614
Density Loss 0.4656471610069275
Density Loss 0.4684572219848633
Density Loss 0.4682452976703644
Density Loss 0.47410017251968384
Density Loss 0.47932615876197815
Density Loss 0.46274158358573914
Density Loss 0.4749678671360016
Density Loss 0.4629645049571991
Density Loss 0.4659246504306793
Density Loss 0.47851869463920593
Density Loss 0.47515320777893066
Density Loss 0.45901745557785034
Density Loss 0.4544401168823242
Density Loss 0.4703439474105835
Density Loss 0.4586043357849121
Density Loss 0.46316131949424744
Density Loss 0.45963361859321594
Density Loss 0.4788045883178711
Iter 1248 | Time 3.0633(2.9384) | Loss 4.449745(4.481739) | NFE Forward 38(38.0) | NFE Backward 222(211.0)
Iter 1249 | Time 2.9323(2.9380) | Loss 4.476859(4.481398) | NFE Forward 38(38.0) | NFE Backward 210(210.9)
Iter 1250 | Time 3.0130(2.9432) | Loss 4.461760(4.480023) | NFE Forward 38(38.0) | NFE Backward 210(210.9)
Iter 1251 | Time 2.9213(2.9417) | Loss 4.480258(4.480040) | NFE Forward 38(38.0) | NFE Backward 210(210.8)
Iter 1252 | Time 2.9216(2.9403) | Loss 4.508622(4.482040) | NFE Forward 38(38.0) | NFE Backward 210(210.8)
Iter 1253 | Time 2.9170(2.9387) | Loss 4.506281(4.483737) | NFE Forward 38(38.0) | NFE Backward 210(210.7)
Iter 1254 | Time 2.9190(2.9373) | Loss 4.462302(4.482237) | NFE Forward 38(38.0) | NFE Backward 210(210.7)
Iter 1255 | Time 2.9229(2.9363) | Loss 4.471877(4.481511) | NFE Forward 38(38.0) | NFE Backward 210(210.6)
Iter 1256 | Time 2.9219(2.9353) | Loss 4.474898(4.481049) | NFE Forward 38(38.0) | NFE Backward 210(210.6)
Iter 1257 | Time 2.8659(2.9304) | Loss 4.484911(4.481319) | NFE Forward 38(38.0) | NFE Backward 204(210.1)
Iter 1258 | Time 2.9383(2.9310) | Loss 4.482083(4.481372) | NFE Forward 38(38.0) | NFE Backward 210(210.1)
Iter 1259 | Time 2.9308(2.9310) | Loss 4.482358(4.481441) | NFE Forward 38(38.0) | NFE Backward 210(210.1)
Iter 1260 | Time 2.9214(2.9303) | Loss 4.492086(4.482187) | NFE Forward 38(38.0) | NFE Backward 210(210.1)
Iter 1261 | Time 2.8621(2.9255) | Loss 4.426350(4.478278) | NFE Forward 38(38.0) | NFE Backward 204(209.7)
Iter 1262 | Time 3.0458(2.9339) | Loss 4.465076(4.477354) | NFE Forward 38(38.0) | NFE Backward 222(210.5)
Iter 1263 | Time 2.8773(2.9300) | Loss 4.434904(4.474382) | NFE Forward 38(38.0) | NFE Backward 204(210.1)
Iter 1264 | Time 2.8768(2.9262) | Loss 4.480757(4.474829) | NFE Forward 38(38.0) | NFE Backward 204(209.6)
Iter 1265 | Time 3.0416(2.9343) | Loss 4.479337(4.475144) | NFE Forward 38(38.0) | NFE Backward 222(210.5)
Iter 1266 | Time 2.9035(2.9322) | Loss 4.464176(4.474376) | NFE Forward 38(38.0) | NFE Backward 210(210.5)
Iter 1267 | Time 2.8420(2.9258) | Loss 4.501740(4.476292) | NFE Forward 38(38.0) | NFE Backward 204(210.0)
Iter 1268 | Time 2.9004(2.9241) | Loss 4.457848(4.475001) | NFE Forward 38(38.0) | NFE Backward 210(210.0)
Iter 1269 | Time 2.9993(2.9293) | Loss 4.450008(4.473251) | NFE Forward 38(38.0) | NFE Backward 210(210.0)
Iter 1270 | Time 2.8982(2.9272) | Loss 4.460540(4.472361) | NFE Forward 38(38.0) | NFE Backward 210(210.0)
Iter 1271 | Time 2.8879(2.9244) | Loss 4.418441(4.468587) | NFE Forward 38(38.0) | NFE Backward 210(210.0)
Iter 1272 | Time 2.8892(2.9219) | Loss 4.502398(4.470954) | NFE Forward 38(38.0) | NFE Backward 210(210.0)
Iter 1273 | Time 2.8335(2.9158) | Loss 4.481099(4.471664) | NFE Forward 38(38.0) | NFE Backward 204(209.6)
Iter 1274 | Time 2.9471(2.9179) | Loss 4.452395(4.470315) | NFE Forward 38(38.0) | NFE Backward 216(210.0)
Iter 1275 | Time 2.8874(2.9158) | Loss 4.463961(4.469870) | NFE Forward 38(38.0) | NFE Backward 210(210.0)
Iter 1276 | Time 2.8887(2.9139) | Loss 4.466672(4.469647) | NFE Forward 38(38.0) | NFE Backward 210(210.0)
Iter 1277 | Time 2.9472(2.9162) | Loss 4.484108(4.470659) | NFE Forward 38(38.0) | NFE Backward 216(210.5)
Iter 1278 | Time 2.8894(2.9144) | Loss 4.496181(4.472445) | NFE Forward 38(38.0) | NFE Backward 210(210.4)
Iter 1279 | Time 2.8253(2.9081) | Loss 4.458930(4.471499) | NFE Forward 38(38.0) | NFE Backward 204(210.0)
Iter 1280 | Time 2.8291(2.9026) | Loss 4.488318(4.472677) | NFE Forward 38(38.0) | NFE Backward 204(209.6)
Iter 1281 | Time 2.8794(2.9010) | Loss 4.487321(4.473702) | NFE Forward 38(38.0) | NFE Backward 210(209.6)
Iter 1282 | Time 2.8795(2.8995) | Loss 4.497070(4.475338) | NFE Forward 38(38.0) | NFE Backward 210(209.6)
Iter 1283 | Time 2.8799(2.8981) | Loss 4.443521(4.473110) | NFE Forward 38(38.0) | NFE Backward 210(209.6)
Iter 1284 | Time 2.8345(2.8936) | Loss 4.495647(4.474688) | NFE Forward 38(38.0) | NFE Backward 204(209.2)
Iter 1285 | Time 2.8209(2.8886) | Loss 4.453813(4.473227) | NFE Forward 38(38.0) | NFE Backward 204(208.9)
Iter 1286 | Time 2.8817(2.8881) | Loss 4.454801(4.471937) | NFE Forward 38(38.0) | NFE Backward 210(209.0)
Iter 1287 | Time 2.9410(2.8918) | Loss 4.456660(4.470868) | NFE Forward 38(38.0) | NFE Backward 216(209.5)
Iter 1288 | Time 2.9949(2.8990) | Loss 4.444063(4.468991) | NFE Forward 38(38.0) | NFE Backward 222(210.3)
Iter 1289 | Time 2.8774(2.8975) | Loss 4.442203(4.467116) | NFE Forward 38(38.0) | NFE Backward 210(210.3)
Iter 1290 | Time 2.8798(2.8962) | Loss 4.501544(4.469526) | NFE Forward 38(38.0) | NFE Backward 210(210.3)
Iter 1291 | Time 2.8760(2.8948) | Loss 4.424911(4.466403) | NFE Forward 38(38.0) | NFE Backward 210(210.3)
Iter 1292 | Time 2.8776(2.8936) | Loss 4.434506(4.464170) | NFE Forward 38(38.0) | NFE Backward 210(210.2)
Iter 1293 | Time 2.8236(2.8887) | Loss 4.421692(4.461197) | NFE Forward 38(38.0) | NFE Backward 204(209.8)
Iter 1294 | Time 2.9439(2.8926) | Loss 4.448484(4.460307) | NFE Forward 38(38.0) | NFE Backward 216(210.2)
Iter 1295 | Time 2.9992(2.9001) | Loss 4.482090(4.461832) | NFE Forward 38(38.0) | NFE Backward 222(211.1)
Iter 1296 | Time 3.0159(2.9082) | Loss 4.467342(4.462217) | NFE Forward 38(38.0) | NFE Backward 222(211.8)
Iter 1297 | Time 2.9001(2.9076) | Loss 4.467847(4.462612) | NFE Forward 38(38.0) | NFE Backward 210(211.7)
Iter 1298 | Time 2.8960(2.9068) | Loss 4.456287(4.462169) | NFE Forward 38(38.0) | NFE Backward 210(211.6)
Iter 1299 | Time 2.8565(2.9033) | Loss 4.509336(4.465471) | NFE Forward 38(38.0) | NFE Backward 204(211.1)
Iter 1300 | Time 2.9742(2.9082) | Loss 4.470792(4.465843) | NFE Forward 38(38.0) | NFE Backward 216(211.4)
[TEST] Iter 1300 | Test Loss 4.477488 | NFE 38
Skipping vis as data dimension is >2
Iter 1301 | Time 2.9057(2.9081) | Loss 4.495718(4.467934) | NFE Forward 38(38.0) | NFE Backward 210(211.3)
Iter 1302 | Time 2.8497(2.9040) | Loss 4.442355(4.466144) | NFE Forward 38(38.0) | NFE Backward 204(210.8)
Iter 1303 | Time 2.7362(2.8922) | Loss 4.475305(4.466785) | NFE Forward 38(38.0) | NFE Backward 192(209.5)
Iter 1304 | Time 2.9121(2.8936) | Loss 4.511681(4.469928) | NFE Forward 38(38.0) | NFE Backward 210(209.5)
Iter 1305 | Time 3.0503(2.9046) | Loss 4.457727(4.469074) | NFE Forward 38(38.0) | NFE Backward 222(210.4)
Iter 1306 | Time 2.8947(2.9039) | Loss 4.506340(4.471682) | NFE Forward 38(38.0) | NFE Backward 210(210.4)
Iter 1307 | Time 2.8910(2.9030) | Loss 4.520888(4.475127) | NFE Forward 38(38.0) | NFE Backward 210(210.3)
Iter 1308 | Time 2.8373(2.8984) | Loss 4.483200(4.475692) | NFE Forward 38(38.0) | NFE Backward 204(209.9)
Iter 1309 | Time 3.0176(2.9067) | Loss 4.459282(4.474543) | NFE Forward 38(38.0) | NFE Backward 222(210.7)
Iter 1310 | Time 2.9342(2.9087) | Loss 4.413487(4.470269) | NFE Forward 38(38.0) | NFE Backward 210(210.7)
Iter 1311 | Time 2.9560(2.9120) | Loss 4.449947(4.468847) | NFE Forward 38(38.0) | NFE Backward 216(211.1)
Iter 1312 | Time 3.0103(2.9189) | Loss 4.475496(4.469312) | NFE Forward 38(38.0) | NFE Backward 222(211.8)
Iter 1313 | Time 3.0096(2.9252) | Loss 4.452622(4.468144) | NFE Forward 38(38.0) | NFE Backward 222(212.5)
Iter 1314 | Time 2.8368(2.9190) | Loss 4.452756(4.467067) | NFE Forward 38(38.0) | NFE Backward 204(211.9)
Iter 1315 | Time 2.8292(2.9127) | Loss 4.467664(4.467108) | NFE Forward 38(38.0) | NFE Backward 204(211.4)
Iter 1316 | Time 2.8307(2.9070) | Loss 4.477771(4.467855) | NFE Forward 38(38.0) | NFE Backward 204(210.9)
Iter 1317 | Time 2.9476(2.9098) | Loss 4.427277(4.465014) | NFE Forward 38(38.0) | NFE Backward 216(211.2)
Iter 1318 | Time 2.8871(2.9082) | Loss 4.478088(4.465930) | NFE Forward 38(38.0) | NFE Backward 210(211.1)
Iter 1319 | Time 2.8485(2.9041) | Loss 4.443499(4.464359) | NFE Forward 38(38.0) | NFE Backward 204(210.6)
Iter 1320 | Time 2.8356(2.8993) | Loss 4.446223(4.463090) | NFE Forward 38(38.0) | NFE Backward 204(210.2)
Iter 1321 | Time 2.8899(2.8986) | Loss 4.506846(4.466153) | NFE Forward 38(38.0) | NFE Backward 210(210.2)
Iter 1322 | Time 2.8952(2.8984) | Loss 4.527739(4.470464) | NFE Forward 38(38.0) | NFE Backward 210(210.2)
Iter 1323 | Time 2.7760(2.8898) | Loss 4.513270(4.473460) | NFE Forward 38(38.0) | NFE Backward 198(209.3)
Iter 1324 | Time 2.8640(2.8880) | Loss 4.476154(4.473649) | NFE Forward 38(38.0) | NFE Backward 204(208.9)
Iter 1325 | Time 2.8397(2.8846) | Loss 4.436385(4.471040) | NFE Forward 38(38.0) | NFE Backward 204(208.6)
Iter 1326 | Time 3.0273(2.8946) | Loss 4.457425(4.470087) | NFE Forward 38(38.0) | NFE Backward 222(209.5)
Iter 1327 | Time 2.8942(2.8946) | Loss 4.400849(4.465241) | NFE Forward 38(38.0) | NFE Backward 210(209.6)
Iter 1328 | Time 2.9018(2.8951) | Loss 4.470025(4.465575) | NFE Forward 38(38.0) | NFE Backward 210(209.6)
Iter 1329 | Time 2.8957(2.8951) | Loss 4.418407(4.462274) | NFE Forward 38(38.0) | NFE Backward 210(209.6)
Iter 1330 | Time 2.9607(2.8997) | Loss 4.495905(4.464628) | NFE Forward 38(38.0) | NFE Backward 216(210.1)
Iter 1331 | Time 2.9041(2.9000) | Loss 4.448763(4.463517) | NFE Forward 38(38.0) | NFE Backward 210(210.1)
Iter 1332 | Time 2.8346(2.8954) | Loss 4.471204(4.464055) | NFE Forward 38(38.0) | NFE Backward 204(209.6)
Iter 1333 | Time 2.8947(2.8954) | Loss 4.435155(4.462032) | NFE Forward 38(38.0) | NFE Backward 210(209.7)
Iter 1334 | Time 2.9182(2.8970) | Loss 4.476615(4.463053) | NFE Forward 38(38.0) | NFE Backward 210(209.7)
Iter 1335 | Time 2.8510(2.8938) | Loss 4.486708(4.464709) | NFE Forward 38(38.0) | NFE Backward 204(209.3)
Iter 1336 | Time 2.8656(2.8918) | Loss 4.450863(4.463740) | NFE Forward 38(38.0) | NFE Backward 204(208.9)
Iter 1337 | Time 2.9075(2.8929) | Loss 4.446476(4.462531) | NFE Forward 38(38.0) | NFE Backward 204(208.6)
Iter 1338 | Time 2.8780(2.8918) | Loss 4.441224(4.461040) | NFE Forward 38(38.0) | NFE Backward 204(208.3)
Iter 1339 | Time 2.8626(2.8898) | Loss 4.465088(4.461323) | NFE Forward 38(38.0) | NFE Backward 204(208.0)
Iter 1340 | Time 2.9177(2.8918) | Loss 4.428475(4.459024) | NFE Forward 38(38.0) | NFE Backward 210(208.1)
Iter 1341 | Time 2.8564(2.8893) | Loss 4.481590(4.460603) | NFE Forward 38(38.0) | NFE Backward 204(207.8)
Iter 1342 | Time 2.9702(2.8949) | Loss 4.426234(4.458198) | NFE Forward 38(38.0) | NFE Backward 216(208.4)
Iter 1343 | Time 2.9044(2.8956) | Loss 4.474553(4.459342) | NFE Forward 38(38.0) | NFE Backward 210(208.5)
Iter 1344 | Time 2.8473(2.8922) | Loss 4.424784(4.456923) | NFE Forward 38(38.0) | NFE Backward 204(208.2)
Iter 1345 | Time 2.9067(2.8932) | Loss 4.471336(4.457932) | NFE Forward 38(38.0) | NFE Backward 210(208.3)
Iter 1346 | Time 2.9213(2.8952) | Loss 4.508677(4.461484) | NFE Forward 38(38.0) | NFE Backward 210(208.4)
Iter 1347 | Time 2.9105(2.8963) | Loss 4.447476(4.460504) | NFE Forward 38(38.0) | NFE Backward 210(208.5)
Iter 1348 | Time 2.8557(2.8934) | Loss 4.485986(4.462288) | NFE Forward 38(38.0) | NFE Backward 204(208.2)
Iter 1349 | Time 2.8458(2.8901) | Loss 4.458731(4.462039) | NFE Forward 38(38.0) | NFE Backward 204(207.9)
Iter 1350 | Time 2.9667(2.8954) | Loss 4.439444(4.460457) | NFE Forward 38(38.0) | NFE Backward 216(208.5)
Iter 1351 | Time 2.8593(2.8929) | Loss 4.448012(4.459586) | NFE Forward 38(38.0) | NFE Backward 204(208.2)
Iter 1352 | Time 2.9236(2.8951) | Loss 4.462276(4.459774) | NFE Forward 38(38.0) | NFE Backward 210(208.3)
Iter 1353 | Time 2.9820(2.9012) | Loss 4.467628(4.460324) | NFE Forward 38(38.0) | NFE Backward 216(208.8)
Iter 1354 | Time 2.8596(2.8982) | Loss 4.482254(4.461859) | NFE Forward 38(38.0) | NFE Backward 204(208.5)
Iter 1355 | Time 2.8715(2.8964) | Loss 4.473534(4.462676) | NFE Forward 38(38.0) | NFE Backward 204(208.2)
Iter 1356 | Time 2.8628(2.8940) | Loss 4.423923(4.459963) | NFE Forward 38(38.0) | NFE Backward 204(207.9)
Iter 1357 | Time 2.8640(2.8919) | Loss 4.454086(4.459552) | NFE Forward 38(38.0) | NFE Backward 204(207.6)
Iter 1358 | Time 2.9726(2.8976) | Loss 4.442932(4.458389) | NFE Forward 38(38.0) | NFE Backward 216(208.2)
Iter 1359 | Time 2.8602(2.8950) | Loss 4.471407(4.459300) | NFE Forward 38(38.0) | NFE Backward 204(207.9)
Iter 1360 | Time 2.8592(2.8924) | Loss 4.464260(4.459647) | NFE Forward 38(38.0) | NFE Backward 204(207.6)
Iter 1361 | Time 2.8436(2.8890) | Loss 4.461417(4.459771) | NFE Forward 38(38.0) | NFE Backward 204(207.4)
Iter 1362 | Time 2.8420(2.8857) | Loss 4.514575(4.463607) | NFE Forward 38(38.0) | NFE Backward 204(207.1)
Iter 1363 | Time 2.8396(2.8825) | Loss 4.431157(4.461336) | NFE Forward 38(38.0) | NFE Backward 204(206.9)
Iter 1364 | Time 2.8949(2.8834) | Loss 4.455803(4.460948) | NFE Forward 38(38.0) | NFE Backward 210(207.1)
Iter 1365 | Time 2.8337(2.8799) | Loss 4.475525(4.461969) | NFE Forward 38(38.0) | NFE Backward 204(206.9)
Iter 1366 | Time 2.8386(2.8770) | Loss 4.415426(4.458711) | NFE Forward 38(38.0) | NFE Backward 204(206.7)
Iter 1367 | Time 2.9553(2.8825) | Loss 4.445687(4.457799) | NFE Forward 38(38.0) | NFE Backward 216(207.4)
Iter 1368 | Time 2.8394(2.8795) | Loss 4.436498(4.456308) | NFE Forward 38(38.0) | NFE Backward 204(207.1)
Iter 1369 | Time 2.9110(2.8817) | Loss 4.424818(4.454104) | NFE Forward 38(38.0) | NFE Backward 210(207.3)
Iter 1370 | Time 2.9853(2.8889) | Loss 4.431105(4.452494) | NFE Forward 38(38.0) | NFE Backward 216(207.9)
Iter 1371 | Time 2.8498(2.8862) | Loss 4.415018(4.449871) | NFE Forward 38(38.0) | NFE Backward 204(207.7)
Iter 1372 | Time 3.0296(2.8962) | Loss 4.501308(4.453471) | NFE Forward 38(38.0) | NFE Backward 222(208.7)
Iter 1373 | Time 2.9651(2.9011) | Loss 4.430676(4.451876) | NFE Forward 38(38.0) | NFE Backward 216(209.2)
Iter 1374 | Time 2.8410(2.8969) | Loss 4.435099(4.450701) | NFE Forward 38(38.0) | NFE Backward 204(208.8)
Iter 1375 | Time 2.8333(2.8924) | Loss 4.467327(4.451865) | NFE Forward 38(38.0) | NFE Backward 204(208.5)
Iter 1376 | Time 3.0061(2.9004) | Loss 4.456584(4.452195) | NFE Forward 38(38.0) | NFE Backward 222(209.4)
Iter 1377 | Time 2.8284(2.8953) | Loss 4.409074(4.449177) | NFE Forward 38(38.0) | NFE Backward 204(209.0)
Iter 1378 | Time 2.8290(2.8907) | Loss 4.432984(4.448043) | NFE Forward 38(38.0) | NFE Backward 204(208.7)
Iter 1379 | Time 2.8278(2.8863) | Loss 4.470223(4.449596) | NFE Forward 38(38.0) | NFE Backward 204(208.4)
Iter 1380 | Time 2.8263(2.8821) | Loss 4.484114(4.452012) | NFE Forward 38(38.0) | NFE Backward 204(208.1)
Iter 1381 | Time 2.9442(2.8864) | Loss 4.436295(4.450912) | NFE Forward 38(38.0) | NFE Backward 216(208.6)
Iter 1382 | Time 2.9669(2.8921) | Loss 4.501871(4.454479) | NFE Forward 38(38.0) | NFE Backward 216(209.1)
Iter 1383 | Time 2.8956(2.8923) | Loss 4.485952(4.456682) | NFE Forward 38(38.0) | NFE Backward 210(209.2)
Iter 1384 | Time 2.8934(2.8924) | Loss 4.480691(4.458363) | NFE Forward 38(38.0) | NFE Backward 210(209.2)
Iter 1385 | Time 2.8379(2.8886) | Loss 4.445169(4.457439) | NFE Forward 38(38.0) | NFE Backward 204(208.9)
Iter 1386 | Time 2.8373(2.8850) | Loss 4.455575(4.457309) | NFE Forward 38(38.0) | NFE Backward 204(208.5)
Iter 1387 | Time 2.9591(2.8902) | Loss 4.429711(4.455377) | NFE Forward 38(38.0) | NFE Backward 216(209.1)
Iter 1388 | Time 3.0196(2.8992) | Loss 4.455823(4.455408) | NFE Forward 38(38.0) | NFE Backward 222(210.0)
Iter 1389 | Time 2.9269(2.9012) | Loss 4.451945(4.455166) | NFE Forward 38(38.0) | NFE Backward 216(210.4)
Iter 1390 | Time 2.8409(2.8969) | Loss 4.426617(4.453167) | NFE Forward 38(38.0) | NFE Backward 204(209.9)
Iter 1391 | Time 2.8406(2.8930) | Loss 4.440930(4.452311) | NFE Forward 38(38.0) | NFE Backward 204(209.5)
Iter 1392 | Time 2.9604(2.8977) | Loss 4.487106(4.454746) | NFE Forward 38(38.0) | NFE Backward 216(210.0)
Iter 1393 | Time 2.9832(2.9037) | Loss 4.456696(4.454883) | NFE Forward 38(38.0) | NFE Backward 216(210.4)
Iter 1394 | Time 3.0190(2.9118) | Loss 4.429554(4.453110) | NFE Forward 38(38.0) | NFE Backward 222(211.2)
Iter 1395 | Time 2.8494(2.9074) | Loss 4.488279(4.455572) | NFE Forward 38(38.0) | NFE Backward 204(210.7)
Iter 1396 | Time 2.8925(2.9064) | Loss 4.414426(4.452691) | NFE Forward 38(38.0) | NFE Backward 204(210.2)
Iter 1397 | Time 2.9622(2.9103) | Loss 4.460518(4.453239) | NFE Forward 38(38.0) | NFE Backward 216(210.6)
Iter 1398 | Time 2.9611(2.9138) | Loss 4.447765(4.452856) | NFE Forward 38(38.0) | NFE Backward 216(211.0)
Iter 1399 | Time 2.9618(2.9172) | Loss 4.452477(4.452830) | NFE Forward 38(38.0) | NFE Backward 216(211.4)
Iter 1400 | Time 2.8487(2.9124) | Loss 4.421512(4.450637) | NFE Forward 38(38.0) | NFE Backward 204(210.9)
[TEST] Iter 1400 | Test Loss 4.514085 | NFE 38
Skipping vis as data dimension is >2
Iter 1401 | Time 2.8997(2.9115) | Loss 4.417106(4.448290) | NFE Forward 38(38.0) | NFE Backward 210(210.8)
Iter 1402 | Time 2.8431(2.9067) | Loss 4.462907(4.449313) | NFE Forward 38(38.0) | NFE Backward 204(210.3)
Iter 1403 | Time 2.9972(2.9131) | Loss 4.464347(4.450366) | NFE Forward 38(38.0) | NFE Backward 216(210.7)
Iter 1404 | Time 2.9865(2.9182) | Loss 4.469440(4.451701) | NFE Forward 38(38.0) | NFE Backward 216(211.1)
Iter 1405 | Time 2.8492(2.9134) | Loss 4.499845(4.455071) | NFE Forward 38(38.0) | NFE Backward 204(210.6)
Iter 1406 | Time 2.8439(2.9085) | Loss 4.453993(4.454996) | NFE Forward 38(38.0) | NFE Backward 204(210.1)
Iter 1407 | Time 2.8468(2.9042) | Loss 4.481725(4.456867) | NFE Forward 38(38.0) | NFE Backward 204(209.7)
Iter 1408 | Time 2.8519(2.9005) | Loss 4.442870(4.455887) | NFE Forward 38(38.0) | NFE Backward 204(209.3)
Iter 1409 | Time 2.9604(2.9047) | Loss 4.466650(4.456640) | NFE Forward 38(38.0) | NFE Backward 216(209.8)
Iter 1410 | Time 2.9648(2.9089) | Loss 4.458744(4.456787) | NFE Forward 38(38.0) | NFE Backward 216(210.2)
Iter 1411 | Time 2.8422(2.9043) | Loss 4.435250(4.455280) | NFE Forward 38(38.0) | NFE Backward 204(209.8)
Iter 1412 | Time 2.8437(2.9000) | Loss 4.439481(4.454174) | NFE Forward 38(38.0) | NFE Backward 204(209.4)
Iter 1413 | Time 2.8578(2.8971) | Loss 4.450525(4.453919) | NFE Forward 38(38.0) | NFE Backward 204(209.0)
Iter 1414 | Time 2.9649(2.9018) | Loss 4.484144(4.456034) | NFE Forward 38(38.0) | NFE Backward 216(209.5)
Iter 1415 | Time 2.9626(2.9061) | Loss 4.453807(4.455878) | NFE Forward 38(38.0) | NFE Backward 216(209.9)
Iter 1416 | Time 2.8391(2.9014) | Loss 4.414805(4.453003) | NFE Forward 38(38.0) | NFE Backward 204(209.5)
Iter 1417 | Time 2.8432(2.8973) | Loss 4.469234(4.454139) | NFE Forward 38(38.0) | NFE Backward 204(209.1)
Iter 1418 | Time 2.8385(2.8932) | Loss 4.457281(4.454359) | NFE Forward 38(38.0) | NFE Backward 204(208.8)
Iter 1419 | Time 2.9587(2.8978) | Loss 4.437971(4.453212) | NFE Forward 38(38.0) | NFE Backward 216(209.3)
Iter 1420 | Time 2.9607(2.9022) | Loss 4.427257(4.451395) | NFE Forward 38(38.0) | NFE Backward 216(209.8)
Iter 1421 | Time 2.9621(2.9064) | Loss 4.440657(4.450644) | NFE Forward 38(38.0) | NFE Backward 216(210.2)
Iter 1422 | Time 2.8977(2.9058) | Loss 4.418939(4.448424) | NFE Forward 38(38.0) | NFE Backward 210(210.2)
Iter 1423 | Time 2.9790(2.9109) | Loss 4.492350(4.451499) | NFE Forward 38(38.0) | NFE Backward 216(210.6)
Iter 1424 | Time 2.8394(2.9059) | Loss 4.478988(4.453423) | NFE Forward 38(38.0) | NFE Backward 204(210.1)
Iter 1425 | Time 2.8395(2.9012) | Loss 4.452015(4.453325) | NFE Forward 38(38.0) | NFE Backward 204(209.7)
Iter 1426 | Time 2.8153(2.8952) | Loss 4.409343(4.450246) | NFE Forward 38(38.0) | NFE Backward 204(209.3)
Iter 1427 | Time 2.8412(2.8915) | Loss 4.430313(4.448851) | NFE Forward 38(38.0) | NFE Backward 204(208.9)
Iter 1428 | Time 2.8429(2.8881) | Loss 4.465786(4.450036) | NFE Forward 38(38.0) | NFE Backward 204(208.6)
Iter 1429 | Time 2.8638(2.8864) | Loss 4.464273(4.451033) | NFE Forward 38(38.0) | NFE Backward 204(208.3)
Iter 1430 | Time 2.8546(2.8841) | Loss 4.425768(4.449264) | NFE Forward 38(38.0) | NFE Backward 204(208.0)
Iter 1431 | Time 2.8541(2.8820) | Loss 4.430360(4.447941) | NFE Forward 38(38.0) | NFE Backward 204(207.7)
Iter 1432 | Time 2.8495(2.8798) | Loss 4.475595(4.449877) | NFE Forward 38(38.0) | NFE Backward 204(207.4)
Iter 1433 | Time 2.8670(2.8789) | Loss 4.458801(4.450501) | NFE Forward 38(38.0) | NFE Backward 204(207.2)
Iter 1434 | Time 2.8760(2.8787) | Loss 4.389606(4.446239) | NFE Forward 38(38.0) | NFE Backward 204(207.0)
Iter 1435 | Time 2.8695(2.8780) | Loss 4.469299(4.447853) | NFE Forward 38(38.0) | NFE Backward 204(206.8)
Iter 1436 | Time 2.8654(2.8771) | Loss 4.463798(4.448969) | NFE Forward 38(38.0) | NFE Backward 204(206.6)
Iter 1437 | Time 2.8646(2.8763) | Loss 4.462757(4.449934) | NFE Forward 38(38.0) | NFE Backward 204(206.4)
Iter 1438 | Time 2.8667(2.8756) | Loss 4.466302(4.451080) | NFE Forward 38(38.0) | NFE Backward 204(206.2)
Iter 1439 | Time 2.9840(2.8832) | Loss 4.460502(4.451739) | NFE Forward 38(38.0) | NFE Backward 216(206.9)
Iter 1440 | Time 2.8667(2.8820) | Loss 4.467012(4.452809) | NFE Forward 38(38.0) | NFE Backward 204(206.7)
Iter 1441 | Time 2.8641(2.8808) | Loss 4.426937(4.450998) | NFE Forward 38(38.0) | NFE Backward 204(206.5)
Iter 1442 | Time 2.8665(2.8798) | Loss 4.476233(4.452764) | NFE Forward 38(38.0) | NFE Backward 204(206.3)
Iter 1443 | Time 2.8638(2.8787) | Loss 4.466300(4.453712) | NFE Forward 38(38.0) | NFE Backward 204(206.2)
Iter 1444 | Time 2.8694(2.8780) | Loss 4.470178(4.454864) | NFE Forward 38(38.0) | NFE Backward 204(206.0)
Iter 1445 | Time 2.9879(2.8857) | Loss 4.452192(4.454677) | NFE Forward 38(38.0) | NFE Backward 216(206.7)
Iter 1446 | Time 2.8637(2.8842) | Loss 4.409022(4.451481) | NFE Forward 38(38.0) | NFE Backward 204(206.5)
Iter 1447 | Time 2.8619(2.8826) | Loss 4.435738(4.450379) | NFE Forward 38(38.0) | NFE Backward 204(206.4)
Iter 1448 | Time 2.8635(2.8813) | Loss 4.399740(4.446834) | NFE Forward 38(38.0) | NFE Backward 204(206.2)
Iter 1449 | Time 2.8693(2.8804) | Loss 4.430775(4.445710) | NFE Forward 38(38.0) | NFE Backward 204(206.0)
Iter 1450 | Time 2.8719(2.8798) | Loss 4.431605(4.444723) | NFE Forward 38(38.0) | NFE Backward 204(205.9)
Iter 1451 | Time 2.8708(2.8792) | Loss 4.457736(4.445634) | NFE Forward 38(38.0) | NFE Backward 204(205.8)
Iter 1452 | Time 2.8684(2.8784) | Loss 4.467620(4.447173) | NFE Forward 38(38.0) | NFE Backward 204(205.6)
Iter 1453 | Time 2.8648(2.8775) | Loss 4.497410(4.450689) | NFE Forward 38(38.0) | NFE Backward 204(205.5)
Iter 1454 | Time 2.9892(2.8853) | Loss 4.448784(4.450556) | NFE Forward 38(38.0) | NFE Backward 216(206.3)
Iter 1455 | Time 2.8786(2.8848) | Loss 4.463421(4.451457) | NFE Forward 38(38.0) | NFE Backward 204(206.1)
Iter 1456 | Time 2.9885(2.8921) | Loss 4.413508(4.448800) | NFE Forward 38(38.0) | NFE Backward 216(206.8)
Iter 1457 | Time 2.8674(2.8904) | Loss 4.448862(4.448804) | NFE Forward 38(38.0) | NFE Backward 204(206.6)
Iter 1458 | Time 2.9843(2.8969) | Loss 4.459159(4.449529) | NFE Forward 38(38.0) | NFE Backward 216(207.3)
Iter 1459 | Time 2.8677(2.8949) | Loss 4.446946(4.449348) | NFE Forward 38(38.0) | NFE Backward 204(207.0)
Iter 1460 | Time 2.8645(2.8928) | Loss 4.423105(4.447511) | NFE Forward 38(38.0) | NFE Backward 204(206.8)
Iter 1461 | Time 2.8644(2.8908) | Loss 4.462525(4.448562) | NFE Forward 38(38.0) | NFE Backward 204(206.6)
Iter 1462 | Time 2.8717(2.8894) | Loss 4.436616(4.447726) | NFE Forward 38(38.0) | NFE Backward 204(206.4)
Iter 1463 | Time 2.8628(2.8876) | Loss 4.438848(4.447105) | NFE Forward 38(38.0) | NFE Backward 204(206.3)
Iter 1464 | Time 2.8725(2.8865) | Loss 4.466290(4.448448) | NFE Forward 38(38.0) | NFE Backward 204(206.1)
Iter 1465 | Time 2.8759(2.8858) | Loss 4.466688(4.449724) | NFE Forward 38(38.0) | NFE Backward 204(206.0)
Iter 1466 | Time 2.9954(2.8935) | Loss 4.459718(4.450424) | NFE Forward 38(38.0) | NFE Backward 216(206.7)
Iter 1467 | Time 2.8591(2.8911) | Loss 4.430160(4.449005) | NFE Forward 38(38.0) | NFE Backward 204(206.5)
Iter 1468 | Time 2.8565(2.8886) | Loss 4.420272(4.446994) | NFE Forward 38(38.0) | NFE Backward 204(206.3)
Iter 1469 | Time 2.8579(2.8865) | Loss 4.468308(4.448486) | NFE Forward 38(38.0) | NFE Backward 204(206.1)
Iter 1470 | Time 2.9767(2.8928) | Loss 4.477847(4.450541) | NFE Forward 38(38.0) | NFE Backward 216(206.8)
Iter 1471 | Time 2.8565(2.8903) | Loss 4.447074(4.450299) | NFE Forward 38(38.0) | NFE Backward 204(206.6)
Iter 1472 | Time 2.8547(2.8878) | Loss 4.441833(4.449706) | NFE Forward 38(38.0) | NFE Backward 204(206.4)
Iter 1473 | Time 2.9383(2.8913) | Loss 4.518209(4.454501) | NFE Forward 38(38.0) | NFE Backward 210(206.7)
Iter 1474 | Time 2.8513(2.8885) | Loss 4.469804(4.455573) | NFE Forward 38(38.0) | NFE Backward 204(206.5)
Iter 1475 | Time 2.8394(2.8851) | Loss 4.439536(4.454450) | NFE Forward 38(38.0) | NFE Backward 204(206.3)
Iter 1476 | Time 2.9697(2.8910) | Loss 4.444077(4.453724) | NFE Forward 38(38.0) | NFE Backward 216(207.0)
Iter 1477 | Time 2.8410(2.8875) | Loss 4.415271(4.451032) | NFE Forward 38(38.0) | NFE Backward 204(206.8)
Iter 1478 | Time 2.8482(2.8847) | Loss 4.449872(4.450951) | NFE Forward 38(38.0) | NFE Backward 204(206.6)
Iter 1479 | Time 2.7864(2.8779) | Loss 4.403006(4.447595) | NFE Forward 38(38.0) | NFE Backward 198(206.0)
Iter 1480 | Time 2.9560(2.8833) | Loss 4.455084(4.448119) | NFE Forward 38(38.0) | NFE Backward 216(206.7)
Iter 1481 | Time 2.8337(2.8799) | Loss 4.411935(4.445586) | NFE Forward 38(38.0) | NFE Backward 204(206.5)
Iter 1482 | Time 2.8482(2.8776) | Loss 4.439410(4.445154) | NFE Forward 38(38.0) | NFE Backward 204(206.3)
Iter 1483 | Time 2.9684(2.8840) | Loss 4.433575(4.444343) | NFE Forward 38(38.0) | NFE Backward 216(207.0)
Iter 1484 | Time 2.8437(2.8812) | Loss 4.414821(4.442277) | NFE Forward 38(38.0) | NFE Backward 204(206.8)
Iter 1485 | Time 2.8715(2.8805) | Loss 4.431736(4.441539) | NFE Forward 38(38.0) | NFE Backward 204(206.6)
Iter 1486 | Time 2.9903(2.8882) | Loss 4.442305(4.441593) | NFE Forward 38(38.0) | NFE Backward 216(207.3)
Iter 1487 | Time 2.8714(2.8870) | Loss 4.407357(4.439196) | NFE Forward 38(38.0) | NFE Backward 204(207.0)
Iter 1488 | Time 2.8664(2.8856) | Loss 4.431455(4.438654) | NFE Forward 38(38.0) | NFE Backward 204(206.8)
Iter 1489 | Time 2.8713(2.8846) | Loss 4.414284(4.436948) | NFE Forward 38(38.0) | NFE Backward 204(206.6)
Iter 1490 | Time 2.8708(2.8836) | Loss 4.452957(4.438069) | NFE Forward 38(38.0) | NFE Backward 204(206.4)
Iter 1491 | Time 2.9924(2.8912) | Loss 4.414430(4.436414) | NFE Forward 38(38.0) | NFE Backward 216(207.1)
Iter 1492 | Time 2.9889(2.8981) | Loss 4.386981(4.432954) | NFE Forward 38(38.0) | NFE Backward 216(207.7)
Iter 1493 | Time 2.8504(2.8947) | Loss 4.420849(4.432106) | NFE Forward 38(38.0) | NFE Backward 204(207.5)
Iter 1494 | Time 2.8422(2.8910) | Loss 4.448108(4.433226) | NFE Forward 38(38.0) | NFE Backward 204(207.2)
Iter 1495 | Time 2.8421(2.8876) | Loss 4.438719(4.433611) | NFE Forward 38(38.0) | NFE Backward 204(207.0)
Iter 1496 | Time 2.8571(2.8855) | Loss 4.400085(4.431264) | NFE Forward 38(38.0) | NFE Backward 204(206.8)
Iter 1497 | Time 2.9096(2.8872) | Loss 4.443277(4.432105) | NFE Forward 38(38.0) | NFE Backward 210(207.0)
Iter 1498 | Time 2.8399(2.8839) | Loss 4.467776(4.434602) | NFE Forward 38(38.0) | NFE Backward 204(206.8)
Density Loss 0.47105681896209717
Density Loss 0.4652237296104431
Density Loss 0.46690449118614197
Density Loss 0.4775720536708832
Density Loss 0.4667896330356598
Density Loss 0.46735700964927673
Density Loss 0.48338332772254944
Density Loss 0.47832489013671875
Density Loss 0.472444623708725
Density Loss 0.46874183416366577
Density Loss 0.46667689085006714
Density Loss 0.462316632270813
Density Loss 0.4870871901512146
Density Loss 0.4481401741504669
Density Loss 0.4793243110179901
Density Loss 0.4830549657344818
Density Loss 0.47337058186531067
Density Loss 0.47296127676963806
Density Loss 0.4780517816543579
Density Loss 0.4843795895576477
Density Loss 0.4835936427116394
Density Loss 0.46794822812080383
Density Loss 0.4653604328632355
Density Loss 0.463851660490036
Density Loss 0.45709335803985596
Density Loss 0.4844958782196045
Density Loss 0.4597161114215851
Density Loss 0.4737776815891266
Density Loss 0.4531027674674988
Density Loss 0.4630637466907501
Density Loss 0.47804656624794006
Density Loss 0.46338963508605957
Density Loss 0.4664900302886963
Density Loss 0.4716891348361969
Density Loss 0.463875412940979
Density Loss 0.48140937089920044
Density Loss 0.4688482880592346
Density Loss 0.47525355219841003
Density Loss 0.4726364016532898
Density Loss 0.4588051736354828
Density Loss 0.47785258293151855
Density Loss 0.4648193120956421
Density Loss 0.48455730080604553
Density Loss 0.487295001745224
Density Loss 0.47836825251579285
Density Loss 0.47560614347457886
Density Loss 0.4721468687057495
Density Loss 0.46924036741256714
Density Loss 0.4614878296852112
Density Loss 0.4708843529224396
Density Loss 0.472723126411438
Density Loss 0.4838908612728119
Density Loss 0.4698618948459625
Density Loss 0.46511542797088623
Density Loss 0.4707304537296295
Density Loss 0.45784276723861694
Density Loss 0.47759169340133667
Density Loss 0.4845808446407318
Density Loss 0.46241337060928345
Density Loss 0.4834180474281311
Density Loss 0.48505038022994995
Density Loss 0.4647437036037445
Density Loss 0.4726004898548126
Density Loss 0.4761083722114563
Density Loss 0.4551854431629181
Density Loss 0.4638514518737793
Density Loss 0.46439236402511597
Density Loss 0.4613115191459656
Density Loss 0.4665128290653229
Density Loss 0.46715623140335083
Density Loss 0.47116711735725403
Density Loss 0.4913933575153351
Density Loss 0.46633490920066833
Density Loss 0.48273059725761414
Density Loss 0.4709407091140747
Density Loss 0.48649588227272034
Density Loss 0.4840378165245056
Density Loss 0.47838592529296875
Density Loss 0.4630143344402313
Density Loss 0.4748290479183197
Density Loss 0.4704382121562958
Density Loss 0.48750418424606323
Density Loss 0.48908787965774536
Density Loss 0.48561447858810425
Density Loss 0.4715576171875
Density Loss 0.48014020919799805
Density Loss 0.48705148696899414
Density Loss 0.4646832048892975
Density Loss 0.4628850519657135
Density Loss 0.46927163004875183
Density Loss 0.4847419857978821
Density Loss 0.46554285287857056
Density Loss 0.46872812509536743
Density Loss 0.49652737379074097
Density Loss 0.4683610200881958
Density Loss 0.48206424713134766
Density Loss 0.4892667829990387
Density Loss 0.46981149911880493
Density Loss 0.471244215965271
Density Loss 0.4868960678577423
Density Loss 0.47011756896972656
Density Loss 0.4768964648246765
Density Loss 0.4658571183681488
Density Loss 0.471183717250824
Density Loss 0.47388654947280884
Density Loss 0.470709890127182
Density Loss 0.45954519510269165
Density Loss 0.47834140062332153
Density Loss 0.4667600393295288
Density Loss 0.4945855438709259
Density Loss 0.46247294545173645
Density Loss 0.4553978443145752
Density Loss 0.45140761137008667
Density Loss 0.46456101536750793
Density Loss 0.4812720715999603
Density Loss 0.4838506281375885
Density Loss 0.47239500284194946
Density Loss 0.4761625826358795
Density Loss 0.477516233921051
Density Loss 0.4618105888366699
Density Loss 0.47365325689315796
Density Loss 0.4807560443878174
Density Loss 0.47785115242004395
Density Loss 0.480119913816452
Density Loss 0.49142733216285706
Density Loss 0.4652777314186096
Density Loss 0.4865970015525818
Density Loss 0.4504205882549286
Density Loss 0.4842342734336853
Density Loss 0.47841814160346985
Density Loss 0.4946427047252655
Density Loss 0.48412302136421204
Density Loss 0.48842889070510864
Density Loss 0.46852225065231323
Density Loss 0.4666716754436493
Density Loss 0.4720368981361389
Density Loss 0.4693606197834015
Density Loss 0.45960700511932373
Density Loss 0.4760323166847229
Density Loss 0.4760885238647461
Density Loss 0.4589863121509552
Density Loss 0.4739832282066345
Density Loss 0.4859234392642975
Density Loss 0.4677797257900238
Density Loss 0.47519442439079285
Density Loss 0.48611193895339966
Density Loss 0.47324138879776
Density Loss 0.47154518961906433
Density Loss 0.5049951076507568
Density Loss 0.47578978538513184
Density Loss 0.4949634075164795
Density Loss 0.48450779914855957
Density Loss 0.48500531911849976
Density Loss 0.46748679876327515
Density Loss 0.4848233163356781
Density Loss 0.47969284653663635
Density Loss 0.4888790547847748
Density Loss 0.47578710317611694
Density Loss 0.47654569149017334
Density Loss 0.47710517048835754
Density Loss 0.49013689160346985
Density Loss 0.4799199104309082
Density Loss 0.4793211817741394
Density Loss 0.48343679308891296
Density Loss 0.4658697247505188
Density Loss 0.47513318061828613
Density Loss 0.4666541814804077
Density Loss 0.47886794805526733
Density Loss 0.49857908487319946
Density Loss 0.47083789110183716
Density Loss 0.4716235101222992
Density Loss 0.46815061569213867
Density Loss 0.4716040790081024
Density Loss 0.4798519015312195
Density Loss 0.4783514142036438
Density Loss 0.49119389057159424
Density Loss 0.48546913266181946
Density Loss 0.4904024302959442
Density Loss 0.4905872046947479
Density Loss 0.47951000928878784
Density Loss 0.4830824136734009
Density Loss 0.46474775671958923
Density Loss 0.4674343764781952
Density Loss 0.4813029170036316
Density Loss 0.46456483006477356
Density Loss 0.47318539023399353
Density Loss 0.4764916002750397
Density Loss 0.4834735095500946
Density Loss 0.47202005982398987
Density Loss 0.47505390644073486
Density Loss 0.48520785570144653
Density Loss 0.4807536005973816
Density Loss 0.48336413502693176
Density Loss 0.4703138470649719
Density Loss 0.48058944940567017
Density Loss 0.46975642442703247
Density Loss 0.48195555806159973
Density Loss 0.4832395315170288
Density Loss 0.46883484721183777
Density Loss 0.48691296577453613
Density Loss 0.4934527277946472
Density Loss 0.4821171164512634
Density Loss 0.4791378676891327
Density Loss 0.4829532206058502
Density Loss 0.48387619853019714
Density Loss 0.48863327503204346
Density Loss 0.4700872302055359
Density Loss 0.48563700914382935
Density Loss 0.4782537817955017
Density Loss 0.48382097482681274
Density Loss 0.4776213765144348
Density Loss 0.4775595963001251
Density Loss 0.4850422143936157
Density Loss 0.4711836874485016
Density Loss 0.47125595808029175
Density Loss 0.48273876309394836
Density Loss 0.48350849747657776
Density Loss 0.4803708493709564
Density Loss 0.477222204208374
Density Loss 0.47145789861679077
Density Loss 0.5097503662109375
Density Loss 0.4858149290084839
Density Loss 0.4866052269935608
Density Loss 0.4847666919231415
Density Loss 0.4746406078338623
Density Loss 0.4742826521396637
Density Loss 0.48624539375305176
Density Loss 0.47740232944488525
Density Loss 0.48378095030784607
Density Loss 0.47848957777023315
Density Loss 0.48382946848869324
Density Loss 0.4708655774593353
Density Loss 0.45614051818847656
Density Loss 0.49412763118743896
Density Loss 0.47195595502853394
Density Loss 0.48282137513160706
Density Loss 0.4716576039791107
Density Loss 0.48046454787254333
Density Loss 0.4804708957672119
Density Loss 0.48005056381225586
Density Loss 0.4814135730266571
Density Loss 0.48329177498817444
Density Loss 0.48611965775489807
Density Loss 0.4851026237010956
Density Loss 0.4934585690498352
Density Loss 0.46289682388305664
Density Loss 0.49115321040153503
Density Loss 0.49850618839263916
Density Loss 0.5028679370880127
Density Loss 0.48708245158195496
Density Loss 0.4908502399921417
Density Loss 0.4801620543003082
Density Loss 0.49972298741340637
Iter 1499 | Time 2.8425(2.8810) | Loss 4.435004(4.434630) | NFE Forward 38(38.0) | NFE Backward 204(206.6)
Iter 1500 | Time 2.9024(2.8825) | Loss 4.429532(4.434273) | NFE Forward 38(38.0) | NFE Backward 210(206.8)
[TEST] Iter 1500 | Test Loss 4.449313 | NFE 38
Skipping vis as data dimension is >2
Iter 1501 | Time 2.7807(2.8753) | Loss 4.484761(4.437807) | NFE Forward 38(38.0) | NFE Backward 198(206.2)
Iter 1502 | Time 2.8421(2.8730) | Loss 4.429320(4.437213) | NFE Forward 38(38.0) | NFE Backward 204(206.1)
Iter 1503 | Time 2.9038(2.8752) | Loss 4.431491(4.436813) | NFE Forward 38(38.0) | NFE Backward 210(206.3)
Iter 1504 | Time 2.9021(2.8771) | Loss 4.453099(4.437953) | NFE Forward 38(38.0) | NFE Backward 210(206.6)
Iter 1505 | Time 2.8401(2.8745) | Loss 4.458953(4.439423) | NFE Forward 38(38.0) | NFE Backward 204(206.4)
Iter 1506 | Time 2.9664(2.8809) | Loss 4.426262(4.438502) | NFE Forward 38(38.0) | NFE Backward 216(207.1)
Iter 1507 | Time 2.8467(2.8785) | Loss 4.412311(4.436668) | NFE Forward 38(38.0) | NFE Backward 204(206.9)
Iter 1508 | Time 2.8434(2.8761) | Loss 4.420503(4.435537) | NFE Forward 38(38.0) | NFE Backward 204(206.7)
Iter 1509 | Time 2.9600(2.8819) | Loss 4.423814(4.434716) | NFE Forward 38(38.0) | NFE Backward 216(207.3)
Iter 1510 | Time 2.8419(2.8791) | Loss 4.461096(4.436563) | NFE Forward 38(38.0) | NFE Backward 204(207.1)
Iter 1511 | Time 2.8482(2.8770) | Loss 4.403235(4.434230) | NFE Forward 38(38.0) | NFE Backward 204(206.9)
Iter 1512 | Time 2.8573(2.8756) | Loss 4.449172(4.435276) | NFE Forward 38(38.0) | NFE Backward 204(206.7)
Iter 1513 | Time 2.8420(2.8732) | Loss 4.443044(4.435819) | NFE Forward 38(38.0) | NFE Backward 204(206.5)
Iter 1514 | Time 2.8597(2.8723) | Loss 4.404656(4.433638) | NFE Forward 38(38.0) | NFE Backward 204(206.3)
Iter 1515 | Time 2.9164(2.8754) | Loss 4.399328(4.431236) | NFE Forward 38(38.0) | NFE Backward 210(206.6)
Iter 1516 | Time 2.8542(2.8739) | Loss 4.421424(4.430549) | NFE Forward 38(38.0) | NFE Backward 204(206.4)
Iter 1517 | Time 2.9323(2.8780) | Loss 4.441412(4.431310) | NFE Forward 38(38.0) | NFE Backward 210(206.6)
Iter 1518 | Time 2.9277(2.8815) | Loss 4.415034(4.430171) | NFE Forward 38(38.0) | NFE Backward 210(206.9)
Iter 1519 | Time 2.9286(2.8848) | Loss 4.448340(4.431442) | NFE Forward 38(38.0) | NFE Backward 210(207.1)
Iter 1520 | Time 2.9300(2.8879) | Loss 4.471436(4.434242) | NFE Forward 38(38.0) | NFE Backward 210(207.3)
Iter 1521 | Time 2.9317(2.8910) | Loss 4.420775(4.433299) | NFE Forward 38(38.0) | NFE Backward 210(207.5)
Iter 1522 | Time 2.9258(2.8934) | Loss 4.408154(4.431539) | NFE Forward 38(38.0) | NFE Backward 210(207.7)
Iter 1523 | Time 2.9835(2.8997) | Loss 4.437277(4.431941) | NFE Forward 38(38.0) | NFE Backward 216(208.2)
Iter 1524 | Time 2.9265(2.9016) | Loss 4.452131(4.433354) | NFE Forward 38(38.0) | NFE Backward 210(208.4)
Iter 1525 | Time 2.9278(2.9034) | Loss 4.435479(4.433503) | NFE Forward 38(38.0) | NFE Backward 210(208.5)
Iter 1526 | Time 2.9298(2.9053) | Loss 4.458641(4.435262) | NFE Forward 38(38.0) | NFE Backward 210(208.6)
Iter 1527 | Time 2.9319(2.9071) | Loss 4.465658(4.437390) | NFE Forward 38(38.0) | NFE Backward 210(208.7)
Iter 1528 | Time 2.9369(2.9092) | Loss 4.386899(4.433856) | NFE Forward 38(38.0) | NFE Backward 210(208.8)
Iter 1529 | Time 2.8709(2.9065) | Loss 4.449911(4.434980) | NFE Forward 38(38.0) | NFE Backward 204(208.4)
Density Loss 0.4930087625980377
Density Loss 0.47550711035728455
Density Loss 0.47090306878089905
Density Loss 0.47726795077323914
Density Loss 0.48117825388908386
Density Loss 0.4699195325374603
Density Loss 0.482453316450119
Density Loss 0.4911080300807953
Density Loss 0.4705395996570587
Density Loss 0.4743882715702057
Density Loss 0.4944862127304077
Density Loss 0.4748810827732086
Density Loss 0.47647958993911743
Density Loss 0.4918229281902313
Density Loss 0.472005158662796
Density Loss 0.4888821244239807
Density Loss 0.49159446358680725
Density Loss 0.48889070749282837
Density Loss 0.4992266595363617
Density Loss 0.47909021377563477
Density Loss 0.47893425822257996
Density Loss 0.47865957021713257
Density Loss 0.49052298069000244
Density Loss 0.4865018427371979
Density Loss 0.47552165389060974
Density Loss 0.5045297145843506
Density Loss 0.4802846610546112
Density Loss 0.4665418863296509
Density Loss 0.4865705966949463
Density Loss 0.48320749402046204
Density Loss 0.4734364151954651
Density Loss 0.47404101490974426
Traceback (most recent call last):
  File "/data/bioinfo/TrajectoryNet/TrajectoryNet-master/TrajectoryNet/main.py", line 519, in <module>
    main(args)
  File "/data/bioinfo/TrajectoryNet/TrajectoryNet-master/TrajectoryNet/main.py", line 502, in main
    train(
  File "/data/bioinfo/TrajectoryNet/TrajectoryNet-master/TrajectoryNet/main.py", line 254, in train
    loss = compute_loss(device, args, model, growth_model, logger, full_data)
  File "/data/bioinfo/TrajectoryNet/TrajectoryNet-master/TrajectoryNet/main.py", line 197, in compute_loss
    int_x = model(x, integration_times=int_t)
  File "/opt/mamba/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/opt/mamba/lib/python3.10/site-packages/TrajectoryNet/lib/layers/container.py", line 21, in forward
    x = self.chain[i](x, reverse=reverse)
  File "/opt/mamba/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/opt/mamba/lib/python3.10/site-packages/TrajectoryNet/lib/layers/cnf.py", line 53, in forward
    state_t = odeint(
  File "/opt/mamba/lib/python3.10/site-packages/torchdiffeq/_impl/adjoint.py", line 129, in odeint_adjoint
    ys = OdeintAdjointMethod.apply(*y0, func, t, flat_params, rtol, atol, method, options)
  File "/opt/mamba/lib/python3.10/site-packages/torch/autograd/function.py", line 506, in apply
    return super().apply(*args, **kwargs)  # type: ignore[misc]
  File "/opt/mamba/lib/python3.10/site-packages/torchdiffeq/_impl/adjoint.py", line 18, in forward
    ans = odeint(func, y0, t, rtol=rtol, atol=atol, method=method, options=options)
  File "/opt/mamba/lib/python3.10/site-packages/torchdiffeq/_impl/odeint.py", line 76, in odeint
    solution = solver.integrate(t)
  File "/opt/mamba/lib/python3.10/site-packages/torchdiffeq/_impl/solvers.py", line 31, in integrate
    y = self.advance(t[i])
  File "/opt/mamba/lib/python3.10/site-packages/torchdiffeq/_impl/dopri5.py", line 90, in advance
    self.rk_state = self._adaptive_dopri5_step(self.rk_state)
KeyboardInterrupt
Error while terminating subprocess (pid=1550):
代码
文本
[20]
%%bash
cd ./TrajectoryNet-master/TrajectoryNet/
python eval.py --save ../results/fig8_results/ --dataset EB-PCA --top_k_reg 0.1 --training_noise 0.0 --max_dim 5

Warning: Clipping dimensionality to 5
integrating backwards
代码
文本

4.2 导入轨迹推断的结果

代码
文本
[15]
zs = np.load('./TrajectoryNet-master/results/fig8_results/backward_trajectories.npy')
代码
文本
[16]
zs.shape
(100, 3332, 5)
代码
文本

TrajectoryNet的输出 zs 是一个张量，对应于100个时间点，最后一个时间点有3332个细胞，每个细胞有5个主成分。为了使这些轨迹对应到PHATE嵌入，我们提取了存储在 phate_operator.graph.data_nu 中的主成分，并使用 StandardScaler 计算均值和标准差。然后，我们将 zs 缩放到前5个主成分的尺度和均值，并绘制从第24天到第27天的一些样本细胞的轨迹。

代码
文本
[17]
scaler = sklearn.preprocessing.StandardScaler()
scaler.fit(phate_operator.graph.data_nu)
StandardScaler()
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
代码
文本
[18]
phate_operator.graph.data_nu.shape
(16821, 100)
代码
文本
[19]
zss = zs * scaler.scale_[:5] + scaler.mean_[:5]
代码
文本
[20]
fig, ax = plt.subplots(1,1)
scprep.plot.scatter2d(phate_operator.graph.data_nu, c=sample_labels, figsize=(30,15), cmap="Spectral",
ticks=True, label_prefix="PC", ax=ax, title='Trajectories for 200 cells from last timepoint')

for i in range(200):
ax.plot(zss[:,i,0], zss[:,i,1])
代码
文本
[21]
trajectory_gene_space = np.dot(zss, phate_operator.graph.data_pca.components_[:5,:])
代码
文本
[22]
trajectory_gene_space.shape
(100, 3332, 17845)
代码
文本
[23]
genes = [x.split(' ')[0] for x in EBT_counts.columns]
代码
文本
[24]
eb_marker_genes = np.loadtxt('./TrajectoryNet-master/data/eb_genes.txt', dtype='str')
代码
文本
[25]
genes_index = {}
代码
文本
[26]
for gene in eb_marker_genes:
genes_index[gene] = genes.index(gene)
代码
文本
[27]
trajectory_eb = trajectory_gene_space[:,:,np.array(list(genes_index.values()))]
代码
文本
[28]
# np.save('./trajectory_eb.npy', trajectory_eb)
np.save('./TrajectoryNet-master/results/fig8_results/trajectory_eb.npy', trajectory_eb)

代码
文本
[29]
trajectory_eb_magic = np.zeros((100,3332,68))
代码
文本
[30]
m_op = magic.MAGIC()
for i in range(100):
trajectory_eb_magic[i,:,:] = m_op.fit_transform(trajectory_eb[i,:,:])
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.43 seconds.
    Calculating affinities...
    Calculated affinities in 0.44 seconds.
  Calculated graph and diffusion operator in 0.88 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.90 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.42 seconds.
    Calculating affinities...
    Calculated affinities in 0.43 seconds.
  Calculated graph and diffusion operator in 0.85 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.87 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.43 seconds.
    Calculating affinities...
    Calculated affinities in 0.44 seconds.
  Calculated graph and diffusion operator in 0.87 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.89 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.42 seconds.
    Calculating affinities...
    Calculated affinities in 0.43 seconds.
  Calculated graph and diffusion operator in 0.86 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.88 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.44 seconds.
    Calculating affinities...
    Calculated affinities in 0.44 seconds.
  Calculated graph and diffusion operator in 0.89 seconds.
  Calculating imputation...
  Calculated imputation in 0.01 seconds.
Calculated MAGIC in 0.91 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.42 seconds.
    Calculating affinities...
    Calculated affinities in 0.43 seconds.
  Calculated graph and diffusion operator in 0.86 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.89 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.42 seconds.
    Calculating affinities...
    Calculated affinities in 0.43 seconds.
  Calculated graph and diffusion operator in 0.87 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.89 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.42 seconds.
    Calculating affinities...
    Calculated affinities in 0.43 seconds.
  Calculated graph and diffusion operator in 0.86 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.88 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.42 seconds.
    Calculating affinities...
    Calculated affinities in 0.43 seconds.
  Calculated graph and diffusion operator in 0.86 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.88 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.42 seconds.
    Calculating affinities...
    Calculated affinities in 0.43 seconds.
  Calculated graph and diffusion operator in 0.86 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.88 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.43 seconds.
    Calculating affinities...
    Calculated affinities in 0.43 seconds.
  Calculated graph and diffusion operator in 0.87 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.89 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.42 seconds.
    Calculating affinities...
    Calculated affinities in 0.43 seconds.
  Calculated graph and diffusion operator in 0.86 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.88 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.43 seconds.
    Calculating affinities...
    Calculated affinities in 0.44 seconds.
  Calculated graph and diffusion operator in 0.88 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.90 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.42 seconds.
    Calculating affinities...
    Calculated affinities in 0.43 seconds.
  Calculated graph and diffusion operator in 0.86 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.88 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.42 seconds.
    Calculating affinities...
    Calculated affinities in 0.43 seconds.
  Calculated graph and diffusion operator in 0.86 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.88 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.42 seconds.
    Calculating affinities...
    Calculated affinities in 0.44 seconds.
  Calculated graph and diffusion operator in 0.86 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.89 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.42 seconds.
    Calculating affinities...
    Calculated affinities in 0.43 seconds.
  Calculated graph and diffusion operator in 0.86 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.88 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.42 seconds.
    Calculating affinities...
    Calculated affinities in 0.43 seconds.
  Calculated graph and diffusion operator in 0.86 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.88 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.43 seconds.
    Calculating affinities...
    Calculated affinities in 0.44 seconds.
  Calculated graph and diffusion operator in 0.88 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.90 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.43 seconds.
    Calculating affinities...
    Calculated affinities in 0.44 seconds.
  Calculated graph and diffusion operator in 0.87 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.89 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.43 seconds.
    Calculating affinities...
    Calculated affinities in 0.44 seconds.
  Calculated graph and diffusion operator in 0.88 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.90 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.43 seconds.
    Calculating affinities...
    Calculated affinities in 0.43 seconds.
  Calculated graph and diffusion operator in 0.87 seconds.
  Calculating imputation...
  Calculated imputation in 0.01 seconds.
Calculated MAGIC in 0.89 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.43 seconds.
    Calculating affinities...
    Calculated affinities in 0.44 seconds.
  Calculated graph and diffusion operator in 0.87 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.90 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.43 seconds.
    Calculating affinities...
    Calculated affinities in 0.44 seconds.
  Calculated graph and diffusion operator in 0.87 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.89 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.44 seconds.
    Calculating affinities...
    Calculated affinities in 0.44 seconds.
  Calculated graph and diffusion operator in 0.89 seconds.
  Calculating imputation...
  Calculated imputation in 0.01 seconds.
Calculated MAGIC in 0.91 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.43 seconds.
    Calculating affinities...
    Calculated affinities in 0.44 seconds.
  Calculated graph and diffusion operator in 0.87 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.89 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.44 seconds.
    Calculating affinities...
    Calculated affinities in 0.45 seconds.
  Calculated graph and diffusion operator in 0.89 seconds.
  Calculating imputation...
  Calculated imputation in 0.01 seconds.
Calculated MAGIC in 0.91 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.43 seconds.
    Calculating affinities...
    Calculated affinities in 0.44 seconds.
  Calculated graph and diffusion operator in 0.88 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.90 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.43 seconds.
    Calculating affinities...
    Calculated affinities in 0.44 seconds.
  Calculated graph and diffusion operator in 0.88 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.90 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.43 seconds.
    Calculating affinities...
    Calculated affinities in 0.44 seconds.
  Calculated graph and diffusion operator in 0.88 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.90 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.43 seconds.
    Calculating affinities...
    Calculated affinities in 0.44 seconds.
  Calculated graph and diffusion operator in 0.88 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.90 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.43 seconds.
    Calculating affinities...
    Calculated affinities in 0.44 seconds.
  Calculated graph and diffusion operator in 0.87 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.90 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.43 seconds.
    Calculating affinities...
    Calculated affinities in 0.44 seconds.
  Calculated graph and diffusion operator in 0.88 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.90 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.43 seconds.
    Calculating affinities...
    Calculated affinities in 0.44 seconds.
  Calculated graph and diffusion operator in 0.88 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.90 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.44 seconds.
    Calculating affinities...
    Calculated affinities in 0.45 seconds.
  Calculated graph and diffusion operator in 0.89 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.91 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.44 seconds.
    Calculating affinities...
    Calculated affinities in 0.45 seconds.
  Calculated graph and diffusion operator in 0.89 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.91 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.44 seconds.
    Calculating affinities...
    Calculated affinities in 0.44 seconds.
  Calculated graph and diffusion operator in 0.89 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.91 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.43 seconds.
    Calculating affinities...
    Calculated affinities in 0.44 seconds.
  Calculated graph and diffusion operator in 0.87 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.89 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.44 seconds.
    Calculating affinities...
    Calculated affinities in 0.44 seconds.
  Calculated graph and diffusion operator in 0.88 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.90 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.43 seconds.
    Calculating affinities...
    Calculated affinities in 0.44 seconds.
  Calculated graph and diffusion operator in 0.88 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.90 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.44 seconds.
    Calculating affinities...
    Calculated affinities in 0.45 seconds.
  Calculated graph and diffusion operator in 0.89 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.91 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.43 seconds.
    Calculating affinities...
    Calculated affinities in 0.44 seconds.
  Calculated graph and diffusion operator in 0.88 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.90 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.43 seconds.
    Calculating affinities...
    Calculated affinities in 0.44 seconds.
  Calculated graph and diffusion operator in 0.88 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.90 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.43 seconds.
    Calculating affinities...
    Calculated affinities in 0.44 seconds.
  Calculated graph and diffusion operator in 0.87 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.89 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.43 seconds.
    Calculating affinities...
    Calculated affinities in 0.44 seconds.
  Calculated graph and diffusion operator in 0.88 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.90 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.43 seconds.
    Calculating affinities...
    Calculated affinities in 0.44 seconds.
  Calculated graph and diffusion operator in 0.87 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.89 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.43 seconds.
    Calculating affinities...
    Calculated affinities in 0.44 seconds.
  Calculated graph and diffusion operator in 0.89 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.91 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.43 seconds.
    Calculating affinities...
    Calculated affinities in 0.44 seconds.
  Calculated graph and diffusion operator in 0.88 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.90 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.43 seconds.
    Calculating affinities...
    Calculated affinities in 0.44 seconds.
  Calculated graph and diffusion operator in 0.88 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.90 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.43 seconds.
    Calculating affinities...
    Calculated affinities in 0.43 seconds.
  Calculated graph and diffusion operator in 0.87 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.89 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.43 seconds.
    Calculating affinities...
    Calculated affinities in 0.44 seconds.
  Calculated graph and diffusion operator in 0.88 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.90 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.44 seconds.
    Calculating affinities...
    Calculated affinities in 0.45 seconds.
  Calculated graph and diffusion operator in 0.89 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.92 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.44 seconds.
    Calculating affinities...
    Calculated affinities in 0.44 seconds.
  Calculated graph and diffusion operator in 0.89 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.91 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.44 seconds.
    Calculating affinities...
    Calculated affinities in 0.44 seconds.
  Calculated graph and diffusion operator in 0.88 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.90 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.44 seconds.
    Calculating affinities...
    Calculated affinities in 0.45 seconds.
  Calculated graph and diffusion operator in 0.89 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.91 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.44 seconds.
    Calculating affinities...
    Calculated affinities in 0.45 seconds.
  Calculated graph and diffusion operator in 0.90 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.92 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.44 seconds.
    Calculating affinities...
    Calculated affinities in 0.45 seconds.
  Calculated graph and diffusion operator in 0.90 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.92 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.44 seconds.
    Calculating affinities...
    Calculated affinities in 0.45 seconds.
  Calculated graph and diffusion operator in 0.90 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.92 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.45 seconds.
    Calculating affinities...
    Calculated affinities in 0.45 seconds.
  Calculated graph and diffusion operator in 0.91 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.93 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.45 seconds.
    Calculating affinities...
    Calculated affinities in 0.45 seconds.
  Calculated graph and diffusion operator in 0.91 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.93 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.45 seconds.
    Calculating affinities...
    Calculated affinities in 0.46 seconds.
  Calculated graph and diffusion operator in 0.91 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.93 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.45 seconds.
    Calculating affinities...
    Calculated affinities in 0.46 seconds.
  Calculated graph and diffusion operator in 0.92 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.94 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.45 seconds.
    Calculating affinities...
    Calculated affinities in 0.46 seconds.
  Calculated graph and diffusion operator in 0.92 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.94 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.45 seconds.
    Calculating affinities...
    Calculated affinities in 0.45 seconds.
  Calculated graph and diffusion operator in 0.91 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.93 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.45 seconds.
    Calculating affinities...
    Calculated affinities in 0.45 seconds.
  Calculated graph and diffusion operator in 0.91 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.93 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.45 seconds.
    Calculating affinities...
    Calculated affinities in 0.46 seconds.
  Calculated graph and diffusion operator in 0.92 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.94 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.45 seconds.
    Calculating affinities...
    Calculated affinities in 0.46 seconds.
  Calculated graph and diffusion operator in 0.92 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.94 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.45 seconds.
    Calculating affinities...
    Calculated affinities in 0.46 seconds.
  Calculated graph and diffusion operator in 0.92 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.94 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.46 seconds.
    Calculating affinities...
    Calculated affinities in 0.47 seconds.
  Calculated graph and diffusion operator in 0.93 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.95 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.46 seconds.
    Calculating affinities...
    Calculated affinities in 0.46 seconds.
  Calculated graph and diffusion operator in 0.93 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.95 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.46 seconds.
    Calculating affinities...
    Calculated affinities in 0.47 seconds.
  Calculated graph and diffusion operator in 0.94 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.96 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.47 seconds.
    Calculating affinities...
    Calculated affinities in 0.48 seconds.
  Calculated graph and diffusion operator in 0.96 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.98 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.48 seconds.
    Calculating affinities...
    Calculated affinities in 0.50 seconds.
  Calculated graph and diffusion operator in 0.99 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 1.01 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.47 seconds.
    Calculating affinities...
    Calculated affinities in 0.48 seconds.
  Calculated graph and diffusion operator in 0.96 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.98 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.48 seconds.
    Calculating affinities...
    Calculated affinities in 0.48 seconds.
  Calculated graph and diffusion operator in 0.97 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.99 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.47 seconds.
    Calculating affinities...
    Calculated affinities in 0.48 seconds.
  Calculated graph and diffusion operator in 0.96 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.98 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.49 seconds.
    Calculating affinities...
    Calculated affinities in 0.49 seconds.
  Calculated graph and diffusion operator in 0.99 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 1.01 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.48 seconds.
    Calculating affinities...
    Calculated affinities in 0.49 seconds.
  Calculated graph and diffusion operator in 0.97 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 0.99 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.48 seconds.
    Calculating affinities...
    Calculated affinities in 0.49 seconds.
  Calculated graph and diffusion operator in 0.98 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 1.00 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.49 seconds.
    Calculating affinities...
    Calculated affinities in 0.49 seconds.
  Calculated graph and diffusion operator in 0.99 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 1.01 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.50 seconds.
    Calculating affinities...
    Calculated affinities in 0.51 seconds.
  Calculated graph and diffusion operator in 1.01 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 1.03 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.50 seconds.
    Calculating affinities...
    Calculated affinities in 0.51 seconds.
  Calculated graph and diffusion operator in 1.01 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 1.03 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.51 seconds.
    Calculating affinities...
    Calculated affinities in 0.51 seconds.
  Calculated graph and diffusion operator in 1.03 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 1.05 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.51 seconds.
    Calculating affinities...
    Calculated affinities in 0.52 seconds.
  Calculated graph and diffusion operator in 1.04 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 1.06 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.52 seconds.
    Calculating affinities...
    Calculated affinities in 0.53 seconds.
  Calculated graph and diffusion operator in 1.05 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 1.07 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.51 seconds.
    Calculating affinities...
    Calculated affinities in 0.52 seconds.
  Calculated graph and diffusion operator in 1.04 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 1.06 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.53 seconds.
    Calculating affinities...
    Calculated affinities in 0.54 seconds.
  Calculated graph and diffusion operator in 1.07 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 1.09 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.53 seconds.
    Calculating affinities...
    Calculated affinities in 0.54 seconds.
  Calculated graph and diffusion operator in 1.08 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 1.11 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.53 seconds.
    Calculating affinities...
    Calculated affinities in 0.54 seconds.
  Calculated graph and diffusion operator in 1.08 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 1.10 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.55 seconds.
    Calculating affinities...
    Calculated affinities in 0.56 seconds.
  Calculated graph and diffusion operator in 1.12 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 1.14 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.56 seconds.
    Calculating affinities...
    Calculated affinities in 0.57 seconds.
  Calculated graph and diffusion operator in 1.13 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 1.15 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.56 seconds.
    Calculating affinities...
    Calculated affinities in 0.57 seconds.
  Calculated graph and diffusion operator in 1.14 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 1.17 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.57 seconds.
    Calculating affinities...
    Calculated affinities in 0.58 seconds.
  Calculated graph and diffusion operator in 1.16 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 1.18 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.58 seconds.
    Calculating affinities...
    Calculated affinities in 0.58 seconds.
  Calculated graph and diffusion operator in 1.17 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 1.19 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.58 seconds.
    Calculating affinities...
    Calculated affinities in 0.59 seconds.
  Calculated graph and diffusion operator in 1.19 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 1.21 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.59 seconds.
    Calculating affinities...
    Calculated affinities in 0.60 seconds.
  Calculated graph and diffusion operator in 1.20 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 1.22 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.60 seconds.
    Calculating affinities...
    Calculated affinities in 0.61 seconds.
  Calculated graph and diffusion operator in 1.22 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 1.24 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.60 seconds.
    Calculating affinities...
    Calculated affinities in 0.61 seconds.
  Calculated graph and diffusion operator in 1.22 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 1.24 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.61 seconds.
    Calculating affinities...
    Calculated affinities in 0.62 seconds.
  Calculated graph and diffusion operator in 1.24 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 1.26 seconds.
Calculating MAGIC...
  Running MAGIC on 3332 cells and 68 genes.
  Calculating graph and diffusion operator...
    Calculating KNN search...
    Calculated KNN search in 0.61 seconds.
    Calculating affinities...
    Calculated affinities in 0.63 seconds.
  Calculated graph and diffusion operator in 1.25 seconds.
  Calculating imputation...
  Calculated imputation in 0.02 seconds.
Calculated MAGIC in 1.27 seconds.
代码
文本
[31]
# np.save('./trajectory_eb_magic.npy', trajectory_eb_magic)
np.save('./TrajectoryNet-master/results/trajectory_eb_magic.npy', trajectory_eb_magic)

代码
文本

4.3 针对感兴趣的基因进行探究

然后，我们在这个数据集中定义了代表4种不同细胞群体的“终点基因”。我们对这些基因的原始计数运行MAGIC，并可视化了估算的表达。

代码
文本
[32]
end_genes = ['PDGFRA ', 'HAND1', 'SOX17', 'ONECUT2', ]
end_points = ['Muscle', 'Cardiac', 'Endothelial', 'Neuronal',]

colors = dict(zip(*[end_genes, [plt.get_cmap('tab10')(i+1) for i in range(len(end_genes))]]))
代码
文本
[33]
other_genes = ['GATA6 ', 'SATB1', 'T ', 'EOMES', 'NANOG', 'TNNT2', 'DLX1', 'TBX18', 'MAP2 ']
genes_of_interest = [*other_genes, *end_genes]

genes_of_interest_end = scprep.select.get_gene_set(EBT_counts, starts_with=end_genes)
genes_of_interest_full = scprep.select.get_gene_set(EBT_counts, starts_with=genes_of_interest)
代码
文本
[34]
genes_mask = EBT_counts.columns.isin(genes_of_interest_full)
genes = EBT_counts.columns[genes_mask]
代码
文本
[35]
inverse = np.dot(zss, phate_operator.graph.data_pca.components_[:5, genes_mask])
end_gene_indexes = [(np.where(genes_of_interest_full == gene)[0][0]) for gene in genes_of_interest_end]
代码
文本
[36]
m_op = magic.MAGIC()
m_op.graph = phate_operator.graph
EBT_magic = m_op.transform(EBT_counts, genes=genes_of_interest_full)
/root/.local/lib/python3.10/site-packages/magic/magic.py:541: UserWarning: Running MAGIC.transform on different data to that which was used for MAGIC.fit may not produce sensible output, unless it comes from the same manifold.
  warnings.warn(
Calculating imputation...
Calculated imputation in 0.03 seconds.
代码
文本
[37]
fig, ax = plt.subplots(1,len(end_genes), figsize=(4*len(end_genes),4))
ax = ax.flatten()
for i in range(len(end_genes)):
scprep.plot.scatter2d(Y_phate,
c=EBT_magic[scprep.select.get_gene_set(EBT_counts, starts_with=end_genes[i])],
ax=ax[i],
title='%s - %s' % (end_points[i], end_genes[i]),
ticks=[],
)
代码
文本

4.4 绘制细胞轨迹

我们将 EBT_5 定义为来自最后一个时间点的计数，然后选择在这个最后时间点中表达最高的每个终点基因的前9个细胞。然后，我们绘制这些细胞的轨迹。

代码
文本
[38]
EBT_5 = EBT_counts[sample_labels == 'Day 24-27']
代码
文本
[39]
masks = {}
top_idxs = {}
for gene in end_genes:
top_idx = np.array(EBT_5[scprep.select.get_gene_set(EBT_counts, starts_with=gene)]).flatten().argsort()[-9:]
top_mask = np.array(pd.Series(range(3332)).isin(top_idx))
masks[gene] = top_mask
top_idxs[gene] = top_idx
print(gene, top_idx)
PDGFRA  [1896 2642  488  748  664  697 1419  300  432]
HAND1 [1959 3195  936 2133  375  668  501 2484 1476]
SOX17 [2053 2595 2664 1988 2353 1129 2823  738  532]
ONECUT2 [2920  555 2283  718 1275 2125 1838 2129 2277]
代码
文本
[40]
fig, ax = plt.subplots(1,1)
scprep.plot.scatter2d(Y_phate, c='Gray', alpha=0.1, ax=ax)
for i, gene in enumerate(end_genes):
scprep.plot.scatter2d(Y_phate[sample_labels=='Day 24-27'][masks[gene]],
ax=ax, c = colors[gene], label=end_points[i], ticks=[])
plt.legend()
代码
文本
[41]
fig, ax = plt.subplots(1,1)
scprep.plot.scatter2d(phate_operator.graph.data_nu, c='Gray', alpha=0.1, ax=ax)
for i, gene in enumerate(end_genes):
scprep.plot.scatter2d(phate_operator.graph.data_nu[sample_labels=='Day 24-27'][masks[gene]], ax=ax,
label='%s - %s' % (end_points[i], end_genes[i]), c=colors[gene], ticks=[])

for gene in end_genes:
for g in top_idxs[gene]:
ax.plot(zss[:,g,0], zss[:,g,1], c=colors[gene])
plt.legend()
代码
文本

4.5 绘制基因表达动态模式

对于这36个细胞（每个终点选择9个基因），我们绘制在100个时间点上的基因表达变化，并确定不同的细胞子集如何显示不同的动态行为。

代码
文本
[42]
fig, ax = plt.subplots(2,2, figsize=(6,6), sharex=True)
ax = ax.flatten()

for i, gene in enumerate(end_genes):
for j, eg in enumerate(end_genes):
for cell in top_idxs[eg]:
ax[i].plot(np.linspace(1,5,100)[::-1], inverse[:,cell,end_gene_indexes[i]], c=colors[eg])
ax[i].set_title(gene)
ax[i].set_yticks([])
ax[i].set_xticks(range(1,6))
代码
文本
[ ]

代码
文本
bioinformatics
生物信息学
Machine Learning
单细胞分析
从算法原理到代码实现
scRNA-seq
bioinformatics生物信息学Machine Learning单细胞分析从算法原理到代码实现scRNA-seq
已赞6
本文被以下合集收录
生物信息学 Notebooks Collection
liyongge
更新于 2024-09-13
33 篇75 人关注
细胞动力学
liyongge
更新于 2024-04-01
6 篇1 人关注
推荐阅读
公开
使用深度生成模型scVI进行单细胞批次效应去除实践
pythonnotebook深度学习机器学习生物信息学单细胞分析中文
pythonnotebook深度学习机器学习生物信息学单细胞分析中文
Gongyq
更新于 2024-09-09
1 赞
公开
ML4CFD | OpenFOAM 和 Basilisk 中的端到端模拟
ML4CFDCFDAI4S
ML4CFDCFDAI4S
JiaweiMiao
发布于 2024-05-03