Bohrium
robot
新建

空间站广场

论文
Notebooks
比赛
课程
Apps
我的主页
我的Notebooks
我的论文库
我的足迹

我的工作空间

任务
节点
文件
数据集
镜像
项目
数据库
公开
Bohrium 帮助文档｜DP-GEN
Bohrium 帮助文档
LBG Utility
DP-GEN
Bohrium 帮助文档LBG UtilityDP-GEN
Bohrium
MileAway
发布于 2023-10-14
推荐镜像 :Basic Image:bohrium-notebook:2023-04-07
推荐机型 :c2_m4_cpu
赞 2
3
12
Bohrium 帮助文档｜DP-GEN(v1)

DP-GEN

本文介绍如何在 Bohrium 平台运行典型的 DP-GEN 任务。

简介

DP-GEN（ Deep Potential Generator，深度势能生成器），致力于解决以下两个问题：

  1. 如何对势能面充分采样，有效覆盖训练可靠模型所需的样本空间？
  2. 如何高效筛选样本，选取训练价值高的构象进行第一性原理计算？

工作流程如下所示：“探索-标记-训练”循环迭代，直至势函数精度达预期要求。

为了方便生成数据以及监测势函数的精度，DP-GEN 还集成了 init_bulk、init_surf、autotest、simplify 等模块。

代码
文本

alt dpgen_scheduler-e2f39723cb340be83e35a4d717d9d59b.png

代码
文本

更多 DP-GEN 介绍请参考 https://github.com/deepmodeling/dpgen

如何在 Bohrium 上运行 DP-GEN 任务

本案例运行完成约需 60 min

步骤一，准备输入数据

首先需要开启管理节点，本案例中镜像选择 dpgen:0.10.6。通过 Web Shell 连接管理节点，执行以下语句确保 DP-GEN 及其依赖已经正确安装：

代码
文本
[1]
! dpgen -h
DeepModeling
------------
Version: 0.11.1
Path:    /opt/conda/lib/python3.8/site-packages/dpgen

Dependency
------------
     numpy     1.22.4   /opt/conda/lib/python3.8/site-packages/numpy
    dpdata     0.2.14   /opt/conda/lib/python3.8/site-packages/dpdata
  pymatgen            unknown version or path
     monty   2022.9.9   /opt/conda/lib/python3.8/site-packages/monty
       ase     3.22.1   /opt/conda/lib/python3.8/site-packages/ase
  paramiko      3.0.0   /opt/conda/lib/python3.8/site-packages/paramiko
 custodian  2022.5.26   /opt/conda/lib/python3.8/site-packages/custodian

Reference
------------
Please cite:
Yuzhi Zhang, Haidi Wang, Weijie Chen, Jinzhe Zeng, Linfeng Zhang, Han Wang, and Weinan E,
DP-GEN: A concurrent learning platform for the generation of reliable deep learning
based potential energy models, Computer Physics Communications, 2020, 107206.
------------

Description
------------
usage: dpgen [-h]
             {init_surf,init_bulk,auto_gen_param,init_reaction,run,run/report,collect,simplify,autotest,db}
             ...

dpgen is a convenient script that uses DeepGenerator to prepare initial data,
drive DeepMDkit and analyze results. This script works based on several sub-
commands with their own options. To see the options for the sub-commands, type
"dpgen sub-command -h".

positional arguments:
  {init_surf,init_bulk,auto_gen_param,init_reaction,run,run/report,collect,simplify,autotest,db}
    init_surf           Generating initial data for surface systems.
    init_bulk           Generating initial data for bulk systems.
    auto_gen_param      auto gen param.json
    init_reaction       Generating initial data for reactive systems.
    run                 Main process of Deep Potential Generator.
    run/report          Report the systems and the thermodynamic conditions of
                        the labeled frames.
    collect             Collect data.
    simplify            Simplify data.
    autotest            Auto-test for Deep Potential.
    db                  Collecting data from DP-GEN.

optional arguments:
  -h, --help            show this help message and exit
代码
文本

若一切正常会打印出 DP-GEN 版本以及依赖的一些库的信息。

代码
文本
示例如下： 
Version: 0.10.6
Date:    Nov-01-2022
Path:    /opt/miniconda/lib/python3.7/site-packages/dpgen

Dependency
------------
     numpy     1.21.6   /opt/miniconda/lib/python3.7/site-packages/numpy
    dpdata     0.2.10   /opt/miniconda/lib/python3.7/site-packages/dpdata
  pymatgen            unknown version or path
     monty   2022.9.9   /opt/miniconda/lib/python3.7/site-packages/monty
       ase     3.22.1   /opt/miniconda/lib/python3.7/site-packages/ase
  paramiko     2.11.0   /opt/miniconda/lib/python3.7/site-packages/paramiko
 custodian  2022.5.26   /opt/miniconda/lib/python3.7/site-packages/custodian

Reference
------------
Please cite:
Yuzhi Zhang, Haidi Wang, Weijie Chen, Jinzhe Zeng, Linfeng Zhang, Han Wang, and Weinan E,
DP-GEN: A concurrent learning platform for the generation of reliable deep learning
based potential energy models, Computer Physics Communications, 2020, 107206.
------------

Description
------------
usage: dpgen [-h]
             {init_surf,init_bulk,auto_gen_param,init_reaction,run,run/report,collect,simplify,autotest,db}
             ...

dpgen is a convenient script that uses DeepGenerator to prepare initial data,
drive DeepMDkit and analyze results. This script works based on several sub-
commands with their own options. To see the options for the sub-commands, type
"dpgen sub-command -h".

positional arguments:
  {init_surf,init_bulk,auto_gen_param,init_reaction,run,run/report,collect,simplify,autotest,db}
    init_surf           Generating initial data for surface systems.
    init_bulk           Generating initial data for bulk systems.
    auto_gen_param      auto gen param.json
    init_reaction       Generating initial data for reactive systems.
    run                 Main process of Deep Potential Generator.
    run/report          Report the systems and the thermodynamic conditions of
                        the labeled frames.
    collect             Collect data.
    simplify            Simplify data.
    autotest            Auto-test for Deep Potential.
    db                  Collecting data from DP-GEN.

optional arguments:
  -h, --help            show this help message and exit
代码
文本

确认无误后便可进行输入文件的准备。以甲烷为例，DP-GEN 的输入文件均已存储至 Bohrium_DP-GEN_run_example 文件夹内，（你可以在左侧点击数据集查看相应文件）：

代码
文本
[9]
# 出于安全考虑，我们没有数据集所在文件夹的写入权限，因此我们将其复制到 `/data/` 目录下:
! cp -nr /bohr/ /data/

# 我们在这里定义一些路径，并切换到工作路径，方便后续调用：
import os

bohr_dataset_url = "/bohr/bohrium-dpgen-1e75/v1/Bohrium_DP-GEN_run_example/" # url 可从左侧数据集复制
work_path = os.path.join("/data", bohr_dataset_url[1:])
os.chdir(work_path)
print(f"当前路径为：{os.getcwd()}")
当前路径为：/data/bohr/bohrium-dpgen-1e75/v1/Bohrium_DP-GEN_run_example
代码
文本

使用 tree . -L 2 命令可以查看输入文件，主要包含以下的内容：

代码
文本
[10]
! tree . -L 2
.
├── CH4.POSCAR
├── CH4.POSCAR.01x01x01
│   ├── 00.place_ele
│   ├── 01.scale_pert
│   ├── 02.md
│   └── param.json
├── INCAR_methane
├── INCAR_methane.md
├── INCAR_methane.rlx
├── POTCAR_C
├── POTCAR_H
├── init.json
├── machine.json
└── run_param.json

4 directories, 10 files
代码
文本

DP-GEN 运行需要的文件说明：

文件/文件夹名称 说明
CH4.POSCAR.01x01x01文件夹 VASP结构文件
machine.json 机器配置文件；包括计算资源、日志等
INCAR_methane VASP计算参数文件；单点能，本教程run过程采用
POTCAR_C、POTCAR_H VASP赝势文件；根据体系元素种类设置
run_param.json 计算参数文件

步骤二，准备配置文件

文件夹内已经包含配置文件 machine.json，您也可以通过以下 Python 代码创建或编辑 machine.json：

代码
文本
[11]
import json

# 设置 job.json 的参数
machine_params = {
"api_version": "1.0",
"deepmd_version": "2.1.5",
"train" :[
{
"command": "dp",
"machine": {
"batch_type": "Bohrium",
"context_type": "BohriumContext",
"local_root" : "./",
"remote_profile":{
"email": "",
"password": "",
"program_id": 0000,
"keep_backup":True,
"input_data":{
"log_file": "00*/train.log",
"grouped":True,
"job_name": "dpgen_train_job",
"disk_size": 100,
"scass_type":"c8_m32_1 * NVIDIA V100",
"checkpoint_files":["00*/checkpoint","00*/model.ckpt*"],
"checkpoint_time":30,
"platform": "ali",
"job_type": "container",
"image_address":"registry.dp.tech/dptech/deepmd-kit:2.1.5-cuda11.6",
"on_demand":0
}
}
},
"resources": {
"number_node": 1,
"cpu_per_node": 8,
"gpu_per_node": 1,
"queue_name": "V100_8_32",
"group_size": 1,
"custom_flags": [],
"strategy": {"if_cuda_multi_devices": True},
"para_deg": 3,
"source_list": []
}

}],

"model_devi":[
{
"command": "lmp -i input.lammps -v restart 0",
"machine": {
"batch_type": "Bohrium",
"context_type": "BohriumContext",
"local_root" : "./",
"remote_profile":{
"email": "",
"password": "",
"program_id":0000,
"keep_backup":True,
"input_data":{
"log_file": "*/model_devi.log",
"grouped":True,
"job_name": "dpgen_model_devi_job",
"disk_size": 200,
"scass_type":"c8_m32_1 * NVIDIA V100",
"platform": "ali",
"job_type": "container",
"image_address":"registry.dp.tech/dptech/deepmd-kit:2.1.5-cuda11.6",
"on_demand":0
}
}
},
"resources": {
"number_node": 1,
"cpu_per_node": 8,
"gpu_per_node": 1,
"queue_name": "V100_8_32",
"group_size": 250,
"source_list": []
}
}],

"fp":[
{
"command": "ulimit -s unlimited; mpirun -np 32 vasp_std",
"machine": {
"batch_type": "Bohrium",
"context_type": "BohriumContext",
"local_root" : "./",
"remote_profile":{
"email": "",
"password": "",
"program_id":0000,
"input_data":{
"api_version":2,
"log_file": "task*/fp.log",
"grouped":True,
"job_name": "dpgen_fp_job",
"disk_size": 100,
"scass_type":"c32_m64_cpu",
"platform": "ali",
"job_type":"container",
"image_address":"您需提供授权凭证到 bohrium@dp.tech 邮箱获取 VASP 镜像",
"on_demand":0
}
}
},
"resources": {
"number_node": 4,
"cpu_per_node": 8,
"gpu_per_node": 0,
"queue_name": "CPU",
"group_size": 10,
"source_list": ["/opt/intel/oneapi/setvars.sh"]
}
}]

}

with open("machine.json", "w") as f:
json.dump(machine_params, f, indent=4, ensure_ascii=False)

print(f"已生成 {os.path.join(os.getcwd(), 'machine.json')}, \n参数为:\n{json.dumps(machine_params, indent=4, ensure_ascii=False)}")
已生成 /data/bohr/bohrium-dpgen-1e75/v1/Bohrium_DP-GEN_run_example/machine.json, 
参数为:
{
    "api_version": "1.0",
    "deepmd_version": "2.1.5",
    "train": [
        {
            "command": "dp",
            "machine": {
                "batch_type": "Bohrium",
                "context_type": "BohriumContext",
                "local_root": "./",
                "remote_profile": {
                    "email": "",
                    "password": "",
                    "program_id": 0,
                    "keep_backup": true,
                    "input_data": {
                        "log_file": "00*/train.log",
                        "grouped": true,
                        "job_name": "dpgen_train_job",
                        "disk_size": 100,
                        "scass_type": "c8_m32_1 * NVIDIA V100",
                        "checkpoint_files": [
                            "00*/checkpoint",
                            "00*/model.ckpt*"
                        ],
                        "checkpoint_time": 30,
                        "platform": "ali",
                        "job_type": "container",
                        "image_address": "registry.dp.tech/dptech/deepmd-kit:2.1.5-cuda11.6",
                        "on_demand": 0
                    }
                }
            },
            "resources": {
                "number_node": 1,
                "cpu_per_node": 8,
                "gpu_per_node": 1,
                "queue_name": "V100_8_32",
                "group_size": 1,
                "custom_flags": [],
                "strategy": {
                    "if_cuda_multi_devices": true
                },
                "para_deg": 3,
                "source_list": []
            }
        }
    ],
    "model_devi": [
        {
            "command": "lmp -i input.lammps -v restart 0",
            "machine": {
                "batch_type": "Bohrium",
                "context_type": "BohriumContext",
                "local_root": "./",
                "remote_profile": {
                    "email": "",
                    "password": "",
                    "program_id": 0,
                    "keep_backup": true,
                    "input_data": {
                        "log_file": "*/model_devi.log",
                        "grouped": true,
                        "job_name": "dpgen_model_devi_job",
                        "disk_size": 200,
                        "scass_type": "c8_m32_1 * NVIDIA V100",
                        "platform": "ali",
                        "job_type": "container",
                        "image_address": "registry.dp.tech/dptech/deepmd-kit:2.1.5-cuda11.6",
                        "on_demand": 0
                    }
                }
            },
            "resources": {
                "number_node": 1,
                "cpu_per_node": 8,
                "gpu_per_node": 1,
                "queue_name": "V100_8_32",
                "group_size": 250,
                "source_list": []
            }
        }
    ],
    "fp": [
        {
            "command": "ulimit -s unlimited; mpirun -np 32 vasp_std",
            "machine": {
                "batch_type": "Bohrium",
                "context_type": "BohriumContext",
                "local_root": "./",
                "remote_profile": {
                    "email": "",
                    "password": "",
                    "program_id": 0,
                    "input_data": {
                        "api_version": 2,
                        "log_file": "task*/fp.log",
                        "grouped": true,
                        "job_name": "dpgen_fp_job",
                        "disk_size": 100,
                        "scass_type": "c32_m64_cpu",
                        "platform": "ali",
                        "job_type": "container",
                        "image_address": "您需提供授权凭证到 bohrium@dp.tech 邮箱获取 VASP 镜像",
                        "on_demand": 0
                    }
                }
            },
            "resources": {
                "number_node": 4,
                "cpu_per_node": 8,
                "gpu_per_node": 0,
                "queue_name": "CPU",
                "group_size": 10,
                "source_list": [
                    "/opt/intel/oneapi/setvars.sh"
                ]
            }
        }
    ]
}
代码
文本

注意：

  1. "project_id":后的 0000 需要替换为您自己的项目ID，可在“项目管理”页查看，共三处位置需要替换。
  2. emails 和 password 需要填入自己的 Bohrium 账户邮箱和密码，共三处位置需要修改。
  3. 根据需要调整 image_address 处填入的镜像地址，在本次示例中训练部分使用 registry.dp.tech/dptech/deepmd-kit:2.1.5-cuda11.6，探索部分使用 registry.dp.tech/dptech/deepmd-kit:2.1.5-cuda11.6，标记部分使用 VASP 镜像（如需使用 VASP 镜像，请提供您的 VASP 授权凭证至 bohrium@dp.tech 邮箱）。
代码
文本

步骤三，提交任务

第一次使用 Lebesgue Utility 需要配置，若您还未配置，请参阅 Bohrium 帮助文档｜Lebesgue Utility）

使用 Lebesgue Utility 提交任务（Lebesgue Utility 是 Bohrium 平台的作业管理系统）：

代码
文本
[ ]
os.system("nohup dpgen run run_param.json machine.json 1>log 2>err&")
代码
文本

注意：DP-GEN 进程需要等待任务完成后进行后续轮次任务的提交，因此请不要关闭管理节点。

查看任务

您可以在监控任务文档中了解如何在 Bohrium 平台查看任务状态。

下载结果

您可以在结果下载文档中了解如何在 Bohrium 平台下载任务结果。

更多 DP-GEN 参数介绍

如果您想在 model_devi 步骤使用自定义的 input.lammps 模板，以实现 plumed 等功能，可参考文档 DP-GEN model_devi步骤 LAMMPS自定义模板

更多DP-GEN参数的详细设定可以参考 DP-GEN源码仓库 与 DP-GEN官方文档

代码
文本
Bohrium 帮助文档
LBG Utility
DP-GEN
Bohrium 帮助文档LBG UtilityDP-GEN
已赞2
本文被以下合集收录
Bohrium 帮助文档｜软件案例
MileAway
更新于 2024-05-12
14 篇7 人关注
DPMD
RamonMi
更新于 2024-07-29
14 篇0 人关注
推荐阅读
公开
【WIP】DeePMD 训练与模拟效果分析指南
DeePMDDP答疑DP-GEN分子动力学
DeePMDDP答疑DP-GEN分子动力学
AnguseZhang
发布于 2023-09-25
14 赞12 转存文件
公开
快速开始 DeepSPIN｜训练NiO深度势能分子动力学模型
DeepSPINTutorial
DeepSPINTutorial
bxu@gscaep.ac.cn
发布于 2023-07-04
16 赞39 转存文件1 评论