新建
基于Transformer网络预测锂电池的老化轨迹
KaiqiYang
JiaweiMiao
Letian
推荐镜像 :Basic Image:bohrium-notebook:2023-04-07
推荐机型 :c12_m46_1 * NVIDIA GPU B
赞 2
1
16
目录
数据集
NASA_data(v1)
基于Transformer网络预测锂电池的老化轨迹
©️ Copyright 2023 @ Authors
作者：
杨凯麒📨
苗嘉伟📨
日期：2023-10-17
共享协议：本作品采用知识共享署名-非商业性使用-相同方式共享 4.0 国际许可协议进行许可。
快速开始：点击上方的 开始连接 按钮，选择 bohrium-notebook:2023-04-07镜像及任意GPU节点配置，稍等片刻即可运行。
主要内容
准确预测锂电池的剩余寿命(RUL)在管理电池健康状态和估计电池状态方面发挥着重要作用。随着电动汽车的快速发展,对预测RUL技术的需求与日俱增。为了预测RUL,本案例中设计了一个基于Transformer的神经网络。首先,电池容量数据通常存在大量噪声,特别是在电池充放电再生过程中。为了缓解这个问题,我们对原始数据进行了去噪自动编码器(DAE)的处理。然后,为了捕获时序信息和学习有用的特征,重构后的序列被输入到Transformer网络中。最后,为了统一去噪和预测两个任务，本案例将其组合到一个统一的框架中。在NASA数据集上的大量实验和与一些现有方法的比较结果表明,本案例中提出的方法在预测RUL方面具有更好的表现。
Transformer是一种神经网络架构，主要用于自然语言处理（NLP）任务，如机器翻译、文本摘要等。它是由Vaswani等人在2017年提出的一种基于自注意力（Self-Attention）机制的深度学习模型。Transformer摒弃了传统的循环神经网络（RNN）和长短时记忆网络（LSTM），利用自注意力机制处理输入序列中的长距离依赖关系。这使得Transformer在处理序列数据时具有更高的并行性和计算效率。此外，Transformer引入了位置编码（Positional Encoding）来捕捉序列中单词的顺序信息。
代码
文本
加载相关的工具包
代码
文本
[2]
import numpy as np
import random
import math
import os
import scipy.io
import matplotlib.pyplot as plt
import torch
import torch.nn as nn
import torch.nn.functional as F
import torchvision
%matplotlib inline
from math import sqrt
from datetime import datetime
from sklearn.metrics import mean_absolute_error
from sklearn.metrics import mean_squared_error
代码
文本
设置设备类型
代码
文本
[3]
device = torch.device('cuda:0' if torch.cuda.is_available() else 'cpu')
代码
文本
载入相关数据
代码
文本
[4]
# 定义str 到 datatime转换函数
def convert_to_time(hmm):
year, month, day, hour, minute, second = int(hmm[0]), int(hmm[1]), int(hmm[2]), int(hmm[3]), int(hmm[4]), int(hmm[5])
return datetime(year=year, month=month, day=day, hour=hour, minute=minute, second=second)
# 载入 .mat 数据
def loadMat(matfile):
data = scipy.io.loadmat(matfile)
filename = matfile.split("/")[-1].split(".")[0]
col = data[filename]
col = col[0][0][0][0]
size = col.shape[0]
data = []
for i in range(size):
k = list(col[i][3][0].dtype.fields.keys())
d1, d2 = {}, {}
if str(col[i][0][0]) != 'impedance':
for j in range(len(k)):
t = col[i][3][0][0][j][0];
l = [t[m] for m in range(len(t))]
d2[k[j]] = l
d1['type'], d1['temp'], d1['time'], d1['data'] = str(col[i][0][0]), int(col[i][1][0]), str(convert_to_time(col[i][2][0])), d2
data.append(d1)
return data
# 获取容量数据
def getBatteryCapacity(Battery):
cycle, capacity = [], []
i = 1
for Bat in Battery:
if Bat['type'] == 'discharge':
capacity.append(Bat['data']['Capacity'][0])
cycle.append(i)
i += 1
return [cycle, capacity]
代码
文本
[5]
Battery_list = ['B0005', 'B0006', 'B0007', 'B0018']
dir_path = '/bohr/NASAdata-3ls8/v1/NASA/1. BatteryAgingARC-FY08Q4/'
Battery = {}
for name in Battery_list:
print('Load Dataset ' + name + '.mat ...')
path = dir_path + name + '.mat'
data = loadMat(path)
# print(len(data))
Battery[name] = getBatteryCapacity(data)
print(len(Battery[name][0])) #各个电池的总循环次数
Load Dataset B0005.mat ... 168 Load Dataset B0006.mat ... 168 Load Dataset B0007.mat ... 168 Load Dataset B0018.mat ... 132
代码
文本
绘制四个电池数据的循环-SOH曲线
代码
文本
[6]
fig, ax = plt.subplots(1, figsize=(12, 8))
color_list = ['b:', 'g--', 'r-.', 'c.']
c = 0
for name,color in zip(Battery_list, color_list):
df_result = Battery[name]
ax.plot(df_result[0], df_result[1]/df_result[1][0], color, label=name) #SOH=Q_cyc/Q_0
ax.set(xlabel='Discharge cycles', ylabel='SOH', title='SOH fading trajectory')
plt.legend()
代码
文本
定义数据处理函数
代码
文本
[7]
def build_sequences(text, window_size):
#将容量衰减的信息看做是文本text信息
x, y = [],[]
for i in range(len(text) - window_size):
sequence = text[i:i+window_size]
target = text[i+1:i+1+window_size]
x.append(sequence)
y.append(target)
return np.array(x), np.array(y)
#将一条数据作为测试集，其余作为训练集
def get_train_test(data_dict, name, window_size=8):
data_sequence=data_dict[name][1]
test_data = data_sequence
test_early, test_rest = data_sequence[:window_size+1], data_sequence[window_size+1:]
test_input = list(test_early)
test_truth = list(test_rest)
n = 0
for k, v in data_dict.items():
if k != name:
data_x, data_y = build_sequences(text=v[1], window_size=window_size)
if n == 0:
train_x, train_y = data_x, data_y
n += 1
else:
train_x, train_y = np.r_[train_x, data_x], np.r_[train_y, data_y]
return train_x, train_y, test_input, test_truth
#定义评估rmse的函数
def evaluation(y_test, y_predict):
mse = mean_squared_error(y_test, y_predict)
rmse = sqrt(mean_squared_error(y_test, y_predict))
return rmse
# 定义随机种子，保证每次训练、预测结果固定
def setup_seed(seed):
np.random.seed(seed) # Numpy module.
random.seed(seed) # Python random module.
os.environ['PYTHONHASHSEED'] = str(seed)
torch.manual_seed(seed)
if torch.cuda.is_available():
torch.cuda.manual_seed(seed)
torch.cuda.manual_seed_all(seed)
torch.backends.cudnn.benchmark = False
torch.backends.cudnn.deterministic = True
代码
文本
定义网络结构
代码
文本
[8]
# 定义自动编码器进行数据去噪
class Autoencoder(nn.Module):
def __init__(self, input_size=48*2, hidden_dim=24*2, noise_level=0.00):
super(Autoencoder, self).__init__()
self.input_size, self.hidden_dim, self.noise_level = input_size, hidden_dim, noise_level
self.fc1 = nn.Linear(self.input_size, self.hidden_dim)
self.fc2 = nn.Linear(self.hidden_dim, self.input_size)
def encoder(self, x):
x = self.fc1(x)
h1 = F.relu(x)
return h1
def mask(self, x):
corrupted_x = x + self.noise_level * torch.randn_like(x)
return corrupted_x
def decoder(self, x):
h2 = self.fc2(x)
return h2
def forward(self, x):
out = self.mask(x)
encode = self.encoder(out)
decode = self.decoder(encode)
return encode, decode
#定义位置编码（Positional Encoding）。位置编码的目的是为了让模型能够学习到输入序列中各个单词之间的相对位置关系。
class PositionalEncoding(nn.Module):
def __init__(self, d_model, dropout=0.0, max_len=16): #初始化一个 PositionalEncoding 对象。其中，`d_model` 表示模型的嵌入维度，`dropout` 表示 dropout 的比率（这里没有用到），`max_len` 表示最大序列长度。
super(PositionalEncoding, self).__init__()
pe = torch.zeros(max_len, d_model) # 生成位置编码矩阵 `pe` ： 初始化一个全零的矩阵，大小为 (max_len, d_model)，用于存储位置编码。
position = torch.arange(0, max_len, dtype=torch.float).unsqueeze(1) #生成位置向量 `position` ： 一个一维张量，包含了从0到 max_len-1 的整数。
#生成分母项 `div_term` ： 一个一维张量，其元素是按照指数递减的，可以理解为一个衰减因子。
div_term = torch.exp(torch.arange(0, d_model, 2).float() * (-math.log(10000.0) / d_model))
#计算位置编码矩阵 `pe` 的各个元素值： 使用正弦函数计算偶数列的值，使用余弦函数计算奇数列的值。
pe[:, 0::2] = torch.sin(position * div_term)
pe[:, 1::2] = torch.cos(position * div_term)
#调整位置编码矩阵 `pe` 的维度： 首先增加一个维度，然后将第0维和第1维进行转置，最后将位置编码矩阵注册为一个缓冲区。
pe = pe.unsqueeze(0).transpose(0, 1)
self.register_buffer('pe', pe)
def forward(self, x):
#实现将位置编码矩阵与输入的词嵌入相加，融合位置信息。输入参数 `x` 是一个词嵌入张量。
x = x + self.pe[-1:, :].squeeze(1) # Fix: change to -1
return x
#定义网络结构
class Net(nn.Module):
def __init__(self, feature_size=16, hidden_dim=32, num_layers=1, nhead=8, dropout=0.0, noise_level=0.01):
super(Net, self).__init__()
self.auto_hidden = int(feature_size/2)
input_size = self.auto_hidden
self.pos = PositionalEncoding(d_model=input_size, max_len=input_size)
#`self.cell` 是一个基于 Transformer 的编码器。它接收编码后的输入数据，并对其进行处理。编码器由多个 Transformer 编码器层组成，每个层包含多头自注意力机制和前馈神经网络。
encoder_layers = nn.TransformerEncoderLayer(d_model=input_size, nhead=nhead, dim_feedforward=hidden_dim, dropout=dropout)
self.cell = nn.TransformerEncoder(encoder_layers, num_layers=num_layers)
self.linear = nn.Linear(input_size, 1)
self.autoencoder = Autoencoder(input_size=feature_size, hidden_dim=self.auto_hidden, noise_level=noise_level)
def forward(self, x):
batch_size, feature_num, feature_size = x.shape
#自编码器用于对输入数据进行降维和去噪。编码后的数据作为 Transformer 编码器的输入。
encode, decode = self.autoencoder(x.reshape(batch_size, -1))# batch_size*seq_len
out = encode.reshape(batch_size, -1, self.auto_hidden)
out = self.pos(out)
out = out.reshape(1, batch_size, -1) # (1, batch_size, feature_size)
out = self.cell(out)
out = out.reshape(batch_size, -1) # (batch_size, hidden_dim)
out = self.linear(out) # out shape: (batch_size, 1)
return out, decode
代码
文本
定义训练函数
代码
文本
[9]
def train(lr=0.01, feature_size=8, hidden_dim=32, num_layers=1, nhead=8, weight_decay=0.0,
EPOCH=1000, seed=0, alpha=0.0, noise_level=0.0, dropout=0.0, test_ind=0):
setup_seed(seed)
name = Battery_list[test_ind]
window_size = feature_size
train_x, train_y, _, _ = get_train_test(Battery, name, window_size)
train_size = len(train_x)
print('sample size: {}'.format(train_size))
model = Net(feature_size=feature_size, hidden_dim=hidden_dim, num_layers=num_layers, nhead=nhead, dropout=dropout,
noise_level=noise_level)
print(model)
print('------------------------------------------------------------------')
model = model.to(device)
optimizer = torch.optim.Adam(model.parameters(), lr=lr, weight_decay=weight_decay)
scheduler = torch.optim.lr_scheduler.ReduceLROnPlateau(optimizer, 'min', factor=0.8, patience=40, verbose=True, min_lr=1e-5)
criterion = nn.MSELoss()
loss_list= [0]
for epoch in range(EPOCH):
X = np.reshape(train_x, (-1, 1, feature_size)).astype(np.float32) # (batch_size, seq_len, input_size)
y = np.reshape(train_y[:,-1],(-1,1)).astype(np.float32) # (batch_size, 1) 自回归！
X, y = torch.from_numpy(X).to(device), torch.from_numpy(y).to(device)
output, decode = model(X)
output = output.reshape(-1, 1)
#loss = criterion(output, y) + alpha * criterion(decode, X.reshape(-1, feature_size))
loss = criterion(output, y)
optimizer.zero_grad() # 对每个训练步清空梯度
loss.backward() # 反向传播计算梯度
optimizer.step() # 梯度优化
scheduler.step(loss)
if (epoch + 1)%200 == 0:
loss_list.append(loss.item())
print('epoch:{:<2d} | loss:{:<6.4f}'.format(epoch+1, loss))
if loss < 0.0001 :
break
model_trained = model
return loss_list, model_trained
代码
文本
设置超参数并开始训练
代码
文本
[10]
import math
window_size = 16 * 3
feature_size = window_size
dropout = 0.0
EPOCH = 20000
nhead = 12
hidden_dim = 256
# input_size=48
num_layers = 1 * 3
lr = 0.0005 # learning rate学习率
weight_decay = 0.0
noise_level = 0.0
alpha = 1e-5
seed = 0
test_ind = 0
print('seed:{}'.format(seed))
loss_list, model_trained = train(lr=lr, feature_size=feature_size,hidden_dim=hidden_dim, num_layers=num_layers, nhead=nhead,
weight_decay=weight_decay, EPOCH=EPOCH, seed=seed, dropout=dropout,
alpha=alpha, noise_level=noise_level,test_ind=test_ind)
print('------------------------------------------------------------------')
seed:0
sample size: 324
Net(
(pos): PositionalEncoding()
(cell): TransformerEncoder(
(layers): ModuleList(
(0): TransformerEncoderLayer(
(self_attn): MultiheadAttention(
(out_proj): NonDynamicallyQuantizableLinear(in_features=24, out_features=24, bias=True)
)
(linear1): Linear(in_features=24, out_features=256, bias=True)
(dropout): Dropout(p=0.0, inplace=False)
(linear2): Linear(in_features=256, out_features=24, bias=True)
(norm1): LayerNorm((24,), eps=1e-05, elementwise_affine=True)
(norm2): LayerNorm((24,), eps=1e-05, elementwise_affine=True)
(dropout1): Dropout(p=0.0, inplace=False)
(dropout2): Dropout(p=0.0, inplace=False)
)
(1): TransformerEncoderLayer(
(self_attn): MultiheadAttention(
(out_proj): NonDynamicallyQuantizableLinear(in_features=24, out_features=24, bias=True)
)
(linear1): Linear(in_features=24, out_features=256, bias=True)
(dropout): Dropout(p=0.0, inplace=False)
(linear2): Linear(in_features=256, out_features=24, bias=True)
(norm1): LayerNorm((24,), eps=1e-05, elementwise_affine=True)
(norm2): LayerNorm((24,), eps=1e-05, elementwise_affine=True)
(dropout1): Dropout(p=0.0, inplace=False)
(dropout2): Dropout(p=0.0, inplace=False)
)
(2): TransformerEncoderLayer(
(self_attn): MultiheadAttention(
(out_proj): NonDynamicallyQuantizableLinear(in_features=24, out_features=24, bias=True)
)
(linear1): Linear(in_features=24, out_features=256, bias=True)
(dropout): Dropout(p=0.0, inplace=False)
(linear2): Linear(in_features=256, out_features=24, bias=True)
(norm1): LayerNorm((24,), eps=1e-05, elementwise_affine=True)
(norm2): LayerNorm((24,), eps=1e-05, elementwise_affine=True)
(dropout1): Dropout(p=0.0, inplace=False)
(dropout2): Dropout(p=0.0, inplace=False)
)
)
)
(linear): Linear(in_features=24, out_features=1, bias=True)
(autoencoder): Autoencoder(
(fc1): Linear(in_features=48, out_features=24, bias=True)
(fc2): Linear(in_features=24, out_features=48, bias=True)
)
)
------------------------------------------------------------------
epoch:200 | loss:0.0072
epoch:400 | loss:0.0012
epoch:600 | loss:0.0008
epoch:800 | loss:0.0007
epoch:1000 | loss:0.0007
epoch:1200 | loss:0.0007
epoch:1400 | loss:0.0007
Epoch 01534: reducing learning rate of group 0 to 4.0000e-04.
epoch:1600 | loss:0.0006
epoch:1800 | loss:0.0006
epoch:2000 | loss:0.0006
Epoch 02081: reducing learning rate of group 0 to 3.2000e-04.
epoch:2200 | loss:0.0006
epoch:2400 | loss:0.0005
epoch:2600 | loss:0.0005
epoch:2800 | loss:0.0005
Epoch 02882: reducing learning rate of group 0 to 2.5600e-04.
epoch:3000 | loss:0.0005
epoch:3200 | loss:0.0005
epoch:3400 | loss:0.0005
Epoch 03491: reducing learning rate of group 0 to 2.0480e-04.
epoch:3600 | loss:0.0005
epoch:3800 | loss:0.0005
Epoch 03847: reducing learning rate of group 0 to 1.6384e-04.
epoch:4000 | loss:0.0005
epoch:4200 | loss:0.0005
epoch:4400 | loss:0.0005
epoch:4600 | loss:0.0004
Epoch 04676: reducing learning rate of group 0 to 1.3107e-04.
epoch:4800 | loss:0.0004
epoch:5000 | loss:0.0004
epoch:5200 | loss:0.0004
epoch:5400 | loss:0.0004
epoch:5600 | loss:0.0004
epoch:5800 | loss:0.0004
epoch:6000 | loss:0.0004
Epoch 06047: reducing learning rate of group 0 to 1.0486e-04.
epoch:6200 | loss:0.0004
epoch:6400 | loss:0.0004
Epoch 06547: reducing learning rate of group 0 to 8.3886e-05.
epoch:6600 | loss:0.0004
epoch:6800 | loss:0.0004
epoch:7000 | loss:0.0004
epoch:7200 | loss:0.0004
epoch:7400 | loss:0.0004
epoch:7600 | loss:0.0004
epoch:7800 | loss:0.0004
epoch:8000 | loss:0.0004
epoch:8200 | loss:0.0004
epoch:8400 | loss:0.0004
epoch:8600 | loss:0.0004
epoch:8800 | loss:0.0004
epoch:9000 | loss:0.0004
epoch:9200 | loss:0.0004
epoch:9400 | loss:0.0004
Epoch 09410: reducing learning rate of group 0 to 6.7109e-05.
epoch:9600 | loss:0.0004
epoch:9800 | loss:0.0004
epoch:10000 | loss:0.0004
epoch:10200 | loss:0.0004
Epoch 10235: reducing learning rate of group 0 to 5.3687e-05.
epoch:10400 | loss:0.0004
epoch:10600 | loss:0.0004
Epoch 10721: reducing learning rate of group 0 to 4.2950e-05.
epoch:10800 | loss:0.0003
epoch:11000 | loss:0.0003
epoch:11200 | loss:0.0003
epoch:11400 | loss:0.0003
epoch:11600 | loss:0.0003
epoch:11800 | loss:0.0003
epoch:12000 | loss:0.0003
epoch:12200 | loss:0.0003
epoch:12400 | loss:0.0003
epoch:12600 | loss:0.0003
epoch:12800 | loss:0.0003
epoch:13000 | loss:0.0003
epoch:13200 | loss:0.0003
epoch:13400 | loss:0.0003
epoch:13600 | loss:0.0003
epoch:13800 | loss:0.0003
epoch:14000 | loss:0.0003
epoch:14200 | loss:0.0003
Epoch 14301: reducing learning rate of group 0 to 3.4360e-05.
epoch:14400 | loss:0.0003
epoch:14600 | loss:0.0003
epoch:14800 | loss:0.0003
epoch:15000 | loss:0.0003
epoch:15200 | loss:0.0003
epoch:15400 | loss:0.0003
Epoch 15426: reducing learning rate of group 0 to 2.7488e-05.
epoch:15600 | loss:0.0003
epoch:15800 | loss:0.0003
epoch:16000 | loss:0.0003
epoch:16200 | loss:0.0003
epoch:16400 | loss:0.0003
epoch:16600 | loss:0.0003
epoch:16800 | loss:0.0003
epoch:17000 | loss:0.0003
epoch:17200 | loss:0.0003
epoch:17400 | loss:0.0003
epoch:17600 | loss:0.0003
epoch:17800 | loss:0.0003
epoch:18000 | loss:0.0003
epoch:18200 | loss:0.0003
epoch:18400 | loss:0.0003
epoch:18600 | loss:0.0003
epoch:18800 | loss:0.0003
epoch:19000 | loss:0.0003
epoch:19200 | loss:0.0003
epoch:19400 | loss:0.0003
epoch:19600 | loss:0.0003
epoch:19800 | loss:0.0003
epoch:20000 | loss:0.0003
------------------------------------------------------------------
代码
文本
训练误差绘图
代码
文本
[11]
epoch_lst = [(i+1)*200 for i in range(int(EPOCH/200))]
logloss = [math.log(t) for t in loss_list[1:]]
plt.plot(epoch_lst, logloss, label='Train loss')
plt.xlabel('Epoch')
plt.ylabel('Ln(Loss)')
plt.legend()
plt.show()
代码
文本
对测试集进行预测
代码
文本
[12]
test_ind = 0
name = Battery_list[test_ind]
#获取测试集数据
_, _, test_input,test_truth = get_train_test(Battery, name, window_size)
y_ = []
rmse = 1
result_list = []
rmse_list = []
test_x = test_input
inputlen = len(test_input)
point_list = []
#自回归预测过程
while (len(test_x) - inputlen) < len(test_truth):
x = np.reshape(np.array(test_x[-feature_size:]),(-1, 1, feature_size)).astype(np.float32)
x = torch.from_numpy(x).to(device) # shape (batch_size,feature_size=1,input_size)
pred, _ = model_trained(x) # pred shape: (batch_size=1, feature_size=1)
next_point = pred.data.cpu().numpy()[0,0]
test_x.append(next_point) # 将预测出的值加入原序列后再去预测下一个点
point_list.append(next_point) # 保存预测出的点
y_.append(point_list) # 保存所有的预测点
rmse = evaluation(y_test=test_truth, y_predict=y_[-1])
rmse_list.append(rmse)
result_list.append(y_[-1])
print('{} | RMSE:{:<6.4f} '.format(name, rmse))
B0005 | RMSE:0.0518
代码
文本
将预测数据和目标曲线作在一张图上比较
代码
文本
[13]
early_data = test_input.copy()
aa = early_data[:window_size+1].copy() # 第一个输入序列
aa = aa + result_list[0] # 加入预测出的序列
battery = Battery[name]
fig, ax = plt.subplots(1, figsize=(12, 8))
ax.plot(battery[0][:-1], battery[1][:-1]/battery[1][0], 'b.-', label=name)
ax.plot(battery[0], aa/battery[1][0], 'r.-', label='Prediction')
ax.set(xlabel='cycles', ylabel='SOH', title='SOH fading trajectory comparasion')
plt.legend()
代码
文本
[ ]
代码
文本
已赞2
本文被以下合集收录
电化学-电池计算
hjchen
更新于 2024-06-13
7 篇9 人关注
Data driven SOH estimation
Samuel
更新于 2024-09-09
2 篇0 人关注
推荐阅读
公开
基于Transformer网络预测锂电池的老化轨迹Letian
更新于 2024-07-16
2 赞3 转存文件
公开
Uni-Mol预测液流电池溶解度zmz
发布于 2023-09-07
9 赞12 转存文件