如何启用混合精确训练，而使用英特尔扩展PyTorch (IPEX)?

Question

我正在使用英特尔扩展Pytorch (Ref - https://github.com/amitrajitbose/cat-v-dog-classifier-pytorch)的Dog-Cat分类器。我想减少模型的训练时间。我怎么在我的代码中混合精确性?参考这个github(https://github.com/intel/intel-extension-for-pytorch)来训练我的模型。

Answer 1

可以使用以下命令启用Intel Extension for PyTorch的混合精度，

# For Float32
model, optimizer = ipex.optimize(model, optimizer=optimizer)
# For BFloat16
model, optimizer = ipex.optimize(model, optimizer=optimizer, dtype=torch.bfloat16) 

请查看链接，https://intel.github.io/intel-extension-for-pytorch/cpu/latest/index.html和https://www.intel.com/content/www/us/en/developer/tools/oneapi/extension-for-pytorch.html了解更多关于PyTorch的英特尔扩展。

Answer 2

要启用混合精度，您可以直接访问github链接获取intel扩展的pytorch(https://github.com/intel/intel-extension-for-pytorch) .

For Float32 you can
# Invoke optimize function against the model object and optimizer object
model, optimizer = ipex.optimize(model, optimizer, dtype=torch.float32)

为BFloat16

# Invoke optimize function against the model object and optimizer object with data type set to torch.bfloat16
model, optimizer = ipex.optimize(model, optimizer, dtype=torch.bfloat16)