Skip to content

IndexError: _Map_base::at (convert pytorch model => onnx => caffe2) #2458

Closed
@zchrissirhcz

Description

@zchrissirhcz

Aim

Convert a pytorch model to onnx then to caffe2.

Environment

Linux(ubuntu16.04)
cuda10.0
miniconda with python 3.7.4

pytorch: build from source (master branch, commit id c300f08)
onnx: tried pip install onnx 1.6 and 1.5, both gives same result

The result of conda env export:

name: base
channels:
  - pytorch
  - https://repo.anaconda.com/pkgs/main
  - defaults
dependencies:
  - _libgcc_mutex=0.1=main
  - asn1crypto=1.2.0=py37_0
  - blas=1.0=mkl
  - bzip2=1.0.8=h7b6447c_0
  - ca-certificates=2019.10.16=0
  - certifi=2019.9.11=py37_0
  - cffi=1.13.2=py37h2e261b9_0
  - chardet=3.0.4=py37_1003
  - cmake=3.14.0=h52cb24c_0
  - conda=4.7.12=py37_0
  - conda-package-handling=1.6.0=py37h7b6447c_0
  - cryptography=2.8=py37h1ba5d50_0
  - expat=2.2.6=he6710b0_0
  - idna=2.8=py37_0
  - intel-openmp=2019.4=243
  - krb5=1.16.1=h173b8e3_7
  - libcurl=7.65.3=h20c2e04_0
  - libedit=3.1.20181209=hc058e9b_0
  - libffi=3.2.1=hd88cf55_4
  - libgcc-ng=9.1.0=hdf63c60_0
  - libgfortran-ng=7.3.0=hdf63c60_0
  - libssh2=1.8.2=h1ba5d50_0
  - libstdcxx-ng=9.1.0=hdf63c60_0
  - magma-cuda100=2.5.1=1
  - mkl=2019.4=243
  - mkl-include=2019.4=243
  - mkl-service=2.3.0=py37he904b0f_0
  - mkl_fft=1.0.15=py37ha843d7b_0
  - mkl_random=1.1.0=py37hd6b4f25_0
  - ncurses=6.1=he6710b0_1
  - ninja=1.9.0=py37hfd86e86_0
  - numpy=1.17.3=py37hd14ec0e_0
  - numpy-base=1.17.3=py37hde5b4d6_0
  - openssl=1.1.1d=h7b6447c_3
  - pip=19.3.1=py37_0
  - pycosat=0.6.3=py37h14c3975_0
  - pycparser=2.19=py37_0
  - pyopenssl=19.0.0=py37_0
  - pysocks=1.7.1=py37_0
  - python=3.7.4=h265db76_1
  - readline=7.0=h7b6447c_5
  - requests=2.22.0=py37_0
  - rhash=1.3.8=h1ba5d50_0
  - ruamel_yaml=0.15.46=py37h14c3975_0
  - setuptools=41.6.0=py37_0
  - six=1.12.0=py37_0
  - sqlite=3.30.0=h7b6447c_0
  - tk=8.6.8=hbc83047_0
  - tqdm=4.36.1=py_0
  - typing=3.6.4=py37_0
  - urllib3=1.24.2=py37_0
  - wheel=0.33.6=py37_0
  - xz=5.2.4=h14c3975_4
  - yaml=0.1.7=had09818_2
  - zlib=1.2.11=h7b6447c_3
  - pip:
    - future==0.18.2
    - onnx==1.5.0
    - pillow==6.2.1
    - protobuf==3.10.0
    - pyyaml==5.1.2
    - torch==1.3.1
    - torchvision==0.4.2
    - typing-extensions==3.7.4.1
prefix: /home/zz/soft/miniconda3

Reproceduce

I followed the instructions and code in the official site: TRANSFERING A MODEL FROM PYTORCH TO CAFFE2 AND MOBILE USING ONNX.

The code I run is:

#!/usr/bin/env python
#coding: utf-8

# Some standard imports
import io
import numpy as np

from torch import nn
import torch.utils.model_zoo as model_zoo
import torch.onnx

# Super Resolution model definition in PyTorch
import torch.nn as nn
import torch.nn.init as init


class SuperResolutionNet(nn.Module):
    def __init__(self, upscale_factor, inplace=False):
        super(SuperResolutionNet, self).__init__()

        self.relu = nn.ReLU(inplace=inplace)
        self.conv1 = nn.Conv2d(1, 64, (5, 5), (1, 1), (2, 2))
        self.conv2 = nn.Conv2d(64, 64, (3, 3), (1, 1), (1, 1))
        self.conv3 = nn.Conv2d(64, 32, (3, 3), (1, 1), (1, 1))
        self.conv4 = nn.Conv2d(32, upscale_factor ** 2, (3, 3), (1, 1), (1, 1))
        self.pixel_shuffle = nn.PixelShuffle(upscale_factor)

        self._initialize_weights()

    def forward(self, x):
        x = self.relu(self.conv1(x))
        x = self.relu(self.conv2(x))
        x = self.relu(self.conv3(x))
        x = self.pixel_shuffle(self.conv4(x))
        return x

    def _initialize_weights(self):
        init.orthogonal_(self.conv1.weight, init.calculate_gain('relu'))
        init.orthogonal_(self.conv2.weight, init.calculate_gain('relu'))
        init.orthogonal_(self.conv3.weight, init.calculate_gain('relu'))
        init.orthogonal_(self.conv4.weight)

# Create the super-resolution model by using the above model definition.
torch_model = SuperResolutionNet(upscale_factor=3)

# Load pretrained model weights
model_url = 'https://s3.amazonaws.com/pytorch/test_data/export/superres_epoch100-44c6958e.pth'
batch_size = 1    # just a random number

# Initialize model with the pretrained weights
map_location = lambda storage, loc: storage
if torch.cuda.is_available():
    map_location = None
torch_model.load_state_dict(model_zoo.load_url(model_url, map_location=map_location))

# set the train mode to false since we will only run the forward pass.
torch_model.train(False)

# Input to the model
x = torch.randn(batch_size, 1, 224, 224, requires_grad=True)

# Export the model
torch_out = torch.onnx._export(torch_model,             # model being run
                               x,                       # model input (or a tuple for multiple inputs)
                               "super_resolution.onnx", # where to save the model (can be a file or file-like object)
                               export_params=True)      # store the trained parameter weights inside the model file






import onnx
import caffe2.python.onnx.backend as onnx_caffe2_backend

# Load the ONNX ModelProto object. model is a standard Python protobuf object
model = onnx.load("super_resolution.onnx")

# prepare the caffe2 backend for executing the model this converts the ONNX model into a
# Caffe2 NetDef that can execute it. Other ONNX backends, like one for CNTK will be
# availiable soon.
prepared_backend = onnx_caffe2_backend.prepare(model)

# run the model in Caffe2

# Construct a map from input names to Tensor data.
# The graph of the model itself contains inputs for all weight parameters, after the input image.
# Since the weights are already embedded, we just need to pass the input image.
# Set the first input.
W = {model.graph.input[0].name: x.data.numpy()}

# Run the Caffe2 net:
c2_out = prepared_backend.run(W)[0]

# Verify the numerical correctness upto 3 decimal places
np.testing.assert_almost_equal(torch_out.data.cpu().numpy(), c2_out, decimal=3)

print("Exported model has been executed on Caffe2 backend, and the result looks good!")


# extract the workspace and the model proto from the internal representation
c2_workspace = prepared_backend.workspace
c2_model = prepared_backend.predict_net

# Now import the caffe2 mobile exporter
from caffe2.python.predictor import mobile_exporter

# call the Export to get the predict_net, init_net. These nets are needed for running things on mobile
init_net, predict_net = mobile_exporter.Export(c2_workspace, c2_model, c2_model.external_input)

# Let's also save the init_net and predict_net to a file that we will later use for running them on mobile
with open('init_net.pb', "wb") as fopen:
    fopen.write(init_net.SerializeToString())
with open('predict_net.pb', "wb") as fopen:
    fopen.write(predict_net.SerializeToString())

Running result:

Traceback (most recent call last):
  File "sr.py", line 82, in <module>
    prepared_backend = onnx_caffe2_backend.prepare(model)
  File "/home/zz/soft/miniconda3/lib/python3.7/site-packages/caffe2/python/onnx/backend.py", line 713, in prepare
    init_net, predict_net = cls._onnx_model_to_caffe2_net(model, device, opset_version, False)
  File "/home/zz/soft/miniconda3/lib/python3.7/site-packages/caffe2/python/onnx/backend.py", line 876, in _onnx_model_to_caffe2_net
    onnx_model = onnx.utils.polish_model(onnx_model)
  File "/home/zz/soft/miniconda3/lib/python3.7/site-packages/onnx/utils.py", line 21, in polish_model
    model = onnx.optimizer.optimize(model)
  File "/home/zz/soft/miniconda3/lib/python3.7/site-packages/onnx/optimizer.py", line 55, in optimize
    optimized_model_str = C.optimize(model_str, passes)
IndexError: _Map_base::at

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @zchrissirhcz@daquexian

        Issue actions

          IndexError: _Map_base::at (convert pytorch model => onnx => caffe2) · Issue #2458 · onnx/onnx