Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] InvalidArgument Error After Optimizing Model with ONNX Runtime #23138

Open
Thrsu opened this issue Dec 18, 2024 · 1 comment
Open

[Bug] InvalidArgument Error After Optimizing Model with ONNX Runtime #23138

Thrsu opened this issue Dec 18, 2024 · 1 comment
Labels
model:transformer issues related to a transformer model: BERT, GPT2, Hugging Face, Longformer, T5, etc.

Comments

@Thrsu
Copy link

Thrsu commented Dec 18, 2024

Describe the issue

After optimizing an ONNX model using ONNX Runtime, I encounter an error when running the optimized model. The error message indicates that the Gelu node expects an input with at least 1 dimension, but the input received has 0 dimensions. This issue does not occur with the original model and it occurs regardless of the optimization level (opt_level=0, opt_level=1, opt_level=99).

Error Message:

Traceback (most recent call last):
  File "/share_container/optfuzz/ONNX/reproduce/Gelu_123.py", line 14, in <module>
    optimized_result = optimized_session.run(output_names, input_data)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/software/onnxruntime/build/Linux/Release/onnxruntime/capi/onnxruntime_inference_collection.py", line 266, in run
    return self._sess.run(output_names, input_feed, run_options)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Non-zero status code returned while running Gelu node. Name:'Gelu' Status Message: Input 0 is expected to have 1 or more dimensions, got 0

To reproduce

  1. Download the model
  2. Run the below script:
import onnx
import onnxruntime as ort
import numpy as np
from onnxruntime.transformers import optimizer

model_path = "Gelu.onnx"
optimized_model_path = f"./opt.onnx"
optimized_model = optimizer.optimize_model(model_path, opt_level=99, use_gpu=True)
optimized_model.save_model_to_file(optimized_model_path)
this_provider_list = ort.get_available_providers()
optimized_session = ort.InferenceSession(optimized_model_path, providers=this_provider_list)
input_data = {'v8_0': np.random.rand(1, 1).astype(np.float32)}
output_names = [output.name for output in optimized_session.get_outputs()]
optimized_result = optimized_session.run(output_names, input_data)

Urgency

No response

Platform

Linux

OS Version

Ubuntu 20.04

ONNX Runtime Installation

Built from Source

ONNX Runtime Version or Commit ID

5c1b7cc

ONNX Runtime API

Python

Architecture

X64

Execution Provider

CUDA

Execution Provider Library Version

No response

@github-actions github-actions bot added the model:transformer issues related to a transformer model: BERT, GPT2, Hugging Face, Longformer, T5, etc. label Dec 18, 2024
@tianleiwu
Copy link
Contributor

tianleiwu commented Dec 18, 2024

If your model need the Gelu fusion, you can remove the Squeeze before Div node as in the following graph. Given a tensor with shape [1], applying squeeze() on axis 0 will remove this dimension, resulting in a tensor with an empty shape []. This signifies a scalar tensor, containing a single element without any dimensions. However, Gelu operator currently does not work on scalar.

Image.

For the Gelu fusion bug, we will take a look to add some checking later to avoid such case. Thanks for reporting.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
model:transformer issues related to a transformer model: BERT, GPT2, Hugging Face, Longformer, T5, etc.
Projects
None yet
Development

No branches or pull requests

2 participants