[Bug][CUDAExecutionProvider] INVALID_ARGUMENT : unsupported conv activation mode "Sigmoid"

[Cookiee235]

Bug Report

The following script run with providers=["CPUExecutionProvider "] successfully. However, inference on the cuda with providers=["CUDAExecutionProvider "] failed.

Traceback (most recent call last):
  File "test.py", line 10, in <module>
    original_session = ort.InferenceSession(model_path, providers=["CUDAExecutionProvider"])
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/software/onnxruntime/build/Linux/Release/onnxruntime/capi/onnxruntime_inference_collection.py", line 465, in __init__
    self._create_inference_session(providers, provider_options, disabled_optimizers)
  File "/software/onnxruntime/build/Linux/Release/onnxruntime/capi/onnxruntime_inference_collection.py", line 537, in _create_inference_session
    sess.initialize_session(providers, provider_options, disabled_optimizers)
onnxruntime.capi.onnxruntime_pybind11_state.RuntimeException: [ONNXRuntimeError] : 6 : RUNTIME_EXCEPTION : Exception during initialization: /software/onnxruntime/onnxruntime/contrib_ops/cuda/fused_conv.cc:68 onnxruntime::contrib::cuda::FusedConv<T>::FusedConv(const onnxruntime::OpKernelInfo&) [with T = float] [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : unsupported conv activation mode "Sigmoid"

To reproduce

Step 1: Download the model via this link

Step 2: run the following script:

import onnx
import onnxruntime as ort

model_path = "sigmoid_crash.onnx"
model = onnx.load(model_path)
onnx.checker.check_model(model)
print(ort.get_available_providers())
original_session = ort.InferenceSession(model_path, providers=["CUDAExecutionProvider"])  # CPUExecutionProvider

Urgency

It's an urgent issue due to the reference to multiple similar bugs.

• ERROR: 2 : INVALID_ARGUMENT : unsupported conv activation mode "Tanh"
• ERROR: 2 : INVALID_ARGUMENT : unsupported conv activation mode "HardSigmoid"
• ERROR: 2 : INVALID_ARGUMENT : unsupported conv activation mode "LeakyRelu"
• ERROR: 2 : INVALID_ARGUMENT : unsupported conv activation mode "Clip"
• ERROR: 2 : INVALID_ARGUMENT : unsupported conv activation mode "Sigmoid"
• ERROR: 2 : INVALID_ARGUMENT : unsupported conv activation mode "Tanh"

Platform

Linux

OS Version

Ubuntu 12.04

ONNX Runtime Installation

Built from Source

ONNX Runtime Version or Commit ID

5c1b7cc

ONNX Runtime API

Python

Architecture

X64

Execution Provider

CUDA

Execution Provider Library Version

No response

[tianleiwu]

Currently only Relu is supported in FusedConv for Cuda:

onnxruntime/onnxruntime/contrib_ops/cuda/fused_conv.cc

Lines 415 to 421 in ae97068

	if (activaton_mode == "Relu") {
	activation_mode_ = cudnnActivationMode_t::CUDNN_ACTIVATION_RELU;
	} else {
	return ORT_MAKE_STATUS(
	StatusCategory::ONNXRUNTIME, StatusCode::INVALID_ARGUMENT,
	"unsupported conv activation mode \"", activaton_mode, "\"");
	}

Need some code change to support other activations that can be mapped to cuDNN:

• CUDNN_ACTIVATION_SIGMOID
• CUDNN_ACTIVATION_TANH
• CUDNN_ACTIVATION_SWISH
• CUDNN_ACTIVATION_CLIPPED_RELU
• CUDNN_ACTIVATION_ELU

To avoid the error, please use the original ONNX model with Conv operator and latest onnxruntime-gpu for inference.

Currently, cuda provider is excluded from Conv and activation fusion:

onnxruntime/onnxruntime/core/optimizer/graph_transformer_utils.cc

Line 341 in ae97068

transformers.emplace_back(std::make_unique<ConvActivationFusion>(cpu_rocm_acl_armnn_js_eps));

A model with FusedConv with Sigmoid is usually optimized for CPU provider, or is optimized by older binary.

[Cookiee235]

@tianleiwu Thank you for your response. I understand now. I'll keep this issue open to track the fix for activations in cuDNN.