Not watched
361
0

#2 算子nn.Conv2d中group参数与dilation参数不兼容问题

Closed
created 3 years ago by zhangximing · 2 comments
| name | about | labels | | ---------- | ------------------------------------- | -------- | | Bug Report | Use this template for reporting a bug | kind/bug | ## Environment - **Hardware Environment(`Ascend`/`GPU`/`CPU`)**: 线下机器 /device ascend - **Software Environment**: -- MindSpore version (source or binary): MindSpore r1.2 -- Python version (e.g., Python 3.7.5): Python 3.7.6 -- OS platform and distribution (e.g., Linux Ubuntu 16.04): eulerosv2r8.aarch64 -- GCC/Compiler version (if compiled from source): 7.5.0 ## Related testcase 测试代码如下 ``` def test(): import numpy as np from mindspore import Tensor from mindspore import dtype as mstype in_channels = 320 shapes = 97 data = Tensor(np.random.rand(10, in_channels, shapes, shapes), mstype.float32) kernel_size = 3 op = nn.Conv2d(in_channels, in_channels, kernel_size, stride=1, dilation=2, group=in_channels, pad_mode='same') featuremap = op(data) print(featuremap) if __name__ == '__main__': test() ``` ## Steps to reproduce the issue 1. 实际使用代码请在gitee有发部分https://gitee.com/mindspore/mindspore/issues/I3I88M 2. 测试出是Conv2d的问题,直接运行上述代码即可复现问题 3. ## Describe the current behavior 对于一下涉及的shape,运行情况不同 | shape | DilConv_3x3 | DilConv_5x5 | | ------------------- | ----------- | ----------- | | (10, 320,97,97) | fail | pass | | (10, 640,49,49) | fail | fail | | (10,1280,25,25) | fail | fail | | (10,2560,13,13) | pass | pass | ### 补充说明 发现是nn.Conv2d 算子可能有问题: 1、不设置group(group=1)、仅设置空洞率(dilation=2)的空洞卷积不报错; 2、设置group(group=input_channels)、不设置空洞率(dilation=1)的depthwise separable 卷积不报错。 可能是conv2d算子的group与dilation之间有什么问题导致出错 ## Describe the expected behavior 以上代码能够跑通 ## Related log / screenshot ``` ssh://root@183.129.171.130:6483/root/archiconda3/envs/wks/bin/python3 -u /root/.pycharm_helpers/pydev/pydevd.py --multiproc --qt-support=auto --client 0.0.0.0 --port 36545 --file /home/HRBEU-MedAI/zhangximing/autodeeplab-mindspore-zhangximing/src/models/test.py Connected to pydev debugger (build 203.7148.72) [WARNING][2021-04-10 21:57:30,528]/root/archiconda3/envs/wks/lib/python3.7/site-packages/mindspore/common/dtype.py:145: DeprecationWarning: `np.str` is a deprecated alias for the builtin `str`. To silence this warning, use `str` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.str_` here. Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations [WARNING][2021-04-10 21:57:31,640]/root/archiconda3/envs/wks/lib/python3.7/site-packages/mindspore/train/checkpoint_pb2.py:23: DeprecationWarning: Call to deprecated create function FileDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool. [WARNING][2021-04-10 21:57:31,640]/root/archiconda3/envs/wks/lib/python3.7/site-packages/mindspore/train/checkpoint_pb2.py:42: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool. [WARNING][2021-04-10 21:57:31,641]/root/archiconda3/envs/wks/lib/python3.7/site-packages/mindspore/train/checkpoint_pb2.py:63: DeprecationWarning: Call to deprecated create function Descriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool. [WARNING][2021-04-10 21:57:31,645]/root/archiconda3/envs/wks/lib/python3.7/site-packages/mindspore/train/print_pb2.py:108: DeprecationWarning: Call to deprecated create function OneofDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool. [WARNING][2021-04-10 21:57:31,658]/root/archiconda3/envs/wks/lib/python3.7/site-packages/mindspore/train/mind_ir_pb2.py:37: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool. [WARNING][2021-04-10 21:57:31,659]/root/archiconda3/envs/wks/lib/python3.7/site-packages/mindspore/train/mind_ir_pb2.py:118: DeprecationWarning: Call to deprecated create function EnumDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool. [WARNING][2021-04-10 21:57:40,624]/root/archiconda3/envs/wks/lib/python3.7/site-packages/mindspore/common/dtype.py:145: DeprecationWarning: `np.str` is a deprecated alias for the builtin `str`. To silence this warning, use `str` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.str_` here. Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations [WARNING][2021-04-10 21:57:41,147]/root/archiconda3/envs/wks/lib/python3.7/site-packages/mindspore/train/checkpoint_pb2.py:23: DeprecationWarning: Call to deprecated create function FileDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool. [WARNING][2021-04-10 21:57:41,147]/root/archiconda3/envs/wks/lib/python3.7/site-packages/mindspore/train/checkpoint_pb2.py:42: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool. [WARNING][2021-04-10 21:57:41,147]/root/archiconda3/envs/wks/lib/python3.7/site-packages/mindspore/train/checkpoint_pb2.py:63: DeprecationWarning: Call to deprecated create function Descriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool. [WARNING][2021-04-10 21:57:41,148]/root/archiconda3/envs/wks/lib/python3.7/site-packages/mindspore/train/print_pb2.py:108: DeprecationWarning: Call to deprecated create function OneofDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool. [WARNING][2021-04-10 21:57:41,153]/root/archiconda3/envs/wks/lib/python3.7/site-packages/mindspore/train/mind_ir_pb2.py:37: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool. [WARNING][2021-04-10 21:57:41,153]/root/archiconda3/envs/wks/lib/python3.7/site-packages/mindspore/train/mind_ir_pb2.py:118: DeprecationWarning: Call to deprecated create function EnumDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool. [ERROR] KERNEL(24249,python3):2021-04-10-21:57:48.131.671 [mindspore/ccsrc/backend/kernel_compiler/tbe/tbe_kernel_parallel_build.cc:99] TbeOpParallelBuild] task compile Failed, task id:8, cause:TBEException:ERROR: [WARNING][2021-04-10 21:57:45,962]/usr/local/Ascend/opp/op_impl/built-in/ai_core/tbe/impl/abs.py:30: DeprecationWarning: register is expired, please replace it with the func register_op_compute [WARNING][2021-04-10 21:57:45,962]/usr/local/Ascend/opp/op_impl/built-in/ai_core/tbe/impl/abs.py:58: DeprecationWarning: te.utils.para_check is expired, please replace it with tbe.common.utils.para_check [WARNING][2021-04-10 21:57:46,052]/root/archiconda3/envs/wks/lib/python3.7/site-packages/topi/cce/util.py:97: DeprecationWarning: check_input_type in topi is expired, please do not use it [WARNING][2021-04-10 21:57:46,131]/usr/local/Ascend/opp/op_impl/built-in/ai_core/tbe/impl/util/util_frac_z.py:18: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3,and in 3.9 it will stop working [WARNING][2021-04-10 21:57:46,344]/root/archiconda3/envs/wks/lib/python3.7/site-packages/te/lang/cce/te_compute/depthwise_conv2d_compute.py:65: DeprecationWarning: te.lang.cce.te_compute.depthwise_conv2d_compute is expired, please replace it with the func tbe.dsl.compute.depthwise_conv2d_compute [WARNING][2021-04-10 21:57:46,345]/root/archiconda3/envs/wks/lib/python3.7/site-packages/topi/cce/util.py:868: DeprecationWarning: check_load3d_w_out_1_support in topi is expired, please do not use it [WARNING][2021-04-10 21:57:46,359]/root/archiconda3/envs/wks/lib/python3.7/site-packages/te/utils/cce.py:37: DeprecationWarning: te.lang.cce.auto_schedule is deprecated, please replace it with the tbe.dsl.auto_schedule Traceback (most recent call last): File "/root/archiconda3/envs/wks/lib/python3.7/site-packages/mindspore/_extends/parallel_compile/tbe_compiler/compiler.py", line 129, in build_op res = op_func(*inputs_args, *outputs_args, *attrs_args, kernel_name=kernel_name) File "/root/archiconda3/envs/wks/lib/python3.7/site-packages/tbe/common/utils/para_check.py", line 529, in _in_wrapper return func(*args, **kwargs) File "/usr/local/Ascend/opp/op_impl/built-in/ai_core/tbe/impl/depthwise_conv2d.py", line 414, in depthwise_conv2d tvm.build_module.build(sch, tensor_list, "cce", name=kernel_name) File "/root/archiconda3/envs/wks/lib/python3.7/site-packages/te/tvm/build_module.py", line 900, in build evaluates=evaluates) File "/root/archiconda3/envs/wks/lib/python3.7/site-packages/te/tvm/build_module.py", line 610, in lower return cce_lower(sch, args, name, binds, simple_mode, evaluates) File "/root/archiconda3/envs/wks/lib/python3.7/site-packages/te/tvm/cce_build_module.py", line 281, in cce_lower arg_list, cfg, evaluates, multi_process) File "/root/archiconda3/envs/wks/lib/python3.7/site-packages/te/tvm/cce_build_module.py", line 80, in cce_static_lower stmt = ir_pass.EmitInsn(stmt) File "/root/archiconda3/envs/wks/lib/python3.7/site-packages/te/tvm/_ffi/_ctypes/function.py", line 209, in __call__ raise get_last_ffi_error() tvm._ffi.base.TVMError: Traceback (most recent call last): [bt] (8) /usr/local/Ascend/fwkacllib/lib64/libtvm.so(+0x14b117c) [0xffff76dbc17c] [bt] (7) /usr/local/Ascend/fwkacllib/lib64/libtvm.so(tvm::ir::IRMutator::Mutate_(tvm::ir::ProducerConsumer const*, tvm::Stmt const&)+0x6c) [0xffff769230ec] [bt] (6) /usr/local/Ascend/fwkacllib/lib64/libtvm.so(tvm::ir::IRMutator::Mutate(tvm::Stmt)+0x6c) [0xffff765f658c] [bt] (5) /usr/local/Ascend/fwkacllib/lib64/libtvm.so(tvm::NodeFunctor<tvm::Stmt (tvm::runtime::ObjectRef const&, tvm::Stmt const&, tvm::ir::IRMutator*)>::operator()(tvm::runtime::ObjectRef const&, tvm::Stmt const&, tvm::ir::IRMutator*) const+0x5c) [0xffff765f628c] [bt] (4) /usr/local/Ascend/fwkacllib/lib64/libtvm.so(+0x14af8c4) [0xffff76dba8c4] [bt] (3) /usr/local/Ascend/fwkacllib/lib64/libtvm.so(tvm::ir::tileir::Tilefy::Mutate_(tvm::ir::AttrStmt const*, tvm::Stmt const&)+0x63c) [0xffff76f4efc4] [bt] (2) /usr/local/Ascend/fwkacllib/lib64/libtvm.so(std::_Function_handler<tvm::Stmt (tvm::Stmt), tvm::ir::tileir::Tilefy::str2function_::{lambda(tvm::Stmt)#4}>::_M_invoke(std::_Any_data const&, tvm::Stmt&&)+0x40) [0xffff76c72a70] [bt] (1) /usr/local/Ascend/fwkacllib/lib64/libtvm.so(tvm::ir::tileir::Tilefy::MutateSetFmatrix(tvm::Stmt const&)+0x66c) [0xffff76f35a1c] [bt] (0) /usr/local/Ascend/fwkacllib/lib64/libtvm.so(dmlc::LogMessageFatal::~LogMessageFatal()+0x4c) [0xffff7658ce6c] File "tilefy_pass.cc", line 2163 TVMError: [EB0000] Check failed: set_fmatrix_shape_[1] % ow == 0 (24 vs. 0) : W tiling unsupportedCurrent IR Stmt:im2col_row_major.local.L1[((((((i1.c*171072) + (i2.c*144)) + (i3.c*144)) + (i4.c*48)) + (i5.c*16)) + i6.c)] = select((((((i4.c*2) + floordiv(i2.c, 97)) < 2) || (((i5.c*2) + floormod(i2.c, 97)) < 2)) || (98 < ((i5.c*2) + floormod(i2.c, 97)))), 0h, fmap.local.L1[((((((i1.c*24832) + (i4.c*3104)) + (i5.c*32)) + (i2.c*16)) + i6.c) - 3136)]) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/root/archiconda3/envs/wks/lib/python3.7/site-packages/mindspore/_extends/parallel_compile/tbe_compiler/compiler.py", line 174, in <module> result = compile_with_json(in_args) File "/root/archiconda3/envs/wks/lib/python3.7/site-packages/mindspore/_extends/parallel_compile/tbe_compiler/compiler.py", line 168, in compile_with_json ret = build_op(op_build, json_str, None) File "/root/archiconda3/envs/wks/lib/python3.7/site-packages/mindspore/_extends/parallel_compile/tbe_compiler/compiler.py", line 135, in build_op raise RuntimeError(e) RuntimeError: Traceback (most recent call last): [bt] (8) /usr/local/Ascend/fwkacllib/lib64/libtvm.so(+0x14b117c) [0xffff76dbc17c] [bt] (7) /usr/local/Ascend/fwkacllib/lib64/libtvm.so(tvm::ir::IRMutator::Mutate_(tvm::ir::ProducerConsumer const*, tvm::Stmt const&)+0x6c) [0xffff769230ec] [bt] (6) /usr/local/Ascend/fwkacllib/lib64/libtvm.so(tvm::ir::IRMutator::Mutate(tvm::Stmt)+0x6c) [0xffff765f658c] [bt] (5) /usr/local/Ascend/fwkacllib/lib64/libtvm.so(tvm::NodeFunctor<tvm::Stmt (tvm::runtime::ObjectRef const&, tvm::Stmt const&, tvm::ir::IRMutator*)>::operator()(tvm::runtime::ObjectRef const&, tvm::Stmt const&, tvm::ir::IRMutator*) const+0x5c) [0xffff765f628c] [bt] (4) /usr/local/Ascend/fwkacllib/lib64/libtvm.so(+0x14af8c4) [0xffff76dba8c4] [bt] (3) /usr/local/Ascend/fwkacllib/lib64/libtvm.so(tvm::ir::tileir::Tilefy::Mutate_(tvm::ir::AttrStmt const*, tvm::Stmt const&)+0x63c) [0xffff76f4efc4] [bt] (2) /usr/local/Ascend/fwkacllib/lib64/libtvm.so(std::_Function_handler<tvm::Stmt (tvm::Stmt), tvm::ir::tileir::Tilefy::str2function_::{lambda(tvm::Stmt)#4}>::_M_invoke(std::_Any_data const&, tvm::Stmt&&)+0x40) [0xffff76c72a70] [bt] (1) /usr/local/Ascend/fwkacllib/lib64/libtvm.so(tvm::ir::tileir::Tilefy::MutateSetFmatrix(tvm::Stmt const&)+0x66c) [0xffff76f35a1c] [bt] (0) /usr/local/Ascend/fwkacllib/lib64/libtvm.so(dmlc::LogMessageFatal::~LogMessageFatal()+0x4c) [0xffff7658ce6c] File "tilefy_pass.cc", line 2163 TVMError: [EB0000] Check failed: set_fmatrix_shape_[1] % ow == 0 (24 vs. 0) : W tiling unsupportedCurrent IR Stmt:im2col_row_major.local.L1[((((((i1.c*171072) + (i2.c*144)) + (i3.c*144)) + (i4.c*48)) + (i5.c*16)) + i6.c)] = select((((((i4.c*2) + floordiv(i2.c, 97)) < 2) || (((i5.c*2) + floormod(i2.c, 97)) < 2)) || (98 < ((i5.c*2) + floormod(i2.c, 97)))), 0h, fmap.local.L1[((((((i1.c*24832) + (i4.c*3104)) + (i5.c*32)) + (i2.c*16)) + i6.c) - 3136)]) input_args: {"SocInfo":{"autoTilingMode":"NO_TUNE","coreNum":"","coreType":"","l1Fusion":"false","l2Fusion":"false","l2Mode":"2","op_debug_level":"","op_impl_mode":"","op_impl_mode_list":[],"socVersion":"Ascend910B"},"impl_path":"","op_info":{"Type":"DepthwiseConv2dNative","attr_desc":[[1,1,1,1],[1,1,2,2],[2,2,2,2],"NCHW",0],"attrs":[{"name":"stride","valid":true,"value":[1,1,1,1]},{"name":"dilation","valid":true,"value":[1,1,2,2]},{"name":"pad_list","valid":true,"value":[2,2,2,2]},{"name":"format","valid":true,"value":"NCHW"},{"name":"offset_a","valid":true,"value":0}],"full_name":"Default/DepthwiseConv2dNative-op8","gen_model":"single","graph_id":1,"inputs":[[{"dtype":"float16","format":"NC1HWC0","name":"x_0","ori_format":"NCHW","ori_shape":[10,320,97,97],"param_type":"required","range":[[10,10],[320,320],[97,97],[97,97]],"shape":[10,20,97,97,16],"valid":true}],[{"dtype":"float16","format":"C1HWNCoC0","name":"filter_0","ori_format":"NCHW","ori_shape":[1,320,3,3],"param_type":"required","range":[[1,1],[320,320],[3,3],[3,3]],"shape":[20,3,3,1,16,16],"valid":true}],[{"name":"bias_optional_2","valid":false}],[{"name":"offset_w_optional_2","valid":false}]],"is_dynamic_shape":false,"kernel_name":"DepthwiseConv2dNative_8172484865844151394_6","module_name":"impl.depthwise_conv2d","name":"depthwise_conv2d","outputs":[[{"dtype":"float16","format":"NC1HWC0","name":"y","ori_format":"NCHW","ori_shape":[10,320,97,97],"param_type":"required","range":[[10,10],[320,320],[97,97],[97,97]],"shape":[10,20,97,97,16],"valid":true}]],"py_module_path":"/usr/local/Ascend/opp/op_impl/built-in/ai_core/tbe","socVersion":"Ascend910B"},"platform":"TBE"} trace: Traceback (most recent call last): File "/root/.pycharm_helpers/pydev/pydevd.py", line 1477, in _exec pydev_imports.execfile(file, globals, locals) # execute the script File "/root/.pycharm_helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile python-BaseException exec(compile(contents+"\n", file, 'exec'), glob, loc) File "/home/HRBEU-MedAI/zhangximing/autodeeplab-mindspore-zhangximing/src/models/test.py", line 56, in <module> test() File "/home/HRBEU-MedAI/zhangximing/autodeeplab-mindspore-zhangximing/src/models/test.py", line 50, in test featuremap = op(data) File "/root/archiconda3/envs/wks/lib/python3.7/site-packages/mindspore/nn/cell.py", line 365, in __call__ output = self.run_construct(cast_inputs, kwargs) File "/root/archiconda3/envs/wks/lib/python3.7/site-packages/mindspore/nn/cell.py", line 323, in run_construct output = self.construct(*cast_inputs, **kwargs) File "/home/HRBEU-MedAI/zhangximing/autodeeplab-mindspore-zhangximing/src/models/test.py", line 34, in construct return self.op(x) File "/root/archiconda3/envs/wks/lib/python3.7/site-packages/mindspore/nn/cell.py", line 365, in __call__ output = self.run_construct(cast_inputs, kwargs) File "/root/archiconda3/envs/wks/lib/python3.7/site-packages/mindspore/nn/cell.py", line 323, in run_construct output = self.construct(*cast_inputs, **kwargs) File "/root/archiconda3/envs/wks/lib/python3.7/site-packages/mindspore/nn/layer/container.py", line 235, in construct input_data = cell(input_data) File "/root/archiconda3/envs/wks/lib/python3.7/site-packages/mindspore/nn/cell.py", line 365, in __call__ output = self.run_construct(cast_inputs, kwargs) File "/root/archiconda3/envs/wks/lib/python3.7/site-packages/mindspore/nn/cell.py", line 323, in run_construct output = self.construct(*cast_inputs, **kwargs) File "/root/archiconda3/envs/wks/lib/python3.7/site-packages/mindspore/nn/layer/conv.py", line 258, in construct output = self.conv2d(x, self.weight) File "/root/archiconda3/envs/wks/lib/python3.7/site-packages/mindspore/ops/primitive.py", line 188, in __call__ return _run_op(self, self.name, args) File "/root/archiconda3/envs/wks/lib/python3.7/site-packages/mindspore/common/api.py", line 75, in wrapper results = fn(*arg, **kwargs) File "/root/archiconda3/envs/wks/lib/python3.7/site-packages/mindspore/ops/primitive.py", line 556, in _run_op output = real_run_op(obj, op_name, args) RuntimeError: mindspore/ccsrc/backend/kernel_compiler/tbe/tbe_kernel_parallel_build.cc:99 TbeOpParallelBuild] task compile Failed, task id:8, cause:TBEException:ERROR: [WARNING][2021-04-10 21:57:45,962]/usr/local/Ascend/opp/op_impl/built-in/ai_core/tbe/impl/abs.py:30: DeprecationWarning: register is expired, please replace it with the func register_op_compute [WARNING][2021-04-10 21:57:45,962]/usr/local/Ascend/opp/op_impl/built-in/ai_core/tbe/impl/abs.py:58: DeprecationWarning: te.utils.para_check is expired, please replace it with tbe.common.utils.para_check [WARNING][2021-04-10 21:57:46,052]/root/archiconda3/envs/wks/lib/python3.7/site-packages/topi/cce/util.py:97: DeprecationWarning: check_input_type in topi is expired, please do not use it [WARNING][2021-04-10 21:57:46,131]/usr/local/Ascend/opp/op_impl/built-in/ai_core/tbe/impl/util/util_frac_z.py:18: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3,and in 3.9 it will stop working [WARNING][2021-04-10 21:57:46,344]/root/archiconda3/envs/wks/lib/python3.7/site-packages/te/lang/cce/te_compute/depthwise_conv2d_compute.py:65: DeprecationWarning: te.lang.cce.te_compute.depthwise_conv2d_compute is expired, please replace it with the func tbe.dsl.compute.depthwise_conv2d_compute [WARNING][2021-04-10 21:57:46,345]/root/archiconda3/envs/wks/lib/python3.7/site-packages/topi/cce/util.py:868: DeprecationWarning: check_load3d_w_out_1_support in topi is expired, please do not use it [WARNING][2021-04-10 21:57:46,359]/root/archiconda3/envs/wks/lib/python3.7/site-packages/te/utils/cce.py:37: DeprecationWarning: te.lang.cce.auto_schedule is deprecated, please replace it with the tbe.dsl.auto_schedule Traceback (most recent call last): File "/root/archiconda3/envs/wks/lib/python3.7/site-packages/mindspore/_extends/parallel_compile/tbe_compiler/compiler.py", line 129, in build_op res = op_func(*inputs_args, *outputs_args, *attrs_args, kernel_name=kernel_name) File "/root/archiconda3/envs/wks/lib/python3.7/site-packages/tbe/common/utils/para_check.py", line 529, in _in_wrapper return func(*args, **kwargs) File "/usr/local/Ascend/opp/op_impl/built-in/ai_core/tbe/impl/depthwise_conv2d.py", line 414, in depthwise_conv2d tvm.build_module.build(sch, tensor_list, "cce", name=kernel_name) File "/root/archiconda3/envs/wks/lib/python3.7/site-packages/te/tvm/build_module.py", line 900, in build evaluates=evaluates) File "/root/archiconda3/envs/wks/lib/python3.7/site-packages/te/tvm/build_module.py", line 610, in lower return cce_lower(sch, args, name, binds, simple_mode, evaluates) File "/root/archiconda3/envs/wks/lib/python3.7/site-packages/te/tvm/cce_build_module.py", line 281, in cce_lower arg_list, cfg, evaluates, multi_process) File "/root/archiconda3/envs/wks/lib/python3.7/site-packages/te/tvm/cce_build_module.py", line 80, in cce_static_lower stmt = ir_pass.EmitInsn(stmt) File "/root/archiconda3/envs/wks/lib/python3.7/site-packages/te/tvm/_ffi/_ctypes/function.py", line 209, in __call__ raise get_last_ffi_error() tvm._ffi.base.TVMError: Traceback (most recent call last): [bt] (8) /usr/local/Ascend/fwkacllib/lib64/libtvm.so(+0x14b117c) [0xffff76dbc17c] [bt] (7) /usr/local/Ascend/fwkacllib/lib64/libtvm.so(tvm::ir::IRMutator::Mutate_(tvm::ir::ProducerConsumer const*, tvm::Stmt const&)+0x6c) [0xffff769230ec] [bt] (6) /usr/local/Ascend/fwkacllib/lib64/libtvm.so(tvm::ir::IRMutator::Mutate(tvm::Stmt)+0x6c) [0xffff765f658c] [bt] (5) /usr/local/Ascend/fwkacllib/lib64/libtvm.so(tvm::NodeFunctor<tvm::Stmt (tvm::runtime::ObjectRef const&, tvm::Stmt const&, tvm::ir::IRMutator*)>::operator()(tvm::runtime::ObjectRef const&, tvm::Stmt const&, tvm::ir::IRMutator*) const+0x5c) [0xffff765f628c] [bt] (4) /usr/local/Ascend/fwkacllib/lib64/libtvm.so(+0x14af8c4) [0xffff76dba8c4] [bt] (3) /usr/local/Ascend/fwkacllib/lib64/libtvm.so(tvm::ir::tileir::Tilefy::Mutate_(tvm::ir::AttrStmt const*, tvm::Stmt const&)+0x63c) [0xffff76f4efc4] [bt] (2) /usr/local/Ascend/fwkacllib/lib64/libtvm.so(std::_Function_handler<tvm::Stmt (tvm::Stmt), tvm::ir::tileir::Tilefy::str2function_::{lambda(tvm::Stmt)#4}>::_M_invoke(std::_Any_data const&, tvm::Stmt&&)+0x40) [0xffff76c72a70] [bt] (1) /usr/local/Ascend/fwkacllib/lib64/libtvm.so(tvm::ir::tileir::Tilefy::MutateSetFmatrix(tvm::Stmt const&)+0x66c) [0xffff76f35a1c] [bt] (0) /usr/local/Ascend/fwkacllib/lib64/libtvm.so(dmlc::LogMessageFatal::~LogMessageFatal()+0x4c) [0xffff7658ce6c] File "tilefy_pass.cc", line 2163 TVMError: [EB0000] Check failed: set_fmatrix_shape_[1] % ow == 0 (24 vs. 0) : W tiling unsupportedCurrent IR Stmt:im2col_row_major.local.L1[((((((i1.c*171072) + (i2.c*144)) + (i3.c*144)) + (i4.c*48)) + (i5.c*16)) + i6.c)] = select((((((i4.c*2) + floordiv(i2.c, 97)) < 2) || (((i5.c*2) + floormod(i2.c, 97)) < 2)) || (98 < ((i5.c*2) + floormod(i2.c, 97)))), 0h, fmap.local.L1[((((((i1.c*24832) + (i4.c*3104)) + (i5.c*32)) + (i2.c*16)) + i6.c) - 3136)]) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/root/archiconda3/envs/wks/lib/python3.7/site-packages/mindspore/_extends/parallel_compile/tbe_compiler/compiler.py", line 174, in <module> result = compile_with_json(in_args) File "/root/archiconda3/envs/wks/lib/python3.7/site-packages/mindspore/_extends/parallel_compile/tbe_compiler/compiler.py", line 168, in compile_with_json ret = build_op(op_build, json_str, None) File "/root/archiconda3/envs/wks/lib/python3.7/site-packages/mindspore/_extends/parallel_compile/tbe_compiler/compiler.py", line 135, in build_op raise RuntimeError(e) RuntimeError: Traceback (most recent call last): [bt] (8) /usr/local/Ascend/fwkacllib/lib64/libtvm.so(+0x14b117c) [0xffff76dbc17c] [bt] (7) /usr/local/Ascend/fwkacllib/lib64/libtvm.so(tvm::ir::IRMutator::Mutate_(tvm::ir::ProducerConsumer const*, tvm::Stmt const&)+0x6c) [0xffff769230ec] [bt] (6) /usr/local/Ascend/fwkacllib/lib64/libtvm.so(tvm::ir::IRMutator::Mutate(tvm::Stmt)+0x6c) [0xffff765f658c] [bt] (5) /usr/local/Ascend/fwkacllib/lib64/libtvm.so(tvm::NodeFunctor<tvm::Stmt (tvm::runtime::ObjectRef const&, tvm::Stmt const&, tvm::ir::IRMutator*)>::operator()(tvm::runtime::ObjectRef const&, tvm::Stmt const&, tvm::ir::IRMutator*) const+0x5c) [0xffff765f628c] [bt] (4) /usr/local/Ascend/fwkacllib/lib64/libtvm.so(+0x14af8c4) [0xffff76dba8c4] [bt] (3) /usr/local/Ascend/fwkacllib/lib64/libtvm.so(tvm::ir::tileir::Tilefy::Mutate_(tvm::ir::AttrStmt const*, tvm::Stmt const&)+0x63c) [0xffff76f4efc4] [bt] (2) /usr/local/Ascend/fwkacllib/lib64/libtvm.so(std::_Function_handler<tvm::Stmt (tvm::Stmt), tvm::ir::tileir::Tilefy::str2function_::{lambda(tvm::Stmt)#4}>::_M_invoke(std::_Any_data const&, tvm::Stmt&&)+0x40) [0xffff76c72a70] [bt] (1) /usr/local/Ascend/fwkacllib/lib64/libtvm.so(tvm::ir::tileir::Tilefy::MutateSetFmatrix(tvm::Stmt const&)+0x66c) [0xffff76f35a1c] [bt] (0) /usr/local/Ascend/fwkacllib/lib64/libtvm.so(dmlc::LogMessageFatal::~LogMessageFatal()+0x4c) [0xffff7658ce6c] File "tilefy_pass.cc", line 2163 TVMError: [EB0000] Check failed: set_fmatrix_shape_[1] % ow == 0 (24 vs. 0) : W tiling unsupportedCurrent IR Stmt:im2col_row_major.local.L1[((((((i1.c*171072) + (i2.c*144)) + (i3.c*144)) + (i4.c*48)) + (i5.c*16)) + i6.c)] = select((((((i4.c*2) + floordiv(i2.c, 97)) < 2) || (((i5.c*2) + floormod(i2.c, 97)) < 2)) || (98 < ((i5.c*2) + floormod(i2.c, 97)))), 0h, fmap.local.L1[((((((i1.c*24832) + (i4.c*3104)) + (i5.c*32)) + (i2.c*16)) + i6.c) - 3136)]) input_args: {"SocInfo":{"autoTilingMode":"NO_TUNE","coreNum":"","coreType":"","l1Fusion":"false","l2Fusion":"false","l2Mode":"2","op_debug_level":"","op_impl_mode":"","op_impl_mode_list":[],"socVersion":"Ascend910B"},"impl_path":"","op_info":{"Type":"DepthwiseConv2dNative","attr_desc":[[1,1,1,1],[1,1,2,2],[2,2,2,2],"NCHW",0],"attrs":[{"name":"stride","valid":true,"value":[1,1,1,1]},{"name":"dilation","valid":true,"value":[1,1,2,2]},{"name":"pad_list","valid":true,"value":[2,2,2,2]},{"name":"format","valid":true,"value":"NCHW"},{"name":"offset_a","valid":true,"value":0}],"full_name":"Default/DepthwiseConv2dNative-op8","gen_model":"single","graph_id":1,"inputs":[[{"dtype":"float16","format":"NC1HWC0","name":"x_0","ori_format":"NCHW","ori_shape":[10,320,97,97],"param_type":"required","range":[[10,10],[320,320],[97,97],[97,97]],"shape":[10,20,97,97,16],"valid":true}],[{"dtype":"float16","format":"C1HWNCoC0","name":"filter_0","ori_format":"NCHW","ori_shape":[1,320,3,3],"param_type":"required","range":[[1,1],[320,320],[3,3],[3,3]],"shape":[20,3,3,1,16,16],"valid":true}],[{"name":"bias_optional_2","valid":false}],[{"name":"offset_w_optional_2","valid":false}]],"is_dynamic_shape":false,"kernel_name":"DepthwiseConv2dNative_8172484865844151394_6","module_name":"impl.depthwise_conv2d","name":"depthwise_conv2d","outputs":[[{"dtype":"float16","format":"NC1HWC0","name":"y","ori_format":"NCHW","ori_shape":[10,320,97,97],"param_type":"required","range":[[10,10],[320,320],[97,97],[97,97]],"shape":[10,20,97,97,16],"valid":true}]],"py_module_path":"/usr/local/Ascend/opp/op_impl/built-in/ai_core/tbe","socVersion":"Ascend910B"},"platform":"TBE"} trace: # ``` ## Special notes for this issue 这是gitee的issues链接: https://gitee.com/mindspore/mindspore/issues/I3I88M
zhangximing added the
mindspore-assistant
label 3 years ago
zhangximing added the
技术
label 3 years ago
zhangximing removed the
技术
label 3 years ago
zhangximing commented 3 years ago
Poster
问题尚未解决,bug尚未修复。目前使用替代方案来取代原来的nn.Conv2d,但是时间复杂度有点高。 ``` class DilSepCov(nn.Cell): def __init__(self, channels, kernel_size, stride, pad_mode='pad', padding=0, dilation=1, has_bias=False, weight_init='HeNormal' ): super(DilSepCov, self).__init__() self.channels = channels self.ops = nn.CellList([]) for i in range(channels): self.ops.append(nn.Conv2d(1, 1, kernel_size=kernel_size, stride=stride, pad_mode=pad_mode, padding=padding, dilation=dilation, has_bias=has_bias, weight_init=weight_init)) self.concat = ops.Concat(axis=1) def construct(self, x): output = () for i in range(1, self.channels): output += (self.ops[i](x[:, i:i+1:1, :, :]), ) output = self.concat(output) return output ```
zhangximing commented 2 years ago
Poster
mindspore r1.5版本已修复此bug,issue关闭
zhangximing closed this issue 2 years ago
Sign in to join this conversation.
No Milestone
No Assignees
1 Participants
Notifications
Due Date

No due date set.

Dependencies

This issue currently doesn't have any dependencies.

Loading…
There is no content yet.