Skip to content

RK3588跑示例ocr模型,输出乱码,但是图片标注正确。 #2218

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
7288Fzq opened this issue Oct 2, 2023 · 4 comments
Closed

Comments

@7288Fzq
Copy link

7288Fzq commented Oct 2, 2023

`blueberry@poodle:~/FastDeploy/examples/vision/ocr/PP-OCR/rockchip/cpp/build$ ./infer_demo ch_PP-OCRv3_det_infer_rk3588_unquantized.rknn ch_ppocr_mobile_v20_cls_infer_rk3588_unquantized.rknn ch_PP-OCRv3_rec_infer_rk3588_unquantized.rknn ppocr_keys_v1.txt 12.jpg 1
[INFO] fastdeploy/runtime/backends/rknpu2/rknpu2_backend.cc(81)::GetSDKAndDeviceVersion rknpu2 runtime version: 1.5.2 (c6b7b351a@2023-08-23T15:28:22)
[INFO] fastdeploy/runtime/backends/rknpu2/rknpu2_backend.cc(82)::GetSDKAndDeviceVersion rknpu2 driver version: 0.8.2
index=0, name=x, n_dims=4, dims=[1, 960, 960, 3], n_elems=2764800, size=5529600, fmt=NHWC, type=FP16, qnt_type=AFFINE, zp=0, scale=1.000000, pass_through=0
index=0, name=sigmoid_0.tmp_0, n_dims=4, dims=[1, 1, 960, 960], n_elems=921600, size=1843200, fmt=NCHW, type=FP32, qnt_type=AFFINE, zp=0, scale=1.000000, pass_through=0
[INFO] fastdeploy/runtime/runtime.cc(367)::CreateRKNPU2Backend Runtime initialized with Backend::RKNPU2 in Device::RKNPU.
[INFO] fastdeploy/runtime/backends/rknpu2/rknpu2_backend.cc(81)::GetSDKAndDeviceVersion rknpu2 runtime version: 1.5.2 (c6b7b351a@2023-08-23T15:28:22)
[INFO] fastdeploy/runtime/backends/rknpu2/rknpu2_backend.cc(82)::GetSDKAndDeviceVersion rknpu2 driver version: 0.8.2
index=0, name=x, n_dims=4, dims=[1, 48, 192, 3], n_elems=27648, size=55296, fmt=NHWC, type=FP16, qnt_type=AFFINE, zp=0, scale=1.000000, pass_through=0
index=0, name=softmax_0.tmp_0, n_dims=2, dims=[1, 2, 0, 0], n_elems=2, size=4, fmt=UNDEFINED, type=FP32, qnt_type=AFFINE, zp=0, scale=1.000000, pass_through=0
[INFO] fastdeploy/runtime/runtime.cc(367)::CreateRKNPU2Backend Runtime initialized with Backend::RKNPU2 in Device::RKNPU.
[INFO] fastdeploy/runtime/backends/rknpu2/rknpu2_backend.cc(81)::GetSDKAndDeviceVersion rknpu2 runtime version: 1.5.2 (c6b7b351a@2023-08-23T15:28:22)
[INFO] fastdeploy/runtime/backends/rknpu2/rknpu2_backend.cc(82)::GetSDKAndDeviceVersion rknpu2 driver version: 0.8.2
index=0, name=x, n_dims=4, dims=[1, 48, 320, 3], n_elems=46080, size=92160, fmt=NHWC, type=FP16, qnt_type=AFFINE, zp=0, scale=1.000000, pass_through=0
index=0, name=softmax_5.tmp_0, n_dims=4, dims=[1, 40, 6625, 1], n_elems=265000, size=530000, fmt=NCHW, type=FP32, qnt_type=AFFINE, zp=0, scale=1.000000, pass_through=0
[INFO] fastdeploy/runtime/runtime.cc(367)::CreateRKNPU2Backend Runtime initialized with Backend::RKNPU2 in Device::RKNPU.
[WARNING] fastdeploy/runtime/backends/rknpu2/rknpu2_backend.cc(420)::InitRKNNTensorMemory The input tensor type != model's inputs type.The input_type need FP16,but inputs[0].type is UINT8
[WARNING] fastdeploy/runtime/backends/rknpu2/rknpu2_backend.cc(420)::InitRKNNTensorMemory The input tensor type != model's inputs type.The input_type need FP16,but inputs[0].type is UINT8
[WARNING] fastdeploy/runtime/backends/rknpu2/rknpu2_backend.cc(420)::InitRKNNTensorMemory The input tensor type != model's inputs type.The input_type need FP16,but inputs[0].type is UINT8
det boxes: [[276,174],[285,173],[285,178],[276,179]]rec text: 洗酸兼滋染斜洗免 rec score:0.996582 cls label: 1 cls score: 0.766602
det boxes: [[43,408],[483,390],[483,431],[44,449]]rec text: 洗洗武斜武酸兼滋龈斜洗免 rec score:0.953939 cls label: 0 cls score: 1.000000
det boxes: [[186,456],[399,448],[399,480],[186,488]]rec text: 冏洗泰权地斜武泰滋龈斜洗 rec score:0.994914 cls label: 0 cls score: 1.000000
det boxes: [[18,501],[513,485],[514,537],[18,554]]rec text: 洗久泰酸兼滋龈斜洗免 rec score:0.992969 cls label: 0 cls score: 1.000000
det boxes: [[78,553],[404,541],[404,573],[78,585]]rec text: 洗鼠▏开泰才钧泰滋龈斜洗免 rec score:0.992751 cls label: 0 cls score: 1.000000

Visualized result saved in ./vis_result.jpg
`
请问这个可能会是什么问题导致的。

image
换了图片后情况一样,标注框准确,但是识别内容错误:
`blueberry@poodle:~/FastDeploy/examples/vision/ocr/PP-OCR/rockchip/cpp/build$ ./infer_demo ch_PP-OCRv3_det_infer_rk3588_unquantized.rknn ch_ppocr_mobile_v20_cls_infer_rk3588_unquantized.rknn ch_PP-OCRv3_rec_infer_rk3588_unquantized.rknn ppocr_keys_v1.txt 14.png 1
[INFO] fastdeploy/runtime/backends/rknpu2/rknpu2_backend.cc(81)::GetSDKAndDeviceVersion rknpu2 runtime version: 1.5.2 (c6b7b351a@2023-08-23T15:28:22)
[INFO] fastdeploy/runtime/backends/rknpu2/rknpu2_backend.cc(82)::GetSDKAndDeviceVersion rknpu2 driver version: 0.8.2
index=0, name=x, n_dims=4, dims=[1, 960, 960, 3], n_elems=2764800, size=5529600, fmt=NHWC, type=FP16, qnt_type=AFFINE, zp=0, scale=1.000000, pass_through=0
index=0, name=sigmoid_0.tmp_0, n_dims=4, dims=[1, 1, 960, 960], n_elems=921600, size=1843200, fmt=NCHW, type=FP32, qnt_type=AFFINE, zp=0, scale=1.000000, pass_through=0
[INFO] fastdeploy/runtime/runtime.cc(367)::CreateRKNPU2Backend Runtime initialized with Backend::RKNPU2 in Device::RKNPU.
[INFO] fastdeploy/runtime/backends/rknpu2/rknpu2_backend.cc(81)::GetSDKAndDeviceVersion rknpu2 runtime version: 1.5.2 (c6b7b351a@2023-08-23T15:28:22)
[INFO] fastdeploy/runtime/backends/rknpu2/rknpu2_backend.cc(82)::GetSDKAndDeviceVersion rknpu2 driver version: 0.8.2
index=0, name=x, n_dims=4, dims=[1, 48, 192, 3], n_elems=27648, size=55296, fmt=NHWC, type=FP16, qnt_type=AFFINE, zp=0, scale=1.000000, pass_through=0
index=0, name=softmax_0.tmp_0, n_dims=2, dims=[1, 2, 0, 0], n_elems=2, size=4, fmt=UNDEFINED, type=FP32, qnt_type=AFFINE, zp=0, scale=1.000000, pass_through=0
[INFO] fastdeploy/runtime/runtime.cc(367)::CreateRKNPU2Backend Runtime initialized with Backend::RKNPU2 in Device::RKNPU.
[INFO] fastdeploy/runtime/backends/rknpu2/rknpu2_backend.cc(81)::GetSDKAndDeviceVersion rknpu2 runtime version: 1.5.2 (c6b7b351a@2023-08-23T15:28:22)
[INFO] fastdeploy/runtime/backends/rknpu2/rknpu2_backend.cc(82)::GetSDKAndDeviceVersion rknpu2 driver version: 0.8.2
index=0, name=x, n_dims=4, dims=[1, 48, 320, 3], n_elems=46080, size=92160, fmt=NHWC, type=FP16, qnt_type=AFFINE, zp=0, scale=1.000000, pass_through=0
index=0, name=softmax_5.tmp_0, n_dims=4, dims=[1, 40, 6625, 1], n_elems=265000, size=530000, fmt=NCHW, type=FP32, qnt_type=AFFINE, zp=0, scale=1.000000, pass_through=0
[INFO] fastdeploy/runtime/runtime.cc(367)::CreateRKNPU2Backend Runtime initialized with Backend::RKNPU2 in Device::RKNPU.
[WARNING] fastdeploy/runtime/backends/rknpu2/rknpu2_backend.cc(420)::InitRKNNTensorMemory The input tensor type != model's inputs type.The input_type need FP16,but inputs[0].type is UINT8
[WARNING] fastdeploy/runtime/backends/rknpu2/rknpu2_backend.cc(420)::InitRKNNTensorMemory The input tensor type != model's inputs type.The input_type need FP16,but inputs[0].type is UINT8
[WARNING] fastdeploy/runtime/backends/rknpu2/rknpu2_backend.cc(420)::InitRKNNTensorMemory The input tensor type != model's inputs type.The input_type need FP16,but inputs[0].type is UINT8
det boxes: [[609,19],[648,19],[648,36],[609,36]]rec text: 男-现才钧吃冈滋染斜洗免 rec score:0.997559 cls label: 0 cls score: 0.998047
det boxes: [[641,66],[682,66],[682,82],[641,82]]rec text: 贪洗窝奶兼盖 rec score:1.000000 cls label: 0 cls score: 1.000000
det boxes: [[262,87],[305,87],[305,103],[262,103]]rec text: 纱盲绽男洗授现才钧吃冈滋染斜洗免 rec score:0.997681 cls label: 0 cls score: 1.000000
det boxes: [[767,84],[811,84],[811,105],[767,105]]rec text: ?酸兼滋染斜洗免 rec score:0.994751 cls label: 0 cls score: 1.000000
det boxes: [[648,99],[693,99],[693,116],[648,116]]rec text: 男洗津地才钧吃冈滋染斜洗免 rec score:0.995718 cls label: 0 cls score: 1.000000
det boxes: [[648,123],[694,123],[694,140],[648,140]]rec text: 盲滋染斜洗 rec score:0.989551 cls label: 0 cls score: 1.000000
det boxes: [[272,163],[291,163],[291,179],[272,179]]rec text: 最显欧成才钧巾滋染斜洗免 rec score:0.998006 cls label: 0 cls score: 0.886230
det boxes: [[642,164],[701,164],[701,181],[642,181]]rec text: 飘窝魏兼盖 rec score:0.974805 cls label: 0 cls score: 1.000000
det boxes: [[42,177],[107,179],[106,195],[41,194]]rec text: 纱盲绽男洗津洗才钧泰滋染斜洗免 rec score:0.988021 cls label: 0 cls score: 1.000000
det boxes: [[266,186],[296,186],[296,204],[266,204]]rec text: 纱盲绽权欧成泰才钧泰滋染斜洗免 rec score:0.998665 cls label: 0 cls score: 0.999512
det boxes: [[466,190],[503,190],[503,207],[466,207]]rec text: 怕酸兼滋染洗洗免 rec score:0.995666 cls label: 0 cls score: 1.000000
det boxes: [[45,201],[103,204],[100,219],[43,216]]rec text: 授授才钧吃冈滋染斜洗免 rec score:0.986461 cls label: 0 cls score: 0.992676
det boxes: [[264,210],[300,210],[300,228],[264,228]]rec text: 壹鼠化洗窝团斜武酸兼滋染斜洗免 rec score:0.997786 cls label: 0 cls score: 1.000000
det boxes: [[265,233],[299,233],[299,251],[265,251]]rec text: 最洗才钧泰滋染斜洗免 rec score:0.997754 cls label: 0 cls score: 1.000000
det boxes: [[630,232],[691,231],[692,248],[631,249]]rec text: 冏个东才钧泰滋染斜洗免 rec score:0.984508 cls label: 0 cls score: 1.000000
det boxes: [[768,239],[867,239],[867,258],[768,258]]rec text: 洗久泰固酸兼滋染斜洗免 rec score:0.976163 cls label: 0 cls score: 1.000000
det boxes: [[631,257],[704,258],[703,275],[630,274]]rec text: 下积滋染斜洗 rec score:0.968343 cls label: 0 cls score: 1.000000
det boxes: [[641,282],[694,282],[694,299],[641,299]]rec text: 授洗伞览洗地酸兼滋染 rec score:0.993066 cls label: 0 cls score: 1.000000
det boxes: [[468,296],[511,296],[511,312],[468,312]]rec text: 纱盲之°叉-显申成泰才钧武喻义金示命栽授叶 rec score:1.000000 cls label: 0 cls score: 1.000000
det boxes: [[473,337],[500,337],[500,353],[473,353]]rec text: 授-洗窝团斜武斜洗免 rec score:1.000000 cls label: 0 cls score: 0.983887
det boxes: [[260,367],[304,367],[304,385],[260,385]]rec text: 冏授洗武斜武酸兼盖滋染斜洗 rec score:0.996657 cls label: 0 cls score: 1.000000
det boxes: [[271,408],[290,411],[286,426],[266,424]]rec text: 魏兼 rec score:1.000000 cls label: 0 cls score: 0.997070
det boxes: [[463,410],[510,410],[510,425],[463,425]]rec text: 壹鼠洗才钧泰滋染斜洗免 rec score:0.993874 cls label: 0 cls score: 1.000000
det boxes: [[636,407],[679,407],[679,428],[636,428]]rec text: ↓祖现地洗盲透酸兼盖滋染洗洗免 rec score:0.997135 cls label: 0 cls score: 1.000000
det boxes: [[559,414],[563,417],[559,420],[554,417]]rec text: 硚冶革中皮魏兼盖 rec score:0.997742 cls label: 0 cls score: 0.615723
det boxes: [[130,418],[151,418],[151,424],[130,424]]rec text: 欧冰才钧泰滋染斜洗免 rec score:0.997558 cls label: 1 cls score: 0.741699
det boxes: [[136,414],[145,414],[145,420],[136,420]]rec text: 洗久泰授东聘地斜武泰滋染斜洗免 rec score:0.997721 cls label: 0 cls score: 0.620117
det boxes: [[268,480],[295,480],[295,497],[268,497]]rec text: 纱盲之°叉-显申成泰才地吃洗'广冶洗寄武 rec score:1.000000 cls label: 0 cls score: 1.000000
det boxes: [[114,420],[161,420],[161,437],[114,437]]rec text: 累•响罩地武洗赵洗5剑 rec score:1.000000 cls label: 0 cls score: 0.553223
det boxes: [[463,481],[511,481],[511,499],[463,499]]rec text: 织窝奶兼盖 rec score:0.995898 cls label: 0 cls score: 1.000000
det boxes: [[647,481],[691,481],[691,503],[647,503]]rec text: 洗地感鼠酸兼滋染斜洗免 rec score:0.996182 cls label: 0 cls score: 1.000000
det boxes: [[688,521],[877,521],[877,533],[688,533]]rec text: 最授东才钧泰滋龈斜洗免 rec score:0.988681 cls label: 0 cls score: 1.000000

Visualized result saved in ./vis_result.jpg
`
image

@ibiz4j
Copy link

ibiz4j commented Oct 5, 2023

1.5.2的toolkit2转换rknn模型有问题,最后支持的版本是1.5.1b19,你可以用1.5.1b19的工具转换模型。但是这个问题还是看下吧,因为板子已经支持半自动的动态输入了,目前的静态resize输入识别率很低,估计变形太严重都丢了

@7288Fzq
Copy link
Author

7288Fzq commented Oct 7, 2023

1.5.2的toolkit2转换rknn模型有问题,最后支持的版本是1.5.1b19,你可以用1.5.1b19的工具转换模型。但是这个问题还是看下吧,因为板子已经支持半自动的动态输入了,目前的静态resize输入识别率很低,估计变形太严重都丢了

是的,您的建议是对的,我当天换了低版本的rknnlite就跑通了。抱歉没给您及时答复。

还想问一下您后面说的“板子已经支持半自动的动态输入了,目前的静态resize输入识别率很低,估计变形太严重都丢了”这句话是什么意思!rknpu2好像都是静态输入的?现在可以动态了吗!我刚接触这方面开发不是很懂。请您赐教,谢谢!

@190948804
Copy link

1.5.2的toolkit2转换rknn模型有问题,最后支持的版本是1.5.1b19,你可以用1.5.1b19的工具转换模型。但是这个问题还是看下吧,因为板子已经支持半自动的动态输入了,目前的静态resize输入识别率很低,估计变形太严重都丢了

特此感谢,1.5.1b19版本模型导出后,再使用1.5.1b19或1.5.2执行识别,就不乱码了。

@equalman
Copy link

equalman commented Feb 1, 2024

1.5.2的toolkit2转换rknn模型有问题,最后支持的版本是1.5.1b19,你可以用1.5.1b19的工具转换模型。但是这个问题还是看下吧,因为板子已经支持半自动的动态输入了,目前的静态resize输入识别率很低,估计变形太严重都丢了

特此感谢,1.5.1b19版本模型导出后,再使用1.5.1b19或1.5.2执行识别,就不乱码了。

1.5.1b19的sdk在哪里下载, https://github.com/rockchip-linux/rknpu2 这里只看到1.5.2版本, https://github.com/rockchip-linux/rknn-toolkit2这里只有1.6.0版本

@paddle-bot paddle-bot bot closed this as completed Feb 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants