Skip to content
This repository was archived by the owner on Oct 23, 2024. It is now read-only.
This repository was archived by the owner on Oct 23, 2024. It is now read-only.

ParserConfig的IdentityHashMap存在内存泄露 #1286

Open
Open
ParserConfig的IdentityHashMap存在内存泄露#1286
@zilong

Description

@zilong
zilong
opened on Jun 22, 2017

String text = "{"modules":{"name": "alibaba"}}";

for (int i=0; i<2; i++) {
        L1<?> r1 = JSONObject.parseObject(text, new TypeReference<L1<L2>>(){});
}
     // debug
L1<?> r1 = JSONObject.parseObject(text, new TypeReference<L1<L2>>(){});

发现了IdentityHashMap的的System.identityHashCode(key);产生了变化，导致了缓存的不命中，后续继续追加到内存里面去，这就是内存爆涨的元凶。

Activity

wuwen5

wuwen5 commented on Jun 22, 2017

@wuwen5
wuwen5
on Jun 22, 2017
Contributor

你使用的版本号是多少？这个问题应该是修复过的.

zilong

zilong commented on Jun 22, 2017

@zilong
zilong
on Jun 22, 2017
Author

1.2.25,1.2.33 都有

kimmking

kimmking commented on Jun 22, 2017

@kimmking
kimmking
on Jun 22, 2017
Contributor

IdentityHashMap的key使用的是Class，如果使用ClassName应该能避免这个问题。
@wuwen5

wuwen5

wuwen5 commented on Jun 23, 2017

@wuwen5
wuwen5
on Jun 23, 2017
Contributor

@kimmking 这个问题应该是在TypeReference解决掉的，IdentityHashMap 最终的key是Type对象类型，固定的，单参和多参的时候是缓存在ConcurrentHashMap中的，这种测试场景下不会一直递增的。

@zilong 建议你再debug仔细检查下，比如循环10次，检查key的地址， 我没有重现你说的问题，如果有问题你可以再提供详细信息.

wuwen5

wuwen5 commented on Jun 23, 2017

@wuwen5
wuwen5
on Jun 23, 2017
Contributor

see #849

FjcD

FjcD commented on Jun 23, 2017

@FjcD
FjcD
on Jun 23, 2017

#1281 好像我的问题和这个差不多，能帮忙看看吗？生产环境跪了几天了，一直没人回复我

wuwen5

wuwen5 commented on Jun 23, 2017

@wuwen5
wuwen5
on Jun 23, 2017 · edited by wuwen5
Contributor

@zilong 确认了这个问题在安卓的环境下存在，不过你之前测试的版本是1.2.24 \1.2.33 确认下你也是在安卓下验证的么？

安卓环境下建议使用带参数的写法.
参考单参数 和多参数的例子. https://github.com/alibaba/fastjson/wiki/TypeReference.

wuwen5
mentioned this on Jun 23, 2017
zilong

zilong commented on Jul 4, 2017

@zilong
zilong
on Jul 4, 2017
Author

IdentityHashMap 是用native Hash的，真有问题，仔细debug就能发现了这个问题了（环境JDK8）。@wuwen5 你找一下高铁

qiutongyeluo

qiutongyeluo commented on Apr 9, 2018

@qiutongyeluo
qiutongyeluo
on Apr 9, 2018 · edited by qiutongyeluo

所以这个解决了么。。。native Hash虽然能避免空指针异常，但是我们override hashCode就没意义了额。也就是说，如果是自己实现ParameterizedType，并使用JSON.parseObject(str, 自定义的type实现) 来解析，还是会造成这种内存泄漏。

magicnian

magicnian commented on Nov 15, 2018

@magicnian
magicnian
on Nov 15, 2018 · edited by magicnian

同样也遇到了相似的问题……
**One instance of "java.lang.ref.Finalizer" loaded by "" occupies 396,992,088 (63.02%) bytes. The instance is referenced by com.alibaba.fastjson.util.IdentityHashMap$Entry @ 0x824d3640 , loaded by "org.apache.catalina.loader.WebappClassLoader @ 0x80627b38".

Keywords
org.apache.catalina.loader.WebappClassLoader @ 0x80627b38
java.lang.ref.Finalizer**
这是从线上拉下的dump文件分析结论

tonycody

tonycody commented on Aug 29, 2019

@tonycody
tonycody
on Aug 29, 2019

`private final static Type typeReference = new TypeReference<ESResultVO>() {
}.getType();

static {
    ParserConfig.getGlobalInstance().getDeserializer(typeReference);
}

@Test
public void json() {
    ExecutorService executorService = ThreadUtil.newExecutor();
    while (true) {
        for (int j = 0; j < 10; j++) {
            executorService.submit(this::test);
        }
    }
}

private void test() {
    ESResultVO<TaskLogVO> json = JSON
        .parseObject("{\n" + "  \"code\": 0,\n" + "  \"message\": null,\n" + "  \"result\": {\n" + "    \"took\": 16,\n" + "    \"count\": 1,\n" + "    \"list\": [\n"
                     + "      {\n" + "        \"appId\": 4,\n" + "        \"bizNo\": \"\",\n" + "        \"elapsed\": 0,\n" + "        \"endTime\": 1567044743242,\n"
                     + "        \"gmtCreate\": 1567044743242,\n" + "        \"gmtUpdate\": 1567044743242,\n" + "        \"guid\": \"1fac95e9-081e-4e7a-965e-3279b306f1c3\",\n"
                     + "        \"id\": \"bc261686-11ef-461d-9ca0-a7aab09112b2\",\n" + "        \"startTime\": 1567044743242,\n" + "        \"status\": 0,\n"
                     + "        \"taskGuid\": \"3c704084-bc66-448d-86ff-f707f3d38317\",\n" + "        \"taskId\": 74,\n"
                     + "        \"traceId\": \"7b979533f94d48908d8fdd0d386a8400\",\n" + "        \"_score\": 0\n" + "      }\n" + "    ]\n" + "  }\n" + "}",
            typeReference);
}`

这么干可以解决这个问题，是个坑啊。最新的1.2.59，依然存在。有时候爆涨很快，有时候很慢，看运气。

rock123gh

rock123gh commented on Aug 30, 2021

@rock123gh
rock123gh
on Aug 30, 2021

1.2.78 依然存在

rookie-ricardo

rookie-ricardo commented on Jun 30, 2022

@rookie-ricardo
rookie-ricardo
on Jun 30, 2022

1.2.78 依然存在

老哥，现在有解决方案吗

rock123gh

rock123gh commented on Jun 30, 2022

@rock123gh
rock123gh
on Jun 30, 2022

1.2.78 依然存在

老哥，现在有解决方案吗

每创建一次classload， IdentityHashMap就会增长，频繁创建，删除classload 就是一直涨，直到OOM，避免频繁创建classload（如果必须，就缓存classload）

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @kimmking@zilong@wuwen5@tonycody@qiutongyeluo

        Issue actions
          ParserConfig的IdentityHashMap存在内存泄露 · Issue #1286 · alibaba/fastjson