SEGV in mongo_cursor_destroy

[superarhow]

Thank you all for the great works.

Recently our program crashed when it tries to retrieve very huge data from mongodb server, here's the back trace:
(gdb) where
0 0x00002aaab24c3385 in mongo_cursor_destroy (cursor=0x10184970) at libmongo/mongo.c:1390
1 0x00002aaab24c3620 in mongo_cursor_get_more (cursor=0x10184970) at libmongo/mongo.c:1234
2 0x00002aaab24c3b38 in mongo_cursor_next (cursor=0x10184970) at libmongo/mongo.c:1362
3 0x00002aaab24cd84d in rmongo_cursor_next (cursor=) at api_mongo.c:309
4 0x00002ba38d25f4ec in do_dotcall (call=0x10f074f8, op=, args=, env=0x1) at dotcode.c:581
5 0x00002ba38d294cae in Rf_eval (e=0x10f074f8, rho=0x11423440) at eval.c:656
6 0x00002ba38d29a53f in Rf_applyClosure (call=0x105410f0, op=0x10f079c8, arglist=0x114233d0, rho=0x10535980, suppliedenv=0xf668118) at eval.c:1043
7 0x00002ba38d29481f in Rf_eval (e=0x105410f0, rho=0x10535980) at eval.c:675
8 0x00002ba38d2985d7 in do_while (call=0x10542be8, op=0xf6345f8, args=0x105410b8, rho=0x10535980) at eval.c:1556
9 0x00002ba38d294a74 in Rf_eval (e=0x10542be8, rho=0x10535980) at eval.c:628
10 0x00002ba38d296924 in do_begin (call=0x10542750, op=0xf636e20, args=0x10542bb0, rho=0x10535980) at eval.c:1632
11 0x00002ba38d294a74 in Rf_eval (e=0x10542750, rho=0x10535980) at eval.c:628
12 0x00002ba38d294a74 in Rf_eval (e=0x10543600, rho=0x10535980) at eval.c:628
13 0x00002ba38d296924 in do_begin (call=0x10554e78, op=0xf636e20, args=0x105435c8, rho=0x10535980) at eval.c:1632
14 0x00002ba38d294a74 in Rf_eval (e=0x10554e78, rho=0x10535980) at eval.c:628
15 0x00002ba38d29a53f in Rf_applyClosure (call=0x1055d6a0, op=0x1055c960, arglist=0x105382f8, rho=0xf6680e0, suppliedenv=0xf668118) at eval.c:1043
16 0x00002ba38d29481f in Rf_eval (e=0x1055d6a0, rho=0xf6680e0) at eval.c:675
17 0x00002ba38d297375 in do_set (call=0x1055d748, op=0xf637018, args=0x1055d710, rho=0xf6680e0) at eval.c:2029
18 0x00002ba38d294a74 in Rf_eval (e=0x1055d748, rho=0xf6680e0) at eval.c:628
19 0x00002ba38d2bbb49 in Rf_ReplIteration (rho=0xf6680e0, savestack=0, browselevel=, state=0x7fffe5df2970) at main.c:257
20 0x00002ba38d2bbf68 in R_ReplConsole (rho=0xf6680e0, savestack=0, browselevel=0) at main.c:306
21 0x00002ba38d2bc434 in run_Rmainloop () at main.c:998
22 0x0000000000bbe91b in r::session::runEmbeddedR(core::FilePath const&, core::FilePath const&, bool, bool, SA_TYPE, r::session::Callbacks const&, r::session::InternalCallbacks*) ()
23 0x0000000000b96641 in r::session::run(r::session::ROptions const&, r::session::RCallbacks const&) ()
24 0x00000000006655d8 in main ()
(gdb) p *cursor
$4 = {reply = 0x2aaab26d7010, conn = 0x10060e90, ns = 0x10fa42f0 "PRV.HDS_PID_HDS_SID__auto_k1mEvent_Rel", flags = 3, seen = 12772, current = {
data = 0x2aaab2ad6f33 <Address 0x2aaab2ad6f33 out of bounds>, cur = 0x0, dataSize = 0, finished = 1, stack = {0 <repeats 32 times>}, stackPos = 0, err = 0,
errstr = 0x0}, err = MONGO_CURSOR_EXHAUSTED, query = 0x7fffe5df0990, fields = 0x10b95690, options = 0, limit = 0, skip = 0}
(gdb) p *cursor->reply
Cannot access memory at address 0x2aaab26d7010
(gdb)

It can reproduce with a high possibility.
I noticed that was a mongo-c-driver bug in v0.7.1 and was fixed at v0.8 (https://github.com/mongodb/mongo-c-driver/commits/v0.8)
I tried to use the code in v0.8 and created a patch to fix it, but I think it is better to use the latest mongo-c-driver.
Thank you again.

10/6 edit to remove "# number" to avoid search engine's problem:)

[schmidb]

Hi, using any mongo-c driver version higher than v.0.7.1 requires a complete re-engineering of the rmongodb package. I am not planing to do that.

[jeroen]

@schmidb what exactly has changed in 0.7.2? How much work would it be to upgrade to 1.0?

[schmidb]

90% changes and a lot of new concepts. It means more or less re-implementing rmongodb.

On 17.09.2014, at 12:07, Jeroen Ooms [email protected] wrote:

@schmidb what exactly has changed in 0.7.2? How much work would it be to upgrade to 1.0?

—
Reply to this email directly or view it on GitHub.

[jeroen]

That's a shame. But the old driver works fine with recent versions of mongodb?

[schmidb]

yes, fully compatible. But there are missing some of the new features.

On 17.09.2014, at 12:43, Jeroen Ooms [email protected] wrote:

That's a shame. But the old driver works fine with recent versions of mongodb?

—
Reply to this email directly or view it on GitHub.

[dselivanov]

@superarhow, can you remove #(NUMBER) from your post? Github referenced this issue to issues wtih id from 0 to 24 :-)

[dselivanov]

I see some API change starting from 0.8.0 version. But seems it is not full API break, like it is in 0.9+. I think we can try rebase rmongodb to 0.8.1.

[jeroen]

FYI I started working on a full rewrite at https://github.com/jeroenooms/mongolite

[dselivanov]

@jeroenooms , great news! I thought about starting new branch based on 1.*+ mongo-c-driver or even mongo-cxx-driver using Rcpp. Prior to my recent contributions to rmongodb I didn't know anything about R's C interface(which is not so easy). I always used Rcpp. So I think new branch or new implementation of R-MongoDB driver using modern C++ interface will be more straightforward. Nevertheless there is still left a great deal of work to do.

[dselivanov]

branch for 0.8.1 version of mongo-c-driver:
https://github.com/mongosoup/rmongodb/tree/rmongodb_2_0