tile38 uses more and more memory

[HkMoyun]

Hello, I only store data for tile38 , about 1000 writes per second， but it uses more and more memory.
Is there any way you can control the growth of memory usage ?
My memory is 32g
operating system is linux

[tidwall]

Tile38 stores all of its data in memory and has a garbage collector that releases unused memory over time. It will use as much memory as it needs in order to maintain the data you give it.

Because it has a garbage collector, the memory will grow and then later shrink, under normal operations.

If those 1000 writes per second are adding new objects to the database then you should see constant memory growth, but if you mostly updating existing then you will see growth then, at a later time, memory shrinks.

A lot will depend on the type of data and the write patterns that you use.

The SERVER command may be useful to you. The output provides the field heap_size which you can graph over time to see the GC memory growth and shrinks.

[HkMoyun]

It's exactly like you said I saw GC working but he can only recycle a small amount of memory .

I try to add an expiration time to each key but memory will still increase . The command I use when writing data is

Here is the Java code:

@Command("SET ?0 ?1 OBJECT ?2")
String setElement(String fleet, String truck, String geoJson);

@Command("EXPIRE ?0 ?1 ?2")
String setExpireIdInKey(String fleet, String truck, int time);

[HkMoyun]

I control the memory that docker can use, but still can't solve my problem .

[tidwall]

What is the output from the SERVER command?

[HkMoyun]

{"ok":true,"stats":{"aof_size":509837447,"avg_item_size":1250,"cpus":6,"heap_released":8798208,"heap_size":1119227344,"http_transport":true,"id":"aed26de11ffc73966679571a7a899beb","in_memory_size":31163007,"max_heap_size":0,"mem_alloc":1119227344,"num_collections":328,"num_hooks":0,"num_objects":894959,"num_points":894959,"num_strings":0,"pid":1,"pointer_size":8,"read_only":false,"threads":17,"version":"1.19.5"},"elapsed":"186.958µs"}

{"ok":true,"stats":{"aof_size":510213462,"avg_item_size":1254,"cpus":6,"heap_released":8798208,"heap_size":1122930016,"http_transport":true,"id":"aed26de11ffc73966679571a7a899beb","in_memory_size":31172989,"max_heap_size":0,"mem_alloc":1122930016,"num_collections":329,"num_hooks":0,"num_objects":895252,"num_points":895252,"num_strings":0,"pid":1,"pointer_size":8,"read_only":false,"threads":17,"version":"1.19.5"},"elapsed":"123.173µs"}

{"ok":true,"stats":{"aof_size":510648392,"avg_item_size":1257,"cpus":6,"heap_released":8798208,"heap_size":1126503632,"http_transport":true,"id":"aed26de11ffc73966679571a7a899beb","in_memory_size":31184720,"max_heap_size":0,"mem_alloc":1126503632,"num_collections":329,"num_hooks":0,"num_objects":895596,"num_points":895596,"num_strings":0,"pid":1,"pointer_size":8,"read_only":false,"threads":17,"version":"1.19.5"},"elapsed":"109.944µs"}

{"ok":true,"stats":{"aof_size":511060169,"avg_item_size":1260,"cpus":6,"heap_released":8798208,"heap_size":1129217264,"http_transport":true,"id":"aed26de11ffc73966679571a7a899beb","in_memory_size":31195497,"max_heap_size":0,"mem_alloc":1129217264,"num_collections":329,"num_hooks":0,"num_objects":895912,"num_points":895912,"num_strings":0,"pid":1,"pointer_size":8,"read_only":false,"threads":17,"version":"1.19.5"},"elapsed":"114.244µs"}

{"ok":true,"stats":{"aof_size":511434202,"avg_item_size":1263,"cpus":6,"heap_released":8798208,"heap_size":1132127672,"http_transport":true,"id":"aed26de11ffc73966679571a7a899beb","in_memory_size":31205414,"max_heap_size":0,"mem_alloc":1132127672,"num_collections":329,"num_hooks":0,"num_objects":896203,"num_points":896203,"num_strings":0,"pid":1,"pointer_size":8,"read_only":false,"threads":17,"version":"1.19.5"},"elapsed":"114.241µs"}

{"ok":true,"stats":{"aof_size":511807010,"avg_item_size":1265,"cpus":6,"heap_released":8798208,"heap_size":1134886912,"http_transport":true,"id":"aed26de11ffc73966679571a7a899beb","in_memory_size":31215397,"max_heap_size":0,"mem_alloc":1134886912,"num_collections":329,"num_hooks":0,"num_objects":896496,"num_points":896496,"num_strings":0,"pid":1,"pointer_size":8,"read_only":false,"threads":17,"version":"1.19.5"},"elapsed":"109.117µs"}

{"ok":true,"stats":{"aof_size":512181019,"avg_item_size":1268,"cpus":6,"heap_released":8798208,"heap_size":1137273216,"http_transport":true,"id":"aed26de11ffc73966679571a7a899beb","in_memory_size":31225384,"max_heap_size":0,"mem_alloc":1137273216,"num_collections":330,"num_hooks":0,"num_objects":896789,"num_points":896789,"num_strings":0,"pid":1,"pointer_size":8,"read_only":false,"threads":17,"version":"1.19.5"},"elapsed":"112.861µs"}

{"ok":true,"stats":{"aof_size":512571578,"avg_item_size":1270,"cpus":6,"heap_released":8798208,"heap_size":1140181024,"http_transport":true,"id":"aed26de11ffc73966679571a7a899beb","in_memory_size":31235618,"max_heap_size":0,"mem_alloc":1140181024,"num_collections":333,"num_hooks":0,"num_objects":897089,"num_points":897089,"num_strings":0,"pid":1,"pointer_size":8,"read_only":false,"threads":17,"version":"1.19.5"},"elapsed":"116.396µs"}

{"ok":true,"stats":{"aof_size":514358799,"avg_item_size":1288,"cpus":6,"heap_released":8798208,"heap_size":1157515040,"http_transport":true,"id":"aed26de11ffc73966679571a7a899beb","in_memory_size":31283059,"max_heap_size":0,"mem_alloc":1157515040,"num_collections":334,"num_hooks":0,"num_objects":898478,"num_points":898478,"num_strings":0,"pid":1,"pointer_size":8,"read_only":false,"threads":17,"version":"1.19.5"},"elapsed":"113.54µs"}

{"ok":true,"stats":{"aof_size":514672125,"avg_item_size":1290,"cpus":6,"heap_released":8798208,"heap_size":1159727184,"http_transport":true,"id":"aed26de11ffc73966679571a7a899beb","in_memory_size":31291463,"max_heap_size":0,"mem_alloc":1159727184,"num_collections":336,"num_hooks":0,"num_objects":898724,"num_points":898724,"num_strings":0,"pid":1,"pointer_size":8,"read_only":false,"threads":17,"version":"1.19.5"},"elapsed":"124.748µs"}

{"ok":true,"stats":{"aof_size":515074367,"avg_item_size":1293,"cpus":6,"heap_released":8798208,"heap_size":1162758376,"http_transport":true,"id":"aed26de11ffc73966679571a7a899beb","in_memory_size":31302328,"max_heap_size":0,"mem_alloc":1162758376,"num_collections":336,"num_hooks":0,"num_objects":899042,"num_points":899042,"num_strings":0,"pid":1,"pointer_size":8,"read_only":false,"threads":17,"version":"1.19.5"},"elapsed":"101.905µs"}

{"ok":true,"stats":{"aof_size":515425579,"avg_item_size":1296,"cpus":6,"heap_released":8798208,"heap_size":1165791656,"http_transport":true,"id":"aed26de11ffc73966679571a7a899beb","in_memory_size":31311640,"max_heap_size":0,"mem_alloc":1165791656,"num_collections":334,"num_hooks":0,"num_objects":899314,"num_points":899314,"num_strings":0,"pid":1,"pointer_size":8,"read_only":false,"threads":17,"version":"1.19.5"},"elapsed":"128.243µs"}

{"ok":true,"stats":{"aof_size":515903167,"avg_item_size":1299,"cpus":6,"heap_released":8798208,"heap_size":1168912528,"http_transport":true,"id":"aed26de11ffc73966679571a7a899beb","in_memory_size":31324187,"max_heap_size":0,"mem_alloc":1168912528,"num_collections":333,"num_hooks":0,"num_objects":899681,"num_points":899681,"num_strings":0,"pid":1,"pointer_size":8,"read_only":false,"threads":17,"version":"1.19.5"},"elapsed":"100.88µs"}

{"ok":true,"stats":{"aof_size":516277470,"avg_item_size":1302,"cpus":6,"heap_released":8798208,"heap_size":1172104544,"http_transport":true,"id":"aed26de11ffc73966679571a7a899beb","in_memory_size":31334066,"max_heap_size":0,"mem_alloc":1172104544,"num_collections":333,"num_hooks":0,"num_objects":899970,"num_points":899970,"num_strings":0,"pid":1,"pointer_size":8,"read_only":false,"threads":17,"version":"1.19.5"},"elapsed":"88.774µs"}

[HkMoyun]

now It seems to be increasing all the time

[tidwall]

now It seems to be increasing all the time

This is normal behavior.

Tile38 has a garbage collector. So every action you perform, including calling SERVER, SET, DEL, etc will increase the allocate memory a little. But at a later point in time, any unused memory will be automatically reused or freed.

Your instance of Tile38 has a heap of 1,172,104,544 bytes (so about 1.1 GB) and 899,970 objects.

Each object takes up a little memory, which includes the object id, the object data, any fields and properties belonging to that object, and it a little extra memory for storing it in the collection btree and spatial rtree.

Your average object size is 1302 bytes.

If you desire you can run GC command to force a garbage collection and you will see the numbers drop.

[HkMoyun]

OK, thank you very much for your support . I do smart transportation and hope Tile38 is getting better and better .

[tidwall]

You're welcome. I wish you the best with your transportation app.

[HkMoyun]

I think I found a problem, I added expiration time to the key in the figure, but it was not deleted after the time, and the TTL command and GET command Can't get him,Is this normal ? Tile38 version is 1.19.5 .

[HkMoyun]

192.168.11.102:9851> KEYS 12PRB06T4U0
{"ok":true,"keys":["12PRB06T4U0"],"elapsed":"4.285µs"}
192.168.11.102:9851>
192.168.11.102:9851> SCAN 12PRB06T4U0 LIMIT 1
{"ok":true,"objects":[{"id":"1582128255917-361-0","object":{"type":"Point","coordinates":[114.060503,22.646221,1582128256020,1582128255917],"crossId":"12PRB06T4U0","crossing":0,"driveAngle":87.47,"entryLane":"","entryRoad":0,"exitLane":"12PRB06T4U012PSB06T4V0002011","exitRoad":0,"fromWay":0,"lat":22.646221,"lng":114.060503,"matchLane":"12PRB06T4U012PSB06T4V0002011","plateColor":0,"plateNumber":"always_left.36","plateType":0,"speed":56.35,"state":0,"time":1582128255917,"trackID":361,"vehicleColor":0,"vehicleType":1}}],"count":1,"cursor":1,"elapsed":"78.044µs"}
192.168.11.102:9851>
192.168.11.102:9851> TTL 12PRB06T4U0 1582128255917-361-0
(error) id not found
192.168.11.102:9851> GET 12PRB06T4U0 1582128255917-361-0
(error) id not found
192.168.11.102:9851>

This is my console. The picture above can't be displayed normally

[tidwall]

Interesting. I'll do some investigation and keep you posted.