[nfbcs] locating hard to find memory leaks

[Littlefield, Tyler]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hello,
Thanks for the reply. I've answered your points inline:
On 10/20/2015 1:18 PM, Bryan Duarte wrote:
> Hello Tyler,
> 
> Although I have never messed with a mud engine before I have done
> work with serializing data for embedded systems. The first question
> I would ask you is what data structure are you using in C++ when
> storing the room objects? If it is a vector the way you are adding
> objects to the grid or list so to speak makes a difference in how
> C++ allocates memory to the vector. A vector allocates memory based
> on a memory alloc function automatically if one is not specified
> when it is initialized. Everything will run fine until that size is
> exceeded. Once the size is exceeded all elements are moved to
> another memory block and the size is reallocated automatically
> based on the STDLIB function. I am not sure about how this function
> allocates memory but for other data structures in java there is a
> default size, then when it grows it takes that number and basically
> doubles it. By the time you are done you could have resized two or
> three times and your structure could be very large even if it is
> not using all of the allocated memory. If you are able I would keep
> track of every object you are storing in the list, check the size, 
> then use the "resize" function to attempt to get rid of any unused 
> memory being allocated.
> 

Thanks a lot for this--it makes sense. I will just call resize on the
vectors once done deserializing, which should clean everything up
quite a lot. I think though that once this is done, the old memory
(from the older vectors) should be reclaimed. I'm not sure if that
would happen automatically though or how to manage that. I store this
data in a vector because for 65000 rooms, you won't have a lot of room
adding and subtracting with the exception of some special cases, so it
should theoretically save some memory.

> Another thing I will suggest is looking into the JSON libraries for
> C++. JSON is a light weight easy to use method for serializing and 
> deserializing information. You can create objects, make calls, and
> it is serializable with a simple function call. When developing
> distributed applications between an embedded system acting as a
> server such as a Raspberry PI, JSON is very easy to use and allows
> for data to be handled even on a small memory device such as the
> PI.
> 

I've thought a lot about using JSON, though the serialization is
pretty deeply embedded so far. I don't mind XML so much because it's
easy to read and there may need to be some editing by hand at times of
files.

> 
> 
> Go Devils!
> 
> Bryan Duarte Software Engineering Graduate student ASU Fulton
> Engineering College QwikEyes CEO
> 
>> On Oct 20, 2015, at 8:46 AM, Littlefield, Tyler via nfbcs 
>> <nfbcs at nfbnet.org <mailto:nfbcs at nfbnet.org>> wrote:
>> 
> Hello all, I am working on my mud engine: 
> http://github.com/sorressean/aspen There appears to be a really
> weird issue when serializing/deserializing a lot of data. When the
> data is loaded, I end up increasing my memory to hold a 250x250
> grid of rooms and exits from 100 KB to around 500 MB. I'm using
> TinyXML for this. Does anyone have any ideas in terms of how to
> track this down? Running valgrind doesn't seem to produce any
> results. This shouldn't be using this much memory at all--the
> objects are compact, which you can understand if you can hold 65000
> rooms plus exits in memory. Any tips here would be awesome. Also if
> you're interested in a run, I provided a world generated grid
> here: https://dl.dropboxusercontent.com/u/10204868/wilderness.tgz 
> to use after you clone aspen: mkdir -p aspen/data/areas put the
> extracted wilderness file there and run. Any tips/advice would be
> greatly appreciated. Thanks,
>> 
>> _______________________________________________ nfbcs mailing
>> list nfbcs at nfbnet.org 
>> http://nfbnet.org/mailman/listinfo/nfbcs_nfbnet.org To
>> unsubscribe, change your list options or get your account info
>> for nfbcs: 
>> http://nfbnet.org/mailman/options/nfbcs_nfbnet.org/bjduarte%40asu.edu
>
>> 
- -- 
Take care,
Ty
twitter: @sorressean
web:http://tysdomain.com
pubkey: http://tysdomain.com/files/pubkey.asc
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBAgAGBQJWJoEzAAoJEAdP60+BYxejpHoIAKFkYFCsHeav+yoW2Wp+gw7C
ZEBhF11hx3zxAroVuRX9zUlp8DmbpucNOfxr7u1So4yeiurNXlskn2qHJA88ejIq
aC8WeDhizU1NsE9XuF+HX3Gbl09oFv9DxNr1NWaJtWmHmaMFD/nKv3C4orAy98mz
ppt+goG7aPzcRNVb3Z39atr7i8Yca8w+gnMlWR1PaioL9fczv+2vXd+b7xQWIJ4m
O4zr/GKHjKkZvGp5IowT4JVOzuOtR43MfERG8tzNXctvrmyMkrxXp0h6kr5MI239
VI4bYF5GRkKhfGJ3R60H3WI/wLo/7zeame7AahW/L4LJrz0cWC2q39Le4pKJhCo=
=YyQU
-----END PGP SIGNATURE-----