[nfbcs] locating hard to find memory leaks

Tue Oct 20 18:10:53 UTC 2015

Wow Tyler, you're the first person I've heard say that XML is easy to read <grin>

Jim

On Tue, Oct 20, 2015 at 06:00:25PM +0000, Littlefield, Tyler via nfbcs wrote:
> I've thought a lot about using JSON, though the serialization is
> pretty deeply embedded so far. I don't mind XML so much because it's
> easy to read and there may need to be some editing by hand at times of
> files.

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Hello,
> Thanks for the reply. I've answered your points inline:
> On 10/20/2015 1:18 PM, Bryan Duarte wrote:
> > Hello Tyler,
> > 
> > Although I have never messed with a mud engine before I have done
> > work with serializing data for embedded systems. The first question
> > I would ask you is what data structure are you using in C++ when
> > storing the room objects? If it is a vector the way you are adding
> > objects to the grid or list so to speak makes a difference in how
> > C++ allocates memory to the vector. A vector allocates memory based
> > on a memory alloc function automatically if one is not specified
> > when it is initialized. Everything will run fine until that size is
> > exceeded. Once the size is exceeded all elements are moved to
> > another memory block and the size is reallocated automatically
> > based on the STDLIB function. I am not sure about how this function
> > allocates memory but for other data structures in java there is a
> > default size, then when it grows it takes that number and basically
> > doubles it. By the time you are done you could have resized two or
> > three times and your structure could be very large even if it is
> > not using all of the allocated memory. If you are able I would keep
> > track of every object you are storing in the list, check the size, 
> > then use the "resize" function to attempt to get rid of any unused 
> > memory being allocated.
> > 
> 
> Thanks a lot for this--it makes sense. I will just call resize on the
> vectors once done deserializing, which should clean everything up
> quite a lot. I think though that once this is done, the old memory
> (from the older vectors) should be reclaimed. I'm not sure if that
> would happen automatically though or how to manage that. I store this
> data in a vector because for 65000 rooms, you won't have a lot of room
> adding and subtracting with the exception of some special cases, so it
> should theoretically save some memory.
> 
> > Another thing I will suggest is looking into the JSON libraries for
> > C++. JSON is a light weight easy to use method for serializing and 
> > deserializing information. You can create objects, make calls, and
> > it is serializable with a simple function call. When developing
> > distributed applications between an embedded system acting as a
> > server such as a Raspberry PI, JSON is very easy to use and allows
> > for data to be handled even on a small memory device such as the
> > PI.
> > 
> 
> I've thought a lot about using JSON, though the serialization is
> pretty deeply embedded so far. I don't mind XML so much because it's
> easy to read and there may need to be some editing by hand at times of
> files.
> 
> > 
> > 
> > Go Devils!
> > 
> > Bryan Duarte Software Engineering Graduate student ASU Fulton
> > Engineering College QwikEyes CEO
> > 
> >> On Oct 20, 2015, at 8:46 AM, Littlefield, Tyler via nfbcs 
> >> <nfbcs at nfbnet.org <mailto:nfbcs at nfbnet.org>> wrote:
> >> 
> > Hello all, I am working on my mud engine: 
> > http://github.com/sorressean/aspen There appears to be a really
> > weird issue when serializing/deserializing a lot of data. When the
> > data is loaded, I end up increasing my memory to hold a 250x250
> > grid of rooms and exits from 100 KB to around 500 MB. I'm using
> > TinyXML for this. Does anyone have any ideas in terms of how to
> > track this down? Running valgrind doesn't seem to produce any
> > results. This shouldn't be using this much memory at all--the
> > objects are compact, which you can understand if you can hold 65000
> > rooms plus exits in memory. Any tips here would be awesome. Also if
> > you're interested in a run, I provided a world generated grid
> > here: https://dl.dropboxusercontent.com/u/10204868/wilderness.tgz 
> > to use after you clone aspen: mkdir -p aspen/data/areas put the
> > extracted wilderness file there and run. Any tips/advice would be
> > greatly appreciated. Thanks,
> >> 
> >> _______________________________________________ nfbcs mailing
> >> list nfbcs at nfbnet.org 
> >> http://nfbnet.org/mailman/listinfo/nfbcs_nfbnet.org To
> >> unsubscribe, change your list options or get your account info
> >> for nfbcs: 
> >> http://nfbnet.org/mailman/options/nfbcs_nfbnet.org/bjduarte%40asu.edu
> >
> >> 
> - -- 
> Take care,
> Ty
> twitter: @sorressean
> web:http://tysdomain.com
> pubkey: http://tysdomain.com/files/pubkey.asc
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2
> 
> iQEcBAEBAgAGBQJWJoEzAAoJEAdP60+BYxejpHoIAKFkYFCsHeav+yoW2Wp+gw7C
> ZEBhF11hx3zxAroVuRX9zUlp8DmbpucNOfxr7u1So4yeiurNXlskn2qHJA88ejIq
> aC8WeDhizU1NsE9XuF+HX3Gbl09oFv9DxNr1NWaJtWmHmaMFD/nKv3C4orAy98mz
> ppt+goG7aPzcRNVb3Z39atr7i8Yca8w+gnMlWR1PaioL9fczv+2vXd+b7xQWIJ4m
> O4zr/GKHjKkZvGp5IowT4JVOzuOtR43MfERG8tzNXctvrmyMkrxXp0h6kr5MI239
> VI4bYF5GRkKhfGJ3R60H3WI/wLo/7zeame7AahW/L4LJrz0cWC2q39Le4pKJhCo=
> =YyQU
> -----END PGP SIGNATURE-----
> 
> _______________________________________________
> nfbcs mailing list
> nfbcs at nfbnet.org
> http://nfbnet.org/mailman/listinfo/nfbcs_nfbnet.org
> To unsubscribe, change your list options or get your account info for nfbcs:
> http://nfbnet.org/mailman/options/nfbcs_nfbnet.org/jbar%40barcore.com
>