Memory Management¶
Memory management is one of the things that aren't fun to deal with. Accordingly, libavg attempts to shield it's users from the intricacies of memory management and usually does a good job. Still, in advanced cases, it can be useful to know how the different node types use CPU and GPU memory.
Allocating and Deleting Nodes¶
Nodes are allocated when createNode(), loadFromFile() or loadFromString() is called. A node is deleted when neither the python program nor libavg hold any references to it anymore. Thus, to delete a node, you must
- not have any python variables referencing it anymore
- remove it from the avg tree, e.g. using
unlink()and - make sure all event callbacks for this node are deleted.
Calling node.unlink(True) takes care of point 3 automatically.
Types of Memory¶
There are three different types of memory used by libavg programs. These are:
- GPU memory: This is memory on the graphics card. All visuals that should appear on the screen must have a copy in GPU memory. In most configurations, this is very fast memory. Using more GPU memory than is available will usually cause a disproportionate slowdown in the application. You can profile GPU memory usage with programs such as Apple's OpenGL Driver Monitor.
- CPU memory: Memory on the motherboard. This is the memory used for all calculations by the processor. Media loaded from disk are initially copied into CPU memory. There is usually more CPU than GPU memory available.
- External storage: Hard disks, networked devices or other storage.
AVGApp supports a graph of CPU memory usage over time that is activated if you press 'm'. If you have an NVidia card, you can additionally activate a GPU memory graph with 'v'.
The Image Cache¶
libavg has an internal two-level image cache that holds images in CPU and GPU memory. Images loaded as node attributes (hrefs, texhrefs or maskhref) are not immediately deleted when the node is deleted and cached instead so they can be reused if needed. This applies to both system memory and textures on the graphics card. Whenever the cache becomes full on either the CPU or GPU, unused images are deleted according to a least-recently-used algorithm. The cache has a configuration interface that can be accessed using player.imageCache. The cache sizes can be set using either this interface or avgrc.
One very positive consequence of having the image cache is that different nodes that have the same href share a single copy of the in-memory bitmap and the texture, saving memory and speeding things up significantly as well.
Loading of Assets from Disk¶
In the case of image nodes, bitmaps are loaded from disk (or taken from the cache) as soon as the node is created. Video nodes open the video file when play() or pause() is called and close it when stop() is called.
By default, media files are loaded from the directory the main python script is in. If the href contains a relative directory, it is relative to this directory. The base directory can be changed by setting the mediadir attribute of a div or avg node. The effective base directory of a node is determined by concatenating the mediadir attributes of all nodes above it in the tree. A node that is not in the tree has the mediadir of the root avg as it's base directory.
When the base directory of a node changes, it's media files get reloaded automatically. This can happen if a mediadir attribute above it in the tree changes or if it is added/removed from the tree. (image nodes don't display error messages on the console if they can't find a file while they are not in the tree. The assumption is that the node's base directory might change as soon as it's added to the tree.)
Memory Usage per Bitmap¶
Most bitmaps use an amount of memory equal to the number of pixels multiplied by four, since each pixel is stored as an RGBA (red, green, blue, alpha) quad. Video images in YUV 4:2:0 color format take only 1.5 bytes per pixel. Most common video codecs use this color format. So, for instance, a 2592x1944 image needs about 20 megabytes, while an image of a 1080p film (1920x1080x1.5) needs about 3,1 megabytes of storage. Adding a mask adds one byte per pixel. Images with mipmaps need 1/3 more memory on the GPU.
In all cases, the relevant size is the size of the image as it is loaded from disk - not the display size.
Internals¶
Note:
The following description assumes a separate (PCI Express or similar, non-motherboard) graphics card. Motherboard graphics chips usually share memory with the CPU, so semantics are a little different.
Also, some drivers keep additional copies of bitmaps in CPU memory.
Image Nodes¶
When a file is loaded into an image node, the bitmap is initially loaded into CPU memory (assuming it hasn't been cached). If the node is part of the avg scene (and playback has started), it is then transfered to GPU memory so it can be displayed. When a node is taken out of the tree, the GPU copy is unreferenced but will probably remain cached; when the node is destroyed, the CPU copy is treated similarly.
Video Nodes¶
Video nodes use different strategies depending on the setting of the threaded attribute. If threaded=False, the video images are loaded when they are needed. There is no queue. CPU and GPU memory for exactly one bitmap is allocated. If threaded=True, video images are loaded into a queue of several bitmaps. The number of bitmaps preloaded can be configured by setting the queuelength attribute, with the default being 8. On the GPU, memory for one bitmap is used. In any case, videos are only decoded and memory allocated for the bitmaps as long as the video state is paused or playing. When stop() is called, all CPU and GPU memory used is freed.