Ideas for Expansion¶

• Ideas for Expansion
• High Priority
  • Mobile Device Support
  • Vector Graphics
  • hrefs
  • Hardware Video Acceleration Support
  • Debugging
  • Layout Engine
• Not High Priority
  • Node Positioning
  • Imaging Effects
  • Text Layout
  • Clean Up Namespaces
  • Media Domains
  • Build System
  • Embedded Web Browser Plugin

This page is an list of modules and features that are still missing from libavg. It is mainly meant as a repository of ideas and an invitation to help coding.

High Priority¶

Mobile Device Support¶

libavg on Android or other mobile devices would be cool. We would have to see which libraries used by libavg work on the target platform and adjust things accordingly.

libavg already runs with OpenGL ES as rendering library, so no change is needed there. libSDL needs to be updated to version 2.0 (with a non-compatible API) to support Android.

Vector Graphics¶

The vector graphics support in libavg is implemented using OpenGL vertex buffer objects, so all rasterizaion is done on the graphics card and the individual vector node classes only have to supply the appropriate vertex positions. Vector graphics can be textured and the textures can be animated, so we have a good initial implementation. Still, we could use some additions:

• pos, angle and pivot currently mean different things for vector nodes and for all other nodes. Some vector nodes have a pos attribute, some don't... this is confusing and should be changed.
• Ellipses are missing.
• Per-Vertex colors would be nice.
• mipmapping is not supported yet for vector nodes.
• The border of VectorNodes is currently centered on the positions. We'd like to be able to position it inside or outside the positions (similar to what stroke-location does in svg).
• Vertexes for Circles and CurveNodes are generated on the CPU, which results in a high CPU load and large vertex buffers that need to be copied to the GPU. Use GPU algorithms instead. See e.g., Loop/Blinn: Rendering Vector Art on the GPU and http://www.mdk.org.pl/2007/10/27/curvy-blues.
• The polygon triangulation algorithm could be faster (probably a lot faster :-)). Possible basis: https://code.google.com/p/poly2tri/
• (while we're at it) CircleNode.r should be CircleNode.radius.
• Vector node strokes don't respond to hit tests.

hrefs¶

ImageNode.href is a misnomer - it should be ImageNode.src. We can get rid of ImageNode.setBitmap and href="canvas:id" by making the src attribute polymorphic:

image.src = "file.png" 
image.src = bitmap
image.src = canvas

Then Canvas.id becomes superfluous and svg.createImageNode() can be replaced with ImageNode(src=svg.renderElement(...)).

Hardware Video Acceleration Support¶

We already support VDPAU (nVidias hardware video decoding framework), but we're missing VA-API (Intel's counterpart) and OpenMax (Android/Raspberry PI) support.

As a bonus, it should be possible to use the video frames (which are already on the graphics card) as textures directly, avoiding the transfer over the bus. Somebody posted some code to do this here http://www.nvnews.net/vbulletin/showthread.php?t=126994.

Debugging¶

1. Scenegraph/Event Processing Visualizer: To help in debugging scenegraph structure and track down causes of event processing bugs, we'd like to support a treeview of the current scenegraph as overlay. The user can expand and collapse subtrees by clicking. The treeview should also show events and the nodes they are routed to.
2. Profiling: Make the frame time view configurable. It should be able to show different profiling zones depending on application need.

Layout Engine¶

We'd like Layout Managers that position their child nodes according to different rules: GridLayout etc.

Not High Priority¶

Node Positioning¶

Node positioning/rotation/scaling works well in simple cases, but there are several shortcomings:

• div nodes can't be scaled,
• positioning a rotated element is a pain and
• it doesn't map well to OpenGL matrix transforms internally.

What libavg needs is a consistent transform hierarchy like those generally used in 3D engines. That means:

• Add a scale (Point2D) attribute, make size read-only.
• Change Node positioning to use pos/rotate/scale consistently.
• Default pivot is (0,0), pivot is used for all transforms (including translation and scale).
• Create a generic Transform class to replace libavg.ui.transform.
• Vector nodes have the same pos/rotate/scale attributes.
• div nodes get the scale attribute as well.
• Possibly add a setAlignment() function so the node is left/center/right-aligned and/or top/center/bottom-aligned. Internally, this would just set the pivot appropriately.

Imaging Effects¶

We need more FXNodes, and we want a list of effects per node.

Text Layout¶

Missing Words node features:

• Tabstops
• Color changes in mid-paragraph
• Irregular layouts (left and right paragraph borders changing from line to line)
• Real soft hyphen support
• Hyphenation using a dictionary

The ability to layout text on a path is missing as well.

To implement these features, we would need to move from using the Pango Freetype backend to the Pango Cairo backend for text rendering.

Clean Up Namespaces¶

• libavg.avg should disappear.

Media Domains¶

The mediadir attribute has some shortcomings.

Instead, we'd like to introduce resource domains: Player manages a table of resource domains that have a name and a path. There is a default domain that is used to find files when no domain is given. src attributes can then use several different protocols:

file:///full/path/to/filename
rsrc:///domain//filename
http://path/to/filename/
filename (default domain)
filename.svg#id_in_svg_file

There should be two new player functions:

player.setResourceDomain(domain, path)       # Sets the path used for a resource domain.
path = player.findFile(domain, resourceName) # Returns the full path of a resource.

Build System¶

We'd love to switch from an autotools-based build system to cmake. Of course, everything that works needs to continue working - that includes the build, but also the automated tests, the continuous build server and things like make distcheck. cmake could also cover the windows build.

Embedded Web Browser Plugin¶

Update: The BrowserNode plugin is being developed here: https://launchpad.net/browsernode.

There is an existing BrowserNode plugin for libavg (http://code.google.com/p/browsernode) which is based on Awesomium 1.08 (under an LGPL license). Awesomium recently changed to a closed source license and so 1.08 will be the last build that is freely available. Unfortunately it now builds against an aging version of Chromium which is increasingly problematic.

The project would then be to look at the existing BrowserNode code, and extend/adapt it to use a different browser embedding framework and enable it to run on Linux/Mac/Windows operating systems.

One potential option is the Berkelium project (http://berkelium.org). Their API is similar to that of libavg and as far as I can tell has similar functionality. Some research is needed here. There may also be some requirements to patch the Berkelium source. This should be in cooperation with their team so that patches go back upstream.

Integration with the Flash plugin is important. This is broadly handled by Berkelium but ensuring that a Flash plugin crash doesn't crash the whole libavg application is important. Consideration should be given to performance. It should be possible to view browser-based video/flash animations at acceptable framerates on average hardware.

It would then be good to extend beyond the existing BrowserNode code base to include things like key-press handling, further JavaScript integration, plugin compatibility etc.