https://public.etherpad-mozilla.org/p/wr-plan

Proposed plan for using WebRender as a Layers backend:
      https://public.etherpad-mozilla.org/p/Gecko-WebRenderer

Serialization bottlenecks
- texture sharing somewhat blocks video
- apz
- widget

https://public.etherpad-mozilla.org/p/wr-plan
Workitems for initial prototype of WRLayerManger
-MaskLayers we need mask image support in WebRender
-bring up c bindings for WebRender
-Widget stuff
-Probably best to start by bringing things up in-process with webrender running synchronously. i.e. make something akin to basic layers

Questions for webrender people:



Questions for us:


------------------

Ideas for software webrender backend:

Invalidation
1. Screenspace tile based invalidation (i.e. has the tile changed)
2. Scroll layer invalidation. (interleaving things are invalidated everytime)
3. 

Invalidation roots
- these would be scrollable things, transformed things. These would retain painted versions, but only what's visible on the screen
- invalidation roots would be largely static
- stuff on top not part of the invalidation root is repainted every composite. i.e. if something is inbetween, we propose repainting everything in that area on every composite
 - e.g. imagine you have some scrollable content that has something inbetween that does not scroll:
      - when the scrollable thing scrolls, we repaint the area 

changing content on a page can be handled by doing something like DASDLBI with per tile. We can do this per invalidation root as we needed.

Don't worry about position fixed content? Can we just redraw in time?



How to convert a display item type to webrender:

  1. Find the type that you want to convert in nsDisplayList.h or in nsDispAlayItemTypesList.h.
  2. Find the nsDisplayItem subclass of that type.
  3. Add an implementation of nsDisplayItemYourType::GetLayerState and make it return LAYER_ACTIVE for webrender layer managers and LAYER_NONE for the other layer managers.
  4. Go to Layers.h and add a subclass of Layer for the type that you want to add.
  5. Go to WebRenderLayerManager.h and add a CreateYourTypeLayer() method (all the other layer managers should return null).
  6. Add nsDisplayItemYourType::BuildLayer implementation that calls aManager->CreateYourTypeLayer() and sets the correct values on the layer.
  7. Before step 6: Figure out what values the layer needs to know about, by reading all the code that is called from nsDisplayItemYourType::Paint.
  8. In WebRenderYourTypeLayer::RenderLayer, add conversion code to build WebRender items of the right type with the right parameters. Might need new bindings in bindings.rs / webrender.h, or even new webrender capabilities.

Matt has already created layer types for Text and Border, for the simple cases.

Converting these things is going to be hard:


SVG
- filters
missing things
- multiple mask images blended together
- clipping to multiple paths

svg patterns
- % patterns make things difficult because we'd need to do layoutish things
- we could flatten out display lists but we might get to many display lists


Idea for supporting text:
The problem is that text rendering can do lots of things that webrender won't support at first. Ideally, you'd have a function CheckTextLayerFeasibility that either returns an array of glyphs or false. But the logic for creating such a check would be really complicated; you'd end up duplicating all of the control flow that is in the regular rendering path.
One idea would be to call the regular text rendering code with a recording DrawTarget during display list construction, and then analyze whether there are only FillGlyphs calls in the recording. If yes, extract the array of glyphs; if no, hold on to the recording DrawTarget and execute the drawing commands during actual painting.
Another idea would be to create a class that initially just wraps a DrawTarget, and that gets passed into text painting code instead of the actual DrawTarget. (This would require lots of type renaming but shouldn't change behavior.) This wrapper class could have an internal (lazily-created) DrawTarget and an array of glyphs. If FillGlyphs is called on the wrapper, the parameters would be added to the glyphs array; if any other drawing method is called, the internal DrawTarget is created, existing recorded FillGlyph calls are executed on it, the FillGlyphs array is discarded, and the current + all future drawing commands is executed directly on this DrawTarget.
In the case where WebRender is disabled, the text rendering code would be called at the normal place during painting, and the wrapper class would just forward all calls to the DrawTarget that backs the PaintedLayer that we're drawing to.

Types: (tag with inactive, started, converted, enabled) 

We should figure out all the places where we draw borders and write down what pieces we need. Create a new nsDisplayBorderSomething class that takes an nsStyleBorder in the constructor? Or a rect, border color + border width?

We should minimize the list of display item types by merging some of our display item classes, e.g. all solid color rects (SOLID_COLOR, FRAMESET_BLANK, CARET), all borders (borders, outlines, focus outlines), ...
These classes need to return different perframekeys for the different parts of an nsIFrame that they're used for. For example, if we have a frame with a border and an outline, and we use the same display item type for both, then DLBI needs to be able to differentiate between the two items.

Display items
  1. Images
    1. How do they get to the webrender process? Currently it looks like ipc-channel copies the data
    2. How do we avoid not using more memory than gecko currently does for images?
      1. i.e. We'd rather not have two copies of every image on a page in memory
    3. Create some image memory stress tests
    4. make sure we handle keeping things around for the unaccelerated side
    5. Current plan is to add and remove images based on Lock/Unlock()
      1. nsDocument::AddImage
      2. nsDocument::RemoveImage
    6. high quality downscaling
  2. SVG
    1. https://public.etherpad-mozilla.org/p/svg-perf
    2. Two possible approaches
      1. Native support for SVG display items
        1. Probably good to bring up servo's SVG renderer as soon as possible
        2. What would the performance of this approach be? (Probably not worse than cairo software rendering)
          1. For svg geometry frame it may be worth special casing the following
      2. Treat it like canvas
  3. Text
    1. Font memory management may be a concern
  4. Fonts
  5. XUL
    1. Most of XUL should be pretty easy
    2. XUL Tree (this could be lowered to other display items ahead of time)
      1. Seperator
      2. Row
      3. Column
      4. DropFeedback
      5. ProgressMeter
      6. CheckBox
      7. Text
      8. Image
      9. Cell
      10. Twisty
  6. Canvas
    1. HW accelerated canvas
      1. texture sharing of some sort (mchang)
  7. Video
    1. DXVA
    2. IOSurfaces
    3. Android stuff
Misc
  1. windows ipc channel port

Servo missing functionality

ImageBridge

WebRender missing capabilities
  1. Subpixel text
  2. elaborate border rendering
  3. svg filters
  4. directwrite
  5. background-repeat: space (https://github.com/servo/webrender/issues/363 )
  6. clipping to text (https://github.com/servo/webrender/issues/360 )
  7. subimage repeat ( https://github.com/servo/servo/issues/13003 )
  8. hit testing (https://github.com/servo/webrender/issues/411)
  9. mask images (https://github.com/servo/webrender/issues/405)
  10. mutliple windows (https://github.com/servo/servo/issues/13338)
  11. timed async images/pick video frame on composition
  12. radial gradients
  13. alpha masks
  14. anti-aliased edges on transforms
  15. blend modes
  16. plane splitting
  17. Color Management (not required for shipping_

Concerns
  1. fatness of WR display items for something like SVG (they'll be much fatter than nsDisplayItems which mostly just contain pointers to the frames)
  2. memory usage overall

Probably worth doing a bit of a WR in gecko prototype:
Overall, nsDisplayItems are very thin and heavily tied to their frames. by the way, I never said that it wan't true.  By the time we actually get to drawing there's quite a bit of stuff that happens inbetween at least for image drawing. It may
make sense to make the cut over to webrender at a lower level in the drawing process.

Code that's being replaced:
    gfx/layers/*
    
bindings:
    https://github.com/servo/webrender/issues/374


Gecko fixes needed:
    - display item per background layer (all that's missing now are tables)
    1. Remove legacy background layer drawing
      1. MathML
      2. XUL Tree
      3. Masks (how to do this a question mark)
      4. Border collapse tables

Risks:

APZ:


Things to be discussed and put in the right place:

How to do SVG:


Android catch up(Jamie + Sotaro)

OMTA(pchang + Sotaro)

webrender features needed from other discussions - glenn / kvark / lsalzman

Threading Model - nical / kvark