exposing CANVAS or something like it to Web Workers

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
41 messages Options
123
Reply | Threaded
Open this post in threaded view
|

exposing CANVAS or something like it to Web Workers

Gregg Tavares (wrk)
I'd like to work on exposing something like CANVAS to web workers.

Ideally how over it works I'd like to be able to

*) get a 2d context in a web worker
*) get a WebGL context in a web worker
*) download images in a web worker and the images with both 2d contexts and WebGL contexts

Any thoughts?


Reply | Threaded
Open this post in threaded view
|

Re: exposing CANVAS or something like it to Web Workers

Anne van Kesteren-4
On Mon, May 14, 2012 at 10:01 PM, Gregg Tavares (勤) <[hidden email]> wrote:

> I'd like to work on exposing something like CANVAS to web workers.
>
> Ideally how over it works I'd like to be able to
>
> *) get a 2d context in a web worker
> *) get a WebGL context in a web worker
> *) download images in a web worker and the images with both 2d contexts and
> WebGL contexts
>
> Any thoughts?

Have we gotten any further with use cases? See
http://lists.w3.org/Archives/Public/public-whatwg-archive/2010Mar/thread.html#msg144
for an old use case thread that went nowhere. Or
http://lists.w3.org/Archives/Public/public-whatwg-archive/2011Feb/thread.html#msg274
where you requested general DOM access which cannot be implemented
everywhere.


--
Anne — Opera Software
http://annevankesteren.nl/
http://www.opera.com/

Reply | Threaded
Open this post in threaded view
|

Re: exposing CANVAS or something like it to Web Workers

Charles Pritchard-2
In reply to this post by Gregg Tavares (wrk)
On 5/14/2012 1:01 PM, Gregg Tavares (勤) wrote:

> I'd like to work on exposing something like CANVAS to web workers.
>
> Ideally how over it works I'd like to be able to
>
> *) get a 2d context in a web worker
> *) get a WebGL context in a web worker
> *) download images in a web worker and the images with both 2d
> contexts and WebGL contexts
>
> Any thoughts?

As far as implementation, I'd love to be able to pass webkit's
document.getCSSCanvasContext('2d') around.
It seems like a safe place to experiment.

I can get a lot done with CanvasPattern as a transferable, without
needing to add Image (or video) into the worker context.

Notes:
1. getCSSCanvasContext is non-standard. It works with CSS image
-webkit-canvas.
2. I heard that a more generic "-moz-element()" paint server is supposed
to replace -webkit-canvas in time.
3. Passing the CSS canvas context would let me render off-frame and
update a canvas visible on the document automatically.

Canvas -should- have toBlob and a typed array buffer for ImageData.
They are both useful for passing image data back to the main frame.


 From my experience with WebGL, I think it should be considered with
added care and lower priority.
There are stability, speed and memory issues. WebGL in workers seems to
augment 2d, are there other big benefits?


-Charles





Reply | Threaded
Open this post in threaded view
|

Re: exposing CANVAS or something like it to Web Workers

Jonas Sicking-2
In reply to this post by Anne van Kesteren-4
On Mon, May 14, 2012 at 1:16 PM, Anne van Kesteren <[hidden email]> wrote:

> On Mon, May 14, 2012 at 10:01 PM, Gregg Tavares (勤) <[hidden email]> wrote:
>> I'd like to work on exposing something like CANVAS to web workers.
>>
>> Ideally how over it works I'd like to be able to
>>
>> *) get a 2d context in a web worker
>> *) get a WebGL context in a web worker
>> *) download images in a web worker and the images with both 2d contexts and
>> WebGL contexts
>>
>> Any thoughts?
>
> Have we gotten any further with use cases? See
> http://lists.w3.org/Archives/Public/public-whatwg-archive/2010Mar/thread.html#msg144
> for an old use case thread that went nowhere. Or
> http://lists.w3.org/Archives/Public/public-whatwg-archive/2011Feb/thread.html#msg274
> where you requested general DOM access which cannot be implemented
> everywhere.

One of the strongest use-cases we have found within mozilla is to
enable compiling applications written in other languages and for other
platforms to the web platform. For example emscripten [1] supports
compiling C/C++/OpenGL programs to javascript + WebGL.

Such programs generally use things like synchronous IO and have their
own event loops, thus not returning to the platform event loop.

So a good strategy for tools like emscripten is to create code which
is intended to run in a worker. That way you don't have to return to
the event loop, and you have access to things like full featured
synchronous XHR (including support for .responseType) without
degrading the user experience.

However a big problem is that you can't do graphics inside a worker.
It would be great to expose canvas since that is the graphics model
these programs have today.

[1] https://github.com/kripken/emscripten

/ Jonas

Reply | Threaded
Open this post in threaded view
|

Re: exposing CANVAS or something like it to Web Workers

Charles Pritchard-2
In reply to this post by Anne van Kesteren-4
On 5/14/2012 1:16 PM, Anne van Kesteren wrote:

> On Mon, May 14, 2012 at 10:01 PM, Gregg Tavares (勤)<[hidden email]>  wrote:
>> I'd like to work on exposing something like CANVAS to web workers.
>>
>> Ideally how over it works I'd like to be able to
>>
>> *) get a 2d context in a web worker
>> *) get a WebGL context in a web worker
>> *) download images in a web worker and the images with both 2d contexts and
>> WebGL contexts
>>
>> Any thoughts?
> Have we gotten any further with use cases? See
> http://lists.w3.org/Archives/Public/public-whatwg-archive/2010Mar/thread.html#msg144
> for an old use case thread that went nowhere. Or

1. Speeding up "onmousemove"-based drawing:
In my drawing projects (based on mouse/pen input), we lose mouse events
/ pen pressure information when the main thread is busy rendering what
the user is drawing.
Processing the drawing commands off-thread would lighten the load.

2. Avoiding blocking during redrawing of complex scenes or pre-rendering
of animations.

With complex scenes, where we're repainting, we don't particularly want
to block the main thread while a scene is loading.
But, we'd also like to use as much horsepower as the user's machine can
lend.

A complex scene may block for a few seconds -- we can of course use
green threading approaches, but that adds quite a bit of extra
guess-work and
does not fully exploit the user's machine for speed.



Reply | Threaded
Open this post in threaded view
|

Re: exposing CANVAS or something like it to Web Workers

Glenn Maynard
In reply to this post by Anne van Kesteren-4
On Mon, May 14, 2012 at 3:01 PM, Gregg Tavares (勤) <[hidden email]> wrote:
I'd like to work on exposing something like CANVAS to web workers.

Ideally how over it works I'd like to be able to

*) get a 2d context in a web worker

I'd recommend not trying to tackle 2d and 3d contexts at once, and only worrying about WebGL to start.

Another issue: rendering in a worker thread onto a canvas which is displayed in the main thread.  This needs to be solved in a way that doesn't cause the asynchronous nature of what's happening to be visible to scripts.  toDataURL and toBlob would probably need to be prohibited on the canvas element.  I'm not sure what the actual API would look like.  This would also require some equivalent to requestAnimationFrame in the worker thread.

*) download images in a web worker and the images with both 2d contexts and WebGL contexts

Don't forget to point people to the recent discussion on this, so the conversation doesn't reboot:

https://www.khronos.org/webgl/public-mailing-list/archives/1205/msg00059.html

In summary, the idea is to provide a much more basic interface for images, which would represent an image in an opaque, non-DOM way, to make it implementable in workers; you'd retrieve it with a method on HTMLImageElement after the image is completely available.  It'd be structured clonable, to hand it to workers.  There should also be an XHR2 responseType mode to retrieve an object like this directly, to allow loading images from workers (again without using the DOM), since workers shouldn't need to rely on a helper in the UI thread just to load textures.

This seems like it would solve this sub-problem pretty well, at least.


On Mon, May 14, 2012 at 3:16 PM, Anne van Kesteren <[hidden email]> wrote:
Have we gotten any further with use cases? See
http://lists.w3.org/Archives/Public/public-whatwg-archive/2010Mar/thread.html#msg144
for an old use case thread that went nowhere. Or
http://lists.w3.org/Archives/Public/public-whatwg-archive/2011Feb/thread.html#msg274
where you requested general DOM access which cannot be implemented
everywhere.

The use case is being able to draw without blocking the UI thread.  WebGL, like OpenGL, tries to be nonblocking by queuing draw operations, but it's impossible to do this in all cases: as soon as you call any method that returns data (eg. getError), WebGL has to synchronously flush the draw queue.  This isn't as severe as synchronous XHR, but it may be done every frame--a 10ms delay at 60Hz is going to have visible effects on the UI.

WebGL is also useful for computational tasks, which may perform much longer drawing operations.  If you're doing this, it's even more important that it not block the UI thread.

Compiling shaders can also take a long time for complex shaders.  Since this is almost always followed by a synchronous call to check the result, it's currently hard to load shaders without affecting the browser UI.

--
Glenn Maynard

Reply | Threaded
Open this post in threaded view
|

Re: exposing CANVAS or something like it to Web Workers

Jonas Sicking-2
On Mon, May 14, 2012 at 3:28 PM, Glenn Maynard <[hidden email]> wrote:

> On Mon, May 14, 2012 at 3:01 PM, Gregg Tavares (勤) <[hidden email]> wrote:
>
>> I'd like to work on exposing something like CANVAS to web workers.
>>
>> Ideally how over it works I'd like to be able to
>>
>> *) get a 2d context in a web worker
>
>
> I'd recommend not trying to tackle 2d and 3d contexts at once, and only
> worrying about WebGL to start.
>
> Another issue: rendering in a worker thread onto a canvas which is displayed
> in the main thread.  This needs to be solved in a way that doesn't cause the
> asynchronous nature of what's happening to be visible to scripts.  toDataURL
> and toBlob would probably need to be prohibited on the canvas element.  I'm
> not sure what the actual API would look like.

If/when we do this, I think it should be done in such a way that the
main window can't access the canvas object at all. Similar to what
happens when an ArrayBuffer is transferred to a Worker using
structured cloning. Once a canvas is transferred to a Worker, any
access to it should throw or return null/0/"". If you want to transfer
pixel data to the main thread, it seems less racy to do that by
getting the pixel data in the Worker which owns the canvas and then
transfer that to the main thread using postMessage.

> This would also require some
> equivalent to requestAnimationFrame in the worker thread.

Agreed!

/ Jonas

Reply | Threaded
Open this post in threaded view
|

Re: exposing CANVAS or something like it to Web Workers

Glenn Maynard
On Mon, May 14, 2012 at 6:42 PM, Jonas Sicking <[hidden email]> wrote:
If/when we do this, I think it should be done in such a way that the
main window can't access the canvas object at all. Similar to what
happens when an ArrayBuffer is transferred to a Worker using
structured cloning. Once a canvas is transferred to a Worker, any
access to it should throw or return null/0/"".

Sort of.  You want to disable the methods on Canvas--toDataURL, toBlob and getContext--but you don't want to disable inherited methods (you should still be able to access eg. parentNode).  I think handling this on a method-by-method basis is fine, since there aren't many of them--just those three, I think.

I still don't have much intuition about how to do this, but another thought: assuming you create a context in the worker which is attached to the canvas, you need to be able to create new contexts in the future if the worker dies, since workers can be killed by the browser at any time.  A mechanism that says "you can create a worker-context for a canvas, but then the canvas element is locked to that worker" isn't good enough (but it's fine to say "you can only have one worker associated with a canvas at a time").

If you want to transfer
pixel data to the main thread, it seems less racy to do that by
getting the pixel data in the Worker which owns the canvas and then
transfer that to the main thread using postMessage.

Right.
 
> This would also require some
> equivalent to requestAnimationFrame in the worker thread.

Agreed!

A tricky bit: you need to know which element to sync to, so the browser knows which monitor's vsync to use.  According to [1] only WebKit's requestAnimationFrame actually takes an element.  (That's surprising; this seems obvious.  Are multi-monitor systems always vsync-locked to each other, which would make this not matter?)  I mention this because this method would need to accept a context in lieu of an element (assuming whatever mechanism we land on it results in having a WebGLContext in the worker but no HTMLCanvasElement, whcih seems likely).

This in mind, it would be good if all Canvas context types had a common base class, even if it's an empty interface.  The return value of HTMLCanvasElement.getContext could be changed to it, instead of object.

[1] https://developer.mozilla.org/en/DOM/window.requestAnimationFrame

--
Glenn Maynard


Reply | Threaded
Open this post in threaded view
|

Re: exposing CANVAS or something like it to Web Workers

Gregg Tavares (wrk)
In reply to this post by Jonas Sicking-2


On Mon, May 14, 2012 at 4:42 PM, Jonas Sicking <[hidden email]> wrote:
On Mon, May 14, 2012 at 3:28 PM, Glenn Maynard <[hidden email]> wrote:
> On Mon, May 14, 2012 at 3:01 PM, Gregg Tavares (勤) <[hidden email]> wrote:
>
>> I'd like to work on exposing something like CANVAS to web workers.
>>
>> Ideally how over it works I'd like to be able to
>>
>> *) get a 2d context in a web worker
>
>
> I'd recommend not trying to tackle 2d and 3d contexts at once, and only
> worrying about WebGL to start.
>
> Another issue: rendering in a worker thread onto a canvas which is displayed
> in the main thread.  This needs to be solved in a way that doesn't cause the
> asynchronous nature of what's happening to be visible to scripts.  toDataURL
> and toBlob would probably need to be prohibited on the canvas element.  I'm
> not sure what the actual API would look like.

If/when we do this, I think it should be done in such a way that the
main window can't access the canvas object at all. Similar to what
happens when an ArrayBuffer is transferred to a Worker using
structured cloning. Once a canvas is transferred to a Worker, any
access to it should throw or return null/0/"". If you want to transfer
pixel data to the main thread, it seems less racy to do that by
getting the pixel data in the Worker which owns the canvas and then
transfer that to the main thread using postMessage.

How about separating the canvasy parts of canvas from canvas and the imagy parts of image from image.

In other words, Imagine canvas is implemented like this

class Canvas : public HTMLElement {
  private:
    CanvasSurface* m_surface;  // everything about canvas that is not HTMLElement
};

And that Image is similarly implemented as

class Image : public HTMLElement {
  private:
    Picture* m_picture;  // everything about Image that is not HTMLElement
}

now imagine you can instantiate inner implementation of these things. The parts that are not HTMLElement

var canvasSurface = new CanvasSurface();
var ctx = canvasSurface.getContext("2d");
var pic = new Picture;
pic.onload = function() {
   ctx.drawImage(pic, 0, );
}

Let's assume you can instantiate these things in either the page or a worker. Nether can be transfered.

Would that work? What problems would that have?

 

 

> This would also require some
> equivalent to requestAnimationFrame in the worker thread.

Agreed!

/ Jonas

Reply | Threaded
Open this post in threaded view
|

Re: exposing CANVAS or something like it to Web Workers

Boris Zbarsky
In reply to this post by Glenn Maynard
On 5/14/12 7:56 PM, Glenn Maynard wrote:
> A tricky bit: you need to know which element to sync to, so the browser
> knows which monitor's vsync to use.  According to [1] only WebKit's
> requestAnimationFrame actually takes an element.  (That's surprising;
> this seems obvious.

Does WebKit actually use the element to determine vsync?  How do they
handle cases when the element spans monitors?

As far as I know WebKit's implementation uses the element to optimize
out callbacks when the element is not visible, but that's it.

Note that Gecko, for example, does not tie requestAnimationFrame
callbacks to vsync.  I can't speak for other UAs.

> I mention this because
> this method would need to accept a context in lieu of an element

What would the context be used for?

-Boris

Reply | Threaded
Open this post in threaded view
|

Re: exposing CANVAS or something like it to Web Workers

Charles Pritchard-2
In reply to this post by Jonas Sicking-2
On May 14, 2012, at 4:42 PM, Jonas Sicking <[hidden email]> wrote:

> On Mon, May 14, 2012 at 3:28 PM, Glenn Maynard <[hidden email]> wrote:
>> On Mon, May 14, 2012 at 3:01 PM, Gregg Tavares (勤) <[hidden email]> wrote:
>>
>>> I'd like to work on exposing something like CANVAS to web workers.
>>>
>>> Ideally how over it works I'd like to be able to
>>>
>>> *) get a 2d context in a web worker
>>
>>
>> I'd recommend not trying to tackle 2d and 3d contexts at once, and only
>> worrying about WebGL to start.
>>
>> Another issue: rendering in a worker thread onto a canvas which is displayed
>> in the main thread.  This needs to be solved in a way that doesn't cause the
>> asynchronous nature of what's happening to be visible to scripts.  toDataURL
>> and toBlob would probably need to be prohibited on the canvas element.  I'm
>> not sure what the actual API would look like.
>
> If/when we do this, I think it should be done in such a way that the
> main window can't access the canvas object at all. Similar to what
> happens when an ArrayBuffer is transferred to a Worker using
> structured cloning. Once a canvas is transferred to a Worker, any
> access to it should throw or return null/0/"". If you want to transfer
> pixel data to the main thread, it seems less racy to do that by
> getting the pixel data in the Worker which owns the canvas and then
> transfer that to the main thread using postMessage.
>
>> This would also require some
>> equivalent to requestAnimationFrame in the worker thread.
>
> Agreed!
>
> / Jonas



I'm a bit lost-- wouldn't we just postMessage from the document over to the web worker when we want a refresh?

I agree that we ought to be transferring pixel data not Canvas contexts; with the possible exception of CSS context.

We could just create new canvas instances inside the worker thread. I'd still prefer to clone CanvasPattern as a means of transferring paint over to the worker, though sending pixel data would work too.

I heard Picture come up-- it seems like that object might have additional semantics for high resolution alternatives that may need to be considered.

-Charles
Reply | Threaded
Open this post in threaded view
|

Re: exposing CANVAS or something like it to Web Workers

Boris Zbarsky
In reply to this post by Gregg Tavares (wrk)
On 5/14/12 8:03 PM, Gregg Tavares (勤) wrote:

> var canvasSurface = new CanvasSurface();
> var ctx = canvasSurface.getContext("2d");
> var pic = new Picture;
> pic.src = "http://someplace.com/someimage.jpg";
> pic.onload = function() {
>     ctx.drawImage(pic, 0, );
> }
>
> Let's assume you can instantiate these things in either the page or a
> worker. Nether can be transfered.
>
> Would that work? What problems would that have?

Two things that come to mind immediately:

1)  Various canvas 2d context methods depend on the styles of the canvas
to define how they actually behave.  Clearly this would need some sort
of changes for Workers anyway; the question is what those changes would
need to be.

2)  How (or maybe whether) this should work with pic.src =
"http://someplace.com/someimage.svg" is an interesting question.

3)  This leaves open the question of how to get data from the worker
back to the main thread to paint there.

#1 is an issue with any proposal that puts a 2d context in a worker.

#2 is only an issue if we do the image loads in workers somehow; if we
wanted to just rasterize the SVG and pass the resulting data to the
worker there would be no problem, obviously.

For #3, being able to transfer a canvas context from the page to a
worker but leave it painting to the canvas may still be desirable...

-Boris

Reply | Threaded
Open this post in threaded view
|

Re: exposing CANVAS or something like it to Web Workers

James Robinson-5
In reply to this post by Boris Zbarsky


On Mon, May 14, 2012 at 5:03 PM, Boris Zbarsky <[hidden email]> wrote:
On 5/14/12 7:56 PM, Glenn Maynard wrote:
A tricky bit: you need to know which element to sync to, so the browser
knows which monitor's vsync to use.  According to [1] only WebKit's
requestAnimationFrame actually takes an element.  (That's surprising;
this seems obvious.

Does WebKit actually use the element to determine vsync?  How do they handle cases when the element spans monitors?

As far as I know WebKit's implementation uses the element to optimize out callbacks when the element is not visible, but that's it.

The element isn't used for anything currently in WebKit.  "Which vsync" is determined by the monitor the tab/window/whatever lands on.  When this spans monitors, something random happens (there aren't many good options). 

Note that Gecko, for example, does not tie requestAnimationFrame callbacks to vsync.  I can't speak for other UAs.

I think you'll want to.

- James
 


I mention this because
this method would need to accept a context in lieu of an element

What would the context be used for?

-Boris


Reply | Threaded
Open this post in threaded view
|

Re: exposing CANVAS or something like it to Web Workers

Charles Pritchard-2
In reply to this post by Gregg Tavares (wrk)
On May 14, 2012, at 5:03 PM, Gregg Tavares (勤) <[hidden email]> wrote:



On Mon, May 14, 2012 at 4:42 PM, Jonas Sicking <[hidden email]> wrote:
On Mon, May 14, 2012 at 3:28 PM, Glenn Maynard <[hidden email]> wrote:
> On Mon, May 14, 2012 at 3:01 PM, Gregg Tavares (勤) <[hidden email]> wrote:
>
>> I'd like to work on exposing something like CANVAS to web workers.
>>
>> Ideally how over it works I'd like to be able to
>>
>> *) get a 2d context in a web worker
>
>
> I'd recommend not trying to tackle 2d and 3d contexts at once, and only
> worrying about WebGL to start.
>
> Another issue: rendering in a worker thread onto a canvas which is displayed
> in the main thread.  This needs to be solved in a way that doesn't cause the
> asynchronous nature of what's happening to be visible to scripts.  toDataURL
> and toBlob would probably need to be prohibited on the canvas element.  I'm
> not sure what the actual API would look like.

If/when we do this, I think it should be done in such a way that the
main window can't access the canvas object at all. Similar to what
happens when an ArrayBuffer is transferred to a Worker using
structured cloning. Once a canvas is transferred to a Worker, any
access to it should throw or return null/0/"". If you want to transfer
pixel data to the main thread, it seems less racy to do that by
getting the pixel data in the Worker which owns the canvas and then
transfer that to the main thread using postMessage.

How about separating the canvasy parts of canvas from canvas and the imagy parts of image from image.

In other words, Imagine canvas is implemented like this

class Canvas : public HTMLElement {
  private:
    CanvasSurface* m_surface;  // everything about canvas that is not HTMLElement
};

And that Image is similarly implemented as

class Image : public HTMLElement {
  private:
    Picture* m_picture;  // everything about Image that is not HTMLElement
}

now imagine you can instantiate inner implementation of these things. The parts that are not HTMLElement

var canvasSurface = new CanvasSurface();
var ctx = canvasSurface.getContext("2d");
var pic = new Picture;
pic.onload = function() {
   ctx.drawImage(pic, 0, );
}

Let's assume you can instantiate these things in either the page or a worker. Nether can be transfered.

Would that work? What problems would that have?


Seems fine. If it works with XHR I'd imagine it'll work with Picture.

Blob uri behavior may come up as an issue. I believe they should "just work". SVG and animated gif would render the same as it would in an Image that has not been added to the dom or is otherwise display: none.

I suspect we may have blob Uris corresponding to live video streams. If those come about and work with onload I'd expect them to work here (simply grabbing the last/first available frame).






 

> This would also require some
> equivalent to requestAnimationFrame in the worker thread.

Agreed!

/ Jonas

Reply | Threaded
Open this post in threaded view
|

Re: exposing CANVAS or something like it to Web Workers

Boris Zbarsky
On 5/14/12 8:21 PM, Charles Pritchard wrote:
> SVG and animated gif would render the same as it would in an
> Image that has not been added to the dom or is otherwise display: none.

I'm not sure that would be workable in a worker for SVG, for the same
reasons that responseXML is not available in workers: it involves DOM
elements.  Unless the SVG rasterization happens on the main thread under
the hood and the raster data is then sent over to the worker.  This
might have ... surprising performance characteristics.

-Boris

Reply | Threaded
Open this post in threaded view
|

Re: exposing CANVAS or something like it to Web Workers

Gregg Tavares (wrk)
In reply to this post by Boris Zbarsky


On Mon, May 14, 2012 at 5:12 PM, Boris Zbarsky <[hidden email]> wrote:
On 5/14/12 8:03 PM, Gregg Tavares (勤) wrote:
var canvasSurface = new CanvasSurface();
var ctx = canvasSurface.getContext("2d");
var pic = new Picture;
pic.src = "http://someplace.com/someimage.jpg";
pic.onload = function() {
   ctx.drawImage(pic, 0, );
}

Let's assume you can instantiate these things in either the page or a
worker. Nether can be transfered.

Would that work? What problems would that have?

Two things that come to mind immediately:

1)  Various canvas 2d context methods depend on the styles of the canvas to define how they actually behave.  Clearly this would need some sort of changes for Workers anyway; the question is what those changes would need to be.

Which methods are these?

 

2)  How (or maybe whether) this should work with pic.src = "http://someplace.com/someimage.svg" is an interesting question.

3)  This leaves open the question of how to get data from the worker back to the main thread to paint there.

I feel that's a somewhat orthogonal issue to getting a Canvas or CanvasSurface in a worker. At least for the WebGL case, WebGL already defines ways to share data safely between contexts so that use case is already covered.

For a 2d context I can certainly imagine lots of ways. Some fast, some slow

slow) pass the result of toDataURL back

slow) pass an ImageData.data back (transfer of ownership)

fast) some how use a CanvasSurface as the source of a video tag

maybe there are some other ideas? But, even if we didn't resolve this, it's still massively useful, at least for WebGL, to be able to instantiate a WebGL context in a worker.



 

#1 is an issue with any proposal that puts a 2d context in a worker.

#2 is only an issue if we do the image loads in workers somehow; if we wanted to just rasterize the SVG and pass the resulting data to the worker there would be no problem, obviously.

For #3, being able to transfer a canvas context from the page to a worker but leave it painting to the canvas may still be desirable...

-Boris


Reply | Threaded
Open this post in threaded view
|

Re: exposing CANVAS or something like it to Web Workers

Charles Pritchard-2
In reply to this post by Boris Zbarsky
On May 14, 2012, at 5:50 PM, Boris Zbarsky <[hidden email]> wrote:

> On 5/14/12 8:21 PM, Charles Pritchard wrote:
>> SVG and animated gif would render the same as it would in an
>> Image that has not been added to the dom or is otherwise display: none.
>
> I'm not sure that would be workable in a worker for SVG, for the same reasons that responseXML is not available in workers: it involves DOM elements.  Unless the SVG rasterization happens on the main thread under the hood and the raster data is then sent over to the worker.  This might have ... surprising performance characteristics.

I agree... Can we get this off the main thread? Svg via image is not quite the same as svg via HTMLDocument (I guess I mean, embedded).

Afaik, svg via image does not have any script controls but it does have xsl things.

I've never tried to abuse the distinction via embedded blob Uris and such.

Put in other words: <img src=image.svg> may use an entirely different implementation than <svg> in an HTML document.
Reply | Threaded
Open this post in threaded view
|

Re: exposing CANVAS or something like it to Web Workers

Boris Zbarsky
In reply to this post by Gregg Tavares (wrk)
On 5/14/12 8:55 PM, Gregg Tavares (勤) wrote:
>     1)  Various canvas 2d context methods depend on the styles of the
>     canvas to define how they actually behave.  Clearly this would need
>     some sort of changes for Workers anyway; the question is what those
>     changes would need to be.
>
> Which methods are these?

Anything involving setting color (e.g. the strokeStyle setter, the
fillStyle setter), due to "currentColor".  Anything involving text
because font styles come from the element or document.

Those are the ones that come to mind offhand, but I haven't looked at
the various recent additions to the 2d context closely.

-Boris

Reply | Threaded
Open this post in threaded view
|

Re: exposing CANVAS or something like it to Web Workers

Boris Zbarsky
In reply to this post by Charles Pritchard-2
On 5/14/12 8:58 PM, Charles Pritchard wrote:
> I agree... Can we get this off the main thread?

"Maybe".

It would be pretty nontrivial in Gecko; last I looked it would be pretty
painful in WebKit too.  Can't speak for other UAs.

> Svg via image is not quite the same as svg via HTMLDocument (I guess I mean, embedded).

Sure, but that doesn't matter.

> Afaik, svg via image does not have any script controls but it does have xsl things.

And CSS.

> Put in other words:<img src=image.svg>  may use an entirely different implementation than<svg>  in an HTML document.

It _may_, but it would likely be pretty buggy.  And at least in Gecko, I
don't think we're willing to duplicate enough of the rendering engine to
do this.

-Boris

Reply | Threaded
Open this post in threaded view
|

Re: exposing CANVAS or something like it to Web Workers

Gregg Tavares (wrk)
In reply to this post by Boris Zbarsky


On Mon, May 14, 2012 at 6:07 PM, Boris Zbarsky <[hidden email]> wrote:
On 5/14/12 8:55 PM, Gregg Tavares (勤) wrote:
   1)  Various canvas 2d context methods depend on the styles of the
   canvas to define how they actually behave.  Clearly this would need
   some sort of changes for Workers anyway; the question is what those
   changes would need to be.

Which methods are these?

Anything involving setting color (e.g. the strokeStyle setter, the fillStyle setter), due to "currentColor".  Anything involving text because font styles come from the element or document.

Good to know.

That doesn't sound like a showstopper though. If a canvas/CanvasSurface is available in workers the simplest solution would just be that "currentColor" defaults to something "black?" or nothing "". Pick one.
 

Those are the ones that come to mind offhand, but I haven't looked at the various recent additions to the 2d context closely.

-Boris

123