WebAssembly Interface Types – HTTP203

WebAssembly Interface Types – HTTP203


JAKE ARCHIBALD: Do
you know what I did? I accidentally took my
family to a strip club. DAS SURMA: Wow. JAKE ARCHIBALD: Too
long a story though, so if you want to hear about
that, go listen to the podcast. DAS SURMA: Da-da-da-da
branding, da-da-da-da branding. [MUSIC PLAYING] I mean, it’s not going to
be surprising that I’ve been working with WebAssembly. JAKE ARCHIBALD: Have you been
working with WebAssembly again? DAS SURMA: I’ve been working
with WebAssembly a little bit. JAKE ARCHIBALD: Excellent. DAS SURMA: Yeah. So there’s a new proposal. Because I think
it’s time that we talk about things that don’t
exist in WebAssembly yet. JAKE ARCHIBALD: We’ve
been doing that recently with other features,
like the infomap thing. DAS SURMA: Yeah, so let’s– JAKE ARCHIBALD: So we’re
carrying on doing that. DAS SURMA: –bring that
good stuff to WebAssembly, talk about stuff that
doesn’t exist yet. JAKE ARCHIBALD: And will
probably change by the time it does exist. DAS SURMA: Yep. JAKE ARCHIBALD: OK,
good, useful stuff. DAS SURMA: As such,
WebAssembly interface types are a new proposal
that basically consists of an explainer. So there’s actually
no spec text in there. It’s a couple of examples. JAKE ARCHIBALD: I
would’ve shipped– like that before. That’s fine. DAS SURMA: Mozilla
wrote a blog post on it, and therefore, it’s real, right? JAKE ARCHIBALD: OK, fair enough. Yeah, if Mozilla
wrote in, I’m happy. DAS SURMA: So I thought
I’d start with the problem. JAKE ARCHIBALD: Right. DAS SURMA: So this
is how you currently use WebAssembly on the web. If you want to load a
module, you fetch it. You throw it into
instantiateStreaming, and you get back an instance. And on this instance you
can call the function or export it from– JAKE ARCHIBALD: Still
really annoys me that– DAS SURMA: I know. It really bugs you, doesn’t it? JAKE ARCHIBALD: That it
takes a promise, yeah. You should just– it
should take a response, and you should await it. Anyway, sorry. So this is how it
normally works, fine. DAS SURMA: And that
will not change. That will continue
to work because web. JAKE ARCHIBALD: Fine. DAS SURMA: And here we’re
calling a function, parameters 1, 2, 3, great. Now, the bit that is a big
pain point to many people is that 1, 2, 3 is
pretty much as capable as WebAssembly is in that it
can only work with numbers. You can only pass numbers in
and only get numbers back. And here’s actually a screenshot
from the spec that says, these are the values
that WebAssembly can do. And they’re integers and
floats in different precisions, and that’s it. That’s all WebAssembly
has to work with. JAKE ARCHIBALD: And that’s
why if you have something that takes a string then you end up
with inscription or something similar, that’s going
to pack that into– DAS SURMA: Exactly so– JAKE ARCHIBALD: –the memory,
the meferance to– meferance to– DAS SURMA: A meferance. JAKE ARCHIBALD:
Meferance to wenry– a reference to memory,
whichever you want. DAS SURMA: Yeah so
let’s say you have an instance of a
module that expects a string in one way or another. JAKE ARCHIBALD: Oh, I’m really
glad I spoiled all of you slides. DAS SURMA: Yes, this was
a perfect segue basically. JAKE ARCHIBALD: Sorry, carry on. DAS SURMA: What
you have to do then is turn that string
into numbers. And so one version of it is
to turn a string into a series of UTF-8 encoded bytes. Because that’s a way to
turn a string into numbers, by agreeing on an encoding. So now you have a
buffer full of numbers, and you have a
length of the buffer. Now, assuming we
are in C land, you have to tell C that,
give me a chunk of memory I can put this in so that C
knows that that piece of memory is now in use and shouldn’t
be used for anything else so that the [INAUDIBLE]
doesn’t start overwriting it. And that’s what malloc
is usually used for. And then you get back an
address in the memory. And then you have to turn
your WebAssembly memory into a Uint8Array so you can
actually write stuff in it. And then you have to use
set to copy in the string. And then eventually,
finally you can call a function that does
something with the string by passing in sad address. This is horrible. JAKE ARCHIBALD: Hang on. Where is– so– DAS SURMA: It is supposed
to be overwhelming. JAKE ARCHIBALD: Yeah,
I’m overwhelmed. So we need to tell– oh, OK,
so this is the memory buffer. And so we’re turning
that into Uint8Array, and then we’re setting that. Right, OK, I follow now. I couldn’t see how the mem8 was
actually part of WebAssembly but it’s because it– DAS SURMA: It’s using exported– JAKE ARCHIBALD:
–memory, right, right. DAS SURMA: So
[INAUDIBLE] has a buffer, and then we turned it into– because that’s just
a series of bits. You decide on how you want to
interpret those series of bits, in this case, a series
of Uint8 of bytes. JAKE ARCHIBALD: Yeah, this is
what we did in Squoosh, right, to get the images
into WebAssembly. DAS SURMA: Yeah, in this
case we take a string. We turn into a UTF-8
encoded byte array and copy that into
the WebAssembly memory at an agreed upon location. Now, this is obviously not a
great developer experience. JAKE ARCHIBALD: No. DAS SURMA: And also,
you need to know what your language expects. Because this is
how C would do it. Rust would do it differently
because they actually encode the length of the string
into a data structure, while C doesn’t. JAKE ARCHIBALD: And that’s
part of how it stops overflows. DAS SURMA: And Go will do
it differently yet again, so it is really
language-dependent. And so what most
languages currently do is that instead of
giving you this, they give you so-called
glue code, which then you use to instantiate the module. And then you get a kind
of wrapped instance that does all this
magic under the hood. So with this, you’re just
able to pass in a string. And the code we just saw is
basically now just hidden away and contains all the
language-specific knowledge. JAKE ARCHIBALD: And
we end up with cases where the glue code can be
just as big, if not bigger, than the [INAUDIBLE]. DAS SURMA: Yeah, we’ve
seen it with the M script. Now, M script does a lot of work
for you, more than just string encoding. So it’s not like the
size is uncalled for, but it turns out
to be quite big. But this is kind of
where you’re like, well, WebAssembly can do
numbers, but strings are extremely common in
pretty much any language. So are probably lists or
dictionaries or Booleans, and all of these need to be
mapped to numbers somehow. But again, every language
does it differently. So can WebAssembly at a status
level maybe make this easier? And that is what
interface types are. The name is not great. It’s actually the third name,
I think, of this proposal. So bear with me. I will explain the
name, and then it will kind of make sense. But really what it is–
it brings more types. JAKE ARCHIBALD: It’s just types. DAS SURMA: Types. JAKE ARCHIBALD: Yeah,
because when I had interface, I was thinking of,
well, TypeScript and how it’s more
of a struct then– DAS SURMA: Yeah. No, it’s types for
the module interface, as in the interface of
the WebAssembly module. JAKE ARCHIBALD: Oh, OK, yeah. DAS SURMA: Interface types. So there is actually
a little graphic– JAKE ARCHIBALD:
Oh, look at this. DAS SURMA: –explain it. So this is– JAKE ARCHIBALD: Is
this how babies happen? [LAUGHTER] DAS SURMA: I mean– JAKE ARCHIBALD: This
is some science book– Right, OK, come on. Explain this to me. DAS SURMA: Right, so
when you use WebAssembly, you usually get the blue bit. The blue bit is just,
like, you have a module. The module has the types
that WebAssembly knows, like the integers and the
floats, and functions. And you can call these functions
with those types, the numbers. Now, interface
types are basically a adapter layer on top
of this core module, and it gives the whole
module a new interface. So if you have an
interface type, the original module will be
completely hidden from you. And instead, a
new interface will be exposed to you
as a developer that can use these high-level types. JAKE ARCHIBALD: Essentially
at the glue code, right? DAS SURMA: Yeah. But now the glue
code, the stuff that translate from high-level types
like strings or dictionaries to low level, is bundled
with the WebAssembly module. JAKE ARCHIBALD: As Wasm? DAS SURMA: As Wasm. JAKE ARCHIBALD: Right, OK, OK. DAS SURMA: So because it is
inherently language-specific, it’s not necessary something
that can be reused. A C module will have to
use different adapter code than a Go module
or a Rust module. And so it’s packaged within
the WebAssembly binary. And whenever you call
a high-level function with a string, then
these adapter codes will run first, translate
all the parameters and/or return values, and
then call the old core module function. So it is actually a
completely abstract layer on top of the old interface. So with that in place, what you
can do is instantiate a module and call it with a string,
which as a function is now much nicer and
much more convenient. There is obviously a question
about progressive enhancement. Because we have now
reached the point where browsers ship
new WebAssembly features at different paces. JAKE ARCHIBALD: Threads, SIMD– all this stuff is on its way. DAS SURMA: And that’s
just the start. I think Chrome is the
only one with threads so far because the
[INAUDIBLE] thing is really complicated for some
browsers to actually ship. SIMD is probably
going to be similar, in that we ship it first and
then others might catch up. But it’s not going to
be a synchronized launch as it was with WebAssembly
and [INAUDIBLE].. JAKE ARCHIBALD: And
even SIMD is like– there’s multiple levels
of SIMD support, right? DAS SURMA: Yeah, there is. JAKE ARCHIBALD: OK. DAS SURMA: Now for the
progressive enhancement, I actually envision
that something like this will be a fairly common
pattern, where you instantiate your module then do a quick
check if this browser has interface type support. And if not, you
basically fall back to the old glue
code pattern, which means that only if you
have an unsupported browser you have to load that
additional piece of JavaScript. JAKE ARCHIBALD: So hang on. So if this instantiate
thing– so this is going to be a fetch
of a resource possibly. DAS SURMA: Yeah. JAKE ARCHIBALD: So
is the idea that– is it compatible in a way
where if an older browser loads the thing with the glue
code, it ignores the code? DAS SURMA: Yeah. JAKE ARCHIBALD: Oh, interesting. DAS SURMA: So it’s bundled
within– it’s a annotation. And so if a browser
doesn’t have support for this special
type of annotation, these sections are just
skipped and ignored. JAKE ARCHIBALD:
Right, that’s smart. DAS SURMA: And
that was very much a core part of
WebAssembly, that they have support for custom
sections, as they’re called, which can give an
arbitrary name. And only if the
browser has like, I know what this name
means, it will actually start respecting that custom
section in the WebAssembly binary. JAKE ARCHIBALD: And it
doesn’t fail if it doesn’t– right, OK. DAS SURMA: And so
in this way, it would basically mean that
only the browser that don’t have interface
type support will actually pay the
cost of the glue code. Or actually, it might not
just be glue code but just a generic interface
type polyfill that is also being worked on. JAKE ARCHIBALD: Oh,
that’ll be good. DAS SURMA: So that
would be kind of nice. JAKE ARCHIBALD: Yeah, right? I like it. DAS SURMA: Now, I’ve
only talked about strings because literally every other
type is in a todo section on the expand. That’s how early we are talking. The mechanism has been kind
of sounded out and kind of explored, but it hasn’t
actually been specified on how other types would work. And that’s what they
still have to figure out to actually make this real. But I still think it’s kind
of interesting to talk about. Now for example, one thing that
would become better is we use high-level types in Squoosh
quite a bit in our WebAssembly modules because we have
codecs that A, take an image– that’s a fairly
high-level type– and usually options for
the coding and quite the substantial set of options. JAKE ARCHIBALD: Yes, almost like
a struct in C++ land and then– DAS SURMA: Yeah, so we have an
encode function, for example, on our mozjpeg
WebAssembly module that takes A, the image, and
B, just a set of options. And these things
would now just work with WebAssembly, which is– JAKE ARCHIBALD:
That’s really nice. DAS SURMA: It’s really cool. And then, of course,
it’s up to the language to define, how does a JSON
object like this actually map to something that
is good in my language? Like in Rust it would probably
be a struct, in C as well. But in Go it might be
something different, or maybe Python at some point. Every language now is in
control in how these things get mapped to their native types. JAKE ARCHIBALD: Nice. DAS SURMA: And another
thing which actually is really nice– because it’s
not just about developers calling functions. It’s also about
this thing that we see in Squoosh, in that
we have image data, and we basically just
pass it back and forth between JavaScript and
different WebAssembly modules. We used AssemblyScript
to rotate images. We use a Rust code
to resize images. Then we use C code to
actually compress the image. And that’s a lot of
jumping back and forth between JavaScript types and
glue code and WebAssembly and back. If interface types
actually become a thing, we could actually live in a
world where this is possible. JAKE ARCHIBALD: Oh, interesting. DAS SURMA: Because these types
are defined completely abstract without an actual
Implementation [INAUDIBLE].. So strings and
lists are just that, and not much more is
defined about them. And so that means now
we can pass a string from AssemblyScript
to Rust without having to go via JavaScript
because both modules will contain the code to translate. JAKE ARCHIBALD: OK, so
this glue code isn’t just about turning a
string into something the language understands. It will do the reverse as well. DAS SURMA: Yeah. JAKE ARCHIBALD: So this
will work for the return types, right, which– DAS SURMA: Yeah, exactly. So there’s the concept of
lowering values and raising values from a high abstraction
to a low abstraction and back. JAKE ARCHIBALD: Nice. DAS SURMA: And so if an
AssemblyScript function returns a string and a Rust
function accepts a string, we can actually do a fast path
and skip the whole JavaScript in between. JAKE ARCHIBALD: So it
would be, I guess– Yeah, like I say, it’s going
to be a finite set of types, right? Because it– DAS SURMA: As far as I
can tell, it’s basically going to be roughly JSON, all
the types that JSON can do, plus a couple of extra bits. Because another old proposal
that has been merged into it this is the WebIDL proposal. So what that would allow us–
this was a perfect set-up– JAKE ARCHIBALD: Oh, excellent. DAS SURMA: –is that with
interface types and a proposal called anyref, which allows
WebAssembly to hold onto anything from JavaScript land
and even call functions on it, we could now do the thing
that so many people want, is have the DOM in WebAssembly,
in that we can just basically give the entire document object
to WebAssembly and be allowed to call functions because now we
have a way to communicate with strings and objects and– JAKE ARCHIBALD: Oh, interesting. DAS SURMA: And so while
you won’t have access to the actual
low-level DOM item, you have a reference to it and
you can call functions on it. And as long as one of these
interface compatible types is being returned or
accepted by the methods, you can actually
communicate with that now. And so something– JAKE ARCHIBALD:
That’s really neat. DAS SURMA: –like
create element– you can call that from
WebAssembly without glue code because strings are a thing. And with anyref you
can call that function. And so now you kind of have DOM
or other APIs in WebAssembly. JAKE ARCHIBALD: Nice. So in Squoosh we use
image data a lot. So I guess that’s one
of the types that’s unlikely to be mapped, so we
would just use [INAUDIBLE].. DAS SURMA: I mean, it maps
very easily to a JSON object. Width height data is
really what image data is, so that would just be a struct
with these three properties. JAKE ARCHIBALD:
And we would just have a very small wrapper at
the end that is just create– DAS SURMA: For example,
I think all the array buffer types will also be
supported by interface types because that’s fairly easy
to map to a piece of memory, right? JAKE ARCHIBALD: Yes. DAS SURMA: It will probably
be a copy because– JAKE ARCHIBALD: WebAssembly
memory, JavaScript memory, yeah. DAS SURMA: But copying
a chunk of memory is a fairly fast
operation anyway. And now that you say that,
there is an upcoming proposal about allowing multiple memories
in a WebAssembly module. So maybe we could even
just mount in memory into WebAssembly so
that it’s no copy but you actually have
a reference to it now. JAKE ARCHIBALD: And I’ve
heard one of the use cases for this is to create– where you could have
a section of memory that is essentially
an m-mapped file. DAS SURMA: Yeah, exactly. So now it wouldn’t
have to be like, oh, we reserve a part
of the old memory but just create a
second piece of memory that is completely independent
from the main memory. And you just have to figure
out which memory to work on with your instructions. So yeah, this is, I
think, really cool. I think most of this
will be taken care of by compilers of languages
who will just basically give you now a module that
suddenly accepts a string, and there’s nothing for you to
do to make use of this new API. But I think the
doors it opens is going to be really
interesting for WebAssembly. JAKE ARCHIBALD: Nice. DAS SURMA: And– going to
be a while though, so– [LAUGHTER] JAKE ARCHIBALD: Look, I
was really happy then, and now you’re saying– DAS SURMA: Yeah, I’m
super excited about this. JAKE ARCHIBALD: Yay! Good, by the time
you retire, you might get to play with this. DAS SURMA: I have no idea
how fast it’s going to be. But I’m going to
keep a close eye out, and I am going to be
tweeting like crazy once we can play
with this in canary. JAKE ARCHIBALD: Looking
forward to that. [MUSIC PLAYING] DAS SURMA: There were
people at View Source. JAKE ARCHIBALD: Good, because
that’s what a conference needs to really work. DAS SURMA: And they were
like, wait, you do a podcast? They were, like, talking
to me about the videos. And I was like– JAKE ARCHIBALD: No
one knows our podcast. DAS SURMA: You don’t watch
the videos if, like– JAKE ARCHIBALD: We need to
advertise the podcast more, just like this.

19 thoughts on “WebAssembly Interface Types – HTTP203”

  1. from my experience, even the tyniest function in wasm is about 15mo so for me wasm is usable only in "computer vision" or other CPU's intensive operations

  2. I don't think the second example is 100% correct: the data returned by `TextEncoder#encode` isn't null terminated, so `ptr` isn't really pointing to a C-string in the usual sense.

  3. "link in the description" …there is no link. I have to use google to find https://developers.google.com/web/shows/http203/podcast

  4. Could the podcast be on some larger podcast platform? Listening to those on the website isn't that convenient on mobile. Although that would be a nice-ish weekend flutter project but still not that useful.

  5. I was hoping for passing interfaces as a wasm value type that can have methods called on it from wasm.
    edit: hmm, seems like maybe on that?

  6. Which of these are not actual terms:
    TypeScript
    TypeAssembly
    WebAssembly
    WebScript
    JavaScript
    JavaType
    AssemblyScript
    AssemblyTypes
    Assembly Interfaces

  7. Hey, what you guys think about AssemblyScript, is it something you can get started with even if you don't know Rust or C deeply?

  8. as i said 2 years ago … WebAssembly IS made to KILL javascript and replace it with C …. and all this because we do NOT learn from history … we have try this with Java ?? and Java failed in the browser … the future is Great, when WebAssmbly fail … we still have JS in the browser, so we can move forward again.

Leave a Reply

Your email address will not be published. Required fields are marked *