Linking in Instantiation stage

Hi, here's from WasmEdge, and I would want to implement the component-model on this runtime.
Hence there's no spec to clearly introduce the steps of instantiation such as WASM core, I may have to ask some basic questions about linking for a better design of our runtime data structure.
Maybe the questions are simple, or are described in the proposal docs, thanks for you all to answer.

The main question is about the import searching when linking.
Take the linking example in this repo:

;; app.wat
(component
  (import "libc" (core module $Libc ...))
  (import "libzip" (core module $Libzip ...))
  (import "libimg" (core module $Libimg ...))

  (import "zipper" (component $Zipper ...))
  (import "imgmgk" (component $Imgmgk ...))

  (core module $Main
    (import "libc" "memory" (memory 1))
    (import "libc" "malloc" (func (param i32) (result i32)))
    (import "zipper" "zip" (func (param i32 i32) (result i32 i32)))
    (import "imgmgk" "transform" (func (param i32 i32) (result i32 i32)))
    ...
    (func (export "run") (param i32 i32) (result i32 i32)
      ...
    )
  )

  ;; ... ignore bellow
)

When instantiating this component, the core:module $Main will be instantiated also.
As above, this component imports "libc", "libzip", and "libimg" modules.
Assume that the validation is passed.
Can we assume that all the imports of the core:module $Main will be reserved (found) in this component imports?

Another example for the nested component.

(component $A
  (import "libc" (core module $Libc ...))
  (component $B
    (core module $Main
      (import "libc" "memory" (memory 1))
      ...
      (func (export "run") (param i32 i32) (result i32 i32)
        ...
      )
    )
  )
  ...
)

For the nested component $B, should this component also import "libc" for the core:module to import?
If the import of "libc" is not necessary in $B, the import searching of the core:module $Main in component $B should access the imports of component $A?

Thank you for answering.

Hi, thanks for asking and excited to hear you're working on the WasmEdge implementation!

Can we assume that all the imports of the core:module $Main will be reserved (found) in this component imports?

A nested module named $M is only instantiated by a subsequent (core instance $name (instantiate $M (with ...)*)) definition (if there is none, no instances are created; $M is dead code). The imports of $M are 100% determined by what is passed in the ... inside the (with ...) expressions. Here, the semantics is just named-parameter-passing (just like the importObject passed to WebAssembly.instantiate. Moreover, there can be 0..N (core instance ...) definitions for $M, creating 0..N instances of $M, each with potentially different arguments supplied for $M's imports. So you basically want to think of $M as a big outer function nesting all the real wasm (func ...)s and the imports of $M are the parameters of this outer function.

On a side note, I've started writing more .wast tests (now that I can actually execute them) in the tests directory. Most of the tests are for async (in the tests/async subdirectory), but I've been meaning to fill in more for pre-async stuff for linking and resource types. E.g., here's a multi-instance test for resource types. Feel free to ask me for any specific .wasts (here or on zulip) if they'd help you.

Hi, thanks for asking and excited to hear you're working on the WasmEdge implementation!

Can we assume that all the imports of the core:module $Main will be reserved (found) in this component imports?

A nested module named $M is only instantiated by a subsequent (core instance $name (instantiate $M (with ...)*)) definition (if there is none, no instances are created; $M is dead code). The imports of $M are 100% determined by what is passed in the ... inside the (with ...) expressions. Here, the semantics is just named-parameter-passing (just like the importObject passed to WebAssembly.instantiate. Moreover, there can be 0..N (core instance ...) definitions for $M, creating 0..N instances of $M, each with potentially different arguments supplied for $M's imports. So you basically want to think of $M as a big outer function nesting all the real wasm (func ...)s and the imports of $M are the parameters of this outer function.

On a side note, I've started writing more .wast tests (now that I can actually execute them) in the tests directory. Most of the tests are for async (in the tests/async subdirectory), but I've been meaning to fill in more for pre-async stuff for linking and resource types. E.g., here's a multi-instance test for resource types. Feel free to ask me for any specific .wasts (here or on zulip) if they'd help you.

Thanks for the reply. The multi-instance test you mentioned is useful, I'll read it then.
The link of the example I took is incorrect, and I've fixed it.

As you mentioned, I think I can treat the import, component, and module section of a component as the descriptions, which describe the declaration of the module or sub-component, and the instance and core:instance sections supply the needed import arguments to instantiate them.
That is, after passing the validation, take the example above:

;; app.wat
(component
  (import "libc" (core module $Libc ...))
  (import "libzip" (core module $Libzip ...))
  (import "libimg" (core module $Libimg ...))

  (import "zipper" (component $Zipper ...))
  (import "imgmgk" (component $Imgmgk ...))

  (core module $Main
    (import "libc" "memory" (memory 1))
    (import "libc" "malloc" (func (param i32) (result i32)))
    (import "zipper" "zip" (func (param i32 i32) (result i32 i32)))
    (import "imgmgk" "transform" (func (param i32 i32) (result i32 i32)))
    ...
    (func (export "run") (param i32 i32) (result i32 i32)
      ...
    )
  )

  (instance $zipper (instantiate (component $Zipper)
    (with "libc" (module $Libc))
    (with "libzip" (module $Libzip))
  ))
  (instance $imgmgk (instantiate (component $Imgmgk)
    (with "libc" (module $Libc))
    (with "libzip" (module $Libzip))
    (with "libimg" (module $Libimg))
  ))

  (core instance $libc (instantiate (module $Libc)))
  (core func $zip (canon lower
    (func $zipper "zip")
    (memory (core memory $libc "memory")) (realloc (func $libc "realloc"))
  ))
  (core func $transform (canon lower
    (func $imgmgk "transform")
    (memory (core memory $libc "memory")) (realloc (func $libc "realloc"))
  ))
  (core instance $main (instantiate (module $Main)
    (with "libc" (instance $libc))
    (with "zipper" (instance (export "zip" (func $zipper "zip"))))
    (with "imgmgk" (instance (export "transform" (func $imgmgk "transform"))))
  ))
  (func $run (param string) (result string) (canon lift
    (core func $main "run")
    (memory (core memory $libc "memory")) (realloc (func $libc "realloc"))
  ))
  (export "run" (func $run))
)

When instantiating the $Main or the component $Zipper (the sub-components or sub-modules), their needed imports are expected to be supplied by the (with ...) expressions. Is it correct?

Another interesting question.
What would the code be like if a component have to import the outside modules, such as wasi?
For example, in core WASM, the module would like:

(module
  (import "wasi" "fd_open" (func (param ...) (result i32)))
  ...
)

And the runtime should help to link with the modules when instantiate it.
In component model proposal, such as the above example, the component seems like described all the needed sub-components into a top component and link as internal.
Would you help to take a simply example of a component import the external modules?
Thanks!

When instantiating the $Main or the component $Zipper (the sub-components or sub-modules), their needed imports are expected to be supplied by the (with ...) expressions. Is it correct?

That's right

What would the code be like if a component have to import the outside modules, such as wasi?

WASI imports appear in a component as imports with instance type. For example, based on this WIT if your component imported now, you'd get:

(component
  (import "wasi:clocks/monotonic-clock@0.3.0" (instance
    (export "now" (func (result u64)))   ;; (i'm ignoring the 'type instant = u64' here)
  ))
  ...
)

Because we're importing an instance (as opposed to a component or core module), you don't have to instantiate it -- it's already instantiated (by the host or by a parent component -- we don't know) and so this import goes straight into the instance index space (with which you can use an alias definition to project out the now function into the func index space, then canon lower it into the core func index space, then reference that from a with when instantiateing the main core module of the component).

When instantiating the $Main or the component $Zipper (the sub-components or sub-modules), their needed imports are expected to be supplied by the (with ...) expressions. Is it correct?

That's right

What would the code be like if a component have to import the outside modules, such as wasi?

WASI imports appear in a component as imports with instance type. For example, based on this WIT if your component imported now, you'd get:

(component
(import "wasi:clocks/monotonic-clock@0.3.0" (instance
(export "now" (func (result u64))) ;; (i'm ignoring the 'type instant = u64' here)
))
...
)
Because we're importing an instance (as opposed to a component or core module), you don't have to instantiate it -- it's already instantiated (by the host or by a parent component -- we don't know) and so this import goes straight into the instance index space (with which you can use an alias definition to project out the now function into the func index space, then canon lower it into the core func index space, then reference that from a with when instantiateing the main core module of the component).

It's an useful information. Thanks for the explaination!
It's more clear to understand this with the concept of index spaces when instantiating or validating (mapping to the context in WASM instantiation).
I realized that this proposal is still in early stage, but I think independent pages or sections for briefly validation and instantiation rules will help a lot.

I realized that this proposal is still in early stage, but I think independent pages or sections for briefly validation and instantiation rules will help a lot.

You're right. The goal is to fully formally define this and #101 contains (in Latex you can render to a pdf) a pretty good sketch of how validation works if you're able to parse the formal language it's using. We'll resume working on this, but switch to SpecTec now that Core WebAssembly spec has switched to SpecTec as well.

I realized that this proposal is still in early stage, but I think independent pages or sections for briefly validation and instantiation rules will help a lot.

You're right. The goal is to fully formally define this and #101 contains (in Latex you can render to a pdf) a pretty good sketch of how validation works if you're able to parse the formal language it's using. We'll resume working on this, but switch to SpecTec now that Core WebAssembly spec has switched to SpecTec as well.

Sorry for another basic question.
Would you help to suggest a tool to convert the text format of component model (for example, this component in the wast file) into binary format? I found that the common tools are mainly used to compose the components into binary files.
To implement the instantiation phase, it's more convenient to have more test data to confirm the correctness of instantiation steps. Aspecially the binary format order of sections may not be the same as the text format. Thanks!

No problem, happy to help. I'd suggest wasm-tools. Given a component.wat (containing a single top-level (component ...)), you can parse+validate+encode it via wasm-tools parse component.wat -o component.wasm.

No problem, happy to help. I'd suggest wasm-tools. Given a component.wat (containing a single top-level (component ...)), you can parse+validate+encode it via wasm-tools parse component.wat -o component.wasm.

Thanks and it help a lot with the test in the test folder.
I used the wasm-tools to translate the example component text format into binary format, but have a question about value section.
Take the exported function in the top component here:

(func (export "run") (alias export $d "run"))

The binary sequence is as following:

// alias section, size = 8, vector size = 1
0x06, 0x08, 0x01,
// alias sort = sort:func, aliastarget = (export, index = 1, name = "run")
0x01, 0x00, 0x01, 0x03, 0x72, 0x75, 0x6e,

// value section, size = 9, vector size = 1
0x0b, 0x09, 0x01,
// value[0]: "run"
0x00, 0x03, 0x72, 0x75, 0x6e, 0x01, 0x00, 0x00

According to the value binary format:

value ::= t:<valtype> len:<core:u32> v:<val(t)>   => (value t v) (where len = ||v||)

Let's focus on the value sequence: 0x00, 0x03, 0x72, 0x75, 0x6e, 0x01, 0x00, 0x00
Would you help to explain how to decode the sequence into the value definition?

According to the value explainer:

Value definitions (in the value index space) are like immutable global definitions in Core WebAssembly except that validation requires them to be consumed exactly once at instantiation-time.

Would you help to explain more detaily about the consuming the value definitions at instantiation time?
Thanks!

Let's focus on the value sequence: 0x00, 0x03, 0x72, 0x75, 0x6e, 0x01, 0x00, 0x00

Actually, I think section 0x0b (type 11) is an export section, rather than a value section---this corresponds to the (export "run") portion of the wat statement you showed. So 0x0b 0x09 says that this is a 9-byte export section. The contents of that section is a vec(<export>), so the 0x01 means that there is only one export. The export rule starts with an exportname', and the 0x00 tells us that it is the first case of an exportname (i.e. one without a version suffix). The rest of that rule tells us that the next (SLEB-encoded) u32 is the length of the name, in this case 3, and we then have the raw bytes of the export name (0x72 0x75 0x6e). With that out of the way, going back to the export rule, we should expect to see a sortidx next, and indeed 0x01 0x00 is the sortidx for the function at index 0 in the function index space (which happens to be the function we just alias export'd into our index space. Finally, we should look for an optional ascribed type in the form of an <externdesc>?, which the final 0x00 tells us is not in fact present.

Would you help to explain more detaily about the consuming the value definitions at instantiation time?

A value can be consumed by feeding it into another component as an instantiation argument, exporting it, or passing it to a function in a start definition (which may in turn produce some values of its own). Any given component must do this exactly once to any value that it gets (either by importing one or by running a start definition that produces one).

Let's focus on the value sequence: 0x00, 0x03, 0x72, 0x75, 0x6e, 0x01, 0x00, 0x00

Actually, I think section 0x0b (type 11) is an export section, rather than a value section---this corresponds to the (export "run") portion of the wat statement you showed. So 0x0b 0x09 says that this is a 9-byte export section. The contents of that section is a vec(<export>), so the 0x01 means that there is only one export. The export rule starts with an exportname', and the 0x00 tells us that it is the first case of an exportname (i.e. one without a version suffix). The rest of that rule tells us that the next (SLEB-encoded) u32 is the length of the name, in this case 3, and we then have the raw bytes of the export name (0x72 0x75 0x6e). With that out of the way, going back to the export rule, we should expect to see a sortidx next, and indeed 0x01 0x00 is the sortidx for the function at index 0 in the function index space (which happens to be the function we just alias export'd into our index space. Finally, we should look for an optional ascribed type in the form of an <externdesc>?, which the final 0x00 tells us is not in fact present.

Would you help to explain more detaily about the consuming the value definitions at instantiation time?

A value can be consumed by feeding it into another component as an instantiation argument, exporting it, or passing it to a function in a start definition (which may in turn produce some values of its own). Any given component must do this exactly once to any value that it gets (either by importing one or by running a start definition that produces one).

Sorry, my mistake. You are right, it's export section.
Thanks for the explaination!

@syntactically

Thanks @syntactically! And just to add: the emojis (like 🪙 for values) next to productions in Binary.md and Explainer.md (for the binary and text formats, resp.) mean that this feature is not yet implemented or officially released in wasm-tools/wasmtime. In the specific case of 🔀 (Preview 3 / async), the feature is implemented, but requires a feature flag to enable.

@lukewagner

Hi @lukewagner , questions about instance section in instantiation phase.

According to the definition:

core:instance       ::= (instance <id>? <core:instancexpr>)
core:instanceexpr   ::= (instantiate <core:moduleidx> <core:instantiatearg>*)
                      | <core:inlineexport>*
core:instantiatearg ::= (with <core:name> (instance <core:instanceidx>))
                      | (with <core:name> (instance <core:inlineexport>*))
core:inlineexport   ::= (export <core:name> <core:sortidx>)

instance       ::= (instance <id>? <instanceexpr>)
instanceexpr   ::= (instantiate <componentidx> <instantiatearg>*)
                 | <inlineexport>*
instantiatearg ::= (with <name> <sortidx>)
                 | (with <name> (instance <inlineexport>*))
inlineexport   ::= (export "<exportname>" <sortidx>)

As mentioned in doc, the instantiatearg has 2 forms: first is the name with sortidx, another is the name with inline exports.
In the test, we can find the inline export case of core:instance:

    (core instance $dm (instantiate $DM (with "" (instance
      (export "R1.resource.drop" (func $R1.resource.drop))
      (export "R2.resource.drop" (func $R2.resource.drop))
      (export "make-R1" (func $make-R1'))
      (export "make-R2" (func $make-R2'))
      (export "get-rep-R1" (func $get-rep-R1'))
      (export "get-rep-R2" (func $get-rep-R2'))
      (export "num-live" (func $num-live'))
    ))))

For the core:instance case, it's simple that construct the inline exports into a core module instance, and supply the core module named as "" to the imports of $DM when instantiation.
We know that when instantiate a core module, the spec only allows importing func, memory, table, and global (and tag in exception-handling proposal) instances.
But the core:sort additionally defined the type, module, and instance cases.
Can we assume that the core:inlineexport for constructing the core module instance for the importing use only contained the func, memory, table, and global cases?

On the other hand is the instance case. The test only covered the with "" (instance ) expression.
Since the sortidx and core:sortidx have several cases, are they mapping onto the index space of the context?
That is, for the component level import/export and instantiation, the with expression accepts the types not only the instance, right?

As my knowledge, for the core:instance case, the inline exports will be constructed into a core module instance, and supplied as imports with the module name in with expression.
But for the instance case, the inline exports will be constructed into a component instance, the with expression can also accepts other entries in the index spaces.

Please help to point out my misunderstanding or give some simple examples. Thanks!

Great questions:

Can we assume that the core:inlineexport for constructing the core module instance for the importing use only contained the func, memory, table, and global cases?

Yes

That is, for the component level import/export and instantiation, the with expression accepts the types not only the instance, right?

Yes. Here's an example of passing component-level funcs and here's an example of passing component-level instances.

@lukewagner

@lukewagner

Thanks for help with the link phase explaination.
I think this may be the last question of implementing the linking part.
After completing this, I will change to canonical ABI part and will create another issue if I meet troubles.
I will close this issue soon after the question being answered.

I found in the tests, the invocation of exported component functions are funcs, not the core funcs. (For example here)
The funcs are all declared by canon lift. With current definitions, there are no other canonical ABI defines the component func, right?
For the export section of the top layer of component, is it possible to export the core func directly and invoke that function?
That is, such as (assert_return (invoke "run") (i32.const 0)) in the wast file, which the "run" is an exported core func of a component?

Another is about the start section.
The start directly defined the funcidx. Does it mean that the function of the start section can only accepts the func, but not core func?

Thanks!

That's great news; good job! Happy to discuss other questions in other issues in the future.

With current definitions, there are no other canonical ABI defines the component func, right?

Depending on how you define "defines" (;-) the functions in the component-level func index space can come from canon lift, import, export or alias. If you repeatedly trace these func indices back to their "originating" definition, the two originating kinds of definitions are canon lift and host-defined functions.

For the export section of the top layer of component, is it possible to export the core func directly and invoke that function? That is, such as (assert_return (invoke "run") (i32.const 0)) in the wast file, which the "run" is an exported core func of a component?

No, core funcs cannot be exported because core func types are not included in componentype.

The start directly defined the funcidx. Does it mean that the function of the start section can only accepts the func, but not core func?

Correct

Depending on how you define "defines" (;-) the functions in the component-level func index space can come from canon lift, import, export or alias. If you repeatedly trace these func indices back to their "originating" definition, the two originating kinds of definitions are canon lift and host-defined functions.

Thanks, that's what I wanted to check.

No, core funcs cannot be exported because core func types are not included in componentype.

It seems much easier that the component-level function exporting will not be mixed with the core WASM level.
For runtime, we should consider the situation because users may instantiate not only components, but traditional WASMs.
So it means that according to the export binary definition (export ::= en:<exportname'> si:<sortidx> ed?:<externdesc>?), the sortidx is valid iff it's not core:sort, right?

So it means that according to the export binary definition (export ::= en:<exportname'> si: ed?:?), the sortidx is valid iff it's not core:sort, right?

Almost; the one core:sort allowed in imports/exports is a core:module (bullet 3 in Binary.md#import-and-export-definitions). Because core:modules are stateless/immutable, it doesn't break shared-nothing to import and export them from components and this feature is useful for factoring out common core wasm code that would otherwise be duplicated between different components (e.g).

Almost; the one core:sort allowed in imports/exports is a core:module (bullet 3 in Binary.md#import-and-export-definitions). Because core:modules are stateless/immutable, it doesn't break shared-nothing to import and export them from components and this feature is useful for factoring out common core wasm code that would otherwise be duplicated between different components (e.g).

Oops, for quickly implementing in runtime, maybe I didn't noticed the detail in documents.
Thanks for pointing it out!

Thanks for your help.
I'll create anothor issue for canonical ABI if I have questions.

Navigation Menu

Repository navigation

Linking in Instantiation stage #547

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Linking in Instantiation stage #547

Description

Activity

lukewagner commented on Jul 24, 2025

q82419 commented on Jul 24, 2025

lukewagner commented on Jul 24, 2025

q82419 commented on Jul 25, 2025

lukewagner commented on Jul 25, 2025

q82419 commented on Jul 28, 2025

lukewagner commented on Jul 28, 2025

q82419 commented on Jul 29, 2025

syntactically commented on Jul 29, 2025

q82419 commented on Jul 29, 2025

lukewagner commented on Jul 29, 2025

q82419 commented on Aug 8, 2025

lukewagner commented on Aug 8, 2025

q82419 commented on Aug 25, 2025

lukewagner commented on Aug 25, 2025

q82419 commented on Aug 26, 2025

lukewagner commented on Aug 26, 2025

q82419 commented on Aug 26, 2025

q82419 commented on Sep 1, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions