Skip to content

Refactor device selection: Rename to computePolicy, remove accelerated, and add fallback #923

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
mingmingtasd wants to merge 2 commits into
base: main
Choose a base branch
from
+18 −24
Open

Refactor device selection: Rename to computePolicy, remove accelerated, and add fallback #923

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
34fa320
define compute policy
mingmingtasd Mar 18, 2026
79249ab
remove CPU in description for fallback
mingmingtasd Apr 24, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 18 additions & 24 deletions index.bs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -980,15 +980,15 @@ WorkerNavigator includes NavigatorML;

## {{ML}} interface ## {#api-ml}
<script type=idl>
enum MLPowerPreference {
enum MLComputePolicy {
"default",
"high-performance",
"low-power"
"low-power",
"fallback"
};

dictionary MLContextOptions {
MLPowerPreference powerPreference = "default";
boolean accelerated = true;
MLComputePolicy computePolicy = "default";
};

[SecureContext, Exposed=(Window, Worker)]
Expand All @@ -1002,18 +1002,18 @@ interface ML {

Note: {{MLContextOptions}} is under active development, and the design is expected to change, informed by further implementation experience and new use cases from the wider web community. The Working Group is considering additional API controls to allow the definition of a fallback device, multiple devices in a preferred order, or an exclusion of a specific device. Other considerations under discussion include error handling, ultimate fallback, and quantized operators. Feedback is welcome on any of these design considerations from web developers, library authors, OS and hardware vendors, and other stakeholders via <a href="https://github.com/webmachinelearning/webnn/labels/device%20selection">GitHub</a>. See [[#privacy]] for additional discussion of fingerprinting considerations.

The <dfn dfn-for=MLContextOptions dfn-type=dict-member>powerPreference</dfn> option is an <dfn dfn-type=enum>MLPowerPreference</dfn> and indicates the application's preference as related to power consumption. It is one of the following:
<dl dfn-for="MLPowerPreference">
The <dfn dfn-for=MLContextOptions dfn-type=dict-member>computePolicy</dfn> option is an <dfn dfn-type=enum>MLComputePolicy</dfn> and indicates the application's policy for the compute device. The policy is designed to be extensible. While the current policy values mainly cover power consumption and performance, future extensions may introduce additional execution polices. It is one of the following:
<dl dfn-for="MLComputePolicy">
<dt>"<dfn enum-value>default</dfn>"</dt>
<dd>Let the user agent select the most suitable behavior.</dd>
<dt>"<dfn enum-value>high-performance</dfn>"</dt>
<dd>Prioritizes execution speed over power consumption.</dd>
<dd>Prioritizes execution speed over other considerations such as power consumption.</dd>
<dt>"<dfn enum-value>low-power</dfn>"</dt>
<dd>Prioritizes power consumption over other considerations such as execution speed.</dd>
<dt>"<dfn enum-value>fallback</dfn>"</dt>
<dd>Prioritizes wider compatibility, more predictable behavior, or improved privacy over other considerations such as execution speed.</dd>
</dl>

The <dfn dfn-for=MLContextOptions dfn-type=dict-member>accelerated</dfn> option indicates the application's preference as related to massively parallel acceleration. This option has less priority than {{MLContextOptions/powerPreference}}. When set to `true` (by default), the underlying platform will attempt to use the available massively parallel accelerators, such as a GPU or NPU, also depending on the {{MLContextOptions/powerPreference}}. When set to `false`, the application indicates it prefers CPU inference. If there is contradictory input, for instance when {{MLContextOptions/powerPreference}} is {{MLPowerPreference/"high-performance"}} and {{MLContextOptions/accelerated}} is `false`, then the implementation will choose the best available match in the underlying platform (for instance a high performance CPU mode, or will ignore {{MLContextOptions/accelerated}} as it has less priority than {{MLContextOptions/powerPreference}}).
Comment thread
fdwr marked this conversation as resolved.

### {{ML/createContext()}} ### {#api-ml-createcontext}

<div dfn-for="ML/createContext(options), ML/createContext(gpuDevice)" dfn-type=argument>
Expand All @@ -1030,15 +1030,12 @@ The <dfn dfn-for=MLContextOptions dfn-type=dict-member>accelerated</dfn> option
1. Let |context| be a new {{MLContext}} in |realm|.
1. If |options| is a {{GPUDevice}} object, then:
1. Set |context|.{{MLContext/[[contextType]]}} to "[=context type/webgpu=]".
1. Set |context|.{{MLContext/[[powerPreference]]}} to {{MLPowerPreference/"default"}}.
1. Set |context|.{{MLContext/[[accelerated]]}} to `true`.
1. Set |context|.{{MLContext/[[computePolicy]]}} to {{MLComputePolicy/"default"}}.
1. Otherwise:
1. Set |context|.{{MLContext/[[contextType]]}} to "[=context type/default=]".
1. Set |context|.{{MLContext/[[lost]]}} to [=a new promise=] in |realm|.
1. If |options|["{{MLContextOptions/powerPreference}}"] [=map/exists=], then set |context|.{{MLContext/[[powerPreference]]}} to |options|["{{MLContextOptions/powerPreference}}"].
1. Otherwise, set |context|.{{MLContext/[[powerPreference]]}} to {{MLPowerPreference/"default"}}.
1. If |options|["{{MLContextOptions/accelerated}}"] [=map/exists=], then set |context|.{{MLContext/[[accelerated]]}} to |options|["{{MLContextOptions/accelerated}}"].
1. Otherwise, set |context|.{{MLContext/[[accelerated]]}} to `true`.
1. If |options|["{{MLContextOptions/computePolicy}}"] [=map/exists=], then set |context|.{{MLContext/[[computePolicy]]}} to |options|["{{MLContextOptions/computePolicy}}"].
1. Otherwise, set |context|.{{MLContext/[[computePolicy]]}} to {{MLComputePolicy/"default"}}.
1. If the user agent cannot support |context|.{{MLContext/[[contextType]]}}, then return failure.
1. Return |context|.
</details>
Expand Down Expand Up @@ -1072,7 +1069,7 @@ The <dfn dfn-for=MLContextOptions dfn-type=dict-member>accelerated</dfn> option
</details>

## {{MLContext}} interface ## {#api-mlcontext}
The {{MLContext}} interface represents a global state of neural network compute workload and execution processes. Each {{MLContext}} object has associated [=context type=] and {{MLPowerPreference}}.
The {{MLContext}} interface represents a global state of neural network compute workload and execution processes. Each {{MLContext}} object has associated [=context type=] and {{MLComputePolicy}}.

<script type=idl>
typedef record<USVString, MLTensor> MLNamedTensors;
Expand All @@ -1098,7 +1095,7 @@ interface MLContext {

undefined destroy();

readonly attribute boolean accelerated;
readonly attribute MLComputePolicy computePolicy;
Copy link
Copy Markdown
Contributor

@huningxin huningxin Apr 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure whether compute policy should remain internal slot as previous power preference. It is not like accelerated attribute that can indicate whether the native AI runtime does fallback or not. @handellm , is there any use case of exposing it?

Copy link
Copy Markdown
Collaborator

@zolkis zolkis Apr 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The intent for exposing it was that the implementation can change it (as it was a context creation hint). The create algorithm should contain clauses for how to do this. Also, after context creation this might be changed by the impl, right? One use case could be to check if the context still has the same policy it was created with.

But one could also argue that the impl got the hint, thank you, it will translate it into a set of internal states, and will take it into account in the best possible way henceforth, so no need for the app meddles with it later. In that case, it could be only an internal slot. The question is what problems might be if exposing it (other than impl complexity to translating back an internal state to the MLComputePolicy).

readonly attribute Promise<MLContextLostInfo> lost;
};
</script>
Expand All @@ -1109,12 +1106,9 @@ interface MLContext {
: <dfn>\[[contextType]]</dfn> of type [=context type=].
::
The {{MLContext}}'s [=context type=].
: <dfn>\[[powerPreference]]</dfn> of type {{MLPowerPreference}}.
::
The {{MLContext}}'s {{MLPowerPreference}}.
: <dfn>\[[accelerated]]</dfn> of type {{boolean}}.
: <dfn>\[[computePolicy]]</dfn> of type {{MLComputePolicy}}.
::
The {{MLContext}}'s processing type (CPU or massively parallel processing).
The {{MLContext}}'s {{MLComputePolicy}}.
: <dfn>\[[lost]]</dfn> of type {{Promise}}<{{MLContextLostInfo}}>.
::
A {{Promise}} that is resolved when the {{MLContext}}'s underlying execution device is no longer available.
Expand All @@ -1135,7 +1129,7 @@ The <dfn>context type</dfn> is the type of the execution context that manages th
</dl>

<div algorithm>
The <dfn attribute for=MLContext>accelerated</dfn> getter steps are to return [=this=].{{MLContext/[[accelerated]]}}.
The <dfn attribute for=MLContext>computePolicy</dfn> getter steps are to return [=this=].{{MLContext/[[computePolicy]]}}.
</div>

<details open algorithm>
Expand Down Expand Up @@ -2199,7 +2193,7 @@ Build a composed graph up to a given output operand into a computational graph a
1. Let |promise| be [=a new promise=] in |realm|.
1. Enqueue the following steps to |graph|.{{MLGraph/[[context]]}}.{{MLContext/[[timeline]]}}:
1. Run these steps, but [=/abort when=] |graph|.{{MLGraph/[[context]]}} [=MLContext/is lost=]:
1. Let |graphImpl| be the result of converting [=this=]'s [=MLGraphBuilder/graph=] with |operands|, |operators|, |inputs|, and |outputs|'s [=map/values=], as well as |graph|.{{MLGraph/[[context]]}}.{{MLContext/[[powerPreference]]}} and |graph|.{{MLGraph/[[context]]}}.{{MLContext/[[accelerated]]}} into an [=implementation-defined=] format which can be interpreted by the underlying platform.
1. Let |graphImpl| be the result of converting [=this=]'s [=MLGraphBuilder/graph=] with |operands|, |operators|, |inputs|, and |outputs|'s [=map/values=], as well as |graph|.{{MLGraph/[[context]]}}.{{MLContext/[[computePolicy]]}} into an [=implementation-defined=] format which can be interpreted by the underlying platform.
1. If the previous step failed, then [=queue an ML task=] with |global| to [=reject=] |promise| with an "{{OperationError}}" {{DOMException}}, and abort these steps.
1. Set |graph|.{{MLGraph/[[implementation]]}} to |graphImpl|.
1. [=Queue an ML task=] with |global| to [=resolve=] |promise| with |graph|.
Expand Down
Loading