Related Pages

Related topics: Core Library, Opsets & OpenVINO IR Format, Model Frontends & Conversion Pipelines, Inference Runtime, Device Plugins & Deployment

Overview, Installation & Quick Start

What is OpenVINO

OpenVINO (Open Visual Inference and Neural network Optimization) is an open-source toolkit maintained by Intel for optimizing, converting, and deploying deep learning models across a wide range of hardware targets. The repository hosts the core C++ runtime, the Python bindings (openvino PyPI package), model converters, the OpenVINO Model Server, and a collection of device plugins (CPU, GPU, NPU, GNA, and ARM via the contrib project). Source: README.md:1-40.

The toolkit's purpose is to take a model trained in frameworks such as PyTorch, TensorFlow, ONNX, or PaddlePaddle and run it efficiently on the user's chosen accelerator with minimal code changes. The current 2026.2 series continues the tradition of bundled release archives and pip wheels published from storage.openvinotoolkit.org. Source: README.md:40-80.

Installation Paths

OpenVINO supports three primary installation channels, each documented under docs/articles_en/get-started/install-openvino/.

1. PyPI (pip) — recommended for Python users

The openvino package on PyPI wraps prebuilt C++ binaries plus the Python API. Installing it is a single command:

pip install openvino==2026.2.1

The wheel metadata declares a Python version range, NumPy constraints, and platform tags via the standard PEP 517 setup.cfg / pyproject.toml files. Source: src/bindings/python/setup.cfg:1-40. Required and optional Python dependencies are listed in constraints.txt, which pins transitive packages tested against the wheel (e.g., NumPy). Source: src/bindings/python/constraints.txt:1-20.

A common pain point surfaced by the community is overly strict NumPy bounds. Issue #11532 reports that pip install openvino rejects NumPy 1.20.x even though the runtime is compatible; maintainers usually relax the upper bound in subsequent releases. When troubleshooting, inspect constraints.txt and use pip install --no-deps openvino if you manage dependencies yourself. Source: docs/articles_en/get-started/install-openvino/install-openvino-pip.rst:1-60.

2. Conda / Anaconda

For users on conda-forge, OpenVINO is also distributed as conda install -c conda-forge openvino. The same wheel configuration files are reused, so dependency resolution follows the rules in setup.cfg. Source: setup.py:1-30.

3. Archive distribution

Standalone archives (.tar.gz for Linux/macOS, .zip for Windows) are published at storage.openvinotoolkit.org/repositories/openvino/packages/. These contain the runtime, samples, demos, and the Model Server, and are useful for non-Python deployments or for systems without internet access. Source: docs/articles_en/get-started/install-openvino.rst:1-40.

Platform notes

• Linux x86_64: full CPU/GPU/NPU support.
• Windows x86_64: full CPU/GPU/NPU support with the standard installer.
• Apple Silicon (M1/M2): official wheels do not yet ship native ARM inference. Issue #11554 documents community-built wheels that bundle the ARM plugin from openvino_contrib. Source: docs/articles_en/get-started/install-openvino/install-openvino-pip.rst:60-120.

Building from Source

Developers contributing to OpenVINO itself use CMake. The top-level CMakeLists.txt defines the project, enables C++17/20, and includes subdirectories for each component. A typical flow:

git clone https://github.com/openvinotoolkit/openvino.git
cd openvino
git submodule update --init --recursive
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release -DENABLE_PYTHON=ON
cmake --build build --parallel

Key toggles include ENABLE_PYTHON, ENABLE_INTEL_GPU, ENABLE_INTEL_NPU, and ENABLE_TESTS. Source: CMakeLists.txt:1-80. Detailed recipes for Linux, Windows, macOS, and Docker are maintained in docs/dev/build.md:1-120. Build prerequisites and supported compilers are documented there as well.

Quick Start: Running Your First Model

Once installed, the standard Python workflow has three steps:

1. Convert the source model to OpenVINO Intermediate Representation (IR) using openvino.convert_model() for ONNX/PyTorch/ TF directly, or ovc (legacy: mo) CLI for legacy conversion.
2. Compile the model for a device with core.compile_model(model, "CPU").
3. Infer by feeding a tensor into the resulting CompiledModel and reading the output dictionary.

import openvino as ov
import numpy as np

core = ov.Core()
model = core.read_model("model.xml")              # IR (.xml + .bin)
compiled = core.compile_model(model, "CPU")
result = compiled([np.zeros((1, 3, 224, 224), dtype=np.float32)])
print(result[compiled.output(0)])

The binding entry points (openvino.Core, openvino.Model, openvino.CompiledModel) are all generated by the Python bindings under src/bindings/python/. Source: src/bindings/python/README.md:1-50.

The table below summarizes the main device keys accepted by compile_model:

Source: docs/articles_en/get-started/install-openvino/install-openvino-pip.rst:120-200.

Where to Go Next

• Samples: docs/notebooks/ contains Jupyter tutorials for object detection, classification, segmentation, LLMs, and YOLO-style models (note 2026.2.1 fixed a YOLO26 GPU compile regression, see release notes).
• Model Server: tools/ovms/ provides an OpenAI/Anthropic-compatible HTTP/gRPC serving layer.
• Operation coverage: Issue #28584 tracks which PyTorch ops are still unsupported; consult it before filing new conversion errors.
• VPU blob versioning: When working with Myriad X, review issue #5156 regarding BLOB_VERSION_MAJOR/MINOR to ensure firmware compatibility.

After completing this quick start, the next logical page in the wiki is the Model Conversion & IR Format guide, which covers openvino.convert_model(), quantization (NNCF), and the ov::Model C++ API.

Related Pages

Related topics: Overview, Installation & Quick Start, Model Frontends & Conversion Pipelines, Inference Runtime, Device Plugins & Deployment

Core Library, Opsets & OpenVINO IR Format

Overview and Scope

The Core Library (src/core/) is the central, device-agnostic component of OpenVINO. It owns the in-memory graph representation used by every plugin (CPU, GPU, NPU, etc.), the operation set versioning system, and the serialization format that connects model conversion to inference. The README in this directory states that the module contains the OpenVINO™ Model representation, the standard set of operations, and the transformations applied to models before execution. Source: src/core/README.md:1-30

The Core Library sits between the frontends (which import models from ONNX, TensorFlow, PyTorch, PaddlePaddle) and the plugins (which compile and execute them). Frontends translate external formats into ov::Model graphs; the Core normalizes and rewrites those graphs via passes; plugins consume the result. This separation allows new operations and new hardware targets to be added without modifying the other layers.

Graph Representation: `ov::Model` and `ov::Node`

The user-facing graph type is ov::Model, declared in model.hpp. A Model is a std::shared_ptr-based container of ov::Node objects, organized into ov::Output<ov::Node> edges, and it carries parameters (graph inputs), results (graph outputs), and a sink vector for side-effect nodes. Source: src/core/include/openvino/core/model.hpp:1-80

Every operation, parameter, and result inherits from ov::Node. Node exposes APIs for:

• Inputs and outputs as ov::Output<T> handles, allowing type-safe traversal.
• Shape and element-type inference, with per-op overrides through validate_and_infer_types().
• Friendly names and a unique description used in error messages and IR dumps.
• A visit_attributes mechanism used by the serializer to walk the node's fields. Source: src/core/include/openvino/core/node.hpp:1-120

Constants (weights) are also Node subclasses (ov::op::v0::Constant) and live inside the same graph; they are referenced by Output edges, not stored separately.

Opsets and Versioned Operations

Opsets are OpenVINO's mechanism for evolving the operation vocabulary without breaking older models. Each opset is a named registry that aliases concrete ov::op::vN::OpName classes to short names such as ov::opset13::Add. The base registry is ov::OpSet, defined in opset.hpp, and it exposes create(op_name) plus a map of available operations. Source: src/core/include/openvino/opsets/opset.hpp:1-60

Concrete opsets extend the registry through OPENVINO_OP / BWDCMP_RTTI macros and via a single OV_OPENSET instantiation. The oldest available opset (ov::opset1) re-exports operations from namespaces ov::op::v0 and ov::op::v1; for example, opset1::Add is an alias for ov::op::v1::Add. Source: src/core/include/openvino/opsets/opset1.hpp:1-40

Newer opsets follow the same pattern but add operations introduced in later versions. opset17.hpp re-exports the modern ov::op::v17 namespace alongside the older ones, giving users one import point for the current vocabulary. Source: src/core/include/openvino/opsets/opset17.hpp:1-30

The diagram below summarizes how a frontend selects an opset when building a graph:

flowchart LR
    A[External model<br/>ONNX / TF / PyTorch] --> B[Frontend importer]
    B --> C{Choose opset}
    C -->|legacy graphs| D[ov::opset1..N-1]
    C -->|current| E[ov::opsetN]
    D --> F[ov::Model graph]
    E --> F[ov::Model graph]
    F --> G[ov::pass::Manager<br/>transformations]
    G --> H[IR v11 serializer]
    G --> I[Plugin compiler]

Opset choice is not just an API convenience: each op version carries its own shape-inference rules, attributes, and validation, so the right alias must be used to round-trip a model losslessly. Community issue #28584 (the PyTorch "Operation list to be supported" thread) is essentially a running discussion of which PyTorch ops map cleanly into which OpenVINO op-version namespace. Source: #28584

Pass Manager and the IR Serialization Pipeline

Before serialization or compilation, the Core runs the model through ov::pass::Manager, declared in pass/manager.hpp. A Manager owns an ordered list of ov::pass::Pass objects and walks the graph with a pattern matcher, applying constant folding, fusing, layout normalization, and device-agnostic cleanups. Source: src/core/include/openvino/pass/manager.hpp:1-70

The serialized output is the OpenVINO Intermediate Representation (IR v11): an XML file describing topology, tensor names, shapes, and element types, plus a binary blob containing weights. The XML is produced by walking each Node's visit_attributes, while the blob is built by deduplicating the Constant tensors encountered during the walk. This round-trip is what allows ov::serialize(model, "model.xml") and core.read_model("model.xml", "model.bin") to be lossless for the supported opset set.

The ov::Extension mechanism lets third parties register custom operations, passes, or frontends without modifying the Core itself. Extensions are added to ov::Core at runtime and participate in the same OpSet lookup used by deserialization, which is why custom ops can be loaded transparently from an IR file. Source: src/core/include/openvino/core/extension.hpp:1-50

Practical Notes for Contributors

• When adding a new operation, create it in op::vN and register it in the corresponding opsets/opsetN.hpp; older opsets must remain unchanged for backward compatibility.
• New transformations belong in src/core/src/pass/ and should be added to an existing pass manager rather than invoked ad hoc, so plugins see a consistent graph.
• Changes to a node's visit_attributes schema require bumping the IR version constants handled by the frontend; this is the same constraint that caused issue #5156, where the VPU team requested explicit BLOB_VERSION_MINOR bumps per release so cached blobs could be invalidated. Source: #5156
• Tight framework pin ranges (such as the numpy<1.20 constraint raised in #11532) often originate in the Core's Python bindings or in frontend dependencies, not in the graph code itself. Source: #11532

Together, the Core Library, the opset registry, and the IR serializer form a stable contract that the rest of OpenVINO — and every supported model format — is built on.

Related Pages

Related topics: Overview, Installation & Quick Start, Core Library, Opsets & OpenVINO IR Format, Inference Runtime, Device Plugins & Deployment

Model Frontends & Conversion Pipelines

OpenVINO model frontends are framework-specific translation layers that convert neural networks from their native training-framework representations — PyTorch, TensorFlow, ONNX, and others — into OpenVINO's Intermediate Representation (IR). The Python entry points openvino.convert_model and openvino.compile_model dispatch to the appropriate frontend based on the input type and model format. Source: src/bindings/python/src/openvino/frontend/pytorch/__init__.py:1-80

Frontend Architecture

Each framework frontend follows a common shape composed of four cooperating pieces:

• A decoder that walks the source computation graph and exposes a uniform node-and-edge model to the rest of the pipeline.
• An input model adapter that feeds framework-specific objects (modules, graphs, functions) into the decoder.
• An operation translator that maps framework ops to OpenVINO opset nodes using a shared op-mapping table.
• An extension hook that lets users register custom operations when a framework op has no direct OpenVINO equivalent.

This separation lets new frameworks reuse the downstream IR generator and serialization stages without duplicating them. Source: src/bindings/python/src/openvino/frontend/pytorch/__init__.py:20-100

PyTorch Frontend

The PyTorch frontend is the most actively extended converter because PyTorch exposes models through several different object types. It currently supports torch.nn.Module, torch.jit.ScriptModule, torch.jit.ScriptFunction, torch.fx.GraphModule, and torch.export.ExportedProgram as inputs. The package __init__ inspects the input and routes it to a specialized decoder. Source: src/bindings/python/src/openvino/frontend/pytorch/__init__.py:40-160

FX Graph Decoder

fx_decoder.py walks a torch.fx.Graph and emits OpenVINO nodes. It examines placeholder, call_function, call_method, and call_module nodes and translates them through the shared op table. The decoder also handles tensor metadata, parameter binding, and the conversion of FX node attributes into OpenVINO constant inputs. Source: src/bindings/python/src/openvino/frontend/pytorch/fx_decoder.py:1-90

TorchDynamo Backend

torchdynamo/backend.py plugs into PyTorch 2.x's compiler stack. When a user calls torch.compile(model, backend="openvino"), the backend intercepts the FX graphs that Dynamo produces and converts them on the fly, allowing graph capture without an explicit convert_model call. This entry point is useful for fallback compilation paths inside larger PyTorch training or inference scripts. Source: src/bindings/python/src/openvino/frontend/pytorch/torchdynamo/backend.py:1-100

Custom Operation Mapping

Two modules cover the extension surface. ov_custom_ops.py registers user-supplied Python functions against specific framework op names, while inlined_extension.py embeds a callable directly into the converted graph as a small subgraph. Custom ops are passed through the extension argument of convert_model and are evaluated during translation rather than at execution time. Source: src/bindings/python/src/openvino/frontend/pytorch/ov_custom_ops.py:1-60 Source: src/bindings/python/src/openvino/frontend/pytorch/inlined_extension.py:1-70

flowchart LR
    A[PyTorch Model] --> B{Entry Point}
    B -->|nn.Module| C[Trace Decoder]
    B -->|FX GraphModule| D[FX Decoder]
    B -->|torch.compile| E[TorchDynamo Backend]
    C --> F[Op Translator]
    D --> F
    E --> F
    F --> G{Extension Registered?}
    G -->|yes| H[Custom / Inlined Op]
    G -->|no| I[OpenVINO Model / IR]
    H --> I

TensorFlow Frontend

The TensorFlow frontend covers tf.keras models, frozen SavedModel directories, and concrete-function graphs. Because TensorFlow models are already static graphs, the decoder does not need to trace execution; it instead reads the graph definition directly and feeds it into the same downstream op translator. Source: src/bindings/python/src/openvino/frontend/tensorflow/__init__.py:1-80

Notable behaviors that distinguish the TensorFlow frontend from the PyTorch one:

• Static graph traversal — there is no symbolic-tracing phase; the graph is consumed as-is.
• Resource variables — a dedicated variable-to-constant pass runs before IR generation so that tf.Variable readers see materialised tensors.
• Control-flow ops — tf.cond, tf.while_loop, and tf.switch are lowered into OpenVINO's loop and conditional ops, with their branch bodies recursively decoded.

Source: src/bindings/python/src/openvino/frontend/tensorflow/__init__.py:40-140

Conversion Pipeline and Extensions

From the user's perspective the pipeline runs in four logical phases regardless of the source framework:

1. Loading — read the input object, detect its framework, and instantiate the matching frontend.
2. Decoding — walk the source graph and emit framework-neutral nodes.
3. Translation — apply the op map, run custom-op extensions, perform shape inference, and produce an ov.Model.
4. Serialization — optionally write .xml and .bin IR files for later use with compile_model.

Users can intervene between phases using PartialShape, input/output overrides, and the extension parameter. The inlined-extension mechanism is especially useful for ops without a direct OpenVINO equivalent that can be expressed as a small composed subgraph. Source: src/bindings/python/src/openvino/frontend/pytorch/inlined_extension.py:20-90

Community context that touches this area:

• Issue #28584 tracks the live list of PyTorch operations awaiting frontend support and reflects how the decoder/translator pair is extended as new ops appear upstream.
• Issue #11532 highlights strict numpy version pinning in the Python frontend wheels, which can affect install paths even when conversion itself works on a wider range of NumPy versions.

Summary

OpenVINO's frontend layer is a modular, per-framework translation pipeline. The PyTorch frontend provides the richest set of entry points (eager modules, FX, TorchDynamo, exported programs) and the most active extension surface, while the TensorFlow frontend consumes static graphs directly. Both share the same downstream IR generator, the same op-mapping infrastructure, and the same custom-op extension mechanism, which keeps the user-facing convert_model API consistent across frameworks.

Related Pages

Related topics: Overview, Installation & Quick Start, Core Library, Opsets & OpenVINO IR Format, Model Frontends & Conversion Pipelines

Inference Runtime, Device Plugins & Deployment

OpenVINO's Inference Runtime is the execution layer that loads optimized neural network models, schedules them onto hardware targets through pluggable device backends, and exposes a unified API across language bindings. The runtime abstracts hardware differences so that the same model can run on CPU, GPU, NPU, VPU, or GNA without code changes. Community discussions (e.g., issue #11554 on Apple silicon support and #28584 on operation coverage) show how device-specific plugins drive most user-facing functionality and bug reports.

Core Architecture and the `ov::Core` Object

At the center of every language binding is the Core class — the entry point for reading models, querying devices, and compiling networks. In the C API, ov_core_create initializes a fresh runtime instance and returns an opaque handle ov_core_t* that downstream functions consume.

Source: src/bindings/c/src/ov_core.cpp:39-65

The Node.js binding wraps the same C primitives through N-API. CoreWrap::New constructs a JavaScript-visible object backed by an std::shared_ptr<ov::Core>, ensuring the runtime survives garbage collection.

Source: src/bindings/js/node/include/core_wrap.hpp:43-79

Python mirrors this design: _ov_api.py exposes Core as the user-facing handle, while the heavy lifting is delegated to the compiled extension _pyopenvino.

Source: src/bindings/python/src/openvino/_ov_api.py:1-50

Reading, Compiling, and Loading Models

The deployment flow follows a consistent three-step pattern across all bindings:

1. Read — parse an .xml/.bin IR pair (or ONNX, Paddle, TensorFlow via frontend plugins) into an ov::Model.
2. Compile — bind the model to a device name ("CPU", "GPU", "NPU", "AUTO", "HETERO", etc.), producing an executable CompiledModel.
3. Infer — create request objects that accept tensors and return results.

The Python API expresses this with the canonical helper: model = core.read_model(path), then compiled = core.compile_model(model, "GPU"). Internally, compile_model calls core.compile_model(model=model, device_name=device_name, config=config) with an optional performance hints dictionary.

Source: src/bindings/python/src/openvino/_ov_api.py:33-42

The C API exposes equivalent functions: ov_core_read_model, ov_core_compile_model, and ov_compiled_model_create_infer_request. Returned objects are opaque handles whose lifecycle is managed by explicit create/free pairs.

Source: src/bindings/c/include/openvino/c/openvino.h:1-120

The Node.js binding translates Core.read_model and Core.compile_model into JavaScript-friendly methods on CoreWrap, surfacing the same compile-time configuration through a plain object.

Source: src/bindings/js/node/src/core_wrap.cpp:39-95

Device Plugins and Multi-Device Execution

Device plugins register themselves at runtime and advertise capabilities such as supported precision, throughput streams, and supported operations. The runtime uses this metadata to route subgraphs to the best-suited device when configurations like "HETERO" or "MULTI" are requested. Users discover available hardware via core.available_devices, which returns the list of registered plugin identifiers.

Source: src/bindings/python/src/openvino/_pyopenvino/__init__.pyi:1-80

The C API mirrors this through ov_core_get_available_devices, returning a string vector that callers can iterate to decide where to compile.

Source: src/bindings/c/src/ov_core.cpp:67-95

Community issue #28584 (Operation list to be supported) is tracked at the plugin level — each device plugin maintains its own allowlist of operations, and new ops become available as plugins are updated. Similarly, the YOLO26 fix shipped in release 2026.2.1 (Fixed issue ID 187077: YOLO26 fails to compile on GPU) shows how GPU plugin coverage evolves between releases.

Asynchronous Inference and Deployment Patterns

For production deployments, OpenVINO provides both synchronous (infer) and asynchronous (start_async, set_callback) request APIs. The Python binding exposes these through InferRequest.infer() and InferRequest.start_async(), with callback registration for completion notification.

Source: src/bindings/python/src/openvino/_ov_api.py:42-50

The compiled model supports get_runtime_model() to inspect the device-specific graph actually executed, and export_model() to serialize a compiled blob for cross-device deployment — a workflow relevant to issue #5156, which requests better VPU blob versioning so users can identify which OpenVINO release produced a given binary.

Deployment Tooling Summary

Source: src/bindings/python/src/openvino/_ov_api.py:1-50, src/bindings/c/include/openvino/c/openvino.h:1-120, src/bindings/js/node/src/core_wrap.cpp:39-95

Across all bindings, the runtime remains the single source of truth: every binding simply adapts the C++ ov::Core API to its host language. This consistency is what enables a model trained and compiled on one platform to be deployed unchanged on another, provided the relevant device plugin is installed — including community-built targets like the ARM plugin referenced in issue #11554.

Doramagic Pitfall Log

Found 6 structured pitfall item(s), including 0 high/blocking item(s). Top priority: Capability evidence risk - Capability evidence risk requires verification.

1. Capability evidence risk: Capability evidence risk requires verification

• Severity: medium
• Finding: README/documentation is current enough for a first validation pass.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: capability.assumptions | https://github.com/openvinotoolkit/openvino

2. Maintenance risk: Maintenance risk requires verification

• Severity: medium
• Finding: Project evidence flags a maintenance risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: evidence.maintainer_signals | https://github.com/openvinotoolkit/openvino

3. Security or permission risk: Security or permission risk requires verification

• Severity: medium
• Finding: no_demo
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: downstream_validation.risk_items | https://github.com/openvinotoolkit/openvino

4. Security or permission risk: Security or permission risk requires verification

• Severity: medium
• Finding: no_demo
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: risks.scoring_risks | https://github.com/openvinotoolkit/openvino

5. Maintenance risk: Maintenance risk requires verification

• Severity: low
• Finding: issue_or_pr_quality=unknown。
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: evidence.maintainer_signals | https://github.com/openvinotoolkit/openvino

6. Maintenance risk: Maintenance risk requires verification

• Severity: low
• Finding: release_recency=unknown。
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: evidence.maintainer_signals | https://github.com/openvinotoolkit/openvino

Community Discussion Evidence

Doramagic exposes project-level community discussion separately from official documentation. Review these links before using openvino with real data or production workflows.

• 2026.2.1 - github / github_release
• 2026.2.0 - github / github_release
• 2026.1.0 - github / github_release
• 2026.0.0 - github / github_release
• 2026.0.1 - github / github_release
• 2025.4.1 - github / github_release
• 2025.4.0 - github / github_release
• 2025.3.0 - github / github_release
• 2025.2.0 - github / github_release
• 2025.1.0 - github / github_release
• Capability evidence risk requires verification - GitHub / issue