Build a baseline you can trust.

Choose a path for your workload

Start with the software-stack guide to choose a host package route and matching application environment. For an existing failure, the troubleshooting log separates historical workarounds from verified upstream outcomes. Image and video users can follow the creative-workflow guide.

For a GGUF language model, the measured configurations in this guide use llama.cpp with its SYCL backend. Start with that route when your goal is to inspect a model, run a single conversation, and understand memory consumption. Our benchmark explorer gives you concrete historical configurations to investigate, not a promise that any current binary will reproduce them.

For a Python application, first inspect its dependencies. A project that expresses its work through PyTorch operations is a different porting problem from one that ships custom NVIDIA kernels. Work through the CUDA-to-Intel decision guide before changing package versions.

Serving many independent requests deserves a separate evaluation. Measure your expected concurrency, response length, and latency target. A fast single-stream microbenchmark does not establish serving performance.

Establish the software boundary

Use the upstream SYCL instructions for the environment and build steps that match your operating system. The document covers Intel GPU drivers, the oneAPI environment, and SYCL builds. The Intel compute-runtime project is the primary source for the compute driver.

Keep a small environment record before you test:

• Operating system, kernel, Intel compute-runtime, and compiler/runtime versions.
• Application commit and whether the checkout contains local changes.
• Model repository and revision, complete weight filename, and quantization.
• Visible devices, intended device selection, and available memory.
• Context, KV types, flash-attention setting, batch size, and concurrency.

The April cohort names oneAPI 2025.3.3; the June cohort names 2026.0. They belong to different environments. Combining their commands into a supposed “known-good” installation would erase the conditions that made those observations useful.

Confirm the intended device

Once you have followed the upstream environment setup, these documentation-checked discovery commands help distinguish runtime visibility from application visibility. Run them in your own prepared llama.cpp checkout; this website does not run them for you.

sycl-ls
./build/bin/llama-cli --list-devices

Expected result: the intended Intel GPU appears in runtime discovery and in the application’s device list. Exact names and device indices vary; the examples in upstream documentation are not a guarantee that your B70 will be named identically.

If runtime discovery finds the card but the application does not, examine which backend was built and which runtime the executable loads. If neither finds the card, investigate the driver, device permissions, and environment setup first. Increasing model offload cannot repair an invisible device.

Record the mapping before testing two cards. A process that only sees one physical card can enumerate it as device zero even when it was the second card in the parent process.

Validate a small, repeatable workload

Choose a model and quantization that leave comfortable room for cache and compute buffers. Use the model’s own prompt format. Follow the inference examples for your selected application revision, and keep the exact launch command with the output.

For an initial correctness check, use a short prompt with an easy-to-inspect answer. Read the output, record the device-offload log, then submit a second prompt to the same long-running instance. Restarting between requests can hide state-related failures.

A successful process exit or HTTP response is insufficient. Look for plausible text, expected stopping behavior, and stability across several different prompts. If a change improves throughput but produces repeated symbols or nonsense, that configuration fails the application test.

The Q8_0 engineering story explains why a first response can pass while a subsequent one fails. That historical regression is a useful test-design lesson; it is not evidence that every current build still has the bug.

Separate correctness from timing

After a working baseline, change one variable at a time. Measure prompt processing and decode separately, retain repeated samples, and disclose warmup. A first request may include compilation, loading, or cache construction that later requests avoid.

A 4,096-token configured context limit is not proof that a 4,096-token prompt succeeded. Likewise, June’s 68.88 decode tok/s measurement used zero prompt tokens and zero depth. It does not describe generation after a full 32K prompt.

Useful acceptance criteria are concrete: the intended GPU was used; several prompts produced sensible output; the second request worked; the longest required input completed; memory remained within the intended device; and repeated timings were saved with the same configuration.

Keep a failure record

Attach the smallest useful reproduction, exact versions, expected output, actual output, and whether the checkout is dirty. Redact local paths or prompts that should not be public. Submit a focused report to the relevant upstream project; use the benchmark repository for corrections to this site’s historical data.