{"id":24579236,"url":"https://github.com/intel/ipex-llm","last_synced_at":"2025-11-13T11:01:14.302Z","repository":{"id":37285213,"uuid":"66823715","full_name":"intel/ipex-llm","owner":"intel","description":"Accelerate local LLM inference and finetuning (LLaMA, Mistral, ChatGLM, Qwen, DeepSeek, Mixtral, Gemma, Phi, MiniCPM, Qwen-VL, MiniCPM-V, etc.) on Intel XPU (e.g., local PC with iGPU and NPU, discrete GPU such as Arc, Flex and Max); seamlessly integrate with llama.cpp, Ollama, HuggingFace, LangChain, LlamaIndex, vLLM, DeepSpeed, Axolotl, etc.","archived":false,"fork":false,"pushed_at":"2025-10-14T06:04:12.000Z","size":237788,"stargazers_count":8424,"open_issues_count":1495,"forks_count":1384,"subscribers_count":260,"default_branch":"main","last_synced_at":"2025-11-01T16:23:01.068Z","etag":null,"topics":["gpu","llm","pytorch","transformers"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/intel.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2016-08-29T07:59:50.000Z","updated_at":"2025-11-01T15:02:52.000Z","dependencies_parsed_at":"2022-07-11T03:18:32.252Z","dependency_job_id":"8f9fdabb-9e59-4c69-9de4-1271bb61ab45","html_url":"https://github.com/intel/ipex-llm","commit_stats":{"total_commits":3417,"total_committers":130,"mean_commits":"26.284615384615385","dds":0.9195200468247,"last_synced_commit":"da9270be2d5fbfb93b67fef51bc19917d88a3424"},"previous_names":["intel-analytics/ipex-llm","intel-analytics/bigdl","intel/ipex-llm"],"tags_count":24,"template":false,"template_full_name":null,"purl":"pkg:github/intel/ipex-llm","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/intel%2Fipex-llm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/intel%2Fipex-llm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/intel%2Fipex-llm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/intel%2Fipex-llm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/intel","download_url":"https://codeload.github.com/intel/ipex-llm/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/intel%2Fipex-llm/sbom","scorecard":{"id":490301,"data":{"date":"2025-01-30T02:40:34Z","repo":{"name":"github.com/intel/ipex-llm","commit":"ee809e71dfa0e1656fa12c1251f4a8db4de4ea06"},"scorecard":{"version":"v4.13.1","commit":"49c0eed3a423f00c872b5c3c9f1bbca9e8aae799"},"score":6.7,"checks":[{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/49c0eed3a423f00c872b5c3c9f1bbca9e8aae799/docs/checks.md#binary-artifacts"}},{"name":"Branch-Protection","score":-1,"reason":"internal error: error during GetBranch(branch-2.4): error during branchesHandler.query: internal error: githubv4.Query: Resource not accessible by integration","details":null,"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/49c0eed3a423f00c872b5c3c9f1bbca9e8aae799/docs/checks.md#branch-protection"}},{"name":"CI-Tests","score":6,"reason":"20 out of 30 merged PRs checked by a CI test -- score normalized to 6","details":null,"documentation":{"short":"Determines if the project runs tests before pull requests are merged.","url":"https://github.com/ossf/scorecard/blob/49c0eed3a423f00c872b5c3c9f1bbca9e8aae799/docs/checks.md#ci-tests"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/49c0eed3a423f00c872b5c3c9f1bbca9e8aae799/docs/checks.md#cii-best-practices"}},{"name":"Code-Review","score":9,"reason":"found 2 unreviewed changesets out of 30 -- score normalized to 9","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/49c0eed3a423f00c872b5c3c9f1bbca9e8aae799/docs/checks.md#code-review"}},{"name":"Contributors","score":10,"reason":"6 different organizations found -- score normalized to 10","details":["Info: contributors work for intel,intel @intel-analytics,nju -\u003e hkust -\u003e intel,seulinux,tencent ai lab,university of illinois urbana-champaign"],"documentation":{"short":"Determines if the project has a set of contributors from multiple organizations (e.g., companies).","url":"https://github.com/ossf/scorecard/blob/49c0eed3a423f00c872b5c3c9f1bbca9e8aae799/docs/checks.md#contributors"}},{"name":"Dangerous-Workflow","score":10,"reason":"no dangerous workflow patterns detected","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/49c0eed3a423f00c872b5c3c9f1bbca9e8aae799/docs/checks.md#dangerous-workflow"}},{"name":"Dependency-Update-Tool","score":0,"reason":"no update tool detected","details":["Warn: tool 'RenovateBot' is not used: Follow the instructions from https://docs.renovatebot.com/configuration-options/. (Low effort)","Warn: tool 'Dependabot' is not used: Follow the instructions from https://docs.github.com/code-security/dependabot/dependabot-version-updates/about-dependabot-version-updates. (Low effort)","Warn: tool 'PyUp' is not used: Follow the instructions from https://docs.pyup.io/docs. (Low effort)","Warn: tool 'Sonatype Lift' is not used: Follow the instructions from https://help.sonatype.com/lift/getting-started. (Low effort)"],"documentation":{"short":"Determines if the project uses a dependency update tool.","url":"https://github.com/ossf/scorecard/blob/49c0eed3a423f00c872b5c3c9f1bbca9e8aae799/docs/checks.md#dependency-update-tool"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no OSSFuzz integration found: Follow the steps in https://github.com/google/oss-fuzz to integrate fuzzing for your project.\nOver time, try to add fuzzing for more functionalities of your project. (High effort)","Warn: no OneFuzz integration found: Follow the steps in https://github.com/microsoft/onefuzz to start fuzzing for your project.\nOver time, try to add fuzzing for more functionalities of your project. (High effort)","Warn: no GoBuiltInFuzzer integration found: Follow the steps in https://go.dev/doc/fuzz/ to enable fuzzing on your project.\nOver time, try to add fuzzing for more functionalities of your project. (Medium effort)","Warn: no PythonAtherisFuzzer integration found: Follow the steps in https://github.com/google/atheris to enable fuzzing on your project.\nOver time, try to add fuzzing for more functionalities of your project. (Medium effort)","Warn: no CLibFuzzer integration found: Follow the steps in https://llvm.org/docs/LibFuzzer.html to enable fuzzing on your project.\nOver time, try to add fuzzing for more functionalities of your project. (Medium effort)","Warn: no CppLibFuzzer integration found: Follow the steps in https://llvm.org/docs/LibFuzzer.html to enable fuzzing on your project.\nOver time, try to add fuzzing for more functionalities of your project. (Medium effort)","Warn: no SwiftLibFuzzer integration found: Follow the steps in https://google.github.io/oss-fuzz/getting-started/new-project-guide/swift-lang/ to enable fuzzing on your project.\nOver time, try to add fuzzing for more functionalities of your project. (Medium effort)","Warn: no RustCargoFuzzer integration found: Follow the steps in https://rust-fuzz.github.io/book/cargo-fuzz.html to enable fuzzing on your project.\nOver time, try to add fuzzing for more functionalities of your project. (Medium effort)","Warn: no JavaJazzerFuzzer integration found: Follow the steps in https://github.com/CodeIntelligenceTesting/jazzer to enable fuzzing on your project.\nOver time, try to add fuzzing for more functionalities of your project. (Medium effort)","Warn: no ClusterFuzzLite integration found: Follow the steps in https://github.com/google/clusterfuzzlite to integrate fuzzing as part of CI.\nOver time, try to add fuzzing for more functionalities of your project. (High effort)","Warn: no HaskellPropertyBasedTesting integration found: Use one of the following frameworks to fuzz your project:\nQuickCheck: https://hackage.haskell.org/package/QuickCheck\nhedgehog: https://hedgehog.qa/\nvalidity: https://github.com/NorfairKing/validity\nsmallcheck: https://hackage.haskell.org/package/smallcheck\nhspec: https://hspec.github.io/\ntasty: https://hackage.haskell.org/package/tasty (High effort)","Warn: no TypeScriptPropertyBasedTesting integration found: Use fast-check: https://github.com/dubzzz/fast-check (High effort)","Warn: no JavaScriptPropertyBasedTesting integration found: Use fast-check: https://github.com/dubzzz/fast-check (High effort)"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/49c0eed3a423f00c872b5c3c9f1bbca9e8aae799/docs/checks.md#fuzzing"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: License file found in expected location: LICENSE:1","Info: FSF or OSI recognized license: LICENSE:1"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/49c0eed3a423f00c872b5c3c9f1bbca9e8aae799/docs/checks.md#license"}},{"name":"Maintained","score":10,"reason":"30 commit(s) out of 30 and 4 issue activity out of 30 found in the last 90 days -- score normalized to 10","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/49c0eed3a423f00c872b5c3c9f1bbca9e8aae799/docs/checks.md#maintained"}},{"name":"Packaging","score":10,"reason":"publishing workflow detected","details":["Info: GitHub/GitLab publishing workflow used in run https://api.github.com/repos/intel/ipex-llm/actions/runs/9709900576: .github/workflows/manually_build.yml:136"],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/49c0eed3a423f00c872b5c3c9f1bbca9e8aae799/docs/checks.md#packaging"}},{"name":"Pinned-Dependencies","score":0,"reason":"dependency not pinned by hash detected -- score normalized to 0","details":["Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/codeql.yml:61: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/codeql.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/codeql.yml:65: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/codeql.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/codeql.yml:93: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/codeql.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/codeql.yml:98: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/codeql.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/codeql.yml:104: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/codeql.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/llm-binary-build.yml:287: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/llm-binary-build.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/llm-binary-build.yml:397: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/llm-binary-build.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/llm-binary-build.yml:417: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/llm-binary-build.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/llm-binary-build.yml:421: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/llm-binary-build.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/llm-binary-build.yml:440: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/llm-binary-build.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/llm-binary-build.yml:57: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/llm-binary-build.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/llm-binary-build.yml:125: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/llm-binary-build.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/llm-binary-build.yml:269: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/llm-binary-build.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/llm-binary-build.yml:332: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/llm-binary-build.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/llm-binary-build.yml:454: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/llm-binary-build.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/llm-binary-build.yml:474: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/llm-binary-build.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/llm-binary-build.yml:478: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/llm-binary-build.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/llm-binary-build.yml:497: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/llm-binary-build.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/llm-binary-build.yml:107: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/llm-binary-build.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/llm-binary-build.yml:195: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/llm-binary-build.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/llm-binary-build.yml:201: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/llm-binary-build.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/llm-binary-build.yml:207: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/llm-binary-build.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/llm-binary-build.yml:227: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/llm-binary-build.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/llm-binary-build.yml:307: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/llm-binary-build.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/llm-binary-build.yml:311: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/llm-binary-build.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/llm-binary-build.yml:318: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/llm-binary-build.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/llm-binary-build.yml:352: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/llm-binary-build.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/llm-binary-build.yml:356: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/llm-binary-build.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/llm-binary-build.yml:383: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/llm-binary-build.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/llm-c-evaluation.yml:109: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/llm-c-evaluation.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/llm-c-evaluation.yml:111: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/llm-c-evaluation.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/llm-c-evaluation.yml:184: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/llm-c-evaluation.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/llm-c-evaluation.yml:195: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/llm-c-evaluation.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/llm-c-evaluation.yml:197: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/llm-c-evaluation.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/llm-c-evaluation.yml:207: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/llm-c-evaluation.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/llm-c-evaluation.yml:222: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/llm-c-evaluation.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/llm-c-evaluation.yml:234: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/llm-c-evaluation.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/llm-c-evaluation.yml:262: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/llm-c-evaluation.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/llm-harness-evaluation.yml:122: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/llm-harness-evaluation.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/llm-harness-evaluation.yml:194: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/llm-harness-evaluation.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/llm-harness-evaluation.yml:214: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/llm-harness-evaluation.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/llm-harness-evaluation.yml:223: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/llm-harness-evaluation.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/llm-harness-evaluation.yml:241: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/llm-harness-evaluation.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/llm-harness-evaluation.yml:263: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/llm-harness-evaluation.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/llm-nightly-test.yml:82: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/llm-nightly-test.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/llm-ppl-evaluation.yml:200: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/llm-ppl-evaluation.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/llm-ppl-evaluation.yml:209: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/llm-ppl-evaluation.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/llm-ppl-evaluation.yml:226: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/llm-ppl-evaluation.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/llm-ppl-evaluation.yml:248: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/llm-ppl-evaluation.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/llm-ppl-evaluation.yml:121: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/llm-ppl-evaluation.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/llm-ppl-evaluation.yml:175: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/llm-ppl-evaluation.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/llm-whisper-evaluation.yml:98: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/llm-whisper-evaluation.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/llm-whisper-evaluation.yml:149: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/llm-whisper-evaluation.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/llm-whisper-evaluation.yml:162: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/llm-whisper-evaluation.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/llm-whisper-evaluation.yml:179: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/llm-whisper-evaluation.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/llm-whisper-evaluation.yml:186: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/llm-whisper-evaluation.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/llm_example_tests.yml:58: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/llm_example_tests.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/llm_performance_tests.yml:113: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/llm_performance_tests.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/llm_performance_tests.yml:459: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/llm_performance_tests.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/llm_performance_tests.yml:542: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/llm_performance_tests.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/llm_tests_for_stable_version_on_arc.yml:45: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/llm_tests_for_stable_version_on_arc.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/llm_tests_for_stable_version_on_arc.yml:169: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/llm_tests_for_stable_version_on_arc.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/llm_tests_for_stable_version_on_spr.yml:42: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/llm_tests_for_stable_version_on_spr.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/llm_tests_for_stable_version_on_spr.yml:100: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/llm_tests_for_stable_version_on_spr.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/llm_unit_tests.yml:123: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/llm_unit_tests.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/llm_unit_tests.yml:311: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/llm_unit_tests.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/python-style-check.yml:42: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/python-style-check.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/release-ipex-llm.yaml:34: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/release-ipex-llm.yaml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/release-pypi.yml:45: update your workflow using https://app.stepsecurity.io/secureworkflow/intel/ipex-llm/release-pypi.yml/main?enable=pin","Warn: containerImage not pinned by hash: docker/llm/finetune/lora/cpu/docker/Dockerfile:1","Warn: containerImage not pinned by hash: docker/llm/finetune/lora/cpu/docker/Dockerfile:10","Warn: containerImage not pinned by hash: docker/llm/finetune/qlora/cpu/docker/Dockerfile:1","Warn: containerImage not pinned by hash: docker/llm/finetune/qlora/cpu/docker/Dockerfile:9","Warn: containerImage not pinned by hash: docker/llm/finetune/qlora/cpu/docker/Dockerfile.k8s:2","Warn: containerImage not pinned by hash: docker/llm/finetune/qlora/cpu/docker/Dockerfile.k8s:10","Warn: containerImage not pinned by hash: docker/llm/finetune/xpu/Dockerfile:1: pin your Docker image by updating intel/oneapi-basekit:2024.0.1-devel-ubuntu22.04 to intel/oneapi-basekit:2024.0.1-devel-ubuntu22.04@sha256:c00c7bc497bc14878eb8ab1d4ba14c05b0e088dc66f01d4c08c414fca5db0702","Warn: containerImage not pinned by hash: docker/llm/inference-cpp/Dockerfile:1: pin your Docker image by updating intel/oneapi-basekit:2024.2.1-0-devel-ubuntu22.04 to intel/oneapi-basekit:2024.2.1-0-devel-ubuntu22.04@sha256:c148163a0476fad50ad46a473c03dc5ca9058fdf5cba69287a4836b3d9ae8bff","Warn: containerImage not pinned by hash: docker/llm/inference/cpu/docker/Dockerfile:1: pin your Docker image by updating ubuntu:22.04 to ubuntu:22.04@sha256:0e5e4a57c2499249aafc3b40fcd541e9a456aab7296681a3994d631587203f97","Warn: containerImage not pinned by hash: docker/llm/inference/xpu/docker/Dockerfile:1: pin your Docker image by updating intel/oneapi:2024.2.1-0-devel-ubuntu22.04 to intel/oneapi:2024.2.1-0-devel-ubuntu22.04@sha256:c148163a0476fad50ad46a473c03dc5ca9058fdf5cba69287a4836b3d9ae8bff","Warn: containerImage not pinned by hash: docker/llm/serving/cpu/docker/Dockerfile:1: pin your Docker image by updating intelanalytics/ipex-llm-cpu:2.2.0-SNAPSHOT to intelanalytics/ipex-llm-cpu:2.2.0-SNAPSHOT@sha256:2f36dff0d74fc161742e0cb7c3023c04fb8fe2c1dfb2b6908d23265d0cba86d6","Warn: containerImage not pinned by hash: docker/llm/serving/xpu/docker/Dockerfile:1: pin your Docker image by updating intel/oneapi-basekit:2024.1.1-devel-ubuntu22.04 to intel/oneapi-basekit:2024.1.1-devel-ubuntu22.04@sha256:be6bc8ccbde26358f9d4163dd60785bb25be452ce764e60da05fff4dbc54db99","Warn: containerImage not pinned by hash: docker/llm/sources/Dockerfile:1: pin your Docker image by updating ubuntu:22.04 to ubuntu:22.04@sha256:0e5e4a57c2499249aafc3b40fcd541e9a456aab7296681a3994d631587203f97","Warn: pipCommand not pinned by hash: docker/llm/finetune/lora/cpu/docker/Dockerfile:21-64","Warn: pipCommand not pinned by hash: docker/llm/finetune/lora/cpu/docker/Dockerfile:21-64","Warn: pipCommand not pinned by hash: docker/llm/finetune/lora/cpu/docker/Dockerfile:21-64","Warn: pipCommand not pinned by hash: docker/llm/finetune/lora/cpu/docker/Dockerfile:21-64","Warn: pipCommand not pinned by hash: docker/llm/finetune/lora/cpu/docker/Dockerfile:21-64","Warn: pipCommand not pinned by hash: docker/llm/finetune/lora/cpu/docker/Dockerfile:21-64","Warn: pipCommand not pinned by hash: docker/llm/finetune/lora/cpu/docker/Dockerfile:21-64","Warn: pipCommand not pinned by hash: docker/llm/finetune/lora/cpu/docker/Dockerfile:21-64","Warn: pipCommand not pinned by hash: docker/llm/finetune/lora/cpu/docker/Dockerfile:21-64","Warn: pipCommand not pinned by hash: docker/llm/finetune/qlora/cpu/docker/Dockerfile:21-59","Warn: pipCommand not pinned by hash: docker/llm/finetune/qlora/cpu/docker/Dockerfile:21-59","Warn: pipCommand not pinned by hash: docker/llm/finetune/qlora/cpu/docker/Dockerfile:21-59","Warn: pipCommand not pinned by hash: docker/llm/finetune/qlora/cpu/docker/Dockerfile:21-59","Warn: pipCommand not pinned by hash: docker/llm/finetune/qlora/cpu/docker/Dockerfile:21-59","Warn: pipCommand not pinned by hash: docker/llm/finetune/qlora/cpu/docker/Dockerfile:21-59","Warn: pipCommand not pinned by hash: docker/llm/finetune/qlora/cpu/docker/Dockerfile:21-59","Warn: pipCommand not pinned by hash: docker/llm/finetune/qlora/cpu/docker/Dockerfile.k8s:22-78","Warn: pipCommand not pinned by hash: docker/llm/finetune/qlora/cpu/docker/Dockerfile.k8s:22-78","Warn: pipCommand not pinned by hash: docker/llm/finetune/qlora/cpu/docker/Dockerfile.k8s:22-78","Warn: pipCommand not pinned by hash: docker/llm/finetune/qlora/cpu/docker/Dockerfile.k8s:22-78","Warn: pipCommand not pinned by hash: docker/llm/finetune/qlora/cpu/docker/Dockerfile.k8s:22-78","Warn: pipCommand not pinned by hash: docker/llm/finetune/qlora/cpu/docker/Dockerfile.k8s:22-78","Warn: pipCommand not pinned by hash: docker/llm/finetune/qlora/cpu/docker/Dockerfile.k8s:22-78","Warn: downloadThenRun not pinned by hash: docker/llm/finetune/xpu/Dockerfile:11-55","Warn: pipCommand not pinned by hash: docker/llm/finetune/xpu/Dockerfile:11-55","Warn: pipCommand not pinned by hash: docker/llm/finetune/xpu/Dockerfile:11-55","Warn: pipCommand not pinned by hash: docker/llm/finetune/xpu/Dockerfile:11-55","Warn: pipCommand not pinned by hash: docker/llm/finetune/xpu/Dockerfile:11-55","Warn: downloadThenRun not pinned by hash: docker/llm/inference-cpp/Dockerfile:15-65","Warn: pipCommand not pinned by hash: docker/llm/inference-cpp/Dockerfile:15-65","Warn: pipCommand not pinned by hash: docker/llm/inference-cpp/Dockerfile:15-65","Warn: pipCommand not pinned by hash: docker/llm/inference-cpp/Dockerfile:15-65","Warn: pipCommand not pinned by hash: docker/llm/inference-cpp/Dockerfile:15-65","Warn: pipCommand not pinned by hash: docker/llm/inference/cpu/docker/Dockerfile:13-69","Warn: pipCommand not pinned by hash: docker/llm/inference/cpu/docker/Dockerfile:13-69","Warn: pipCommand not pinned by hash: docker/llm/inference/cpu/docker/Dockerfile:13-69","Warn: pipCommand not pinned by hash: docker/llm/inference/cpu/docker/Dockerfile:13-69","Warn: pipCommand not pinned by hash: docker/llm/inference/cpu/docker/Dockerfile:13-69","Warn: pipCommand not pinned by hash: docker/llm/inference/cpu/docker/Dockerfile:13-69","Warn: pipCommand not pinned by hash: docker/llm/inference/cpu/docker/Dockerfile:13-69","Warn: pipCommand not pinned by hash: docker/llm/inference/cpu/docker/Dockerfile:13-69","Warn: pipCommand not pinned by hash: docker/llm/inference/cpu/docker/Dockerfile:13-69","Warn: pipCommand not pinned by hash: docker/llm/inference/cpu/docker/Dockerfile:13-69","Warn: pipCommand not pinned by hash: docker/llm/inference/cpu/docker/Dockerfile:13-69","Warn: pipCommand not pinned by hash: docker/llm/inference/cpu/docker/Dockerfile:13-69","Warn: pipCommand not pinned by hash: docker/llm/inference/cpu/docker/Dockerfile:13-69","Warn: pipCommand not pinned by hash: docker/llm/inference/cpu/docker/Dockerfile:13-69","Warn: pipCommand not pinned by hash: docker/llm/inference/cpu/docker/Dockerfile:13-69","Warn: downloadThenRun not pinned by hash: docker/llm/inference/xpu/docker/Dockerfile:18-93","Warn: pipCommand not pinned by hash: docker/llm/inference/xpu/docker/Dockerfile:18-93","Warn: pipCommand not pinned by hash: docker/llm/inference/xpu/docker/Dockerfile:18-93","Warn: pipCommand not pinned by hash: docker/llm/inference/xpu/docker/Dockerfile:18-93","Warn: pipCommand not pinned by hash: docker/llm/inference/xpu/docker/Dockerfile:18-93","Warn: pipCommand not pinned by hash: docker/llm/inference/xpu/docker/Dockerfile:18-93","Warn: pipCommand not pinned by hash: docker/llm/inference/xpu/docker/Dockerfile:18-93","Warn: pipCommand not pinned by hash: docker/llm/inference/xpu/docker/Dockerfile:18-93","Warn: pipCommand not pinned by hash: docker/llm/inference/xpu/docker/Dockerfile:18-93","Warn: pipCommand not pinned by hash: docker/llm/inference/xpu/docker/Dockerfile:18-93","Warn: pipCommand not pinned by hash: docker/llm/inference/xpu/docker/Dockerfile:18-93","Warn: pipCommand not pinned by hash: docker/llm/inference/xpu/docker/Dockerfile:18-93","Warn: pipCommand not pinned by hash: docker/llm/inference/xpu/docker/Dockerfile:18-93","Warn: pipCommand not pinned by hash: docker/llm/serving/cpu/docker/Dockerfile:13-34","Warn: pipCommand not pinned by hash: docker/llm/serving/cpu/docker/Dockerfile:13-34","Warn: pipCommand not pinned by hash: docker/llm/serving/cpu/docker/Dockerfile:13-34","Warn: pipCommand not pinned by hash: docker/llm/serving/cpu/docker/Dockerfile:13-34","Warn: pipCommand not pinned by hash: docker/llm/serving/cpu/docker/Dockerfile:13-34","Warn: downloadThenRun not pinned by hash: docker/llm/serving/xpu/docker/Dockerfile:17-100","Warn: pipCommand not pinned by hash: docker/llm/serving/xpu/docker/Dockerfile:17-100","Warn: pipCommand not pinned by hash: docker/llm/serving/xpu/docker/Dockerfile:17-100","Warn: pipCommand not pinned by hash: docker/llm/serving/xpu/docker/Dockerfile:17-100","Warn: pipCommand not pinned by hash: docker/llm/serving/xpu/docker/Dockerfile:17-100","Warn: pipCommand not pinned by hash: docker/llm/serving/xpu/docker/Dockerfile:17-100","Warn: pipCommand not pinned by hash: docker/llm/serving/xpu/docker/Dockerfile:17-100","Warn: pipCommand not pinned by hash: docker/llm/serving/xpu/docker/Dockerfile:17-100","Warn: downloadThenRun not pinned by hash: docker/llm/serving/xpu/docker/Dockerfile:17-100","Warn: pipCommand not pinned by hash: docker/llm/serving/xpu/docker/Dockerfile:17-100","Warn: pipCommand not pinned by hash: docker/llm/serving/xpu/docker/Dockerfile:17-100","Warn: pipCommand not pinned by hash: docker/llm/serving/xpu/docker/Dockerfile:17-100","Warn: pipCommand not pinned by hash: docker/llm/serving/xpu/docker/Dockerfile:17-100","Warn: pipCommand not pinned by hash: docker/llm/serving/xpu/docker/Dockerfile:17-100","Warn: pipCommand not pinned by hash: docker/llm/serving/xpu/docker/Dockerfile:17-100","Warn: pipCommand not pinned by hash: python/llm/example/CPU/Deepspeed-AutoTP/install.sh:14","Warn: pipCommand not pinned by hash: python/llm/example/CPU/Deepspeed-AutoTP/install.sh:15","Warn: pipCommand not pinned by hash: python/llm/example/CPU/Deepspeed-AutoTP/install.sh:19","Warn: pipCommand not pinned by hash: python/llm/example/CPU/Deepspeed-AutoTP/install.sh:23","Warn: pipCommand not pinned by hash: .github/workflows/llm-c-evaluation.yml:117","Warn: pipCommand not pinned by hash: .github/workflows/llm-c-evaluation.yml:118","Warn: pipCommand not pinned by hash: .github/workflows/llm-c-evaluation.yml:119","Warn: pipCommand not pinned by hash: .github/workflows/llm-c-evaluation.yml:162","Warn: pipCommand not pinned by hash: .github/workflows/llm-c-evaluation.yml:163","Warn: pipCommand not pinned by hash: .github/workflows/llm-c-evaluation.yml:164","Warn: pipCommand not pinned by hash: .github/workflows/llm-c-evaluation.yml:165","Warn: pipCommand not pinned by hash: .github/workflows/llm-c-evaluation.yml:166","Warn: pipCommand not pinned by hash: .github/workflows/llm-c-evaluation.yml:167","Warn: pipCommand not pinned by hash: .github/workflows/llm-c-evaluation.yml:203","Warn: pipCommand not pinned by hash: .github/workflows/llm-c-evaluation.yml:204","Warn: pipCommand not pinned by hash: .github/workflows/llm-c-evaluation.yml:240","Warn: pipCommand not pinned by hash: .github/workflows/llm-c-evaluation.yml:241","Warn: pipCommand not pinned by hash: .github/workflows/llm-c-evaluation.yml:242","Warn: pipCommand not pinned by hash: .github/workflows/llm-c-evaluation.yml:243","Warn: pipCommand not pinned by hash: .github/workflows/llm-harness-evaluation.yml:129","Warn: pipCommand not pinned by hash: .github/workflows/llm-harness-evaluation.yml:130","Warn: pipCommand not pinned by hash: .github/workflows/llm-harness-evaluation.yml:131","Warn: pipCommand not pinned by hash: .github/workflows/llm-harness-evaluation.yml:150","Warn: pipCommand not pinned by hash: .github/workflows/llm-harness-evaluation.yml:166","Warn: pipCommand not pinned by hash: .github/workflows/llm-harness-evaluation.yml:220","Warn: pipCommand not pinned by hash: .github/workflows/llm-harness-evaluation.yml:221","Warn: pipCommand not pinned by hash: .github/workflows/llm-harness-evaluation.yml:247","Warn: pipCommand not pinned by hash: .github/workflows/llm-harness-evaluation.yml:248","Warn: pipCommand not pinned by hash: .github/workflows/llm-harness-evaluation.yml:249","Warn: pipCommand not pinned by hash: .github/workflows/llm-nightly-test.yml:88","Warn: pipCommand not pinned by hash: .github/workflows/llm-nightly-test.yml:89","Warn: pipCommand not pinned by hash: .github/workflows/llm-ppl-evaluation.yml:127","Warn: pipCommand not pinned by hash: .github/workflows/llm-ppl-evaluation.yml:128","Warn: pipCommand not pinned by hash: .github/workflows/llm-ppl-evaluation.yml:129","Warn: pipCommand not pinned by hash: .github/workflows/llm-ppl-evaluation.yml:151","Warn: pipCommand not pinned by hash: .github/workflows/llm-ppl-evaluation.yml:206","Warn: pipCommand not pinned by hash: .github/workflows/llm-ppl-evaluation.yml:207","Warn: pipCommand not pinned by hash: .github/workflows/llm-ppl-evaluation.yml:232","Warn: pipCommand not pinned by hash: .github/workflows/llm-ppl-evaluation.yml:233","Warn: pipCommand not pinned by hash: .github/workflows/llm-ppl-evaluation.yml:234","Warn: pipCommand not pinned by hash: .github/workflows/llm-whisper-evaluation.yml:105","Warn: pipCommand not pinned by hash: .github/workflows/llm-whisper-evaluation.yml:106","Warn: pipCommand not pinned by hash: .github/workflows/llm-whisper-evaluation.yml:107","Warn: pipCommand not pinned by hash: .github/workflows/llm-whisper-evaluation.yml:108","Warn: pipCommand not pinned by hash: .github/workflows/llm-whisper-evaluation.yml:109","Warn: pipCommand not pinned by hash: .github/workflows/llm-whisper-evaluation.yml:110","Warn: pipCommand not pinned by hash: .github/workflows/llm-whisper-evaluation.yml:111","Warn: pipCommand not pinned by hash: .github/workflows/llm-whisper-evaluation.yml:112","Warn: pipCommand not pinned by hash: .github/workflows/llm-whisper-evaluation.yml:196","Warn: pipCommand not pinned by hash: .github/workflows/llm-whisper-evaluation.yml:205","Warn: pipCommand not pinned by hash: .github/workflows/llm_example_tests.yml:63","Warn: pipCommand not pinned by hash: .github/workflows/llm_example_tests.yml:64","Warn: pipCommand not pinned by hash: .github/workflows/llm_performance_tests.yml:466","Warn: pipCommand not pinned by hash: .github/workflows/llm_performance_tests.yml:467","Warn: pipCommand not pinned by hash: .github/workflows/llm_performance_tests.yml:468","Warn: pipCommand not pinned by hash: .github/workflows/llm_performance_tests.yml:469","Warn: pipCommand not pinned by hash: .github/workflows/llm_performance_tests.yml:470","Warn: pipCommand not pinned by hash: .github/workflows/llm_performance_tests.yml:471","Warn: pipCommand not pinned by hash: .github/workflows/llm_performance_tests.yml:472","Warn: pipCommand not pinned by hash: .github/workflows/llm_performance_tests.yml:488","Warn: pipCommand not pinned by hash: .github/workflows/llm_performance_tests.yml:511","Warn: pipCommand not pinned by hash: .github/workflows/llm_performance_tests.yml:549","Warn: pipCommand not pinned by hash: .github/workflows/llm_performance_tests.yml:550","Warn: pipCommand not pinned by hash: .github/workflows/llm_performance_tests.yml:551","Warn: pipCommand not pinned by hash: .github/workflows/llm_performance_tests.yml:552","Warn: pipCommand not pinned by hash: .github/workflows/llm_performance_tests.yml:569","Warn: pipCommand not pinned by hash: .github/workflows/llm_performance_tests.yml:591","Warn: pipCommand not pinned by hash: .github/workflows/llm_performance_tests.yml:123","Warn: pipCommand not pinned by hash: .github/workflows/llm_performance_tests.yml:124","Warn: pipCommand not pinned by hash: .github/workflows/llm_performance_tests.yml:125","Warn: pipCommand not pinned by hash: .github/workflows/llm_performance_tests.yml:126","Warn: pipCommand not pinned by hash: .github/workflows/llm_performance_tests.yml:127","Warn: pipCommand not pinned by hash: .github/workflows/llm_performance_tests.yml:128","Warn: pipCommand not pinned by hash: .github/workflows/llm_performance_tests.yml:129","Warn: pipCommand not pinned by hash: .github/workflows/llm_performance_tests.yml:130","Warn: pipCommand not pinned by hash: .github/workflows/llm_performance_tests.yml:149","Warn: pipCommand not pinned by hash: .github/workflows/llm_performance_tests.yml:159","Warn: pipCommand not pinned by hash: .github/workflows/llm_performance_tests.yml:160","Warn: pipCommand not pinned by hash: .github/workflows/llm_performance_tests.yml:171","Warn: pipCommand not pinned by hash: .github/workflows/llm_performance_tests.yml:208","Warn: pipCommand not pinned by hash: .github/workflows/llm_performance_tests.yml:240","Warn: pipCommand not pinned by hash: .github/workflows/llm_performance_tests.yml:241","Warn: pipCommand not pinned by hash: .github/workflows/llm_performance_tests.yml:257","Warn: pipCommand not pinned by hash: .github/workflows/llm_performance_tests.yml:258","Warn: pipCommand not pinned by hash: .github/workflows/llm_performance_tests.yml:291","Warn: pipCommand not pinned by hash: .github/workflows/llm_performance_tests.yml:292","Warn: pipCommand not pinned by hash: .github/workflows/llm_performance_tests.yml:293","Warn: pipCommand not pinned by hash: .github/workflows/llm_performance_tests.yml:317","Warn: pipCommand not pinned by hash: .github/workflows/llm_performance_tests.yml:331","Warn: pipCommand not pinned by hash: .github/workflows/llm_tests_for_stable_version_on_arc.yml:54","Warn: pipCommand not pinned by hash: .github/workflows/llm_tests_for_stable_version_on_arc.yml:55","Warn: pipCommand not pinned by hash: .github/workflows/llm_tests_for_stable_version_on_arc.yml:56","Warn: pipCommand not pinned by hash: .github/workflows/llm_tests_for_stable_version_on_arc.yml:57","Warn: pipCommand not pinned by hash: .github/workflows/llm_tests_for_stable_version_on_arc.yml:58","Warn: pipCommand not pinned by hash: .github/workflows/llm_tests_for_stable_version_on_arc.yml:59","Warn: pipCommand not pinned by hash: .github/workflows/llm_tests_for_stable_version_on_arc.yml:60","Warn: pipCommand not pinned by hash: .github/workflows/llm_tests_for_stable_version_on_arc.yml:109","Warn: pipCommand not pinned by hash: .github/workflows/llm_tests_for_stable_version_on_arc.yml:146","Warn: pipCommand not pinned by hash: .github/workflows/llm_tests_for_stable_version_on_arc.yml:178","Warn: pipCommand not pinned by hash: .github/workflows/llm_tests_for_stable_version_on_arc.yml:179","Warn: pipCommand not pinned by hash: .github/workflows/llm_tests_for_stable_version_on_arc.yml:180","Warn: pipCommand not pinned by hash: .github/workflows/llm_tests_for_stable_version_on_arc.yml:181","Warn: pipCommand not pinned by hash: .github/workflows/llm_tests_for_stable_version_on_arc.yml:182","Warn: pipCommand not pinned by hash: .github/workflows/llm_tests_for_stable_version_on_arc.yml:183","Warn: pipCommand not pinned by hash: .github/workflows/llm_tests_for_stable_version_on_arc.yml:184","Warn: pipCommand not pinned by hash: .github/workflows/llm_tests_for_stable_version_on_arc.yml:214","Warn: pipCommand not pinned by hash: .github/workflows/llm_tests_for_stable_version_on_arc.yml:231","Warn: pipCommand not pinned by hash: .github/workflows/llm_tests_for_stable_version_on_spr.yml:49","Warn: pipCommand not pinned by hash: .github/workflows/llm_tests_for_stable_version_on_spr.yml:50","Warn: pipCommand not pinned by hash: .github/workflows/llm_tests_for_stable_version_on_spr.yml:51","Warn: pipCommand not pinned by hash: .github/workflows/llm_tests_for_stable_version_on_spr.yml:52","Warn: pipCommand not pinned by hash: .github/workflows/llm_tests_for_stable_version_on_spr.yml:53","Warn: pipCommand not pinned by hash: .github/workflows/llm_tests_for_stable_version_on_spr.yml:54","Warn: pipCommand not pinned by hash: .github/workflows/llm_tests_for_stable_version_on_spr.yml:55","Warn: pipCommand not pinned by hash: .github/workflows/llm_tests_for_stable_version_on_spr.yml:77","Warn: pipCommand not pinned by hash: .github/workflows/llm_tests_for_stable_version_on_spr.yml:107","Warn: pipCommand not pinned by hash: .github/workflows/llm_tests_for_stable_version_on_spr.yml:108","Warn: pipCommand not pinned by hash: .github/workflows/llm_tests_for_stable_version_on_spr.yml:109","Warn: pipCommand not pinned by hash: .github/workflows/llm_tests_for_stable_version_on_spr.yml:110","Warn: pipCommand not pinned by hash: .github/workflows/llm_tests_for_stable_version_on_spr.yml:111","Warn: pipCommand not pinned by hash: .github/workflows/llm_tests_for_stable_version_on_spr.yml:112","Warn: pipCommand not pinned by hash: .github/workflows/llm_tests_for_stable_version_on_spr.yml:113","Warn: pipCommand not pinned by hash: .github/workflows/llm_tests_for_stable_version_on_spr.yml:135","Warn: pipCommand not pinned by hash: .github/workflows/llm_unit_tests.yml:129","Warn: pipCommand not pinned by hash: .github/workflows/llm_unit_tests.yml:130","Warn: pipCommand not pinned by hash: .github/workflows/llm_unit_tests.yml:251","Warn: pipCommand not pinned by hash: .github/workflows/llm_unit_tests.yml:256","Warn: pipCommand not pinned by hash: .github/workflows/llm_unit_tests.yml:257","Warn: pipCommand not pinned by hash: .github/workflows/llm_unit_tests.yml:258","Warn: pipCommand not pinned by hash: .github/workflows/llm_unit_tests.yml:263","Warn: pipCommand not pinned by hash: .github/workflows/llm_unit_tests.yml:264","Warn: pipCommand not pinned by hash: .github/workflows/llm_unit_tests.yml:265","Warn: pipCommand not pinned by hash: .github/workflows/llm_unit_tests.yml:266","Warn: pipCommand not pinned by hash: .github/workflows/llm_unit_tests.yml:267","Warn: pipCommand not pinned by hash: .github/workflows/llm_unit_tests.yml:318","Warn: pipCommand not pinned by hash: .github/workflows/llm_unit_tests.yml:319","Warn: pipCommand not pinned by hash: .github/workflows/llm_unit_tests.yml:320","Warn: pipCommand not pinned by hash: .github/workflows/llm_unit_tests.yml:414","Warn: pipCommand not pinned by hash: .github/workflows/llm_unit_tests.yml:422","Warn: pipCommand not pinned by hash: .github/workflows/llm_unit_tests.yml:423","Warn: pipCommand not pinned by hash: .github/workflows/llm_unit_tests.yml:438","Warn: pipCommand not pinned by hash: .github/workflows/llm_unit_tests.yml:453","Warn: pipCommand not pinned by hash: .github/workflows/llm_unit_tests.yml:454","Warn: pipCommand not pinned by hash: .github/workflows/llm_unit_tests.yml:455","Warn: pipCommand not pinned by hash: .github/workflows/llm_unit_tests.yml:466","Warn: pipCommand not pinned by hash: .github/workflows/llm_unit_tests.yml:467","Warn: pipCommand not pinned by hash: .github/workflows/llm_unit_tests.yml:473","Warn: pipCommand not pinned by hash: .github/workflows/llm_unit_tests.yml:474","Warn: pipCommand not pinned by hash: .github/workflows/llm_unit_tests.yml:475","Warn: pipCommand not pinned by hash: .github/workflows/llm_unit_tests.yml:478","Warn: pipCommand not pinned by hash: .github/workflows/llm_unit_tests.yml:483","Warn: pipCommand not pinned by hash: .github/workflows/llm_unit_tests.yml:488","Warn: pipCommand not pinned by hash: .github/workflows/llm_unit_tests.yml:489","Warn: pipCommand not pinned by hash: .github/workflows/python-style-check.yml:48","Warn: pipCommand not pinned by hash: .github/workflows/release-ipex-llm.yaml:40","Warn: pipCommand not pinned by hash: .github/workflows/release-ipex-llm.yaml:41","Warn: pipCommand not pinned by hash: .github/workflows/release-ipex-llm.yaml:42","Warn: pipCommand not pinned by hash: .github/workflows/release-ipex-llm.yaml:43","Warn: pipCommand not pinned by hash: .github/workflows/release-pypi.yml:51","Warn: pipCommand not pinned by hash: .github/workflows/release-pypi.yml:52","Warn: pipCommand not pinned by hash: .github/workflows/release-pypi.yml:53","Warn: pipCommand not pinned by hash: .github/workflows/release-pypi.yml:54","Info:  54 out of 107 GitHub-owned GitHubAction dependencies pinned","Info:   1 out of  17 third-party GitHubAction dependencies pinned","Info:   0 out of  13 containerImage dependencies pinned","Info:   2 out of 240 pipCommand dependencies pinned","Info:   0 out of   5 downloadThenRun dependencies pinned"],"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/49c0eed3a423f00c872b5c3c9f1bbca9e8aae799/docs/checks.md#pinned-dependencies"}},{"name":"SAST","score":7,"reason":"SAST tool detected but not run on all commits","details":["Warn: 0 commits out of 30 are checked with a SAST tool","Info: SAST tool detected: CodeQL"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/49c0eed3a423f00c872b5c3c9f1bbca9e8aae799/docs/checks.md#sast"}},{"name":"Security-Policy","score":10,"reason":"security policy file detected","details":["Info: security policy file detected: SECURITY.md:1","Info: Found linked content: SECURITY.md:1","Info: Found disclosure, vulnerability, and/or timelines in security policy: SECURITY.md:1","Info: Found text in security policy: SECURITY.md:1"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/49c0eed3a423f00c872b5c3c9f1bbca9e8aae799/docs/checks.md#security-policy"}},{"name":"Signed-Releases","score":8,"reason":"1 out of 1 artifacts are signed or have provenance","details":["Warn: release artifact v2.4.0 does not have provenance: https://api.github.com/repos/intel/ipex-llm/releases/129083386","Info: signed release artifact: bigdl-llm-portable-2.4.0.zip.asc: https://api.github.com/repos/intel/ipex-llm/releases/assets/154149977"],"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/49c0eed3a423f00c872b5c3c9f1bbca9e8aae799/docs/checks.md#signed-releases"}},{"name":"Token-Permissions","score":10,"reason":"GitHub workflow tokens follow principle of least privilege","details":["Info: topLevel 'contents' permission set to 'read': .github/workflows/codeql.yml:15","Info: jobLevel 'contents' permission set to 'read': .github/workflows/codeql.yml:43","Info: jobLevel 'packages' permission set to 'read': .github/workflows/codeql.yml:39","Info: jobLevel 'actions' permission set to 'read': .github/workflows/codeql.yml:42","Info: topLevel 'contents' permission set to 'read': .github/workflows/llm-binary-build.yml:9","Info: topLevel 'contents' permission set to 'read': .github/workflows/llm-c-evaluation.yml:9","Info: topLevel 'contents' permission set to 'read': .github/workflows/llm-harness-evaluation.yml:9","Info: topLevel 'contents' permission set to 'read': .github/workflows/llm-nightly-test.yml:9","Info: topLevel 'contents' permission set to 'read': .github/workflows/llm-ppl-evaluation.yml:9","Info: topLevel 'contents' permission set to 'read': .github/workflows/llm-whisper-evaluation.yml:9","Info: topLevel 'contents' permission set to 'read': .github/workflows/llm_example_tests.yml:9","Info: topLevel 'contents' permission set to 'read': .github/workflows/llm_performance_tests.yml:9","Info: topLevel 'contents' permission set to 'read': .github/workflows/llm_tests_for_stable_version_on_arc.yml:9","Info: topLevel 'contents' permission set to 'read': .github/workflows/llm_tests_for_stable_version_on_spr.yml:9","Info: topLevel 'contents' permission set to 'read': .github/workflows/llm_unit_tests.yml:9","Info: topLevel 'contents' permission set to 'read': .github/workflows/manually_build.yml:57","Info: topLevel 'contents' permission set to 'read': .github/workflows/manually_build_for_testing.yml:37","Info: topLevel 'contents' permission set to 'read': .github/workflows/python-style-check.yml:9","Info: topLevel 'contents' permission set to 'read': .github/workflows/release-ipex-llm.yaml:13","Info: topLevel 'contents' permission set to 'read': .github/workflows/release-pypi.yml:26","Info: topLevel permissions set to 'read-all': .github/workflows/scorecard.yml:20","Info: no jobLevel write permissions found"],"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/49c0eed3a423f00c872b5c3c9f1bbca9e8aae799/docs/checks.md#token-permissions"}},{"name":"Vulnerabilities","score":0,"reason":"38 existing vulnerabilities detected","details":["Warn: Project is vulnerable to: GHSA-fj7x-q9j7-g6q6 / PYSEC-2024-48","Warn: Project is vulnerable to: GHSA-26jh-r8g2-6fpr","Warn: Project is vulnerable to: GHSA-279j-x4gx-hfrh / PYSEC-2024-219","Warn: Project is vulnerable to: GHSA-34rf-p3r3-58x2","Warn: Project is vulnerable to: GHSA-37qc-qgx6-9xjv / PYSEC-2024-197","Warn: Project is vulnerable to: GHSA-3c67-5hwx-f6wx / PYSEC-2024-196","Warn: Project is vulnerable to: GHSA-3f95-mxq2-2f63","Warn: Project is vulnerable to: GHSA-3gf9-wv65-gwh9","Warn: Project is vulnerable to: GHSA-3qqg-pgqq-3695 / PYSEC-2023-90","Warn: Project is vulnerable to: GHSA-3x5j-9vwr-8rr5 / PYSEC-2023-16","Warn: Project is vulnerable to: GHSA-48cq-79qq-6f7x","Warn: Project is vulnerable to: GHSA-4q3c-cj7g-jcwf / PYSEC-2024-217","Warn: Project is vulnerable to: GHSA-576c-3j53-r9jj / PYSEC-2024-215","Warn: Project is vulnerable to: GHSA-6qm2-wpxq-7qh2 / PYSEC-2023-249","Warn: Project is vulnerable to: GHSA-6v6g-j5fq-hpvw / PYSEC-2024-184","Warn: Project is vulnerable to: GHSA-77xq-6g77-h274 / PYSEC-2024-213","Warn: Project is vulnerable to: GHSA-89v2-pqfv-c5r9 / PYSEC-2024-214","Warn: Project is vulnerable to: GHSA-8c87-gvhj-xm8m / PYSEC-2024-216","Warn: Project is vulnerable to: GHSA-973g-55hp-3frw","Warn: Project is vulnerable to: GHSA-f3h9-8phc-6gvh","Warn: Project is vulnerable to: GHSA-f8xq-q7px-wg8c / PYSEC-2022-229","Warn: Project is vulnerable to: GHSA-g6c9-f4xm-9j4x","Warn: Project is vulnerable to: GHSA-g9cj-cfpp-4g2x","Warn: Project is vulnerable to: GHSA-gqvf-3hgp-5hxv / PYSEC-2023-255","Warn: Project is vulnerable to: GHSA-gvv6-33j7-884g / PYSEC-2024-220","Warn: Project is vulnerable to: GHSA-hm3c-93pg-4cxw / PYSEC-2024-198","Warn: Project is vulnerable to: GHSA-hmx6-r76c-85g9","Warn: Project is vulnerable to: GHSA-j2jg-fq62-7c3h","Warn: Project is vulnerable to: GHSA-j757-pf57-f8r4 / PYSEC-2024-199","Warn: Project is vulnerable to: GHSA-m842-4qm8-7gpq","Warn: Project is vulnerable to: GHSA-qh6x-j82h-vpf9","Warn: Project is vulnerable to: GHSA-r364-m2j9-mf4h","Warn: Project is vulnerable to: GHSA-rhq2-3vr9-6mcr / PYSEC-2021-873","Warn: Project is vulnerable to: GHSA-v4q9-qgqf-7jwp","Warn: Project is vulnerable to: GHSA-xh2x-3mrm-fwqm / PYSEC-2024-218","Warn: Project is vulnerable to: GHSA-xp76-357g-9wqq / PYSEC-2019-156","Warn: Project is vulnerable to: PYSEC-2023-102","Warn: Project is vulnerable to: PYSEC-2023-114"],"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/49c0eed3a423f00c872b5c3c9f1bbca9e8aae799/docs/checks.md#vulnerabilities"}}]},"last_synced_at":"2025-08-19T18:57:23.954Z","repository_id":37285213,"created_at":"2025-08-19T18:57:23.954Z","updated_at":"2025-08-19T18:57:23.954Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":284202058,"owners_count":26964370,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-11-13T02:00:06.582Z","response_time":61,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["gpu","llm","pytorch","transformers"],"created_at":"2025-01-24T00:01:59.707Z","updated_at":"2025-11-13T11:01:14.288Z","avatar_url":"https://github.com/intel.png","language":"Python","funding_links":[],"categories":["Python","Deployment and Serving","Chatbots \u0026 Virtual Companions"],"sub_categories":[],"readme":"#  💫 Intel® LLM Library for PyTorch* \n\u003cp\u003e\n  \u003cb\u003e\u003c English\u003c/b\u003e | \u003ca href='./README.zh-CN.md'\u003e中文\u003c/a\u003e \u003e\n\u003c/p\u003e\n\n**`IPEX-LLM`** is an LLM acceleration library for Intel [GPU](docs/mddocs/Quickstart/install_windows_gpu.md) *(e.g., local PC with iGPU, discrete GPU such as Arc, Flex and Max)*, [NPU](docs/mddocs/Quickstart/npu_quickstart.md) and CPU [^1].\n\u003e [!NOTE]\n\u003e - *`IPEX-LLM` provides seamless integration with [llama.cpp](docs/mddocs/Quickstart/llamacpp_portable_zip_gpu_quickstart.md), [Ollama](docs/mddocs/Quickstart/ollama_portable_zip_quickstart.md), [vLLM](docs/mddocs/Quickstart/vLLM_quickstart.md), [HuggingFace transformers](python/llm/example/GPU/HuggingFace), [LangChain](python/llm/example/GPU/LangChain), [LlamaIndex](python/llm/example/GPU/LlamaIndex), [Text-Generation-WebUI](docs/mddocs/Quickstart/webui_quickstart.md), [DeepSpeed-AutoTP](python/llm/example/GPU/Deepspeed-AutoTP), [FastChat](docs/mddocs/Quickstart/fastchat_quickstart.md), [Axolotl](docs/mddocs/Quickstart/axolotl_quickstart.md), [HuggingFace PEFT](python/llm/example/GPU/LLM-Finetuning), [HuggingFace TRL](python/llm/example/GPU/LLM-Finetuning/DPO), [AutoGen](python/llm/example/CPU/Applications/autogen), [ModeScope](python/llm/example/GPU/ModelScope-Models), etc.* \n\u003e - ***70+ models** have been optimized/verified on `ipex-llm` (e.g., Llama, Phi, Mistral, Mixtral, DeepSeek, Qwen, ChatGLM, MiniCPM, Qwen-VL, MiniCPM-V and more), with state-of-art **LLM optimizations**, **XPU acceleration** and **low-bit (FP8/FP6/FP4/INT4) support**; see the complete list [here](#verified-models).*\n\n## Latest Update 🔥 \n- [2025/05] You can now run ***DeepSeek V3/R1 671B*** and ***Qwen3MoE 235B*** models with just 1 or 2 Intel Arc GPU (such as A770 or B580) using [FlashMoE](docs/mddocs/Quickstart/flashmoe_quickstart.md) in `ipex-llm`.\n- [2025/04] We released `ipex-llm 2.2.0`, which includes [Ollama Portable Zip](docs/mddocs/Quickstart/ollama_portable_zip_quickstart.md) and [llama.cpp Portable Zip](docs/mddocs/Quickstart/llamacpp_portable_zip_gpu_quickstart.md).\n  \u003e ⚠️ **Warning (for llama.cpp Portable Zip)**  \n  \u003e `mmap`-based model loading in *llama.cpp* may leak data via side-channels in multi-tenant or shared-host environments.  \n  \u003e To disable `mmap`, add:  \n  \u003e ```bash\n  \u003e --no-mmap\n  \u003e ```\n- [2025/04] We added support of [PyTorch 2.6](docs/mddocs/Quickstart/install_pytorch26_gpu.md) for Intel GPU.\n- [2025/03] We added support for **Gemma3** model in the latest [llama.cpp Portable Zip](https://github.com/intel/ipex-llm/issues/12963#issuecomment-2724032898).\n- [2025/03] We can now run **DeepSeek-R1-671B-Q4_K_M** with 1 or 2 Arc A770 on Xeon using the latest [llama.cpp Portable Zip](docs/mddocs/Quickstart/llamacpp_portable_zip_gpu_quickstart.md#flashmoe-for-deepseek-v3r1).\n- [2025/02] We added support of [llama.cpp Portable Zip](https://github.com/ipex-llm/ipex-llm/releases/tag/v2.3.0-nightly) for Intel **GPU** (both [Windows](docs/mddocs/Quickstart/llamacpp_portable_zip_gpu_quickstart.md#windows-quickstart) and [Linux](docs/mddocs/Quickstart/llamacpp_portable_zip_gpu_quickstart.md#linux-quickstart)) and **NPU** ([Windows](docs/mddocs/Quickstart/llama_cpp_npu_portable_zip_quickstart.md) only).\n- [2025/02] We added support of [Ollama Portable Zip](https://github.com/ipex-llm/ipex-llm/releases/tag/v2.3.0-nightly) to directly run Ollama on Intel **GPU** for both [Windows](docs/mddocs/Quickstart/ollama_portable_zip_quickstart.md#windows-quickstart) and [Linux](docs/mddocs/Quickstart/ollama_portable_zip_quickstart.md#linux-quickstart) (***without the need of manual installations***).\n- [2025/02] We added support for running [vLLM 0.6.6](docs/mddocs/DockerGuides/vllm_docker_quickstart.md) on Intel Arc GPUs.\n- [2025/01] We added the guide for running `ipex-llm` on Intel Arc [B580](docs/mddocs/Quickstart/bmg_quickstart.md) GPU.\n- [2025/01] We added support for running [Ollama 0.5.4](docs/mddocs/Quickstart/ollama_quickstart.md) on Intel GPU.\n- [2024/12] We added both ***Python*** and ***C++*** support for Intel Core Ultra [NPU](docs/mddocs/Quickstart/npu_quickstart.md) (including 100H, 200V, 200K and 200H series).\n\n\u003cdetails\u003e\u003csummary\u003eMore updates\u003c/summary\u003e\n\u003cbr/\u003e\n\n- [2024/11] We added support for running [vLLM 0.6.2](docs/mddocs/DockerGuides/vllm_docker_quickstart.md) on Intel Arc GPUs.\n- [2024/07] We added support for running Microsoft's **GraphRAG** using local LLM on Intel GPU; see the quickstart guide [here](docs/mddocs/Quickstart/graphrag_quickstart.md).\n- [2024/07] We added extensive support for Large Multimodal Models, including [StableDiffusion](python/llm/example/GPU/HuggingFace/Multimodal/StableDiffusion), [Phi-3-Vision](python/llm/example/GPU/HuggingFace/Multimodal/phi-3-vision), [Qwen-VL](python/llm/example/GPU/HuggingFace/Multimodal/qwen-vl), and [more](python/llm/example/GPU/HuggingFace/Multimodal).\n- [2024/07] We added **FP6** support on Intel [GPU](python/llm/example/GPU/HuggingFace/More-Data-Types). \n- [2024/06] We added experimental **NPU** support for Intel Core Ultra processors; see the examples [here](python/llm/example/NPU/HF-Transformers-AutoModels). \n- [2024/06] We added extensive support of **pipeline parallel** [inference](python/llm/example/GPU/Pipeline-Parallel-Inference), which makes it easy to run large-sized LLM using 2 or more Intel GPUs (such as Arc).\n- [2024/06] We added support for running **RAGFlow** with `ipex-llm` on Intel [GPU](docs/mddocs/Quickstart/ragflow_quickstart.md).\n- [2024/05] `ipex-llm` now supports **Axolotl** for LLM finetuning on Intel GPU; see the quickstart [here](docs/mddocs/Quickstart/axolotl_quickstart.md). \n- [2024/05] You can now easily run `ipex-llm` inference, serving and finetuning using the **Docker** [images](#docker).\n- [2024/05] You can now install `ipex-llm` on Windows using just \"*[one command](docs/mddocs/Quickstart/install_windows_gpu.md#install-ipex-llm)*\".\n- [2024/04] You can now run **Open WebUI** on Intel GPU using `ipex-llm`; see the quickstart [here](docs/mddocs/Quickstart/open_webui_with_ollama_quickstart.md).\n- [2024/04] You can now run **Llama 3** on Intel GPU using `llama.cpp` and `ollama` with `ipex-llm`; see the quickstart [here](docs/mddocs/Quickstart/llama3_llamacpp_ollama_quickstart.md).\n- [2024/04] `ipex-llm` now supports **Llama 3** on both Intel [GPU](python/llm/example/GPU/HuggingFace/LLM/llama3) and [CPU](python/llm/example/CPU/HF-Transformers-AutoModels/Model/llama3).\n- [2024/04] `ipex-llm` now provides C++ interface, which can be used as an accelerated backend for running [llama.cpp](docs/mddocs/Quickstart/llama_cpp_quickstart.md) and [ollama](docs/mddocs/Quickstart/ollama_quickstart.md) on Intel GPU.\n- [2024/03] `bigdl-llm` has now become `ipex-llm` (see the migration guide [here](docs/mddocs/Quickstart/bigdl_llm_migration.md)); you may find the original `BigDL` project [here](https://github.com/intel-analytics/bigdl-2.x).\n- [2024/02] `ipex-llm` now supports directly loading model from [ModelScope](python/llm/example/GPU/ModelScope-Models) ([魔搭](python/llm/example/CPU/ModelScope-Models)).\n- [2024/02] `ipex-llm` added initial **INT2** support (based on llama.cpp [IQ2](python/llm/example/GPU/HuggingFace/Advanced-Quantizations/GGUF-IQ2) mechanism), which makes it possible to run large-sized LLM (e.g., Mixtral-8x7B) on Intel GPU with 16GB VRAM.\n- [2024/02] Users can now use `ipex-llm` through [Text-Generation-WebUI](https://github.com/intel-analytics/text-generation-webui) GUI.\n- [2024/02] `ipex-llm` now supports *[Self-Speculative Decoding](docs/mddocs/Inference/Self_Speculative_Decoding.md)*, which in practice brings **~30% speedup** for FP16 and BF16 inference latency on Intel [GPU](python/llm/example/GPU/Speculative-Decoding) and [CPU](python/llm/example/CPU/Speculative-Decoding) respectively.\n- [2024/02] `ipex-llm` now supports a comprehensive list of LLM **finetuning** on Intel GPU (including [LoRA](python/llm/example/GPU/LLM-Finetuning/LoRA), [QLoRA](python/llm/example/GPU/LLM-Finetuning/QLoRA), [DPO](python/llm/example/GPU/LLM-Finetuning/DPO), [QA-LoRA](python/llm/example/GPU/LLM-Finetuning/QA-LoRA) and [ReLoRA](python/llm/example/GPU/LLM-Finetuning/ReLora)).\n- [2024/01] Using `ipex-llm` [QLoRA](python/llm/example/GPU/LLM-Finetuning/QLoRA), we managed to finetune LLaMA2-7B in **21 minutes** and LLaMA2-70B in **3.14 hours** on 8 Intel Max 1550 GPU for [Standford-Alpaca](python/llm/example/GPU/LLM-Finetuning/QLoRA/alpaca-qlora) (see the blog [here](https://www.intel.com/content/www/us/en/developer/articles/technical/finetuning-llms-on-intel-gpus-using-bigdl-llm.html)). \n- [2023/12] `ipex-llm` now supports [ReLoRA](python/llm/example/GPU/LLM-Finetuning/ReLora) (see *[\"ReLoRA: High-Rank Training Through Low-Rank Updates\"](https://arxiv.org/abs/2307.05695)*).\n- [2023/12] `ipex-llm` now supports [Mixtral-8x7B](python/llm/example/GPU/HuggingFace/LLM/mixtral) on both Intel [GPU](python/llm/example/GPU/HuggingFace/LLM/mixtral) and [CPU](python/llm/example/CPU/HF-Transformers-AutoModels/Model/mixtral). \n- [2023/12] `ipex-llm` now supports [QA-LoRA](python/llm/example/GPU/LLM-Finetuning/QA-LoRA) (see *[\"QA-LoRA: Quantization-Aware Low-Rank Adaptation of Large Language Models\"](https://arxiv.org/abs/2309.14717)*). \n- [2023/12] `ipex-llm` now supports [FP8 and FP4 inference](python/llm/example/GPU/HuggingFace/More-Data-Types) on Intel ***GPU***.\n- [2023/11] Initial support for directly loading [GGUF](python/llm/example/GPU/HuggingFace/Advanced-Quantizations/GGUF), [AWQ](python/llm/example/GPU/HuggingFace/Advanced-Quantizations/AWQ) and [GPTQ](python/llm/example/GPU/HuggingFace/Advanced-Quantizations/GPTQ) models into `ipex-llm` is available.\n- [2023/11] `ipex-llm` now supports [vLLM continuous batching](python/llm/example/GPU/vLLM-Serving) on both Intel [GPU](python/llm/example/GPU/vLLM-Serving) and [CPU](python/llm/example/CPU/vLLM-Serving).\n- [2023/10] `ipex-llm` now supports [QLoRA finetuning](python/llm/example/GPU/LLM-Finetuning/QLoRA) on both Intel [GPU](python/llm/example/GPU/LLM-Finetuning/QLoRA) and [CPU](python/llm/example/CPU/QLoRA-FineTuning).\n- [2023/10] `ipex-llm` now supports [FastChat serving](python/llm/src/ipex_llm/llm/serving) on on both Intel CPU and GPU.\n- [2023/09] `ipex-llm` now supports [Intel GPU](python/llm/example/GPU) (including iGPU, Arc, Flex and MAX).\n- [2023/09] `ipex-llm` [tutorial](https://github.com/intel-analytics/ipex-llm-tutorial) is released.\n \n\u003c/details\u003e \n\n## `ipex-llm` Demo\n\nSee demos of running local LLMs *on Intel Core Ultra iGPU, Intel Core Ultra NPU, single-card Arc GPU, or multi-card Arc GPUs* using `ipex-llm` below.\n\n\u003ctable width=\"100%\"\u003e\n  \u003ctr\u003e\n    \u003ctd align=\"center\" colspan=\"1\"\u003e\u003cstrong\u003eIntel Core Ultra iGPU\u003c/strong\u003e\u003c/td\u003e\n    \u003ctd align=\"center\" colspan=\"1\"\u003e\u003cstrong\u003eIntel Core Ultra NPU\u003c/strong\u003e\u003c/td\u003e\n    \u003ctd align=\"center\" colspan=\"1\"\u003e\u003cstrong\u003e2-Card Intel Arc dGPUs\u003c/strong\u003e\u003c/td\u003e\n    \u003ctd align=\"center\" colspan=\"1\"\u003e\u003cstrong\u003eIntel Xeon + Arc dGPU\u003c/strong\u003e\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003e\n      \u003ca href=\"https://llm-assets.readthedocs.io/en/latest/_images/mtl_mistral-7B_q4_k_m_ollama.gif\" target=\"_blank\"\u003e\n        \u003cimg src=\"https://llm-assets.readthedocs.io/en/latest/_images/mtl_mistral-7B_q4_k_m_ollama.gif\" width=100%; /\u003e\n      \u003c/a\u003e\n    \u003c/td\u003e\n    \u003ctd\u003e\n      \u003ca href=\"https://llm-assets.readthedocs.io/en/latest/_images/npu_llama3.2-3B.gif\" target=\"_blank\"\u003e\n        \u003cimg src=\"https://llm-assets.readthedocs.io/en/latest/_images/npu_llama3.2-3B.gif\" width=100%; /\u003e\n      \u003c/a\u003e\n    \u003c/td\u003e\n    \u003ctd\u003e\n      \u003ca href=\"https://llm-assets.readthedocs.io/en/latest/_images/2arc_DeepSeek-R1-Distill-Qwen-32B-Q4_K_M.gif\" target=\"_blank\"\u003e\n        \u003cimg src=\"https://llm-assets.readthedocs.io/en/latest/_images/2arc_DeepSeek-R1-Distill-Qwen-32B-Q4_K_M.gif\" width=100%; /\u003e\n      \u003c/a\u003e\n    \u003c/td\u003e\n    \u003ctd\u003e\n      \u003ca href=\"https://llm-assets.readthedocs.io/en/latest/_images/FlashMoE-Qwen3-235B.gif\" target=\"_blank\"\u003e\n        \u003cimg src=\"https://llm-assets.readthedocs.io/en/latest/_images/FlashMoE-Qwen3-235B.gif\" width=100%; /\u003e\n      \u003c/a\u003e\n    \u003c/td\u003e    \n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd align=\"center\" width=\"25%\"\u003e\n      \u003ca href=\"docs/mddocs/Quickstart/ollama_portable_zip_quickstart.md\"\u003eOllama \u003cbr\u003e (Mistral-7B, Q4_K) \u003c/a\u003e\n    \u003c/td\u003e\n    \u003ctd align=\"center\" width=\"25%\"\u003e\n      \u003ca href=\"docs/mddocs/Quickstart/npu_quickstart.md\"\u003eHuggingFace \u003cbr\u003e (Llama3.2-3B, SYM_INT4)\u003c/a\u003e\n    \u003c/td\u003e\n    \u003ctd align=\"center\" width=\"25%\"\u003e\n      \u003ca href=\"docs/mddocs/Quickstart/llamacpp_portable_zip_gpu_quickstart.md\"\u003ellama.cpp \u003cbr\u003e (DeepSeek-R1-Distill-Qwen-32B, Q4_K)\u003c/a\u003e\n    \u003c/td\u003e\n    \u003ctd align=\"center\" width=\"25%\"\u003e\n      \u003ca href=\"docs/mddocs/Quickstart/flashmoe_quickstart.md\"\u003eFlashMoE \u003cbr\u003e (Qwen3MoE-235B, Q4_K) \u003c/a\u003e\n    \u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\n\u003c!--\nSee the demo of running [*Text-Generation-WebUI*](https://ipex-llm.readthedocs.io/en/latest/doc/LLM/Quickstart/webui_quickstart.html), [*local RAG using LangChain-Chatchat*](https://ipex-llm.readthedocs.io/en/latest/doc/LLM/Quickstart/chatchat_quickstart.html), [*llama.cpp*](https://ipex-llm.readthedocs.io/en/latest/doc/LLM/Quickstart/llamacpp_portable_zip_gpu_quickstart.md) and [*Ollama*](https://ipex-llm.readthedocs.io/en/latest/doc/LLM/Quickstart/ollama_quickstart.html) *(on either Intel Core Ultra laptop or Arc GPU)* with `ipex-llm`  below.\n\n\u003ctable width=\"100%\"\u003e\n  \u003ctr\u003e\n    \u003ctd align=\"center\" colspan=\"2\"\u003e\u003cstrong\u003eIntel Core Ultra Laptop\u003c/strong\u003e\u003c/td\u003e\n    \u003ctd align=\"center\" colspan=\"2\"\u003e\u003cstrong\u003eIntel Arc GPU\u003c/strong\u003e\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003e\n      \u003cvideo src=\"https://private-user-images.githubusercontent.com/1931082/319632616-895d56cd-e74b-4da1-b4d1-2157df341424.mp4?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTIyNDE4MjUsIm5iZiI6MTcxMjI0MTUyNSwicGF0aCI6Ii8xOTMxMDgyLzMxOTYzMjYxNi04OTVkNTZjZC1lNzRiLTRkYTEtYjRkMS0yMTU3ZGYzNDE0MjQubXA0P1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI0MDQwNCUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNDA0MDRUMTQzODQ1WiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9Y2JmYzkxYWFhMGYyN2MxYTkxOTI3MGQ2NTFkZDY4ZjFjYjg3NmZhY2VkMzVhZTU2OGEyYjhjNzI5YTFhOGNhNSZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QmYWN0b3JfaWQ9MCZrZXlfaWQ9MCZyZXBvX2lkPTAifQ.Ga8mmCAO62DFCNzU1fdoyC_4MzqhDHzjZedzmi_2L-I\" width=100% controls /\u003e\n    \u003c/td\u003e\n    \u003ctd\u003e\n      \u003cvideo src=\"https://private-user-images.githubusercontent.com/1931082/319625142-68da379e-59c6-4308-88e8-c17e40baba7b.mp4?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTIyNDA2MzQsIm5iZiI6MTcxMjI0MDMzNCwicGF0aCI6Ii8xOTMxMDgyLzMxOTYyNTE0Mi02OGRhMzc5ZS01OWM2LTQzMDgtODhlOC1jMTdlNDBiYWJhN2IubXA0P1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI0MDQwNCUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNDA0MDRUMTQxODU0WiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9NzYwOWI4MmQxZjFhMjJlNGNhZTA3MGUyZDE4OTA0N2Q2YjQ4NTcwN2M2MTY1ODAwZmE3OTIzOWI0Y2U3YzYwNyZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QmYWN0b3JfaWQ9MCZrZXlfaWQ9MCZyZXBvX2lkPTAifQ.g0bYAj3J8IJci7pLzoJI6QDalyzXzMYtQkDY7aqZMc4\" width=100% controls /\u003e\n    \u003c/td\u003e\n    \u003ctd\u003e\n      \u003cvideo src=\"https://private-user-images.githubusercontent.com/1931082/319625685-ff13b099-bcda-48f1-b11b-05421e7d386d.mp4?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTIyNDA4MTcsIm5iZiI6MTcxMjI0MDUxNywicGF0aCI6Ii8xOTMxMDgyLzMxOTYyNTY4NS1mZjEzYjA5OS1iY2RhLTQ4ZjEtYjExYi0wNTQyMWU3ZDM4NmQubXA0P1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI0MDQwNCUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNDA0MDRUMTQyMTU3WiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9MWQ3MmEwZGRkNGVlY2RkNjAzMTliODM1NDEzODU3NWQ0ZGE4MjYyOGEyZjdkMjBiZjI0MjllYTU4ODQ4YzM0NCZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QmYWN0b3JfaWQ9MCZrZXlfaWQ9MCZyZXBvX2lkPTAifQ.OFxex8Yj6WyqJKMi6B1Q19KkmbYqYCg1rD49wUwxdXQ\" width=100% controls /\u003e\n    \u003c/td\u003e\n    \u003ctd\u003e\n      \u003cvideo src=\"https://private-user-images.githubusercontent.com/1931082/325939544-2fc0ad5e-9ac7-4f95-b7b9-7885a8738443.mp4?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTQxMjYwODAsIm5iZiI6MTcxNDEyNTc4MCwicGF0aCI6Ii8xOTMxMDgyLzMyNTkzOTU0NC0yZmMwYWQ1ZS05YWM3LTRmOTUtYjdiOS03ODg1YTg3Mzg0NDMubXA0P1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI0MDQyNiUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNDA0MjZUMTAwMzAwWiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9YjZlZDE4YjFjZWJkMzQ4NmY3ZjNlMmRiYWUzMDYxMTI3YzcxYjRiYjgwNmE2NDliMjMwOTI0NWJhMDQ1NDY1YyZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QmYWN0b3JfaWQ9MCZrZXlfaWQ9MCZyZXBvX2lkPTAifQ.WfA2qwr8EP9W7a3oOYcKqaqsEKDlAkF254zbmn9dVv0\" width=100% controls /\u003e\n    \u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd align=\"center\" width=\"25%\"\u003e\n      \u003ca href=\"https://ipex-llm.readthedocs.io/en/latest/doc/LLM/Quickstart/webui_quickstart.html\"\u003eText-Generation-WebUI\u003c/a\u003e\n    \u003c/td\u003e\n    \u003ctd align=\"center\" width=\"25%\"\u003e\n      \u003ca href=\"https://ipex-llm.readthedocs.io/en/latest/doc/LLM/Quickstart/chatchat_quickstart.html\"\u003eLocal RAG using LangChain-Chatchat\u003c/a\u003e\n    \u003c/td\u003e\n    \u003ctd align=\"center\" width=\"25%\"\u003e\n      \u003ca href=\"https://ipex-llm.readthedocs.io/en/latest/doc/LLM/Quickstart/llamacpp_portable_zip_gpu_quickstart.md\"\u003ellama.cpp\u003c/a\u003e\n    \u003c/td\u003e\n    \u003ctd align=\"center\" width=\"25%\"\u003e\n      \u003ca href=\"https://ipex-llm.readthedocs.io/en/latest/doc/LLM/Quickstart/ollama_portable_zip_quickstart.md\"\u003eOllama\u003c/a\u003e\n    \u003c/td\u003e  \u003c/tr\u003e\n\u003c/table\u003e\n--\u003e\n\n## `ipex-llm` Performance\nSee the **Token Generation Speed** on *Intel Core Ultra* and *Intel Arc GPU* below[^1] (and refer to [[2]](https://www.intel.com/content/www/us/en/developer/articles/technical/accelerate-meta-llama3-with-intel-ai-solutions.html)[[3]](https://www.intel.com/content/www/us/en/developer/articles/technical/accelerate-microsoft-phi-3-models-intel-ai-soln.html)[[4]](https://www.intel.com/content/www/us/en/developer/articles/technical/intel-ai-solutions-accelerate-alibaba-qwen2-llms.html) for more details).\n\n\u003ctable width=\"100%\"\u003e\n  \u003ctr\u003e\n    \u003ctd\u003e\n      \u003ca href=\"https://llm-assets.readthedocs.io/en/latest/_images/MTL_perf.jpg\" target=\"_blank\"\u003e\n        \u003cimg src=\"https://llm-assets.readthedocs.io/en/latest/_images/MTL_perf.jpg\" width=100%; /\u003e\n      \u003c/a\u003e\n    \u003c/td\u003e\n    \u003ctd\u003e\n      \u003ca href=\"https://llm-assets.readthedocs.io/en/latest/_images/Arc_perf.jpg\" target=\"_blank\"\u003e\n        \u003cimg src=\"https://llm-assets.readthedocs.io/en/latest/_images/Arc_perf.jpg\" width=100%; /\u003e\n      \u003c/a\u003e\n    \u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\nYou may follow the [Benchmarking Guide](docs/mddocs/Quickstart/benchmark_quickstart.md) to run `ipex-llm` performance benchmark yourself.\n\n## Model Accuracy\nPlease see the **Perplexity** result below (tested on Wikitext dataset using the script [here](https://github.com/intel-analytics/ipex-llm/tree/main/python/llm/dev/benchmark/perplexity)).\n|Perplexity                 |sym_int4\t|q4_k\t  |fp6\t  |fp8_e5m2 |fp8_e4m3 |fp16   |\n|---------------------------|---------|-------|-------|---------|---------|-------|\n|Llama-2-7B-chat-hf\t        |6.364 \t  |6.218 \t|6.092 \t|6.180 \t  |6.098    |6.096  | \n|Mistral-7B-Instruct-v0.2\t  |5.365 \t  |5.320 \t|5.270 \t|5.273 \t  |5.246\t   |5.244  |\n|Baichuan2-7B-chat\t         |6.734    |6.727\t |6.527\t |6.539\t   |6.488\t   |6.508  |\n|Qwen1.5-7B-chat\t           |8.865 \t  |8.816 \t|8.557 \t|8.846 \t  |8.530    |8.607  | \n|Llama-3.1-8B-Instruct\t     |6.705\t   |6.566\t |6.338\t |6.383\t   |6.325\t   |6.267  |\n|gemma-2-9b-it\t             |7.541\t   |7.412\t |7.269\t |7.380\t   |7.268\t   |7.270  |\n|Baichuan2-13B-Chat\t        |6.313\t   |6.160\t |6.070\t |6.145\t   |6.086\t   |6.031  |\n|Llama-2-13b-chat-hf\t       |5.449\t   |5.422\t |5.341\t |5.384\t   |5.332\t   |5.329  |\n|Qwen1.5-14B-Chat\t          |7.529\t   |7.520\t |7.367\t |7.504\t   |7.297\t   |7.334  |\n\n[^1]: Performance varies by use, configuration and other factors. `ipex-llm` may not optimize to the same degree for non-Intel products. Learn more at www.Intel.com/PerformanceIndex.\n\n## `ipex-llm` Quickstart\n\n### Use\n- [Ollama](docs/mddocs/Quickstart/ollama_portable_zip_quickstart.md): running **Ollama** on Intel GPU ***without the need of manual installations***\n- [llama.cpp](docs/mddocs/Quickstart/llamacpp_portable_zip_gpu_quickstart.md): running **llama.cpp** on Intel GPU ***without the need of manual installations***\n- [Arc B580](docs/mddocs/Quickstart/bmg_quickstart.md): running `ipex-llm` on Intel Arc **B580** GPU for Ollama, llama.cpp, PyTorch, HuggingFace, etc.\n- [NPU](docs/mddocs/Quickstart/npu_quickstart.md): running `ipex-llm` on Intel **NPU** in both Python/C++ or [llama.cpp](docs/mddocs/Quickstart/llama_cpp_npu_portable_zip_quickstart.md) API.\n- [PyTorch/HuggingFace](docs/mddocs/Quickstart/install_windows_gpu.md): running **PyTorch**, **HuggingFace**, **LangChain**, **LlamaIndex**, etc. (*using Python interface of `ipex-llm`*) on Intel GPU for [Windows](docs/mddocs/Quickstart/install_windows_gpu.md) and [Linux](docs/mddocs/Quickstart/install_linux_gpu.md)\n- [vLLM](docs/mddocs/Quickstart/vLLM_quickstart.md): running `ipex-llm` in **vLLM** on both Intel [GPU](docs/mddocs/DockerGuides/vllm_docker_quickstart.md) and [CPU](docs/mddocs/DockerGuides/vllm_cpu_docker_quickstart.md)\n- [FastChat](docs/mddocs/Quickstart/fastchat_quickstart.md): running `ipex-llm` in **FastChat** serving on on both Intel GPU and CPU\n- [Serving on multiple Intel GPUs](docs/mddocs/Quickstart/deepspeed_autotp_fastapi_quickstart.md): running `ipex-llm` **serving on multiple Intel GPUs** by leveraging DeepSpeed AutoTP and FastAPI\n- [Text-Generation-WebUI](docs/mddocs/Quickstart/webui_quickstart.md): running `ipex-llm` in `oobabooga` **WebUI**\n- [Axolotl](docs/mddocs/Quickstart/axolotl_quickstart.md): running `ipex-llm` in **Axolotl** for LLM finetuning\n- [Benchmarking](docs/mddocs/Quickstart/benchmark_quickstart.md): running  (latency and throughput) **benchmarks** for `ipex-llm` on Intel CPU and GPU\n\n### Docker\n- [GPU Inference in C++](docs/mddocs/DockerGuides/docker_cpp_xpu_quickstart.md): running `llama.cpp`, `ollama`, etc., with `ipex-llm` on Intel GPU\n- [GPU Inference in Python](docs/mddocs/DockerGuides/docker_pytorch_inference_gpu.md) : running HuggingFace `transformers`, `LangChain`, `LlamaIndex`, `ModelScope`, etc. with `ipex-llm` on Intel GPU\n- [vLLM on GPU](docs/mddocs/DockerGuides/vllm_docker_quickstart.md): running `vLLM` serving with `ipex-llm` on Intel GPU\n- [vLLM on CPU](docs/mddocs/DockerGuides/vllm_cpu_docker_quickstart.md): running `vLLM` serving with `ipex-llm` on Intel CPU  \n- [FastChat on GPU](docs/mddocs/DockerGuides/fastchat_docker_quickstart.md): running `FastChat` serving with `ipex-llm` on Intel GPU\n- [VSCode on GPU](docs/mddocs/DockerGuides/docker_run_pytorch_inference_in_vscode.md): running and developing `ipex-llm` applications in Python using VSCode on Intel GPU\n\n### Applications\n- [GraphRAG](docs/mddocs/Quickstart/graphrag_quickstart.md): running Microsoft's `GraphRAG` using local LLM with `ipex-llm`\n- [RAGFlow](docs/mddocs/Quickstart/ragflow_quickstart.md): running `RAGFlow` (*an open-source RAG engine*) with `ipex-llm`\n- [LangChain-Chatchat](docs/mddocs/Quickstart/chatchat_quickstart.md): running `LangChain-Chatchat` (*Knowledge Base QA using RAG pipeline*) with `ipex-llm`\n- [Coding copilot](docs/mddocs/Quickstart/continue_quickstart.md): running `Continue` (coding copilot in VSCode) with `ipex-llm`\n- [Open WebUI](docs/mddocs/Quickstart/open_webui_with_ollama_quickstart.md): running `Open WebUI` with `ipex-llm`\n- [PrivateGPT](docs/mddocs/Quickstart/privateGPT_quickstart.md): running `PrivateGPT` to interact with documents with `ipex-llm`\n- [Dify platform](docs/mddocs/Quickstart/dify_quickstart.md): running `ipex-llm` in `Dify`(*production-ready LLM app development platform*)\n\n### Install \n- [Windows GPU](docs/mddocs/Quickstart/install_windows_gpu.md): installing `ipex-llm` on Windows with Intel GPU\n- [Linux GPU](docs/mddocs/Quickstart/install_linux_gpu.md): installing `ipex-llm` on Linux with Intel GPU\n- *For more details, please refer to the [full installation guide](docs/mddocs/Overview/install.md)*\n\n### Code Examples\n- #### Low bit inference\n  - [INT4 inference](python/llm/example/GPU/HuggingFace/LLM): **INT4** LLM inference on Intel [GPU](python/llm/example/GPU/HuggingFace/LLM) and [CPU](python/llm/example/CPU/HF-Transformers-AutoModels/Model)\n  - [FP8/FP6/FP4 inference](python/llm/example/GPU/HuggingFace/More-Data-Types): **FP8**, **FP6** and **FP4** LLM inference on Intel [GPU](python/llm/example/GPU/HuggingFace/More-Data-Types)\n  - [INT8 inference](python/llm/example/GPU/HuggingFace/More-Data-Types): **INT8** LLM inference on Intel [GPU](python/llm/example/GPU/HuggingFace/More-Data-Types) and [CPU](python/llm/example/CPU/HF-Transformers-AutoModels/More-Data-Types)\n  - [INT2 inference](python/llm/example/GPU/HuggingFace/Advanced-Quantizations/GGUF-IQ2): **INT2** LLM inference (based on llama.cpp IQ2 mechanism) on Intel [GPU](python/llm/example/GPU/HuggingFace/Advanced-Quantizations/GGUF-IQ2)\n- #### FP16/BF16 inference\n  - **FP16** LLM inference on Intel [GPU](python/llm/example/GPU/Speculative-Decoding), with possible [self-speculative decoding](docs/mddocs/Inference/Self_Speculative_Decoding.md) optimization\n  - **BF16** LLM inference on Intel [CPU](python/llm/example/CPU/Speculative-Decoding), with possible [self-speculative decoding](docs/mddocs/Inference/Self_Speculative_Decoding.md) optimization\n- #### Distributed inference\n  - **Pipeline Parallel** inference on Intel [GPU](python/llm/example/GPU/Pipeline-Parallel-Inference)\n  - **DeepSpeed AutoTP** inference on Intel [GPU](python/llm/example/GPU/Deepspeed-AutoTP)\n- #### Save and load\n  - [Low-bit models](python/llm/example/CPU/HF-Transformers-AutoModels/Save-Load): saving and loading `ipex-llm` low-bit models (INT4/FP4/FP6/INT8/FP8/FP16/etc.)\n  - [GGUF](python/llm/example/GPU/HuggingFace/Advanced-Quantizations/GGUF): directly loading GGUF models into `ipex-llm`\n  - [AWQ](python/llm/example/GPU/HuggingFace/Advanced-Quantizations/AWQ): directly loading AWQ models into `ipex-llm`\n  - [GPTQ](python/llm/example/GPU/HuggingFace/Advanced-Quantizations/GPTQ): directly loading GPTQ models into `ipex-llm`\n- #### Finetuning\n  - LLM finetuning on Intel [GPU](python/llm/example/GPU/LLM-Finetuning), including [LoRA](python/llm/example/GPU/LLM-Finetuning/LoRA), [QLoRA](python/llm/example/GPU/LLM-Finetuning/QLoRA), [DPO](python/llm/example/GPU/LLM-Finetuning/DPO), [QA-LoRA](python/llm/example/GPU/LLM-Finetuning/QA-LoRA) and [ReLoRA](python/llm/example/GPU/LLM-Finetuning/ReLora)\n  - QLoRA finetuning on Intel [CPU](python/llm/example/CPU/QLoRA-FineTuning)\n- #### Integration with community libraries\n  - [HuggingFace transformers](python/llm/example/GPU/HuggingFace)\n  - [Standard PyTorch model](python/llm/example/GPU/PyTorch-Models)\n  - [LangChain](python/llm/example/GPU/LangChain)\n  - [LlamaIndex](python/llm/example/GPU/LlamaIndex)\n  - [DeepSpeed-AutoTP](python/llm/example/GPU/Deepspeed-AutoTP)\n  - [Axolotl](docs/mddocs/Quickstart/axolotl_quickstart.md)\n  - [HuggingFace PEFT](python/llm/example/GPU/LLM-Finetuning/HF-PEFT)\n  - [HuggingFace TRL](python/llm/example/GPU/LLM-Finetuning/DPO)\n  - [AutoGen](python/llm/example/CPU/Applications/autogen)\n  - [ModeScope](python/llm/example/GPU/ModelScope-Models)\n- [Tutorials](https://github.com/intel-analytics/ipex-llm-tutorial)\n\n## API Doc\n- [HuggingFace Transformers-style API (Auto Classes)](docs/mddocs/PythonAPI/transformers.md)\n- [API for arbitrary PyTorch Model](https://github.com/intel-analytics/ipex-llm/blob/main/docs/mddocs/PythonAPI/optimize.md)\n\n## FAQ\n- [FAQ \u0026 Trouble Shooting](docs/mddocs/Overview/FAQ/faq.md)\n\n## Verified Models\nOver 70 models have been optimized/verified on `ipex-llm`, including *LLaMA/LLaMA2, Mistral, Mixtral, Gemma, LLaVA, Whisper, ChatGLM2/ChatGLM3, Baichuan/Baichuan2, Qwen/Qwen-1.5, InternLM* and more; see the list below.\n  \n| Model      | CPU Example                                  | GPU Example                                  | NPU Example                                  |\n|------------|----------------------------------------------|----------------------------------------------|----------------------------------------------|\n| LLaMA  | [link1](python/llm/example/CPU/Native-Models), [link2](python/llm/example/CPU/HF-Transformers-AutoModels/Model/vicuna) |[link](python/llm/example/GPU/HuggingFace/LLM/vicuna)|\n| LLaMA 2    | [link1](python/llm/example/CPU/Native-Models), [link2](python/llm/example/CPU/HF-Transformers-AutoModels/Model/llama2) | [link](python/llm/example/GPU/HuggingFace/LLM/llama2)  | [Python link](python/llm/example/NPU/HF-Transformers-AutoModels/LLM), [C++ link](python/llm/example/NPU/HF-Transformers-AutoModels/LLM/CPP_Examples) |\n| LLaMA 3    | [link](python/llm/example/CPU/HF-Transformers-AutoModels/Model/llama3) | [link](python/llm/example/GPU/HuggingFace/LLM/llama3)  | [Python link](python/llm/example/NPU/HF-Transformers-AutoModels/LLM), [C++ link](python/llm/example/NPU/HF-Transformers-AutoModels/LLM/CPP_Examples) |\n| LLaMA 3.1    | [link](python/llm/example/CPU/HF-Transformers-AutoModels/Model/llama3.1) | [link](python/llm/example/GPU/HuggingFace/LLM/llama3.1)  |\n| LLaMA 3.2    |  | [link](python/llm/example/GPU/HuggingFace/LLM/llama3.2)  | [Python link](python/llm/example/NPU/HF-Transformers-AutoModels/LLM), [C++ link](python/llm/example/NPU/HF-Transformers-AutoModels/LLM/CPP_Examples) |\n| LLaMA 3.2-Vision    |  | [link](python/llm/example/GPU/PyTorch-Models/Model/llama3.2-vision/)  |\n| ChatGLM    | [link](python/llm/example/CPU/HF-Transformers-AutoModels/Model/chatglm)   |    | \n| ChatGLM2   | [link](python/llm/example/CPU/HF-Transformers-AutoModels/Model/chatglm2)  | [link](python/llm/example/GPU/HuggingFace/LLM/chatglm2)   |\n| ChatGLM3   | [link](python/llm/example/CPU/HF-Transformers-AutoModels/Model/chatglm3)  | [link](python/llm/example/GPU/HuggingFace/LLM/chatglm3)   |\n| GLM-4      | [link](python/llm/example/CPU/HF-Transformers-AutoModels/Model/glm4)      | [link](python/llm/example/GPU/HuggingFace/LLM/glm4)       |\n| GLM-4V     | [link](python/llm/example/CPU/HF-Transformers-AutoModels/Model/glm-4v)    | [link](python/llm/example/GPU/HuggingFace/Multimodal/glm-4v)     |\n| GLM-Edge   |  | [link](python/llm/example/GPU/HuggingFace/LLM/glm-edge) | [Python link](python/llm/example/NPU/HF-Transformers-AutoModels/LLM) |\n| GLM-Edge-V   |  | [link](python/llm/example/GPU/HuggingFace/Multimodal/glm-edge-v) |\n| Mistral    | [link](python/llm/example/CPU/HF-Transformers-AutoModels/Model/mistral)   | [link](python/llm/example/GPU/HuggingFace/LLM/mistral)    |\n| Mixtral    | [link](python/llm/example/CPU/HF-Transformers-AutoModels/Model/mixtral)   | [link](python/llm/example/GPU/HuggingFace/LLM/mixtral)    |\n| Falcon     | [link](python/llm/example/CPU/HF-Transformers-AutoModels/Model/falcon)    | [link](python/llm/example/GPU/HuggingFace/LLM/falcon)     |\n| MPT        | [link](python/llm/example/CPU/HF-Transformers-AutoModels/Model/mpt)       | [link](python/llm/example/GPU/HuggingFace/LLM/mpt)        |\n| Dolly-v1   | [link](python/llm/example/CPU/HF-Transformers-AutoModels/Model/dolly_v1)  | [link](python/llm/example/GPU/HuggingFace/LLM/dolly-v1)   | \n| Dolly-v2   | [link](python/llm/example/CPU/HF-Transformers-AutoModels/Model/dolly_v2)  | [link](python/llm/example/GPU/HuggingFace/LLM/dolly-v2)   | \n| Replit Code| [link](python/llm/example/CPU/HF-Transformers-AutoModels/Model/replit)    | [link](python/llm/example/GPU/HuggingFace/LLM/replit)     |\n| RedPajama  | [link1](python/llm/example/CPU/Native-Models), [link2](python/llm/example/CPU/HF-Transformers-AutoModels/Model/redpajama) |    | \n| Phoenix    | [link1](python/llm/example/CPU/Native-Models), [link2](python/llm/example/CPU/HF-Transformers-AutoModels/Model/phoenix)   |    | \n| StarCoder  | [link1](python/llm/example/CPU/Native-Models), [link2](python/llm/example/CPU/HF-Transformers-AutoModels/Model/starcoder) | [link](python/llm/example/GPU/HuggingFace/LLM/starcoder) | \n| Baichuan   | [link](python/llm/example/CPU/HF-Transformers-AutoModels/Model/baichuan)  | [link](python/llm/example/GPU/HuggingFace/LLM/baichuan)   |\n| Baichuan2  | [link](python/llm/example/CPU/HF-Transformers-AutoModels/Model/baichuan2) | [link](python/llm/example/GPU/HuggingFace/LLM/baichuan2)  | [Python link](python/llm/example/NPU/HF-Transformers-AutoModels/LLM) |\n| InternLM   | [link](python/llm/example/CPU/HF-Transformers-AutoModels/Model/internlm)  | [link](python/llm/example/GPU/HuggingFace/LLM/internlm)   |\n| InternVL2   |   | [link](python/llm/example/GPU/HuggingFace/Multimodal/internvl2)   |\n| Qwen       | [link](python/llm/example/CPU/HF-Transformers-AutoModels/Model/qwen)      | [link](python/llm/example/GPU/HuggingFace/LLM/qwen)       |\n| Qwen1.5 | [link](python/llm/example/CPU/HF-Transformers-AutoModels/Model/qwen1.5) | [link](python/llm/example/GPU/HuggingFace/LLM/qwen1.5) |\n| Qwen2 | [link](python/llm/example/CPU/HF-Transformers-AutoModels/Model/qwen2) | [link](python/llm/example/GPU/HuggingFace/LLM/qwen2) | [Python link](python/llm/example/NPU/HF-Transformers-AutoModels/LLM), [C++ link](python/llm/example/NPU/HF-Transformers-AutoModels/LLM/CPP_Examples) |\n| Qwen2.5 |  | [link](python/llm/example/GPU/HuggingFace/LLM/qwen2.5) | [Python link](python/llm/example/NPU/HF-Transformers-AutoModels/LLM), [C++ link](python/llm/example/NPU/HF-Transformers-AutoModels/LLM/CPP_Examples) |\n| Qwen-VL    | [link](python/llm/example/CPU/HF-Transformers-AutoModels/Model/qwen-vl)   | [link](python/llm/example/GPU/HuggingFace/Multimodal/qwen-vl)    |\n| Qwen2-VL    || [link](python/llm/example/GPU/HuggingFace/Multimodal/qwen2-vl)    |\n| Qwen2-Audio    |  | [link](python/llm/example/GPU/HuggingFace/Multimodal/qwen2-audio)    |\n| Aquila     | [link](python/llm/example/CPU/HF-Transformers-AutoModels/Model/aquila)    | [link](python/llm/example/GPU/HuggingFace/LLM/aquila)     |\n| Aquila2     | [link](python/llm/example/CPU/HF-Transformers-AutoModels/Model/aquila2)    | [link](python/llm/example/GPU/HuggingFace/LLM/aquila2)     |\n| MOSS       | [link](python/llm/example/CPU/HF-Transformers-AutoModels/Model/moss)      |    | \n| Whisper    | [link](python/llm/example/CPU/HF-Transformers-AutoModels/Model/whisper)   | [link](python/llm/example/GPU/HuggingFace/Multimodal/whisper)    |\n| Phi-1_5    | [link](python/llm/example/CPU/HF-Transformers-AutoModels/Model/phi-1_5)   | [link](python/llm/example/GPU/HuggingFace/LLM/phi-1_5)    |\n| Flan-t5    | [link](python/llm/example/CPU/HF-Transformers-AutoModels/Model/flan-t5)   | [link](python/llm/example/GPU/HuggingFace/LLM/flan-t5)    |\n| LLaVA      | [link](python/llm/example/CPU/PyTorch-Models/Model/llava)                 | [link](python/llm/example/GPU/PyTorch-Models/Model/llava)                  |\n| CodeLlama  | [link](python/llm/example/CPU/HF-Transformers-AutoModels/Model/codellama) | [link](python/llm/example/GPU/HuggingFace/LLM/codellama)  |\n| Skywork      | [link](python/llm/example/CPU/HF-Transformers-AutoModels/Model/skywork)                 |    |\n| InternLM-XComposer  | [link](python/llm/example/CPU/HF-Transformers-AutoModels/Model/internlm-xcomposer)   |    |\n| WizardCoder-Python | [link](python/llm/example/CPU/HF-Transformers-AutoModels/Model/wizardcoder-python) | |\n| CodeShell | [link](python/llm/example/CPU/HF-Transformers-AutoModels/Model/codeshell) | |\n| Fuyu      | [link](python/llm/example/CPU/HF-Transformers-AutoModels/Model/fuyu) | |\n| Distil-Whisper | [link](python/llm/example/CPU/HF-Transformers-AutoModels/Model/distil-whisper) | [link](python/llm/example/GPU/HuggingFace/Multimodal/distil-whisper) |\n| Yi | [link](python/llm/example/CPU/HF-Transformers-AutoModels/Model/yi) | [link](python/llm/example/GPU/HuggingFace/LLM/yi) |\n| BlueLM | [link](python/llm/example/CPU/HF-Transformers-AutoModels/Model/bluelm) | [link](python/llm/example/GPU/HuggingFace/LLM/bluelm) |\n| Mamba | [link](python/llm/example/CPU/PyTorch-Models/Model/mamba) | [link](python/llm/example/GPU/PyTorch-Models/Model/mamba) |\n| SOLAR | [link](python/llm/example/CPU/HF-Transformers-AutoModels/Model/solar) | [link](python/llm/example/GPU/HuggingFace/LLM/solar) |\n| Phixtral | [link](python/llm/example/CPU/HF-Transformers-AutoModels/Model/phixtral) | [link](python/llm/example/GPU/HuggingFace/LLM/phixtral) |\n| InternLM2 | [link](python/llm/example/CPU/HF-Transformers-AutoModels/Model/internlm2) | [link](python/llm/example/GPU/HuggingFace/LLM/internlm2) |\n| RWKV4 |  | [link](python/llm/example/GPU/HuggingFace/LLM/rwkv4) |\n| RWKV5 |  | [link](python/llm/example/GPU/HuggingFace/LLM/rwkv5) |\n| Bark | [link](python/llm/example/CPU/PyTorch-Models/Model/bark) | [link](python/llm/example/GPU/PyTorch-Models/Model/bark) |\n| SpeechT5 |  | [link](python/llm/example/GPU/PyTorch-Models/Model/speech-t5) |\n| DeepSeek-MoE | [link](python/llm/example/CPU/HF-Transformers-AutoModels/Model/deepseek-moe) |  |\n| Ziya-Coding-34B-v1.0 | [link](python/llm/example/CPU/HF-Transformers-AutoModels/Model/ziya) | |\n| Phi-2 | [link](python/llm/example/CPU/HF-Transformers-AutoModels/Model/phi-2) | [link](python/llm/example/GPU/HuggingFace/LLM/phi-2) |\n| Phi-3 | [link](python/llm/example/CPU/HF-Transformers-AutoModels/Model/phi-3) | [link](python/llm/example/GPU/HuggingFace/LLM/phi-3) |\n| Phi-3-vision | [link](python/llm/example/CPU/HF-Transformers-AutoModels/Model/phi-3-vision) | [link](python/llm/example/GPU/HuggingFace/Multimodal/phi-3-vision) |\n| Yuan2 | [link](python/llm/example/CPU/HF-Transformers-AutoModels/Model/yuan2) | [link](python/llm/example/GPU/HuggingFace/LLM/yuan2) |\n| Gemma | [link](python/llm/example/CPU/HF-Transformers-AutoModels/Model/gemma) | [link](python/llm/example/GPU/HuggingFace/LLM/gemma) |\n| Gemma2 |  | [link](python/llm/example/GPU/HuggingFace/LLM/gemma2) |\n| DeciLM-7B | [link](python/llm/example/CPU/HF-Transformers-AutoModels/Model/deciLM-7b) | [link](python/llm/example/GPU/HuggingFace/LLM/deciLM-7b) |\n| Deepseek | [link](python/llm/example/CPU/HF-Transformers-AutoModels/Model/deepseek) | [link](python/llm/example/GPU/HuggingFace/LLM/deepseek) |\n| StableLM | [link](python/llm/example/CPU/HF-Transformers-AutoModels/Model/stablelm) | [link](python/llm/example/GPU/HuggingFace/LLM/stablelm) |\n| CodeGemma | [link](python/llm/example/CPU/HF-Transformers-AutoModels/Model/codegemma) | [link](python/llm/example/GPU/HuggingFace/LLM/codegemma) |\n| Command-R/cohere | [link](python/llm/example/CPU/HF-Transformers-AutoModels/Model/cohere) | [link](python/llm/example/GPU/HuggingFace/LLM/cohere) |\n| CodeGeeX2 | [link](python/llm/example/CPU/HF-Transformers-AutoModels/Model/codegeex2) | [link](python/llm/example/GPU/HuggingFace/LLM/codegeex2) |\n| MiniCPM | [link](python/llm/example/CPU/HF-Transformers-AutoModels/Model/minicpm) | [link](python/llm/example/GPU/HuggingFace/LLM/minicpm) | [Python link](python/llm/example/NPU/HF-Transformers-AutoModels/LLM), [C++ link](python/llm/example/NPU/HF-Transformers-AutoModels/LLM/CPP_Examples) |\n| MiniCPM3 |  | [link](python/llm/example/GPU/HuggingFace/LLM/minicpm3) |\n| MiniCPM-V |  | [link](python/llm/example/GPU/HuggingFace/Multimodal/MiniCPM-V) |\n| MiniCPM-V-2 | [link](python/llm/example/CPU/HF-Transformers-AutoModels/Model/minicpm-v-2) | [link](python/llm/example/GPU/HuggingFace/Multimodal/MiniCPM-V-2) |\n| MiniCPM-Llama3-V-2_5 |  | [link](python/llm/example/GPU/HuggingFace/Multimodal/MiniCPM-Llama3-V-2_5) | [Python link](python/llm/example/NPU/HF-Transformers-AutoModels/Multimodal) |\n| MiniCPM-V-2_6 | [link](python/llm/example/CPU/HF-Transformers-AutoModels/Model/minicpm-v-2_6) | [link](python/llm/example/GPU/HuggingFace/Multimodal/MiniCPM-V-2_6) | [Python link](python/llm/example/NPU/HF-Transformers-AutoModels/Multimodal) |\n| MiniCPM-o-2_6 | | [link](python/llm/example/GPU/HuggingFace/Multimodal/MiniCPM-o-2_6/) |\n| Janus-Pro | | [link](python/llm/example/GPU/HuggingFace/Multimodal/janus-pro/) |\n| Moonlight | |[link](python/llm/example/GPU/HuggingFace/LLM/moonlight/) |\n| StableDiffusion | | [link](python/llm/example/GPU/HuggingFace/Multimodal/StableDiffusion) |\n| Bce-Embedding-Base-V1 | | | [Python link](python/llm/example/NPU/HF-Transformers-AutoModels/Embedding) |\n| Speech_Paraformer-Large | | | [Python link](python/llm/example/NPU/HF-Transformers-AutoModels/Multimodal) |\n\n## Get Support\n- Please report a bug or raise a feature request by opening a [Github Issue](https://github.com/intel-analytics/ipex-llm/issues)\n- Please report a vulnerability by opening a draft [GitHub Security Advisory](https://github.com/intel-analytics/ipex-llm/security/advisories)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fintel%2Fipex-llm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fintel%2Fipex-llm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fintel%2Fipex-llm/lists"}