Intel Releases OpenVINO 2024.1 With More Gen AI & LLM Features

[phoronix]

Phoronix: Intel Releases OpenVINO 2024.1 With More Gen AI & LLM Features

Intel engineers have just released OpenVINO 2024.1, the newest feature release for this excellent open-source AI toolkit that continues expanding its features and capabilities particularly around Generative AI "GenAI" and Large Language Models (LLMs)...

https://www.phoronix.com/news/Intel-OpenVINO-2024.1-Released

[pong]

Looks interesting.

Since they released it the same day as the ubuntu 24.04LTS, I thought they'd probably (hopefully) have support for that OS ready in
the updates for openvino 2024.1 though I don't see any corresponding ubuntu24 docker images from openvino at this time.
Hopefully soon.

I'll also have to see whether the ubuntu24 targeted openvino / one-api / level-zero compute related packages are up on the intel repo., too.

[Eirikr1848]

Hmmm does this support using multiple neural compute sticks for local LLM?

A couple NCS2, 128GB RAM + Arc A770 16GB + … a Ryzen 5600X3D could yield interesting results of all can be pooled.