Apple has finally debuted CoreAI, which is a successor to its CoreML engine that had reigned supreme for around 9 years, bringing with it format-agnostic inferencing and support for large-model memory footprints. Even so, initial tests are painting a much more nuanced picture of Apple's new AI framework and, in turn, its on-device models. New benchmark tests show Apple's CoreAI "converges to a near-tie [with MLX] at a realistic 8B" model size for decoding For the benefit of those who might not be aware, Apple launched its CoreML machine-learning framework back in 2017 to primarily run smaller, static machine-learning tasks […]

Read full article at https://wccftech.com/apples-new-coreai-engine-barely-edges-out-its-own-mlx-framework-at-realistic-8b-model-sizes-despite-being-2-47x-faster-on-tiny-models/