clip : use smart pointer (⚠️ breaking change)

[ngxson]

Summarize this change

Breaking change to clip.cpp

The struct clip_image_f32_batch now become an opaque pointer. To access the underlay data, use these API calls:

size_t clip_image_f32_batch_n_images(const struct clip_image_f32_batch * batch); // equivalent to batch->size()
size_t clip_image_f32_batch_nx(const struct clip_image_f32_batch * batch, int idx); // equivalent to batch[idx]->nx
size_t clip_image_f32_batch_ny(const struct clip_image_f32_batch * batch, int idx); // equivalent to batch[idx]->ny
clip_image_f32 * clip_image_f32_get_img(const struct clip_image_f32_batch * batch, int idx); // equivalent to batch[idx]->data

Note: The struct clip_image_u8_batch is also became an opaque pointer, but I cannot find any code actually using it. If this change affect you, please leave a comment in this PR.

Note 2: If your existing code initialize clip_image_f32_batch on heap, here is how to migrate:

// old code:
clip_image_f32_batch batch_f32;
clip_image_preprocess(ctx.ctx_clip, img_u8, &batch_f32);
clip_image_batch_encode(ctx.ctx_clip, ctx.n_threads, &batch_f32, ...);
clip_image_f32_batch_free(&batch_f32);

// new code:
clip_image_f32_batch * batch_f32 = clip_image_f32_init();
clip_image_preprocess(ctx.ctx_clip, img_u8, batch_f32); // no more "&"
clip_image_batch_encode(ctx.ctx_clip, ctx.n_threads, batch_f32, ...); // no more "&"
clip_image_f32_batch_free(batch_f32); // no more "&"

Test

OK:   llama-gemma3-cli ggml-org/gemma-3-4b-it-GGUF:Q4_K_M
OK:   llama-llava-cli cmp-nct/Yi-VL-6B-GGUF:Q5_K
OK:   llama-llava-cli guinmoon/MobileVLM-3B-GGUF:Q4_K_M
OK:   llama-llava-cli THUDM/glm-edge-v-5b-gguf:Q4_K_M
OK:   llama-llava-cli second-state/Llava-v1.5-7B-GGUF:Q2_K
OK:   llama-llava-cli cjpais/llava-1.6-mistral-7b-gguf:Q3_K
OK:   llama-llava-cli ibm-research/granite-vision-3.2-2b-GGUF:Q4_K_M
OK:   llama-minicpmv-cli second-state/MiniCPM-Llama3-V-2_5-GGUF:Q2_K
OK:   llama-minicpmv-cli openbmb/MiniCPM-V-2_6-gguf:Q2_K
OK:   llama-minicpmv-cli openbmb/MiniCPM-o-2_6-gguf:Q4_0
OK:   llama-qwen2vl-cli bartowski/Qwen2-VL-2B-Instruct-GGUF:Q4_K_M

[zhouwg]

you are a highly-skilled and an interesting AI tech expert.hope can get help/guidance from you or collaborate with you in the future.

[ggerganov]

Inheritance from STL classes is generally not recommended. Although it's tempting, I think it would be better to avoid establishing this practice, as it can lead to subtle issues.

The recommended way is to use composition instead.

[ngxson]

Thanks, that's good to know! (And I will also need to do the same on my llama_batch_ext PR)

With the help of gemini 3.5 pro, I added the composite in f8c5394

[ggerganov]

I was thinking about something much simpler like:

struct clip_image_u8_batch {
    std::vector<clip_image_u8_ptr> entries;
};

// ...
// end then using directly the `entries` member in the code
batch.entries.clear();
batch.entries.push_back(...);

[ngxson]

Yes thanks, I simplify this in f727124

The clip_image_u8_ptr is also no longer being initialized by default, I think it's not a big deal, reduce the complexity of the wrapper

[ggerganov]

(And I will also need to do the same on my llama_batch_ext PR)

Yup, I will get back to this PR after some refactoring in libllama. We had a discussion with @slaren how to improve on it.

[acbits]

Thanks, that's good to know! (And I will also need to do the same on my llama_batch_ext PR)

With the help of gemini 3.5 pro, I added the composite in f8c5394

Inheriting from std::unique_ptr is very strange usage of C++ pointers.

Also, std::shared_ptr might be a better option than std::unique_ptr as the former avoids dangling references. The overhead is usually not significant.

[ngxson]

Tbh unique_ptr gives me more confident when writing code, kinda rust-like experience

[acbits]

Tbh unique_ptr gives me more confident when writing code, kinda rust-like experience

No. They are not equivalent to rust semantics.

[ngxson]

I said rust-like experience, not rust-equivalent semantics. unique_ptr is useful in my case because the compiler will complain about unique_ptr being non-copiable, so I have total control over the life cycle of the object owned by unique_ptr

[acbits]

Makes sense. Just be aware of
.get() usage as it could create dangling pointers despite unique ownership.