In the era of big models, where will general visual models go?

You may have noticed that the pace of visual AI in the past two years is a bit like squeezing the subway in the morning: crowded, chaotic, but indispensable. In the context of the continuous high-frequency words "big model era", the general visual model is no longer just a noun in scientific research papers, but like the old acquaintance sitting in a corner coffee shop, it seems familiar but is becoming increasingly difficult to define. Their direction is not only related to technological breakthroughs, but also to the "reshuffle" of semantic understanding, product integration and even business logic.

Where is the "general vision model" common?

Don't rush to give a definition. I remember that in mid-2023, when Meta Vision Transformer (ViT) became popular, a friend said in a WeChat group, "To put it bluntly, it is the GPT in the visual world, you can see anything, but you can't see anything." It sounds like a complaint, but I feel more and more that this sentence is intriguing.

The so-called "universal" is a false proposition if it is not implemented in the scenario. It can identify cats, dogs, cars, boats, and boats, and can divide images and understand depth, and add some language skills. It sounds like a panacea - but if you really use it for industrial testing, it may not even be able to tell a screw. The universality of the big model is a hypothesis to some extent: we hope it understands everything, but it just doesn't specialize in anything.

A re-division of "semantic territory"

OpenAI has created CLIP, Meta has launched SAM, Google has launched Gemini Vision... This round of visual models has made it not just that you can see clearly, but that you can "speak out what you see." From recognition to understanding , from pixel layer to semantic layer, is the real leap behind this wave of general visual models.

But the question also follows: Is the fusion of multimodal really "integrated"? Or is it just the language model playing the side of the "translated" image? I once talked to a former colleague who was engaged in multimodal research and development. He said that the current general visual model is actually "pretending to understand what you said, but in fact it just learned how to reply to you."

"Bottleneck Bias" in General Visual Models

Among many public test data, such as ImageNet, COCO, etc., the performance of the general model is amazing. But you change your thinking - use these models to deal with a local scenario, such as the night recognition task of coastal port monitoring in Guangdong, and the accuracy rate becomes "inconsistent".

This is not a bug, it is an inevitable result of design logic. Universality and professionalism are one thing that goes up and down . The more you want it to be "complete", the harder it will be to refine it in a certain field. The deviation of data distribution, the difference in lighting conditions, the blur of semantic boundaries... These variables have become three mountains pressing on the head of the general model.

A "industry impatient" moment is approaching

To be honest, the model team is also anxious. I recently attended a closed-door AI vision entrepreneurship meeting. The most discussed thing is not "We can still call out how many great models we can call out", but "Why should users still believe that we can solve practical problems?"

An industrial vision project team even said bluntly: "We now prefer small models + rule engine combinations. The large models are too heavy and the reasoning is too slow, so users don't buy it." Isn't it a bit ironic? The "Vision of Unification" that once boasted to heaven has now been pulled back to the ground by the real user's "Don't block my device, thank God."

The dispute between "specialized but not connected" vs. "specialized but not connected"

Have you noticed a phenomenon: more and more AI projects are turning to the "special model route". Medical images have BioGPT+Vision modules, smart security has a customized Light-ViT combination, and even e-commerce platforms prefer small models that accurately identify product materials.

This actually reveals a new market orientation: "pseudo-universality" is no longer sexy , and everyone is more concerned about ROI, whether it can be truly implemented. Not to mention the "thousand-model war", some companies have quietly cut off the original planned general model deployment and switched to edge computing and back-end lightweight fusion.

The fork of the technical route: multimodal or task expert?

There are currently two trends in the industry: one advocates continuing to promote the unified model of "visual-language-sound-action" and create a "super center"; the other believes that instead of building a giant beast to swallow the sky, it is better to let each model perform its own duties and work hard.

What do you think, do you ask me? I prefer the latter, at least at the implementation level. Just like a car does not include all the functions of transportation, but has trucks, cars, and motorcycles, each with its own advantages in scenarios.

However, it is not that general models have no prospects. They are still suitable for use in the "cold start stage", especially in new scenarios without labeling data, which can quickly build prototypes and feedback mechanisms, and the value of universality will be amplified. But in the period of refined operation, professional models are still available.

Future direction: "Mixed Model" may be a grayscale solution

We see more and more platforms starting to try "model combination strategy": use general model rough screening on the front end, and in-depth analysis of expert models in the back end calling field. Just like the double-explorer partner in the movie, one advocates synesthesia and the other relies on the rules of thumb, and only when combined is truly efficient.

There are even teams trying out the "Prompt Engineering" of visual models, simulating expert behavior by adjusting context input. This method does not rely on new training, but rather a secondary activation of the general model - flexible, but also full of experimental risks.

Technology is the wave, and scene is the coastline

To put it bluntly, all imaginations of "general" will eventually hit the wall of "specific needs". No model can be universal forever, just like no one can speak all languages. Instead of obsessing with "big and complete", it is better to build a visual ecosystem of "flexible integration + on-demand matching". This may be the real way out for general visual models in the next few years.

But then again, have you ever thought about it - when a big model can look at pictures, read texts, decode sounds, deduce cause and effect, and output code... At that time, is "vision" itself no longer a separate concept?

Or, will the future of general visual models be that "visual models no longer exist"?

When you take a photo with your mobile phone one day, AI will not only tell you what it is, but also remind you that "the last time you saw this type of item was in Shanghai, and then you were looking for a project reference" - that is the real universal intelligence. And today's model is just still on the way.

What do you think? Would you rather use a visual model that "knows everything but is always a little slower", or would you rather believe in those "specialized but fast" model assistants?

Welcome to leave a message, let’s continue to talk - this matter is not so quickly, but technology should not only determine the direction.