Opening article
Hugging Face has added a new integration to JetBrains PyCharm that lets developers insert pre‑trained models directly into their code. By right‑clicking in the editor and selecting "Insert HF Model", users can choose from a list of image‑text‑to‑text models, such as microsoft/Phi-3.5-vision-instruct, and automatically paste sample code or view the full model card. The integration also provides a cache manager accessed via a 🤗 icon, allowing users to see, clear, or reuse downloaded models. The feature is available only in PyCharm Professional and is promoted as a quick way to add state‑of‑the‑art open‑source AI to applications.
It’s a Tuesday morning. As a Transformers maintainer, I’m doing the same thing I do most weekday mornings: Opening PyCharm , loading up the Transformers codebase and gazing lovingly at the chat template documentation while ignoring the 50 user issues I was pinged on that day. But this time, something feels different:
Something is… wait! Computer! Enhance! Is that..? Those user issues are definitely not getting responses today. Let’s talk about the Hugging Face integration in PyCharm.
I could introduce this integration by just listing features, but that’s boring and there’s documentation for that. Instead, let’s walk through how we’d use it all in practice. Let’s say I’m writing a Python app, and I decide I want the app to be able to chat with users. Not just text chat, though – we want the users to be able to paste in images too, and for the app to naturally chat about them as well.
At Hugging Face, we tend to think of open-source AI as being a natural extension of the open-source philosophy: Open software solves problems for developers and users, creating new abilities for them to integrate into their code, and open models do the same. There is a tendency to be blinded by complexity, and to focus too much on the implementation details because they’re all so novel and exciting, but models exist to do stuff for you. If you abstract away the details of architecture and training, they’re fundamentally functions - tools in your code that will transform a certain kind of input into a certain kind of output.
If you’re not super-familiar with the current state-of-the-art in machine learning, this might seem like a terrifying demand, but don’t fear. Simply right click in your code, and select “Insert HF Model”. You’ll get a dialog box: Chatting with both images and text is called “image-text-to-text”: the user can supply images and text, and the model outputs text. Scroll down on the left until you find it. By default, the model list will be sorted by Likes – but remember, older models often have a lot of likes built up even if they’re not really the state of the art anymore. We can check how old models are by seeing the date they were last updated, just under the model name. Let’s pick something that’s both recent and popular: microsoft/Phi-3.5-vision-instruct .
You can select “Use Model” for some model categories to automatically paste some basic code into your notebook, but what often works better is to scroll through the Model Card on the right and grab any sample code. You can see the full model card to the right of the dialog box, exactly as it's shown on Hugging Face Hub. Let’s do that and paste it into our code!
Your office cybersecurity person might complain about you copying a chunk of random text from the internet and running it without even reading all of it, but if that happens just call them a nerd and continue regardless. And behold: We now have a working model that can happily chat about images - in this case, it reads and comments on screenshots of a Microsoft slide deck. Feel free to play around with this example. Try your own chat, or your own images. Once you get it working, simply wrap this code into a class and it’s ready to go in your app. We just got state of the art open-source machine learning in ten minutes without even opening a web browser.
By embedding model selection and documentation directly into the IDE, developers can accelerate AI integration, reduce friction in model adoption, and improve code quality through easier review and reuse.