# clip-interrogator
*Want to figure out what a good prompt might be to create new images like an existing one? The **CLIP Interrogator** is here to get you answers!*
## About
The **CLIP Interrogator** is a prompt engineering tool that combines OpenAI's [CLIP](https://openai.com/blog/clip/) and Salesforce's [BLIP](https://blog.salesforceairesearch.com/blip-bootstrapping-language-image-pretraining/) to optimize text prompts to match a given image. Use the resulting prompts with text-to-image models like [Stable Diffusion](https://github.com/CompVis/stable-diffusion) on [DreamStudio](https://beta.dreamstudio.ai/) to create cool art!
## Using as a library
Create and activate a Python virtual environment
```bash
python3 -m venv ci_env
(for linux ) source ci_env/bin/activate
(for windows) .\ci_env\Scripts\activate
```
Install with PIP
```
# install torch with GPU support for example:
pip3 install torch torchvision --extra-index-url https://download.pytorch.org/whl/cu117
# install clip-interrogator
pip install clip-interrogator==0.5.4
# or for very latest WIP with BLIP2 support
#pip install clip-interrogator==0.6.0
```
You can then use it in your script
```python
from PIL import Image
from clip_interrogator import Config, Interrogator
image = Image.open(image_path).convert('RGB')
ci = Interrogator(Config(clip_model_name="ViT-L-14/openai"))
print(ci.interrogate(image))
```
CLIP Interrogator uses OpenCLIP which supports many different pretrained CLIP models. For the best prompts for
Stable Diffusion 1.X use `ViT-L-14/openai` for clip_model_name. For Stable Diffusion 2.0 use `ViT-H-14/laion2b_s32b_b79k`
## Configuration
The `Config` object lets you configure CLIP Interrogator's processing.
* `clip_model_name`: which of the OpenCLIP pretrained CLIP models to use
* `cache_path`: path where to save precomputed text embeddings
* `download_cache`: when True will download the precomputed embeddings from huggingface
* `chunk_size`: batch size for CLIP, use smaller for lower VRAM
* `quiet`: when True no progress bars or text output will be displayed
On systems with low VRAM you can call `config.apply_low_vram_defaults()` to reduce the amount of VRAM needed (at the cost of some speed and quality). The default settings use about 6.3GB of VRAM and the low VRAM settings use about 2.7GB.
See the [run_cli.py](https://github.com/pharmapsychotic/clip-interrogator/blob/main/run_cli.py) and [run_gradio.py](https://github.com/pharmapsychotic/clip-interrogator/blob/main/run_gradio.py) for more examples on using Config and Interrogator classes.
## Ranking against your own list of terms (requires version 0.6.0)
```python
from clip_interrogator import Config, Interrogator, LabelTable, load_list
from PIL import Image
ci = Interrogator(Config(blip_model_type=None))
image = Image.open(image_path).convert('RGB')
table = LabelTable(load_list('terms.txt'), 'terms', ci)
best_match = table.rank(ci.image_to_features(image), top_count=1)[0]
print(best_match)
```
没有合适的资源?快使用搜索试试~ 我知道了~
多模态大模型应用-使用CLIP+BLIP基于图像获取提示词-Image-to-Prompt-附项目源码-优质大模型应用实战
共17个文件
txt:6个
py:6个
toml:1个
1.该资源内容由用户上传,如若侵权请联系客服进行举报
2.虚拟产品一经售出概不退款(资源遇到问题,请及时私信上传者)
2.虚拟产品一经售出概不退款(资源遇到问题,请及时私信上传者)
版权申诉
0 下载量 159 浏览量
2024-05-08
10:03:19
上传
评论 1
收藏 781KB ZIP 举报
温馨提示
多模态大模型应用_使用CLIP+BLIP基于图像获取提示词_Image_to_Prompt_附项目源码_优质大模型应用实战
资源推荐
资源详情
资源评论
收起资源包目录
多模态大模型应用_使用CLIP+BLIP基于图像获取提示词_Image_to_Prompt_附项目源码_优质大模型应用实战.zip (17个子文件)
多模态大模型应用_使用CLIP+BLIP基于图像获取提示词_Image_to_Prompt_附项目源码_优质大模型应用实战
setup.py 1KB
run_gradio.py 4KB
clip_interrogator.ipynb 12KB
predict.py 2KB
cog.yaml 334B
clip_interrogator
__init__.py 167B
data
artists.txt 81KB
mediums.txt 2KB
flavors.txt 1.72MB
negative.txt 418B
movements.txt 3KB
clip_interrogator.py 20KB
pyproject.toml 80B
requirements.txt 106B
run_cli.py 3KB
MANIFEST.in 242B
README.md 3KB
共 17 条
- 1
资源评论
极智视界
- 粉丝: 2w+
- 资源: 1459
下载权益
C知道特权
VIP文章
课程特权
开通VIP
上传资源 快速赚钱
- 我的内容管理 展开
- 我的资源 快来上传第一个资源
- 我的收益 登录查看自己的收益
- 我的积分 登录查看自己的积分
- 我的C币 登录后查看C币余额
- 我的收藏
- 我的下载
- 下载帮助
最新资源
- Panoply软件是大名鼎鼎的NASA下属的GISS研究所开发的可视化软件,该软件可以实现对地学常用数据的读取,其中包括netC
- 一些高质量的学习Ruby的资源清单.zip
- 基于STM32智能家居(智能云)
- 适合江苏地带的别墅小院子图纸D038-两层-11.04&11.94米-施工图.dwg
- 农村小别墅图纸四合院图纸D037-两层-13.20&12.90米-施工图.dwg
- 农村别墅系列图纸cad图纸D035-两层-10.20&14.70米- 施工图.dwg
- Verilog硬件仿真过程中出现的X态总结
- 中国各省份2012-2020年政府财政支出占GDP的比重
- 旅游管理系统毕业设计+计算机专业本科毕业设计+Vue+JS框架
- 农村小别墅二层小楼图纸D034-两层-11.40&14.70米- 施工图.dwg
资源上传下载、课程学习等过程中有任何疑问或建议,欢迎提出宝贵意见哦~我们会及时处理!
点击此处反馈
安全验证
文档复制为VIP权益,开通VIP直接复制
信息提交成功