@@ -310,141 +310,6 @@ Datasets & transform refactoring
310310 * Paper: [ ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoders] ( http://arxiv.org/abs/2301.00808 )
311311 * Reference impl: https://github.com/facebookresearch/ConvNeXt-V2 (NOTE: weights currently CC-BY-NC)
312312
313- ### Dec 23, 2022 🎄☃
314- * Add FlexiViT models and weights from https://github.com/google-research/big_vision (check out paper at https://arxiv.org/abs/2212.08013 )
315- * NOTE currently resizing is static on model creation, on-the-fly dynamic / train patch size sampling is a WIP
316- * Many more models updated to multi-weight and downloadable via HF hub now (convnext, efficientnet, mobilenet, vision_transformer* , beit)
317- * More model pretrained tag and adjustments, some model names changed (working on deprecation translations, consider main branch DEV branch right now, use 0.6.x for stable use)
318- * More ImageNet-12k (subset of 22k) pretrain models popping up:
319- * ` efficientnet_b5.in12k_ft_in1k ` - 85.9 @ 448x448
320- * ` vit_medium_patch16_gap_384.in12k_ft_in1k ` - 85.5 @ 384x384
321- * ` vit_medium_patch16_gap_256.in12k_ft_in1k ` - 84.5 @ 256x256
322- * ` convnext_nano.in12k_ft_in1k ` - 82.9 @ 288x288
323-
324- ### Dec 8, 2022
325- * Add 'EVA l' to ` vision_transformer.py ` , MAE style ViT-L/14 MIM pretrain w/ EVA-CLIP targets, FT on ImageNet-1k (w/ ImageNet-22k intermediate for some)
326- * original source: https://github.com/baaivision/EVA
327-
328- | model | top1 | param_count | gmac | macts | hub |
329- | :------------------------------------------| -----:| ------------:| ------:| ------:| :----------------------------------------|
330- | eva_large_patch14_336.in22k_ft_in22k_in1k | 89.2 | 304.5 | 191.1 | 270.2 | [ link] ( https://huggingface.co/BAAI/EVA ) |
331- | eva_large_patch14_336.in22k_ft_in1k | 88.7 | 304.5 | 191.1 | 270.2 | [ link] ( https://huggingface.co/BAAI/EVA ) |
332- | eva_large_patch14_196.in22k_ft_in22k_in1k | 88.6 | 304.1 | 61.6 | 63.5 | [ link] ( https://huggingface.co/BAAI/EVA ) |
333- | eva_large_patch14_196.in22k_ft_in1k | 87.9 | 304.1 | 61.6 | 63.5 | [ link] ( https://huggingface.co/BAAI/EVA ) |
334-
335- ### Dec 6, 2022
336- * Add 'EVA g', BEiT style ViT-g/14 model weights w/ both MIM pretrain and CLIP pretrain to ` beit.py ` .
337- * original source: https://github.com/baaivision/EVA
338- * paper: https://arxiv.org/abs/2211.07636
339-
340- | model | top1 | param_count | gmac | macts | hub |
341- | :-----------------------------------------| -------:| --------------:| -------:| --------:| :----------------------------------------|
342- | eva_giant_patch14_560.m30m_ft_in22k_in1k | 89.8 | 1014.4 | 1906.8 | 2577.2 | [ link] ( https://huggingface.co/BAAI/EVA ) |
343- | eva_giant_patch14_336.m30m_ft_in22k_in1k | 89.6 | 1013 | 620.6 | 550.7 | [ link] ( https://huggingface.co/BAAI/EVA ) |
344- | eva_giant_patch14_336.clip_ft_in1k | 89.4 | 1013 | 620.6 | 550.7 | [ link] ( https://huggingface.co/BAAI/EVA ) |
345- | eva_giant_patch14_224.clip_ft_in1k | 89.1 | 1012.6 | 267.2 | 192.6 | [ link] ( https://huggingface.co/BAAI/EVA ) |
346-
347- ### Dec 5, 2022
348-
349- * Pre-release (` 0.8.0dev0 ` ) of multi-weight support (` model_arch.pretrained_tag ` ). Install with ` pip install --pre timm `
350- * vision_transformer, maxvit, convnext are the first three model impl w/ support
351- * model names are changing with this (previous _ 21k, etc. fn will merge), still sorting out deprecation handling
352- * bugs are likely, but I need feedback so please try it out
353- * if stability is needed, please use 0.6.x pypi releases or clone from [ 0.6.x branch] ( https://github.com/rwightman/pytorch-image-models/tree/0.6.x )
354- * Support for PyTorch 2.0 compile is added in train/validate/inference/benchmark, use ` --torchcompile ` argument
355- * Inference script allows more control over output, select k for top-class index + prob json, csv or parquet output
356- * Add a full set of fine-tuned CLIP image tower weights from both LAION-2B and original OpenAI CLIP models
357-
358- | model | top1 | param_count | gmac | macts | hub |
359- | :-------------------------------------------------| -------:| --------------:| -------:| --------:| :-------------------------------------------------------------------------------------|
360- | vit_huge_patch14_clip_336.laion2b_ft_in12k_in1k | 88.6 | 632.5 | 391 | 407.5 | [ link] ( https://huggingface.co/timm/vit_huge_patch14_clip_336.laion2b_ft_in12k_in1k ) |
361- | vit_large_patch14_clip_336.openai_ft_in12k_in1k | 88.3 | 304.5 | 191.1 | 270.2 | [ link] ( https://huggingface.co/timm/vit_large_patch14_clip_336.openai_ft_in12k_in1k ) |
362- | vit_huge_patch14_clip_224.laion2b_ft_in12k_in1k | 88.2 | 632 | 167.4 | 139.4 | [ link] ( https://huggingface.co/timm/vit_huge_patch14_clip_224.laion2b_ft_in12k_in1k ) |
363- | vit_large_patch14_clip_336.laion2b_ft_in12k_in1k | 88.2 | 304.5 | 191.1 | 270.2 | [ link] ( https://huggingface.co/timm/vit_large_patch14_clip_336.laion2b_ft_in12k_in1k ) |
364- | vit_large_patch14_clip_224.openai_ft_in12k_in1k | 88.2 | 304.2 | 81.1 | 88.8 | [ link] ( https://huggingface.co/timm/vit_large_patch14_clip_224.openai_ft_in12k_in1k ) |
365- | vit_large_patch14_clip_224.laion2b_ft_in12k_in1k | 87.9 | 304.2 | 81.1 | 88.8 | [ link] ( https://huggingface.co/timm/vit_large_patch14_clip_224.laion2b_ft_in12k_in1k ) |
366- | vit_large_patch14_clip_224.openai_ft_in1k | 87.9 | 304.2 | 81.1 | 88.8 | [ link] ( https://huggingface.co/timm/vit_large_patch14_clip_224.openai_ft_in1k ) |
367- | vit_large_patch14_clip_336.laion2b_ft_in1k | 87.9 | 304.5 | 191.1 | 270.2 | [ link] ( https://huggingface.co/timm/vit_large_patch14_clip_336.laion2b_ft_in1k ) |
368- | vit_huge_patch14_clip_224.laion2b_ft_in1k | 87.6 | 632 | 167.4 | 139.4 | [ link] ( https://huggingface.co/timm/vit_huge_patch14_clip_224.laion2b_ft_in1k ) |
369- | vit_large_patch14_clip_224.laion2b_ft_in1k | 87.3 | 304.2 | 81.1 | 88.8 | [ link] ( https://huggingface.co/timm/vit_large_patch14_clip_224.laion2b_ft_in1k ) |
370- | vit_base_patch16_clip_384.laion2b_ft_in12k_in1k | 87.2 | 86.9 | 55.5 | 101.6 | [ link] ( https://huggingface.co/timm/vit_base_patch16_clip_384.laion2b_ft_in12k_in1k ) |
371- | vit_base_patch16_clip_384.openai_ft_in12k_in1k | 87 | 86.9 | 55.5 | 101.6 | [ link] ( https://huggingface.co/timm/vit_base_patch16_clip_384.openai_ft_in12k_in1k ) |
372- | vit_base_patch16_clip_384.laion2b_ft_in1k | 86.6 | 86.9 | 55.5 | 101.6 | [ link] ( https://huggingface.co/timm/vit_base_patch16_clip_384.laion2b_ft_in1k ) |
373- | vit_base_patch16_clip_384.openai_ft_in1k | 86.2 | 86.9 | 55.5 | 101.6 | [ link] ( https://huggingface.co/timm/vit_base_patch16_clip_384.openai_ft_in1k ) |
374- | vit_base_patch16_clip_224.laion2b_ft_in12k_in1k | 86.2 | 86.6 | 17.6 | 23.9 | [ link] ( https://huggingface.co/timm/vit_base_patch16_clip_224.laion2b_ft_in12k_in1k ) |
375- | vit_base_patch16_clip_224.openai_ft_in12k_in1k | 85.9 | 86.6 | 17.6 | 23.9 | [ link] ( https://huggingface.co/timm/vit_base_patch16_clip_224.openai_ft_in12k_in1k ) |
376- | vit_base_patch32_clip_448.laion2b_ft_in12k_in1k | 85.8 | 88.3 | 17.9 | 23.9 | [ link] ( https://huggingface.co/timm/vit_base_patch32_clip_448.laion2b_ft_in12k_in1k ) |
377- | vit_base_patch16_clip_224.laion2b_ft_in1k | 85.5 | 86.6 | 17.6 | 23.9 | [ link] ( https://huggingface.co/timm/vit_base_patch16_clip_224.laion2b_ft_in1k ) |
378- | vit_base_patch32_clip_384.laion2b_ft_in12k_in1k | 85.4 | 88.3 | 13.1 | 16.5 | [ link] ( https://huggingface.co/timm/vit_base_patch32_clip_384.laion2b_ft_in12k_in1k ) |
379- | vit_base_patch16_clip_224.openai_ft_in1k | 85.3 | 86.6 | 17.6 | 23.9 | [ link] ( https://huggingface.co/timm/vit_base_patch16_clip_224.openai_ft_in1k ) |
380- | vit_base_patch32_clip_384.openai_ft_in12k_in1k | 85.2 | 88.3 | 13.1 | 16.5 | [ link] ( https://huggingface.co/timm/vit_base_patch32_clip_384.openai_ft_in12k_in1k ) |
381- | vit_base_patch32_clip_224.laion2b_ft_in12k_in1k | 83.3 | 88.2 | 4.4 | 5 | [ link] ( https://huggingface.co/timm/vit_base_patch32_clip_224.laion2b_ft_in12k_in1k ) |
382- | vit_base_patch32_clip_224.laion2b_ft_in1k | 82.6 | 88.2 | 4.4 | 5 | [ link] ( https://huggingface.co/timm/vit_base_patch32_clip_224.laion2b_ft_in1k ) |
383- | vit_base_patch32_clip_224.openai_ft_in1k | 81.9 | 88.2 | 4.4 | 5 | [ link] ( https://huggingface.co/timm/vit_base_patch32_clip_224.openai_ft_in1k ) |
384-
385- * Port of MaxViT Tensorflow Weights from official impl at https://github.com/google-research/maxvit
386- * There was larger than expected drops for the upscaled 384/512 in21k fine-tune weights, possible detail missing, but the 21k FT did seem sensitive to small preprocessing
387-
388- | model | top1 | param_count | gmac | macts | hub |
389- | :-----------------------------------| -------:| --------------:| -------:| --------:| :-----------------------------------------------------------------------|
390- | maxvit_xlarge_tf_512.in21k_ft_in1k | 88.5 | 475.8 | 534.1 | 1413.2 | [ link] ( https://huggingface.co/timm/maxvit_xlarge_tf_512.in21k_ft_in1k ) |
391- | maxvit_xlarge_tf_384.in21k_ft_in1k | 88.3 | 475.3 | 292.8 | 668.8 | [ link] ( https://huggingface.co/timm/maxvit_xlarge_tf_384.in21k_ft_in1k ) |
392- | maxvit_base_tf_512.in21k_ft_in1k | 88.2 | 119.9 | 138 | 704 | [ link] ( https://huggingface.co/timm/maxvit_base_tf_512.in21k_ft_in1k ) |
393- | maxvit_large_tf_512.in21k_ft_in1k | 88 | 212.3 | 244.8 | 942.2 | [ link] ( https://huggingface.co/timm/maxvit_large_tf_512.in21k_ft_in1k ) |
394- | maxvit_large_tf_384.in21k_ft_in1k | 88 | 212 | 132.6 | 445.8 | [ link] ( https://huggingface.co/timm/maxvit_large_tf_384.in21k_ft_in1k ) |
395- | maxvit_base_tf_384.in21k_ft_in1k | 87.9 | 119.6 | 73.8 | 332.9 | [ link] ( https://huggingface.co/timm/maxvit_base_tf_384.in21k_ft_in1k ) |
396- | maxvit_base_tf_512.in1k | 86.6 | 119.9 | 138 | 704 | [ link] ( https://huggingface.co/timm/maxvit_base_tf_512.in1k ) |
397- | maxvit_large_tf_512.in1k | 86.5 | 212.3 | 244.8 | 942.2 | [ link] ( https://huggingface.co/timm/maxvit_large_tf_512.in1k ) |
398- | maxvit_base_tf_384.in1k | 86.3 | 119.6 | 73.8 | 332.9 | [ link] ( https://huggingface.co/timm/maxvit_base_tf_384.in1k ) |
399- | maxvit_large_tf_384.in1k | 86.2 | 212 | 132.6 | 445.8 | [ link] ( https://huggingface.co/timm/maxvit_large_tf_384.in1k ) |
400- | maxvit_small_tf_512.in1k | 86.1 | 69.1 | 67.3 | 383.8 | [ link] ( https://huggingface.co/timm/maxvit_small_tf_512.in1k ) |
401- | maxvit_tiny_tf_512.in1k | 85.7 | 31 | 33.5 | 257.6 | [ link] ( https://huggingface.co/timm/maxvit_tiny_tf_512.in1k ) |
402- | maxvit_small_tf_384.in1k | 85.5 | 69 | 35.9 | 183.6 | [ link] ( https://huggingface.co/timm/maxvit_small_tf_384.in1k ) |
403- | maxvit_tiny_tf_384.in1k | 85.1 | 31 | 17.5 | 123.4 | [ link] ( https://huggingface.co/timm/maxvit_tiny_tf_384.in1k ) |
404- | maxvit_large_tf_224.in1k | 84.9 | 211.8 | 43.7 | 127.4 | [ link] ( https://huggingface.co/timm/maxvit_large_tf_224.in1k ) |
405- | maxvit_base_tf_224.in1k | 84.9 | 119.5 | 24 | 95 | [ link] ( https://huggingface.co/timm/maxvit_base_tf_224.in1k ) |
406- | maxvit_small_tf_224.in1k | 84.4 | 68.9 | 11.7 | 53.2 | [ link] ( https://huggingface.co/timm/maxvit_small_tf_224.in1k ) |
407- | maxvit_tiny_tf_224.in1k | 83.4 | 30.9 | 5.6 | 35.8 | [ link] ( https://huggingface.co/timm/maxvit_tiny_tf_224.in1k ) |
408-
409- ### Oct 15, 2022
410- * Train and validation script enhancements
411- * Non-GPU (ie CPU) device support
412- * SLURM compatibility for train script
413- * HF datasets support (via ReaderHfds)
414- * TFDS/WDS dataloading improvements (sample padding/wrap for distributed use fixed wrt sample count estimate)
415- * in_chans !=3 support for scripts / loader
416- * Adan optimizer
417- * Can enable per-step LR scheduling via args
418- * Dataset 'parsers' renamed to 'readers', more descriptive of purpose
419- * AMP args changed, APEX via ` --amp-impl apex ` , bfloat16 supportedf via ` --amp-dtype bfloat16 `
420- * main branch switched to 0.7.x version, 0.6x forked for stable release of weight only adds
421- * master -> main branch rename
422-
423- ### Oct 10, 2022
424- * More weights in ` maxxvit ` series, incl first ConvNeXt block based ` coatnext ` and ` maxxvit ` experiments:
425- * ` coatnext_nano_rw_224 ` - 82.0 @ 224 (G) -- (uses ConvNeXt conv block, no BatchNorm)
426- * ` maxxvit_rmlp_nano_rw_256 ` - 83.0 @ 256, 83.7 @ 320 (G) (uses ConvNeXt conv block, no BN)
427- * ` maxvit_rmlp_small_rw_224 ` - 84.5 @ 224, 85.1 @ 320 (G)
428- * ` maxxvit_rmlp_small_rw_256 ` - 84.6 @ 256, 84.9 @ 288 (G) -- could be trained better, hparams need tuning (uses ConvNeXt block, no BN)
429- * ` coatnet_rmlp_2_rw_224 ` - 84.6 @ 224, 85 @ 320 (T)
430- * NOTE: official MaxVit weights (in1k) have been released at https://github.com/google-research/maxvit -- some extra work is needed to port and adapt since my impl was created independently of theirs and has a few small differences + the whole TF same padding fun.
431-
432- ### Sept 23, 2022
433- * LAION-2B CLIP image towers supported as pretrained backbones for fine-tune or features (no classifier)
434- * vit_base_patch32_224_clip_laion2b
435- * vit_large_patch14_224_clip_laion2b
436- * vit_huge_patch14_224_clip_laion2b
437- * vit_giant_patch14_224_clip_laion2b
438-
439- ### Sept 7, 2022
440- * Hugging Face [ ` timm ` docs] ( https://huggingface.co/docs/hub/timm ) home now exists, look for more here in the future
441- * Add BEiT-v2 weights for base and large 224x224 models from https://github.com/microsoft/unilm/tree/master/beit2
442- * Add more weights in ` maxxvit ` series incl a ` pico ` (7.5M params, 1.9 GMACs), two ` tiny ` variants:
443- * ` maxvit_rmlp_pico_rw_256 ` - 80.5 @ 256, 81.3 @ 320 (T)
444- * ` maxvit_tiny_rw_224 ` - 83.5 @ 224 (G)
445- * ` maxvit_rmlp_tiny_rw_256 ` - 84.2 @ 256, 84.8 @ 320 (T)
446-
447-
448313## Introduction
449314
450315Py** T** orch ** Im** age ** M** odels (` timm ` ) is a collection of image models, layers, utilities, optimizers, schedulers, data-loaders / augmentations, and reference training / validation scripts that aim to pull together a wide variety of SOTA models with ability to reproduce ImageNet training results.
0 commit comments