huggingface/transformers
huggingface/transformers
Activity
Last release
Open issues
Open PRs
License
release notes
release notes
Published 9/6/2023
PatchContains breaking changesFalcon is a class of causal decoder-only models built by TII. The largest Falcon checkpoints have been trained on >=1T tokens of text, with a particular emphasis on the RefinedWeb corpus. They are made available under the Apache 2.0 license.
Falcon’s architecture is modern and optimized for inference, with multi-query attention and support for efficient attention variants like FlashAttention. Both ‘base’ models trained only as causal language models as well as ‘instruct’ models that have received further fine-tuning are available.
Falcon] Remove SDPA for falcon to support earlier versions of PyTorch (< 2.0) by @younesbelkada in #25947Code Llama, is a family of large language models for code based on Llama 2, providing state-of-the-art performance among open models, infilling capabilities, support for large input contexts, and zero-shot instruction following ability for programming tasks.
CodeLlama] Add support for CodeLlama by @ArthurZucker in #25740CodeLlama] Fix CI by @ArthurZucker in #25890ViTDet reuses the ViT model architecture, adapted to object detection.
DINO v2 is the next iteration of the DINO model. It is added as a backbone class, allowing it to be re-used in downstream models.
VITS (Variational Inference with adversarial learning for end-to-end Text-to-Speech) is an end-to-end speech synthesis model that predicts a speech waveform conditional on an input text sequence. It is a conditional variational autoencoder (VAE) comprised of a posterior encoder, decoder, and conditional prior.
Refactor] Move third-party related utility files into integrations/ folder 🚨🚨🚨 by @younesbelkada in #25599Moves all third party libs (outside HF ecosystem) related utility files inside integrations/ instead of having them in transformers directly.
In order to get the previous usage you should be changing your call to the following:
- from transformers.deepspeed import HfDeepSpeedConfig
+ from transformers.integrations import HfDeepSpeedConfig
TRANSFORMERS_TEST_BACKEND by @vvvm23 in #25655SPM] Patch spm Llama and T5 by @ArthurZucker in #25656GPTNeo] Add input_embeds functionality to gpt_neo Causal LM by @ArthurZucker in #25664utils/documentation_tests.txt by @ydshieh in #25680pad_token check condition by @ydshieh in #25685inputs_embeds by @gante in #25687configuration_gpt2.py by @susnato in #25676LlamaTokenizer] make unk_token_length a property by @ArthurZucker in #25689test_batch_generation for bloom by @ydshieh in #25718PEFT] Fix peft version by @younesbelkada in #25710AutoGPTQ] Add correct installation of GPTQ library + fix slow tests by @younesbelkada in #25713do_sample=False when temperature=0.0 by @gante in #25722from_pretrained] Simpler code for peft by @ArthurZucker in #25726from_pretrained] Fix failing PEFT tests by @younesbelkada in #25733visual_question_answering.md to Korean by @wonhyeongseo in #25679PEFT] Fix PeftConfig save pretrained when calling add_adapter by @younesbelkada in #25738Sentencepiece] make sure legacy do not require protobuf by @ArthurZucker in #25684HammingDiversityLogitsProcessor by @gante in #25756LlamaFamiliy] add a tip about dtype by @ArthurZucker in #25794hidden_act by @stas00 in #25787Docs] More clarifications on BT + FA by @younesbelkada in #25823LlamaTokenizer] tokenize nits. by @ArthurZucker in #25793model_memory_anatomy.md to Korean by @mjk0618 in #25755add_new_pipeline.md to Korean by @heuristicwave in #25498community.md to Korean by @sim-so in #25674Pop2Piano checkpoints by @susnato in #25827generate() return True in can_generate() by @gante in #25838generation_strategies.md by @gante in #25874stage3_gather_16bit_weights_on_model_save=False by @pacman100 in #25817TokenizerFast] can_save_slow_tokenizer as a property for when vocab_file's folder was removed by @ArthurZucker in #25626InstructBlip] FINAL Fix instructblip test by @younesbelkada in #25887setup.py by @ydshieh in #25893is_tensor by @sgugger in #25871ViTDet by @ydshieh in #25913The following contributors have made significant changes to the library over the last release:
release notes
Published 9/6/2023
PatchContains breaking changesFalcon is a class of causal decoder-only models built by TII. The largest Falcon checkpoints have been trained on >=1T tokens of text, with a particular emphasis on the RefinedWeb corpus. They are made available under the Apache 2.0 license.
Falcon’s architecture is modern and optimized for inference, with multi-query attention and support for efficient attention variants like FlashAttention. Both ‘base’ models trained only as causal language models as well as ‘instruct’ models that have received further fine-tuning are available.
Falcon] Remove SDPA for falcon to support earlier versions of PyTorch (< 2.0) by @younesbelkada in #25947Code Llama, is a family of large language models for code based on Llama 2, providing state-of-the-art performance among open models, infilling capabilities, support for large input contexts, and zero-shot instruction following ability for programming tasks.
CodeLlama] Add support for CodeLlama by @ArthurZucker in #25740CodeLlama] Fix CI by @ArthurZucker in #25890ViTDet reuses the ViT model architecture, adapted to object detection.
DINO v2 is the next iteration of the DINO model. It is added as a backbone class, allowing it to be re-used in downstream models.
VITS (Variational Inference with adversarial learning for end-to-end Text-to-Speech) is an end-to-end speech synthesis model that predicts a speech waveform conditional on an input text sequence. It is a conditional variational autoencoder (VAE) comprised of a posterior encoder, decoder, and conditional prior.
Refactor] Move third-party related utility files into integrations/ folder 🚨🚨🚨 by @younesbelkada in #25599Moves all third party libs (outside HF ecosystem) related utility files inside integrations/ instead of having them in transformers directly.
In order to get the previous usage you should be changing your call to the following:
- from transformers.deepspeed import HfDeepSpeedConfig
+ from transformers.integrations import HfDeepSpeedConfig
TRANSFORMERS_TEST_BACKEND by @vvvm23 in #25655SPM] Patch spm Llama and T5 by @ArthurZucker in #25656GPTNeo] Add input_embeds functionality to gpt_neo Causal LM by @ArthurZucker in #25664utils/documentation_tests.txt by @ydshieh in #25680pad_token check condition by @ydshieh in #25685inputs_embeds by @gante in #25687configuration_gpt2.py by @susnato in #25676LlamaTokenizer] make unk_token_length a property by @ArthurZucker in #25689test_batch_generation for bloom by @ydshieh in #25718PEFT] Fix peft version by @younesbelkada in #25710AutoGPTQ] Add correct installation of GPTQ library + fix slow tests by @younesbelkada in #25713do_sample=False when temperature=0.0 by @gante in #25722from_pretrained] Simpler code for peft by @ArthurZucker in #25726from_pretrained] Fix failing PEFT tests by @younesbelkada in #25733visual_question_answering.md to Korean by @wonhyeongseo in #25679PEFT] Fix PeftConfig save pretrained when calling add_adapter by @younesbelkada in #25738Sentencepiece] make sure legacy do not require protobuf by @ArthurZucker in #25684HammingDiversityLogitsProcessor by @gante in #25756LlamaFamiliy] add a tip about dtype by @ArthurZucker in #25794hidden_act by @stas00 in #25787Docs] More clarifications on BT + FA by @younesbelkada in #25823LlamaTokenizer] tokenize nits. by @ArthurZucker in #25793model_memory_anatomy.md to Korean by @mjk0618 in #25755add_new_pipeline.md to Korean by @heuristicwave in #25498community.md to Korean by @sim-so in #25674Pop2Piano checkpoints by @susnato in #25827generate() return True in can_generate() by @gante in #25838generation_strategies.md by @gante in #25874stage3_gather_16bit_weights_on_model_save=False by @pacman100 in #25817TokenizerFast] can_save_slow_tokenizer as a property for when vocab_file's folder was removed by @ArthurZucker in #25626InstructBlip] FINAL Fix instructblip test by @younesbelkada in #25887setup.py by @ydshieh in #25893is_tensor by @sgugger in #25871ViTDet by @ydshieh in #25913The following contributors have made significant changes to the library over the last release:
🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.