Bohrium
robot
新建

空间站广场

论文
Notebooks
比赛
课程
Apps
我的主页
我的Notebooks
我的论文库
我的足迹

我的工作空间

任务
节点
文件
数据集
镜像
项目
数据库
公开
Hugging Face Tutorials: Summary of the tasks
Transformers
Transformers
爱学习的王一博
发布于 2024-05-18
推荐镜像 :Basic Image:bohrium-notebook:2023-04-07
推荐机型 :c12_m46_1 * NVIDIA GPU B
赞 1
Summary of the tasks
Sequence Classification
Extractive Question Answering
Language Modeling
Masked Language Modeling
Causal Language Modeling
Text Generation
Named Entity Recognition
Summarization
Translation
Audio classification
Automatic speech recognition
Image classification
[2]
# Transformers installation
! pip install transformers datasets
# To install from source instead of the last release, comment the command above and uncomment the following one.
# ! pip install git+https://github.com/huggingface/transformers.git
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Requirement already satisfied: transformers in /opt/conda/lib/python3.8/site-packages (4.27.1)
Requirement already satisfied: datasets in /opt/conda/lib/python3.8/site-packages (2.10.1)
Requirement already satisfied: requests in /opt/conda/lib/python3.8/site-packages (from transformers) (2.28.2)
Requirement already satisfied: pyyaml>=5.1 in /opt/conda/lib/python3.8/site-packages (from transformers) (6.0)
Requirement already satisfied: tokenizers!=0.11.3,<0.14,>=0.11.1 in /opt/conda/lib/python3.8/site-packages (from transformers) (0.13.2)
Requirement already satisfied: filelock in /opt/conda/lib/python3.8/site-packages (from transformers) (3.9.0)
Requirement already satisfied: regex!=2019.12.17 in /opt/conda/lib/python3.8/site-packages (from transformers) (2022.6.2)
Requirement already satisfied: numpy>=1.17 in /opt/conda/lib/python3.8/site-packages (from transformers) (1.22.4)
Requirement already satisfied: packaging>=20.0 in /opt/conda/lib/python3.8/site-packages (from transformers) (23.0)
Requirement already satisfied: tqdm>=4.27 in /opt/conda/lib/python3.8/site-packages (from transformers) (4.64.1)
Requirement already satisfied: huggingface-hub<1.0,>=0.11.0 in /opt/conda/lib/python3.8/site-packages (from transformers) (0.13.2)
Requirement already satisfied: fsspec[http]>=2021.11.1 in /opt/conda/lib/python3.8/site-packages (from datasets) (2023.1.0)
Requirement already satisfied: aiohttp in /opt/conda/lib/python3.8/site-packages (from datasets) (3.8.4)
Requirement already satisfied: pyarrow>=6.0.0 in /opt/conda/lib/python3.8/site-packages (from datasets) (9.0.0)
Requirement already satisfied: multiprocess in /opt/conda/lib/python3.8/site-packages (from datasets) (0.70.14)
Requirement already satisfied: dill<0.3.7,>=0.3.0 in /opt/conda/lib/python3.8/site-packages (from datasets) (0.3.6)
Requirement already satisfied: responses<0.19 in /opt/conda/lib/python3.8/site-packages (from datasets) (0.18.0)
Requirement already satisfied: pandas in /opt/conda/lib/python3.8/site-packages (from datasets) (1.5.3)
Requirement already satisfied: xxhash in /opt/conda/lib/python3.8/site-packages (from datasets) (3.2.0)
Requirement already satisfied: aiosignal>=1.1.2 in /opt/conda/lib/python3.8/site-packages (from aiohttp->datasets) (1.3.1)
Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /opt/conda/lib/python3.8/site-packages (from aiohttp->datasets) (4.0.2)
Requirement already satisfied: yarl<2.0,>=1.0 in /opt/conda/lib/python3.8/site-packages (from aiohttp->datasets) (1.8.2)
Requirement already satisfied: charset-normalizer<4.0,>=2.0 in /opt/conda/lib/python3.8/site-packages (from aiohttp->datasets) (3.0.1)
Requirement already satisfied: multidict<7.0,>=4.5 in /opt/conda/lib/python3.8/site-packages (from aiohttp->datasets) (6.0.4)
Requirement already satisfied: frozenlist>=1.1.1 in /opt/conda/lib/python3.8/site-packages (from aiohttp->datasets) (1.3.3)
Requirement already satisfied: attrs>=17.3.0 in /opt/conda/lib/python3.8/site-packages (from aiohttp->datasets) (22.1.0)
Requirement already satisfied: typing-extensions>=3.7.4.3 in /opt/conda/lib/python3.8/site-packages (from huggingface-hub<1.0,>=0.11.0->transformers) (4.5.0)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /opt/conda/lib/python3.8/site-packages (from requests->transformers) (1.26.14)
Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.8/site-packages (from requests->transformers) (3.4)
Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.8/site-packages (from requests->transformers) (2022.12.7)
Requirement already satisfied: python-dateutil>=2.8.1 in /opt/conda/lib/python3.8/site-packages (from pandas->datasets) (2.8.2)
Requirement already satisfied: pytz>=2020.1 in /opt/conda/lib/python3.8/site-packages (from pandas->datasets) (2022.7)
Requirement already satisfied: six>=1.5 in /opt/conda/lib/python3.8/site-packages (from python-dateutil>=2.8.1->pandas->datasets) (1.16.0)
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
代码
文本
[3]
import os
os.environ['HTTP_PROXY'] = 'http://ga.dp.tech:8118'
os.environ['HTTPS_PROXY'] = 'http://ga.dp.tech:8118'
代码
文本
[4]
#import os
#os.environ["HF_HOME"] ="/share/hfcache"
代码
文本

Summary of the tasks

代码
文本

This page shows the most frequent use-cases when using the library. The models available allow for many different configurations and a great versatility in use-cases. The most simple ones are presented here, showcasing usage for tasks such as image classification, question answering, sequence classification, named entity recognition and others.

These examples leverage auto-models, which are classes that will instantiate a model according to a given checkpoint, automatically selecting the correct model architecture. Please check the AutoModel documentation for more information. Feel free to modify the code to be more specific and adapt it to your specific use-case.

In order for a model to perform well on a task, it must be loaded from a checkpoint corresponding to that task. These checkpoints are usually pre-trained on a large corpus of data and fine-tuned on a specific task. This means the following:

  • Not all models were fine-tuned on all tasks. If you want to fine-tune a model on a specific task, you can leverage one of the run_$TASK.py scripts in the examples directory.
  • Fine-tuned models were fine-tuned on a specific dataset. This dataset may or may not overlap with your use-case and domain. As mentioned previously, you may leverage the examples scripts to fine-tune your model, or you may create your own training script.

In order to do an inference on a task, several mechanisms are made available by the library:

  • Pipelines: very easy-to-use abstractions, which require as little as two lines of code.
  • Direct model use: Less abstractions, but more flexibility and power via a direct access to a tokenizer (PyTorch/TensorFlow) and full inference capacity.

Both approaches are showcased here.

All tasks presented here leverage pre-trained checkpoints that were fine-tuned on specific tasks. Loading a checkpoint that was not fine-tuned on a specific task would load only the base transformer layers and not the additional head that is used for the task, initializing the weights of that head randomly.

This would produce random output.

代码
文本

Sequence Classification

代码
文本

Sequence classification is the task of classifying sequences according to a given number of classes. An example of sequence classification is the GLUE dataset, which is entirely based on that task. If you would like to fine-tune a model on a GLUE sequence classification task, you may leverage the run_glue.py, run_tf_glue.py, run_tf_text_classification.py or run_xnli.py scripts.

Here is an example of using pipelines to do sentiment analysis: identifying if a sequence is positive or negative. It leverages a fine-tuned model on sst2, which is a GLUE task.

This returns a label ("POSITIVE" or "NEGATIVE") alongside a score, as follows:

代码
文本
[5]
from transformers import pipeline

classifier = pipeline("sentiment-analysis")

result = classifier("I hate you")[0]
print(f"label: {result['label']}, with score: {round(result['score'], 4)}")
2024-05-18 10:13:04.856760: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/mpi/gcc/openmpi-4.1.0rc5/lib:/usr/local/nccl-rdma-sharp-plugins/lib:/usr/local/nvidia/lib64:/usr/local/cuda/lib64:/usr/local/nvidia/lib:/usr/local/nvidia/lib64
2024-05-18 10:13:04.856900: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/mpi/gcc/openmpi-4.1.0rc5/lib:/usr/local/nccl-rdma-sharp-plugins/lib:/usr/local/nvidia/lib64:/usr/local/cuda/lib64:/usr/local/nvidia/lib:/usr/local/nvidia/lib64
2024-05-18 10:13:04.856914: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
No model was supplied, defaulted to distilbert-base-uncased-finetuned-sst-2-english and revision af0f99b (https://huggingface.co/distilbert-base-uncased-finetuned-sst-2-english).
Using a pipeline without specifying a model name and revision in production is not recommended.
label: NEGATIVE, with score: 0.9991
代码
文本
[6]
result = classifier("I love you")[0]
print(f"label: {result['label']}, with score: {round(result['score'], 4)}")
label: POSITIVE, with score: 0.9999
代码
文本

Here is an example of doing a sequence classification using a model to determine if two sequences are paraphrases of each other. The process is the following:

  1. Instantiate a tokenizer and a model from the checkpoint name. The model is identified as a BERT model and loads it with the weights stored in the checkpoint.
  2. Build a sequence from the two sentences, with the correct model-specific separators, token type ids and attention masks (which will be created automatically by the tokenizer).
  3. Pass this sequence through the model so that it is classified in one of the two available classes: 0 (not a paraphrase) and 1 (is a paraphrase).
  4. Compute the softmax of the result to get probabilities over the classes.
  5. Print the results.
代码
文本
[7]
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

tokenizer = AutoTokenizer.from_pretrained("bert-base-cased-finetuned-mrpc")
model = AutoModelForSequenceClassification.from_pretrained("bert-base-cased-finetuned-mrpc")

classes = ["not paraphrase", "is paraphrase"]

sequence_0 = "The company HuggingFace is based in New York City"
sequence_1 = "Apples are especially bad for your health"
sequence_2 = "HuggingFace's headquarters are situated in Manhattan"

# The tokenizer will automatically add any model specific separators (i.e. <CLS> and <SEP>) and tokens to
# the sequence, as well as compute the attention masks.
paraphrase = tokenizer(sequence_0, sequence_2, return_tensors="pt")
not_paraphrase = tokenizer(sequence_0, sequence_1, return_tensors="pt")

paraphrase_classification_logits = model(**paraphrase).logits
not_paraphrase_classification_logits = model(**not_paraphrase).logits

paraphrase_results = torch.softmax(paraphrase_classification_logits, dim=1).tolist()[0]
not_paraphrase_results = torch.softmax(not_paraphrase_classification_logits, dim=1).tolist()[0]

# Should be paraphrase
for i in range(len(classes)):
print(f"{classes[i]}: {int(round(paraphrase_results[i] * 100))}%")
not paraphrase: 10%
is paraphrase: 90%
代码
文本
[8]
# Should not be paraphrase
for i in range(len(classes)):
print(f"{classes[i]}: {int(round(not_paraphrase_results[i] * 100))}%")
not paraphrase: 94%
is paraphrase: 6%
代码
文本
[ ]
from transformers import AutoTokenizer, TFAutoModelForSequenceClassification
import tensorflow as tf

tokenizer = AutoTokenizer.from_pretrained("bert-base-cased-finetuned-mrpc")
model = TFAutoModelForSequenceClassification.from_pretrained("bert-base-cased-finetuned-mrpc")

classes = ["not paraphrase", "is paraphrase"]

sequence_0 = "The company HuggingFace is based in New York City"
sequence_1 = "Apples are especially bad for your health"
sequence_2 = "HuggingFace's headquarters are situated in Manhattan"

# The tokenizer will automatically add any model specific separators (i.e. <CLS> and <SEP>) and tokens to
# the sequence, as well as compute the attention masks.
paraphrase = tokenizer(sequence_0, sequence_2, return_tensors="tf")
not_paraphrase = tokenizer(sequence_0, sequence_1, return_tensors="tf")

paraphrase_classification_logits = model(paraphrase).logits
not_paraphrase_classification_logits = model(not_paraphrase).logits

paraphrase_results = tf.nn.softmax(paraphrase_classification_logits, axis=1).numpy()[0]
not_paraphrase_results = tf.nn.softmax(not_paraphrase_classification_logits, axis=1).numpy()[0]

# Should be paraphrase
for i in range(len(classes)):
print(f"{classes[i]}: {int(round(paraphrase_results[i] * 100))}%")
2024-05-18 10:16:24.874813: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcuda.so.1'; dlerror: /usr/lib/x86_64-linux-gnu/libcuda.so.1: file too short; LD_LIBRARY_PATH: /usr/mpi/gcc/openmpi-4.1.0rc5/lib:/usr/local/nccl-rdma-sharp-plugins/lib:/usr/local/nvidia/lib64:/usr/local/cuda/lib64:/usr/local/nvidia/lib:/usr/local/nvidia/lib64
2024-05-18 10:16:24.876324: W tensorflow/compiler/xla/stream_executor/cuda/cuda_driver.cc:265] failed call to cuInit: UNKNOWN ERROR (303)
All PyTorch model weights were used when initializing TFBertForSequenceClassification.

All the weights of TFBertForSequenceClassification were initialized from the PyTorch model.
If your task is similar to the task the model of the checkpoint was trained on, you can already use TFBertForSequenceClassification for predictions without further training.
not paraphrase: 10%
is paraphrase: 90%
代码
文本
[10]
# Should not be paraphrase
for i in range(len(classes)):
print(f"{classes[i]}: {int(round(not_paraphrase_results[i] * 100))}%")
not paraphrase: 94%
is paraphrase: 6%
代码
文本

Extractive Question Answering

代码
文本

Extractive Question Answering is the task of extracting an answer from a text given a question. An example of a question answering dataset is the SQuAD dataset, which is entirely based on that task. If you would like to fine-tune a model on a SQuAD task, you may leverage the run_qa.py and run_tf_squad.py scripts.

Here is an example of using pipelines to do question answering: extracting an answer from a text given a question. It leverages a fine-tuned model on SQuAD.

代码
文本
[11]
from transformers import pipeline

question_answerer = pipeline("question-answering")

context = r"""
Extractive Question Answering is the task of extracting an answer from a text given a question. An example of a
question answering dataset is the SQuAD dataset, which is entirely based on that task. If you would like to fine-tune
a model on a SQuAD task, you may leverage the examples/pytorch/question-answering/run_squad.py script.
"""
No model was supplied, defaulted to distilbert-base-cased-distilled-squad and revision 626af31 (https://huggingface.co/distilbert-base-cased-distilled-squad).
Using a pipeline without specifying a model name and revision in production is not recommended.
代码
文本

This returns an answer extracted from the text, a confidence score, alongside "start" and "end" values, which are the positions of the extracted answer in the text.

代码
文本
[13]
result = question_answerer(question="What is extractive question answering?", context=context)
print(
f"Answer: '{result['answer']}', score: {round(result['score'], 4)}, start: {result['start']}, end: {result['end']}"
)
Answer: 'the task of extracting an answer from a text given a question', score: 0.6177, start: 34, end: 95
代码
文本
[14]
result = question_answerer(question="What is a good example of a question answering dataset?", context=context)
print(
f"Answer: '{result['answer']}', score: {round(result['score'], 4)}, start: {result['start']}, end: {result['end']}"
)
Answer: 'SQuAD dataset', score: 0.5152, start: 147, end: 160
代码
文本

Here is an example of question answering using a model and a tokenizer. The process is the following:

  1. Instantiate a tokenizer and a model from the checkpoint name. The model is identified as a BERT model and loads it with the weights stored in the checkpoint.
  2. Define a text and a few questions.
  3. Iterate over the questions and build a sequence from the text and the current question, with the correct model-specific separators, token type ids and attention masks.
  4. Pass this sequence through the model. This outputs a range of scores across the entire sequence tokens (question and text), for both the start and end positions.
  5. Compute the softmax of the result to get probabilities over the tokens.
  6. Fetch the tokens from the identified start and stop values, convert those tokens to a string.
  7. Print the results.
代码
文本
[15]
from transformers import AutoTokenizer, AutoModelForQuestionAnswering
import torch

tokenizer = AutoTokenizer.from_pretrained("bert-large-uncased-whole-word-masking-finetuned-squad")
model = AutoModelForQuestionAnswering.from_pretrained("bert-large-uncased-whole-word-masking-finetuned-squad")

text = r"""
🤗 Transformers (formerly known as pytorch-transformers and pytorch-pretrained-bert) provides general-purpose
architectures (BERT, GPT-2, RoBERTa, XLM, DistilBert, XLNet…) for Natural Language Understanding (NLU) and Natural
Language Generation (NLG) with over 32+ pretrained models in 100+ languages and deep interoperability between
TensorFlow 2.0 and PyTorch.
"""

questions = [
"How many pretrained models are available in 🤗 Transformers?",
"What does 🤗 Transformers provide?",
"🤗 Transformers provides interoperability between which frameworks?",
]

for question in questions:
inputs = tokenizer(question, text, add_special_tokens=True, return_tensors="pt")
input_ids = inputs["input_ids"].tolist()[0]

outputs = model(**inputs)
answer_start_scores = outputs.start_logits
answer_end_scores = outputs.end_logits

# Get the most likely beginning of answer with the argmax of the score
answer_start = torch.argmax(answer_start_scores)
# Get the most likely end of answer with the argmax of the score
answer_end = torch.argmax(answer_end_scores) + 1

answer = tokenizer.convert_tokens_to_string(
tokenizer.convert_ids_to_tokens(input_ids[answer_start:answer_end])
)

print(f"Question: {question}")
print(f"Answer: {answer}")
Question: How many pretrained models are available in 🤗 Transformers?
Answer: over 32 +
Question: What does 🤗 Transformers provide?
Answer: general - purpose architectures
Question: 🤗 Transformers provides interoperability between which frameworks?
Answer: tensorflow 2. 0 and pytorch
代码
文本
[16]
from transformers import AutoTokenizer, TFAutoModelForQuestionAnswering
import tensorflow as tf

tokenizer = AutoTokenizer.from_pretrained("bert-large-uncased-whole-word-masking-finetuned-squad")
model = TFAutoModelForQuestionAnswering.from_pretrained("bert-large-uncased-whole-word-masking-finetuned-squad")

text = r"""
🤗 Transformers (formerly known as pytorch-transformers and pytorch-pretrained-bert) provides general-purpose
architectures (BERT, GPT-2, RoBERTa, XLM, DistilBert, XLNet…) for Natural Language Understanding (NLU) and Natural
Language Generation (NLG) with over 32+ pretrained models in 100+ languages and deep interoperability between
TensorFlow 2.0 and PyTorch.
"""

questions = [
"How many pretrained models are available in 🤗 Transformers?",
"What does 🤗 Transformers provide?",
"🤗 Transformers provides interoperability between which frameworks?",
]

for question in questions:
inputs = tokenizer(question, text, add_special_tokens=True, return_tensors="tf")
input_ids = inputs["input_ids"].numpy()[0]

outputs = model(inputs)
answer_start_scores = outputs.start_logits
answer_end_scores = outputs.end_logits

# Get the most likely beginning of answer with the argmax of the score
answer_start = tf.argmax(answer_start_scores, axis=1).numpy()[0]
# Get the most likely end of answer with the argmax of the score
answer_end = tf.argmax(answer_end_scores, axis=1).numpy()[0] + 1

answer = tokenizer.convert_tokens_to_string(
tokenizer.convert_ids_to_tokens(input_ids[answer_start:answer_end])
)

print(f"Question: {question}")
print(f"Answer: {answer}")
All PyTorch model weights were used when initializing TFBertForQuestionAnswering.

All the weights of TFBertForQuestionAnswering were initialized from the PyTorch model.
If your task is similar to the task the model of the checkpoint was trained on, you can already use TFBertForQuestionAnswering for predictions without further training.
Question: How many pretrained models are available in 🤗 Transformers?
Answer: over 32 +
Question: What does 🤗 Transformers provide?
Answer: general - purpose architectures
Question: 🤗 Transformers provides interoperability between which frameworks?
Answer: tensorflow 2. 0 and pytorch
代码
文本

Language Modeling

代码
文本

Language modeling is the task of fitting a model to a corpus, which can be domain specific. All popular transformer-based models are trained using a variant of language modeling, e.g. BERT with masked language modeling, GPT-2 with causal language modeling.

Language modeling can be useful outside of pretraining as well, for example to shift the model distribution to be domain-specific: using a language model trained over a very large corpus, and then fine-tuning it to a news dataset or on scientific papers e.g. LysandreJik/arxiv-nlp.

代码
文本

Masked Language Modeling

代码
文本

Masked language modeling is the task of masking tokens in a sequence with a masking token, and prompting the model to fill that mask with an appropriate token. This allows the model to attend to both the right context (tokens on the right of the mask) and the left context (tokens on the left of the mask). Such a training creates a strong basis for downstream tasks requiring bi-directional context, such as SQuAD (question answering, see Lewis, Lui, Goyal et al., part 4.2). If you would like to fine-tune a model on a masked language modeling task, you may leverage the run_mlm.py script.

Here is an example of using pipelines to replace a mask from a sequence:

代码
文本
[17]
from transformers import pipeline

unmasker = pipeline("fill-mask")
No model was supplied, defaulted to distilroberta-base and revision ec58a5b (https://huggingface.co/distilroberta-base).
Using a pipeline without specifying a model name and revision in production is not recommended.
代码
文本

This outputs the sequences with the mask filled, the confidence score, and the token id in the tokenizer vocabulary:

代码
文本
[18]
from pprint import pprint

pprint(
unmasker(
f"HuggingFace is creating a {unmasker.tokenizer.mask_token} that the community uses to solve NLP tasks."
)
)
[{'score': 0.17927584052085876,
  'sequence': 'HuggingFace is creating a tool that the community uses to solve '
              'NLP tasks.',
  'token': 3944,
  'token_str': ' tool'},
 {'score': 0.11349426209926605,
  'sequence': 'HuggingFace is creating a framework that the community uses to '
              'solve NLP tasks.',
  'token': 7208,
  'token_str': ' framework'},
 {'score': 0.05243551358580589,
  'sequence': 'HuggingFace is creating a library that the community uses to '
              'solve NLP tasks.',
  'token': 5560,
  'token_str': ' library'},
 {'score': 0.03493541106581688,
  'sequence': 'HuggingFace is creating a database that the community uses to '
              'solve NLP tasks.',
  'token': 8503,
  'token_str': ' database'},
 {'score': 0.02860243059694767,
  'sequence': 'HuggingFace is creating a prototype that the community uses to '
              'solve NLP tasks.',
  'token': 17715,
  'token_str': ' prototype'}]
代码
文本

Here is an example of doing masked language modeling using a model and a tokenizer. The process is the following:

  1. Instantiate a tokenizer and a model from the checkpoint name. The model is identified as a DistilBERT model and loads it with the weights stored in the checkpoint.
  2. Define a sequence with a masked token, placing the tokenizer.mask_token instead of a word.
  3. Encode that sequence into a list of IDs and find the position of the masked token in that list.
  4. Retrieve the predictions at the index of the mask token: this tensor has the same size as the vocabulary, and the values are the scores attributed to each token. The model gives higher score to tokens it deems probable in that context.
  5. Retrieve the top 5 tokens using the PyTorch topk or TensorFlow top_k methods.
  6. Replace the mask token by the tokens and print the results
代码
文本
[19]
from transformers import AutoModelForMaskedLM, AutoTokenizer
import torch

tokenizer = AutoTokenizer.from_pretrained("distilbert-base-cased")
model = AutoModelForMaskedLM.from_pretrained("distilbert-base-cased")

sequence = (
"Distilled models are smaller than the models they mimic. Using them instead of the large "
f"versions would help {tokenizer.mask_token} our carbon footprint."
)

inputs = tokenizer(sequence, return_tensors="pt")
mask_token_index = torch.where(inputs["input_ids"] == tokenizer.mask_token_id)[1]

token_logits = model(**inputs).logits
mask_token_logits = token_logits[0, mask_token_index, :]

top_5_tokens = torch.topk(mask_token_logits, 5, dim=1).indices[0].tolist()

for token in top_5_tokens:
print(sequence.replace(tokenizer.mask_token, tokenizer.decode([token])))
Distilled models are smaller than the models they mimic. Using them instead of the large versions would help reduce our carbon footprint.
Distilled models are smaller than the models they mimic. Using them instead of the large versions would help increase our carbon footprint.
Distilled models are smaller than the models they mimic. Using them instead of the large versions would help decrease our carbon footprint.
Distilled models are smaller than the models they mimic. Using them instead of the large versions would help offset our carbon footprint.
Distilled models are smaller than the models they mimic. Using them instead of the large versions would help improve our carbon footprint.
代码
文本
[20]
from transformers import TFAutoModelForMaskedLM, AutoTokenizer
import tensorflow as tf

tokenizer = AutoTokenizer.from_pretrained("distilbert-base-cased")
model = TFAutoModelForMaskedLM.from_pretrained("distilbert-base-cased")

sequence = (
"Distilled models are smaller than the models they mimic. Using them instead of the large "
f"versions would help {tokenizer.mask_token} our carbon footprint."
)

inputs = tokenizer(sequence, return_tensors="tf")
mask_token_index = tf.where(inputs["input_ids"] == tokenizer.mask_token_id)[0, 1]

token_logits = model(**inputs).logits
mask_token_logits = token_logits[0, mask_token_index, :]

top_5_tokens = tf.math.top_k(mask_token_logits, 5).indices.numpy()

for token in top_5_tokens:
print(sequence.replace(tokenizer.mask_token, tokenizer.decode([token])))
All PyTorch model weights were used when initializing TFDistilBertForMaskedLM.

All the weights of TFDistilBertForMaskedLM were initialized from the PyTorch model.
If your task is similar to the task the model of the checkpoint was trained on, you can already use TFDistilBertForMaskedLM for predictions without further training.
Distilled models are smaller than the models they mimic. Using them instead of the large versions would help reduce our carbon footprint.
Distilled models are smaller than the models they mimic. Using them instead of the large versions would help increase our carbon footprint.
Distilled models are smaller than the models they mimic. Using them instead of the large versions would help decrease our carbon footprint.
Distilled models are smaller than the models they mimic. Using them instead of the large versions would help offset our carbon footprint.
Distilled models are smaller than the models they mimic. Using them instead of the large versions would help improve our carbon footprint.
代码
文本

This prints five sequences, with the top 5 tokens predicted by the model.

代码
文本

Causal Language Modeling

代码
文本

Causal language modeling is the task of predicting the token following a sequence of tokens. In this situation, the model only attends to the left context (tokens on the left of the mask). Such a training is particularly interesting for generation tasks. If you would like to fine-tune a model on a causal language modeling task, you may leverage the run_clm.py script.

Usually, the next token is predicted by sampling from the logits of the last hidden state the model produces from the input sequence.

Here is an example of using the tokenizer and model and leveraging the top_k_top_p_filtering() method to sample the next token following an input sequence of tokens.

代码
文本
[22]
from transformers import AutoModelForCausalLM, AutoTokenizer, top_k_top_p_filtering
import torch
from torch import nn

tokenizer = AutoTokenizer.from_pretrained("gpt2")
model = AutoModelForCausalLM.from_pretrained("gpt2")

sequence = f"Hugging Face is based in DUMBO, New York City, and"

inputs = tokenizer(sequence, return_tensors="pt")
input_ids = inputs["input_ids"]

# get logits of last hidden state
next_token_logits = model(**inputs).logits[:, -1, :]

# filter
filtered_next_token_logits = top_k_top_p_filtering(next_token_logits, top_k=50, top_p=1.0)

# sample
probs = nn.functional.softmax(filtered_next_token_logits, dim=-1)
next_token = torch.multinomial(probs, num_samples=1)

generated = torch.cat([input_ids, next_token], dim=-1)

resulting_string = tokenizer.decode(generated.tolist()[0])
print(resulting_string)
Hugging Face is based in DUMBO, New York City, and was
代码
文本

Here is an example of using the tokenizer and model and leveraging the tf_top_k_top_p_filtering() method to sample the next token following an input sequence of tokens.

代码
文本
[23]
from transformers import TFAutoModelForCausalLM, AutoTokenizer, tf_top_k_top_p_filtering
import tensorflow as tf

tokenizer = AutoTokenizer.from_pretrained("gpt2")
model = TFAutoModelForCausalLM.from_pretrained("gpt2")

sequence = f"Hugging Face is based in DUMBO, New York City, and"

inputs = tokenizer(sequence, return_tensors="tf")
input_ids = inputs["input_ids"]

# get logits of last hidden state
next_token_logits = model(**inputs).logits[:, -1, :]

# filter
filtered_next_token_logits = tf_top_k_top_p_filtering(next_token_logits, top_k=50, top_p=1.0)

# sample
next_token = tf.random.categorical(filtered_next_token_logits, dtype=tf.int32, num_samples=1)

generated = tf.concat([input_ids, next_token], axis=1)

resulting_string = tokenizer.decode(generated.numpy().tolist()[0])
print(resulting_string)
All PyTorch model weights were used when initializing TFGPT2LMHeadModel.

All the weights of TFGPT2LMHeadModel were initialized from the PyTorch model.
If your task is similar to the task the model of the checkpoint was trained on, you can already use TFGPT2LMHeadModel for predictions without further training.
Hugging Face is based in DUMBO, New York City, and the
代码
文本

This outputs a (hopefully) coherent next token following the original sequence, which in our case is the word is or features.

In the next section, we show how generation.GenerationMixin.generate() can be used to generate multiple tokens up to a specified length instead of one token at a time.

代码
文本

Text Generation

代码
文本

In text generation (a.k.a open-ended text generation) the goal is to create a coherent portion of text that is a continuation from the given context. The following example shows how GPT-2 can be used in pipelines to generate text. As a default all models apply Top-K sampling when used in pipelines, as configured in their respective configurations (see gpt-2 config for example).

代码
文本
[24]
from transformers import pipeline

text_generator = pipeline("text-generation")
print(text_generator("As far as I am concerned, I will", max_length=50, do_sample=False))
No model was supplied, defaulted to gpt2 and revision 6c0e608 (https://huggingface.co/gpt2).
Using a pipeline without specifying a model name and revision in production is not recommended.
/opt/conda/lib/python3.8/site-packages/transformers/generation/utils.py:1201: UserWarning: You have modified the pretrained model configuration to control generation. This is a deprecated strategy to control generation and will be removed soon, in a future version. Please use a generation configuration file (see https://huggingface.co/docs/transformers/main_classes/text_generation)
  warnings.warn(
Setting `pad_token_id` to `eos_token_id`:50256 for open-end generation.
[{'generated_text': 'As far as I am concerned, I will be the first to admit that I am not a fan of the idea of a "free market." I think that the idea of a free market is a bit of a stretch. I think that the idea'}]
代码
文本

Here, the model generates a random text with a total maximal length of 50 tokens from context "As far as I am concerned, I will". Behind the scenes, the pipeline object calls the method PreTrainedModel.generate() to generate text. The default arguments for this method can be overridden in the pipeline, as is shown above for the arguments max_length and do_sample.

Below is an example of text generation using XLNet and its tokenizer, which includes calling generate() directly:

代码
文本
[25]
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("xlnet-base-cased")
tokenizer = AutoTokenizer.from_pretrained("xlnet-base-cased")

# Padding text helps XLNet with short prompts - proposed by Aman Rusia in https://github.com/rusiaaman/XLNet-gen#methodology
PADDING_TEXT = """In 1991, the remains of Russian Tsar Nicholas II and his family
(except for Alexei and Maria) are discovered.
The voice of Nicholas's young son, Tsarevich Alexei Nikolaevich, narrates the
remainder of the story. 1883 Western Siberia,
a young Grigori Rasputin is asked by his father and a group of men to perform magic.
Rasputin has a vision and denounces one of the men as a horse thief. Although his
father initially slaps him for making such an accusation, Rasputin watches as the
man is chased outside and beaten. Twenty years later, Rasputin sees a vision of
the Virgin Mary, prompting him to become a priest. Rasputin quickly becomes famous,
with people, even a bishop, begging for his blessing. <eod> </s> <eos>"""

prompt = "Today the weather is really nice and I am planning on "
inputs = tokenizer(PADDING_TEXT + prompt, add_special_tokens=False, return_tensors="pt")["input_ids"]

prompt_length = len(tokenizer.decode(inputs[0]))
outputs = model.generate(inputs, max_length=250, do_sample=True, top_p=0.95, top_k=60)
generated = prompt + tokenizer.decode(outputs[0])[prompt_length + 1 :]

print(generated)
Today the weather is really nice and I am planning on enjoying my time at D.L. University. I had been thinking about going to Penn. in September as I went to the National Mall. I thought that it was the perfect opportunity, but the weather was a little bit slow. I went to Penn. in August with the intention of going to Pittsburgh in September. I had been thinking about going to Penn. and
代码
文本
[26]
from transformers import TFAutoModelForCausalLM, AutoTokenizer

model = TFAutoModelForCausalLM.from_pretrained("xlnet-base-cased")
tokenizer = AutoTokenizer.from_pretrained("xlnet-base-cased")

# Padding text helps XLNet with short prompts - proposed by Aman Rusia in https://github.com/rusiaaman/XLNet-gen#methodology
PADDING_TEXT = """In 1991, the remains of Russian Tsar Nicholas II and his family
(except for Alexei and Maria) are discovered.
The voice of Nicholas's young son, Tsarevich Alexei Nikolaevich, narrates the
remainder of the story. 1883 Western Siberia,
a young Grigori Rasputin is asked by his father and a group of men to perform magic.
Rasputin has a vision and denounces one of the men as a horse thief. Although his
father initially slaps him for making such an accusation, Rasputin watches as the
man is chased outside and beaten. Twenty years later, Rasputin sees a vision of
the Virgin Mary, prompting him to become a priest. Rasputin quickly becomes famous,
with people, even a bishop, begging for his blessing. <eod> </s> <eos>"""

prompt = "Today the weather is really nice and I am planning on "
inputs = tokenizer(PADDING_TEXT + prompt, add_special_tokens=False, return_tensors="tf")["input_ids"]

prompt_length = len(tokenizer.decode(inputs[0]))
outputs = model.generate(inputs, max_length=250, do_sample=True, top_p=0.95, top_k=60)
generated = prompt + tokenizer.decode(outputs[0])[prompt_length + 1 :]

print(generated)
/opt/conda/lib/python3.8/site-packages/keras/initializers/initializers_v2.py:120: UserWarning: The initializer TruncatedNormal is unseeded and being called multiple times, which will return identical values  each time (even if the initializer is unseeded). Please update your code to provide a seed to the initializer, or avoid using the same initalizer instance more than once.
  warnings.warn(
All model checkpoint layers were used when initializing TFXLNetLMHeadModel.

All the layers of TFXLNetLMHeadModel were initialized from the model checkpoint at xlnet-base-cased.
If your task is similar to the task the model of the checkpoint was trained on, you can already use TFXLNetLMHeadModel for predictions without further training.
Today the weather is really nice and I am planning on riding a river again. I also feel like I am going to be going out and looking up the best restaurants. I’m feeling really good.<eop> Thanks for reading my article. I do hope you enjoy the piece. It has been wonderful to write it. I also really enjoyed what I was in and how my work was put together. I definitely like the way
代码
文本

Text generation is currently possible with GPT-2, OpenAi-GPT, CTRL, XLNet, Transfo-XL and Reformer in PyTorch and for most models in Tensorflow as well. As can be seen in the example above XLNet and Transfo-XL often need to be padded to work well. GPT-2 is usually a good choice for open-ended text generation because it was trained on millions of webpages with a causal language modeling objective.

For more information on how to apply different decoding strategies for text generation, please also refer to our text generation blog post here.

代码
文本

Named Entity Recognition

代码
文本

Named Entity Recognition (NER) is the task of classifying tokens according to a class, for example, identifying a token as a person, an organisation or a location. An example of a named entity recognition dataset is the CoNLL-2003 dataset, which is entirely based on that task. If you would like to fine-tune a model on an NER task, you may leverage the run_ner.py script.

Here is an example of using pipelines to do named entity recognition, specifically, trying to identify tokens as belonging to one of 9 classes:

  • O, Outside of a named entity
  • B-MIS, Beginning of a miscellaneous entity right after another miscellaneous entity
  • I-MIS, Miscellaneous entity
  • B-PER, Beginning of a person's name right after another person's name
  • I-PER, Person's name
  • B-ORG, Beginning of an organisation right after another organisation
  • I-ORG, Organisation
  • B-LOC, Beginning of a location right after another location
  • I-LOC, Location

It leverages a fine-tuned model on CoNLL-2003, fine-tuned by @stefan-it from dbmdz.

代码
文本
[27]
from transformers import pipeline

ner_pipe = pipeline("ner")

sequence = """Hugging Face Inc. is a company based in New York City. Its headquarters are in DUMBO,
therefore very close to the Manhattan Bridge which is visible from the window."""
No model was supplied, defaulted to dbmdz/bert-large-cased-finetuned-conll03-english and revision f2482bf (https://huggingface.co/dbmdz/bert-large-cased-finetuned-conll03-english).
Using a pipeline without specifying a model name and revision in production is not recommended.
代码
文本

This outputs a list of all words that have been identified as one of the entities from the 9 classes defined above. Here are the expected results:

代码
文本
[28]
for entity in ner_pipe(sequence):
print(entity)
{'entity': 'I-ORG', 'score': 0.99957865, 'index': 1, 'word': 'Hu', 'start': 0, 'end': 2}
{'entity': 'I-ORG', 'score': 0.9909764, 'index': 2, 'word': '##gging', 'start': 2, 'end': 7}
{'entity': 'I-ORG', 'score': 0.9982224, 'index': 3, 'word': 'Face', 'start': 8, 'end': 12}
{'entity': 'I-ORG', 'score': 0.9994879, 'index': 4, 'word': 'Inc', 'start': 13, 'end': 16}
{'entity': 'I-LOC', 'score': 0.9994344, 'index': 11, 'word': 'New', 'start': 40, 'end': 43}
{'entity': 'I-LOC', 'score': 0.99931955, 'index': 12, 'word': 'York', 'start': 44, 'end': 48}
{'entity': 'I-LOC', 'score': 0.9993794, 'index': 13, 'word': 'City', 'start': 49, 'end': 53}
{'entity': 'I-LOC', 'score': 0.98625827, 'index': 19, 'word': 'D', 'start': 79, 'end': 80}
{'entity': 'I-LOC', 'score': 0.95142686, 'index': 20, 'word': '##UM', 'start': 80, 'end': 82}
{'entity': 'I-LOC', 'score': 0.933659, 'index': 21, 'word': '##BO', 'start': 82, 'end': 84}
{'entity': 'I-LOC', 'score': 0.9761654, 'index': 28, 'word': 'Manhattan', 'start': 114, 'end': 123}
{'entity': 'I-LOC', 'score': 0.9914629, 'index': 29, 'word': 'Bridge', 'start': 124, 'end': 130}
代码
文本

Note how the tokens of the sequence "Hugging Face" have been identified as an organisation, and "New York City", "DUMBO" and "Manhattan Bridge" have been identified as locations.

Here is an example of doing named entity recognition, using a model and a tokenizer. The process is the following:

  1. Instantiate a tokenizer and a model from the checkpoint name. The model is identified as a BERT model and loads it with the weights stored in the checkpoint.
  2. Define a sequence with known entities, such as "Hugging Face" as an organisation and "New York City" as a location.
  3. Split words into tokens so that they can be mapped to predictions. We use a small hack by, first, completely encoding and decoding the sequence, so that we're left with a string that contains the special tokens.
  4. Encode that sequence into IDs (special tokens are added automatically).
  5. Retrieve the predictions by passing the input to the model and getting the first output. This results in a distribution over the 9 possible classes for each token. We take the argmax to retrieve the most likely class for each token.
  6. Zip together each token with its prediction and print it.
代码
文本
[29]
from transformers import AutoModelForTokenClassification, AutoTokenizer
import torch

model = AutoModelForTokenClassification.from_pretrained("dbmdz/bert-large-cased-finetuned-conll03-english")
tokenizer = AutoTokenizer.from_pretrained("bert-base-cased")

sequence = (
"Hugging Face Inc. is a company based in New York City. Its headquarters are in DUMBO, "
"therefore very close to the Manhattan Bridge."
)

inputs = tokenizer(sequence, return_tensors="pt")
tokens = inputs.tokens()

outputs = model(**inputs).logits
predictions = torch.argmax(outputs, dim=2)
代码
文本
[30]
from transformers import TFAutoModelForTokenClassification, AutoTokenizer
import tensorflow as tf

model = TFAutoModelForTokenClassification.from_pretrained("dbmdz/bert-large-cased-finetuned-conll03-english")
tokenizer = AutoTokenizer.from_pretrained("bert-base-cased")

sequence = (
"Hugging Face Inc. is a company based in New York City. Its headquarters are in DUMBO, "
"therefore very close to the Manhattan Bridge."
)

inputs = tokenizer(sequence, return_tensors="tf")
tokens = inputs.tokens()

outputs = model(**inputs)[0]
predictions = tf.argmax(outputs, axis=2)
All PyTorch model weights were used when initializing TFBertForTokenClassification.

All the weights of TFBertForTokenClassification were initialized from the PyTorch model.
If your task is similar to the task the model of the checkpoint was trained on, you can already use TFBertForTokenClassification for predictions without further training.
代码
文本

This outputs a list of each token mapped to its corresponding prediction. Differently from the pipeline, here every token has a prediction as we didn't remove the "0"th class, which means that no particular entity was found on that token.

In the above example, predictions is an integer that corresponds to the predicted class. We can use the model.config.id2label property in order to recover the class name corresponding to the class number, which is illustrated below:

代码
文本
[31]
for token, prediction in zip(tokens, predictions[0].numpy()):
print((token, model.config.id2label[prediction]))
('[CLS]', 'O')
('Hu', 'I-ORG')
('##gging', 'I-ORG')
('Face', 'I-ORG')
('Inc', 'I-ORG')
('.', 'O')
('is', 'O')
('a', 'O')
('company', 'O')
('based', 'O')
('in', 'O')
('New', 'I-LOC')
('York', 'I-LOC')
('City', 'I-LOC')
('.', 'O')
('Its', 'O')
('headquarters', 'O')
('are', 'O')
('in', 'O')
('D', 'I-LOC')
('##UM', 'I-LOC')
('##BO', 'I-LOC')
(',', 'O')
('therefore', 'O')
('very', 'O')
('close', 'O')
('to', 'O')
('the', 'O')
('Manhattan', 'I-LOC')
('Bridge', 'I-LOC')
('.', 'O')
('[SEP]', 'O')
代码
文本

Summarization

代码
文本

Summarization is the task of summarizing a document or an article into a shorter text. If you would like to fine-tune a model on a summarization task, you may leverage the run_summarization.py script.

An example of a summarization dataset is the CNN / Daily Mail dataset, which consists of long news articles and was created for the task of summarization. If you would like to fine-tune a model on a summarization task, various approaches are described in this document.

Here is an example of using the pipelines to do summarization. It leverages a Bart model that was fine-tuned on the CNN / Daily Mail data set.

代码
文本
[32]
from transformers import pipeline

summarizer = pipeline("summarization")

ARTICLE = """ New York (CNN)When Liana Barrientos was 23 years old, she got married in Westchester County, New York.
A year later, she got married again in Westchester County, but to a different man and without divorcing her first husband.
Only 18 days after that marriage, she got hitched yet again. Then, Barrientos declared "I do" five more times, sometimes only within two weeks of each other.
In 2010, she married once more, this time in the Bronx. In an application for a marriage license, she stated it was her "first and only" marriage.
Barrientos, now 39, is facing two criminal counts of "offering a false instrument for filing in the first degree," referring to her false statements on the
2010 marriage license application, according to court documents.
Prosecutors said the marriages were part of an immigration scam.
On Friday, she pleaded not guilty at State Supreme Court in the Bronx, according to her attorney, Christopher Wright, who declined to comment further.
After leaving court, Barrientos was arrested and charged with theft of service and criminal trespass for allegedly sneaking into the New York subway through an emergency exit, said Detective
Annette Markowski, a police spokeswoman. In total, Barrientos has been married 10 times, with nine of her marriages occurring between 1999 and 2002.
All occurred either in Westchester County, Long Island, New Jersey or the Bronx. She is believed to still be married to four men, and at one time, she was married to eight men at once, prosecutors say.
Prosecutors said the immigration scam involved some of her husbands, who filed for permanent residence status shortly after the marriages.
Any divorces happened only after such filings were approved. It was unclear whether any of the men will be prosecuted.
The case was referred to the Bronx District Attorney\'s Office by Immigration and Customs Enforcement and the Department of Homeland Security\'s
Investigation Division. Seven of the men are from so-called "red-flagged" countries, including Egypt, Turkey, Georgia, Pakistan and Mali.
Her eighth husband, Rashid Rajput, was deported in 2006 to his native Pakistan after an investigation by the Joint Terrorism Task Force.
If convicted, Barrientos faces up to four years in prison. Her next court appearance is scheduled for May 18.
"""
No model was supplied, defaulted to sshleifer/distilbart-cnn-12-6 and revision a4f8f3e (https://huggingface.co/sshleifer/distilbart-cnn-12-6).
Using a pipeline without specifying a model name and revision in production is not recommended.
代码
文本

Because the summarization pipeline depends on the PreTrainedModel.generate() method, we can override the default arguments of PreTrainedModel.generate() directly in the pipeline for max_length and min_length as shown below. This outputs the following summary:

代码
文本
[33]
print(summarizer(ARTICLE, max_length=130, min_length=30, do_sample=False))
[{'summary_text': ' Liana Barrientos, 39, is charged with two counts of "offering a false instrument for filing in the first degree" In total, she has been married 10 times, with nine of her marriages occurring between 1999 and 2002 . At one time, she was married to eight men at once, prosecutors say .'}]
代码
文本

Here is an example of doing summarization using a model and a tokenizer. The process is the following:

  1. Instantiate a tokenizer and a model from the checkpoint name. Summarization is usually done using an encoder-decoder model, such as Bart or T5.
  2. Define the article that should be summarized.
  3. Add the T5 specific prefix "summarize: ".
  4. Use the PreTrainedModel.generate() method to generate the summary.

In this example we use Google's T5 model. Even though it was pre-trained only on a multi-task mixed dataset (including CNN / Daily Mail), it yields very good results.

代码
文本
[34]
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

model = AutoModelForSeq2SeqLM.from_pretrained("t5-base")
tokenizer = AutoTokenizer.from_pretrained("t5-base")

# T5 uses a max_length of 512 so we cut the article to 512 tokens.
inputs = tokenizer("summarize: " + ARTICLE, return_tensors="pt", max_length=512, truncation=True)
outputs = model.generate(
inputs["input_ids"], max_length=150, min_length=40, length_penalty=2.0, num_beams=4, early_stopping=True
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))
/opt/conda/lib/python3.8/site-packages/transformers/models/t5/tokenization_t5_fast.py:155: FutureWarning: This tokenizer was incorrectly instantiated with a model max length of 512 which will be corrected in Transformers v5.
For now, this behavior is kept to avoid breaking backwards compatibility when padding/encoding with `truncation is True`.
- Be aware that you SHOULD NOT rely on t5-base automatically truncating your input to 512 when padding/encoding.
- If you want to encode/pad to sequences longer than 512 you can either instantiate this tokenizer with `model_max_length` or pass `max_length` when encoding/padding.
- To avoid this warning, please instantiate this tokenizer with `model_max_length` set to your preferred value.
  warnings.warn(
prosecutors say the marriages were part of an immigration scam. if convicted, barrientos faces two criminal counts of "offering a false instrument for filing in the first degree" she has been married 10 times, nine of them between 1999 and 2002.
代码
文本
[35]
from transformers import TFAutoModelForSeq2SeqLM, AutoTokenizer

model = TFAutoModelForSeq2SeqLM.from_pretrained("t5-base")
tokenizer = AutoTokenizer.from_pretrained("t5-base")

# T5 uses a max_length of 512 so we cut the article to 512 tokens.
inputs = tokenizer("summarize: " + ARTICLE, return_tensors="tf", max_length=512)
outputs = model.generate(
inputs["input_ids"], max_length=150, min_length=40, length_penalty=2.0, num_beams=4, early_stopping=True
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))
All PyTorch model weights were used when initializing TFT5ForConditionalGeneration.

All the weights of TFT5ForConditionalGeneration were initialized from the PyTorch model.
If your task is similar to the task the model of the checkpoint was trained on, you can already use TFT5ForConditionalGeneration for predictions without further training.
Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.
prosecutors say the marriages were part of an immigration scam. if convicted, barrientos faces two criminal counts of "offering a false instrument for filing in the first degree" she has been married 10 times, nine of them between 1999 and 2002.
代码
文本

Translation

代码
文本

Translation is the task of translating a text from one language to another. If you would like to fine-tune a model on a translation task, you may leverage the run_translation.py script.

An example of a translation dataset is the WMT English to German dataset, which has sentences in English as the input data and the corresponding sentences in German as the target data. If you would like to fine-tune a model on a translation task, various approaches are described in this document.

Here is an example of using the pipelines to do translation. It leverages a T5 model that was only pre-trained on a multi-task mixture dataset (including WMT), yet, yielding impressive translation results.

代码
文本
[36]
from transformers import pipeline

translator = pipeline("translation_en_to_de")
print(translator("Hugging Face is a technology company based in New York and Paris", max_length=40))
No model was supplied, defaulted to t5-base and revision 686f1db (https://huggingface.co/t5-base).
Using a pipeline without specifying a model name and revision in production is not recommended.
[{'translation_text': 'Hugging Face ist ein Technologieunternehmen mit Sitz in New York und Paris.'}]
代码
文本

Because the translation pipeline depends on the PreTrainedModel.generate() method, we can override the default arguments of PreTrainedModel.generate() directly in the pipeline as is shown for max_length above.

Here is an example of doing translation using a model and a tokenizer. The process is the following:

  1. Instantiate a tokenizer and a model from the checkpoint name. Summarization is usually done using an encoder-decoder model, such as Bart or T5.
  2. Define the article that should be summarized.
  3. Add the T5 specific prefix "translate English to German: "
  4. Use the PreTrainedModel.generate() method to perform the translation.
代码
文本
[37]
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

model = AutoModelForSeq2SeqLM.from_pretrained("t5-base")
tokenizer = AutoTokenizer.from_pretrained("t5-base")

inputs = tokenizer(
"translate English to German: Hugging Face is a technology company based in New York and Paris",
return_tensors="pt",
)
outputs = model.generate(inputs["input_ids"], max_length=40, num_beams=4, early_stopping=True)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Hugging Face ist ein Technologieunternehmen mit Sitz in New York und Paris.
代码
文本
[38]
from transformers import TFAutoModelForSeq2SeqLM, AutoTokenizer

model = TFAutoModelForSeq2SeqLM.from_pretrained("t5-base")
tokenizer = AutoTokenizer.from_pretrained("t5-base")

inputs = tokenizer(
"translate English to German: Hugging Face is a technology company based in New York and Paris",
return_tensors="tf",
)
outputs = model.generate(inputs["input_ids"], max_length=40, num_beams=4, early_stopping=True)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))
All PyTorch model weights were used when initializing TFT5ForConditionalGeneration.

All the weights of TFT5ForConditionalGeneration were initialized from the PyTorch model.
If your task is similar to the task the model of the checkpoint was trained on, you can already use TFT5ForConditionalGeneration for predictions without further training.
Hugging Face ist ein Technologieunternehmen mit Sitz in New York und Paris.
代码
文本
[46]
!apt update
!apt install ffmpeg -y
Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease                         
Get:2 https://deb.nodesource.com/node_18.x focal InRelease [4583 B]            m
Get:3 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB]      
Get:4 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB]        
Get:5 https://deb.nodesource.com/node_18.x focal/main amd64 Packages [776 B]   
Get:6 https://developer.download.nvidia.cn/compute/cuda/repos/ubuntu2004/x86_64  InRelease [1581 B][33m
Get:7 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB]      
Get:8 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [1205 kB]
Get:9 https://developer.download.nvidia.cn/compute/cuda/repos/ubuntu2004/x86_64  Packages [1520 kB]
Get:10 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1502 kB]m
Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [3608 kB]
Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [4143 kB]
Get:13 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [3669 kB]33m
Get:14 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [3758 kB]
Get:15 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [29.8 kB]3m
Get:16 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [32.5 kB][33m
Get:17 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [55.2 kB]
Get:18 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [28.6 kB]
Fetched 19.9 MB in 5s (4028 kB/s)[33m                   
Reading package lists... Done
Building dependency tree       
Reading state information... Done
163 packages can be upgraded. Run 'apt list --upgradable' to see them.
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following additional packages will be installed:
  i965-va-driver intel-media-va-driver libaacs0 libaom0 libass9 libasyncns0
  libavc1394-0 libavcodec58 libavdevice58 libavfilter7 libavformat58
  libavresample4 libavutil56 libbdplus0 libbluray2 libbs2b0 libcaca0
  libcdio-cdda2 libcdio-paranoia2 libcdio18 libchromaprint1 libcodec2-0.9
  libdc1394-22 libflac8 libflite1 libgme0 libgsm1 libiec61883-0 libigdgmm11
  libjack-jackd2-0 liblilv-0-0 libmp3lame0 libmpg123-0 libmysofa1 libnorm1
  libopenal-data libopenal1 libopenjp2-7 libopenmpt0 libopus0 libpgm-5.2-0
  libpostproc55 libpulse0 libraw1394-11 librubberband2 libsamplerate0
  libsdl2-2.0-0 libserd-0-0 libshine3 libslang2 libsnappy1v5 libsndfile1
  libsndio7.0 libsodium23 libsord-0-0 libsoxr0 libspeex1 libsratom-0-0
  libssh-gcrypt-4 libswresample3 libswscale5 libtheora0 libtwolame0 libva-drm2
  libva-x11-2 libva2 libvdpau1 libvidstab1.1 libvorbisenc2 libvpx6 libwavpack1
  libx264-155 libx265-179 libxvidcore4 libzmq5 libzvbi-common libzvbi0
  mesa-va-drivers mesa-vdpau-drivers ocl-icd-libopencl1 va-driver-all
  vdpau-driver-all
Suggested packages:
  ffmpeg-doc i965-va-driver-shaders libbluray-bdj jackd2 libportaudio2
  opus-tools pulseaudio libraw1394-doc serdi sndiod sordi speex opencl-icd
  libvdpau-va-gl1 nvidia-vdpau-driver nvidia-legacy-340xx-vdpau-driver
  nvidia-legacy-304xx-vdpau-driver
The following NEW packages will be installed:
  ffmpeg i965-va-driver intel-media-va-driver libaacs0 libaom0 libass9
  libasyncns0 libavc1394-0 libavcodec58 libavdevice58 libavfilter7
  libavformat58 libavresample4 libavutil56 libbdplus0 libbluray2 libbs2b0
  libcaca0 libcdio-cdda2 libcdio-paranoia2 libcdio18 libchromaprint1
  libcodec2-0.9 libdc1394-22 libflac8 libflite1 libgme0 libgsm1 libiec61883-0
  libigdgmm11 libjack-jackd2-0 liblilv-0-0 libmp3lame0 libmpg123-0 libmysofa1
  libnorm1 libopenal-data libopenal1 libopenjp2-7 libopenmpt0 libopus0
  libpgm-5.2-0 libpostproc55 libpulse0 libraw1394-11 librubberband2
  libsamplerate0 libsdl2-2.0-0 libserd-0-0 libshine3 libslang2 libsnappy1v5
  libsndfile1 libsndio7.0 libsodium23 libsord-0-0 libsoxr0 libspeex1
  libsratom-0-0 libssh-gcrypt-4 libswresample3 libswscale5 libtheora0
  libtwolame0 libva-drm2 libva-x11-2 libva2 libvdpau1 libvidstab1.1
  libvorbisenc2 libvpx6 libwavpack1 libx264-155 libx265-179 libxvidcore4
  libzmq5 libzvbi-common libzvbi0 mesa-va-drivers mesa-vdpau-drivers
  ocl-icd-libopencl1 va-driver-all vdpau-driver-all
0 upgraded, 83 newly installed, 0 to remove and 163 not upgraded.
Need to get 50.0 MB of archives.
After this operation, 221 MB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 libslang2 amd64 2.3.2-4 [429 kB]
Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 libsodium23 amd64 1.0.18-1 [150 kB]
Get:3 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libaom0 amd64 1.0.0.errata1-3+deb11u1build0.20.04.1 [1161 kB]
Get:4 http://archive.ubuntu.com/ubuntu focal/universe amd64 libva2 amd64 2.7.0-2 [53.5 kB]
Get:5 http://archive.ubuntu.com/ubuntu focal/universe amd64 libva-drm2 amd64 2.7.0-2 [7044 B]
Get:6 http://archive.ubuntu.com/ubuntu focal/universe amd64 libva-x11-2 amd64 2.7.0-2 [11.9 kB]
Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 libvdpau1 amd64 1.3-1ubuntu2 [25.6 kB]
Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 ocl-icd-libopencl1 amd64 2.2.11-1ubuntu1 [30.3 kB]
Get:9 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libavutil56 amd64 7:4.2.7-0ubuntu0.1 [241 kB]
Get:10 http://archive.ubuntu.com/ubuntu focal/universe amd64 libcodec2-0.9 amd64 0.9.2-2 [7886 kB]
Get:11 http://archive.ubuntu.com/ubuntu focal/universe amd64 libgsm1 amd64 1.0.18-2 [24.4 kB]
Get:12 http://archive.ubuntu.com/ubuntu focal/main amd64 libmp3lame0 amd64 3.100-3 [133 kB]
Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libopenjp2-7 amd64 2.3.1-1ubuntu4.20.04.1 [141 kB]
Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 libopus0 amd64 1.3.1-0ubuntu1 [191 kB]
Get:15 http://archive.ubuntu.com/ubuntu focal/universe amd64 libshine3 amd64 3.1.1-2 [23.2 kB]
Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 libsnappy1v5 amd64 1.1.8-1build1 [16.7 kB]
Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libspeex1 amd64 1.2~rc1.2-1.1ubuntu1.20.04.1 [53.2 kB]
Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 libsoxr0 amd64 0.1.3-2build1 [78.0 kB]
Get:19 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libswresample3 amd64 7:4.2.7-0ubuntu0.1 [57.1 kB]
Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 libtheora0 amd64 1.1.1+dfsg.1-15ubuntu2 [162 kB]
Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 libtwolame0 amd64 0.4.0-2 [47.6 kB]
Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 libvorbisenc2 amd64 1.3.6-2ubuntu1 [70.7 kB]
Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libvpx6 amd64 1.8.2-1ubuntu0.2 [820 kB]
Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libwavpack1 amd64 5.2.0-1ubuntu0.1 [77.3 kB]
Get:25 http://archive.ubuntu.com/ubuntu focal/universe amd64 libx264-155 amd64 2:0.155.2917+git0a84d98-2 [521 kB]
Get:26 http://archive.ubuntu.com/ubuntu focal/universe amd64 libx265-179 amd64 3.2.1-1build1 [1060 kB]
Get:27 http://archive.ubuntu.com/ubuntu focal/universe amd64 libxvidcore4 amd64 2:1.3.7-1 [201 kB]
Get:28 http://archive.ubuntu.com/ubuntu focal/universe amd64 libzvbi-common all 0.2.35-17 [32.5 kB]
Get:29 http://archive.ubuntu.com/ubuntu focal/universe amd64 libzvbi0 amd64 0.2.35-17 [237 kB]
Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libavcodec58 amd64 7:4.2.7-0ubuntu0.1 [4878 kB]
Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 libraw1394-11 amd64 2.1.2-1 [30.7 kB]
Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 libavc1394-0 amd64 0.5.4-5 [16.2 kB]
Get:33 http://archive.ubuntu.com/ubuntu focal/universe amd64 libass9 amd64 1:0.14.0-2 [88.0 kB]
Get:34 http://archive.ubuntu.com/ubuntu focal/universe amd64 libbluray2 amd64 1:1.2.0-1 [138 kB]
Get:35 http://archive.ubuntu.com/ubuntu focal/universe amd64 libchromaprint1 amd64 1.4.3-3build1 [37.6 kB]
Get:36 http://archive.ubuntu.com/ubuntu focal/universe amd64 libgme0 amd64 0.6.2-1build1 [123 kB]
Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 libmpg123-0 amd64 1.25.13-1 [124 kB]
Get:38 http://archive.ubuntu.com/ubuntu focal/universe amd64 libopenmpt0 amd64 0.4.11-1build1 [599 kB]
Get:39 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssh-gcrypt-4 amd64 0.9.3-2ubuntu2.5 [202 kB]
Get:40 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libavformat58 amd64 7:4.2.7-0ubuntu0.1 [985 kB]
Get:41 http://archive.ubuntu.com/ubuntu focal/universe amd64 libbs2b0 amd64 3.1.0+dfsg-2.2build1 [10.2 kB]
Get:42 http://archive.ubuntu.com/ubuntu focal/universe amd64 libflite1 amd64 2.1-release-3 [12.8 MB]
Get:43 http://archive.ubuntu.com/ubuntu focal/universe amd64 libserd-0-0 amd64 0.30.2-1 [46.6 kB]m
Get:44 http://archive.ubuntu.com/ubuntu focal/universe amd64 libsord-0-0 amd64 0.16.4-1 [19.5 kB]
Get:45 http://archive.ubuntu.com/ubuntu focal/universe amd64 libsratom-0-0 amd64 0.6.4-1 [16.9 kB]
Get:46 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 liblilv-0-0 amd64 0.24.6-1ubuntu0.1 [40.6 kB]
Get:47 http://archive.ubuntu.com/ubuntu focal/universe amd64 libmysofa1 amd64 1.0~dfsg0-1 [39.2 kB]
Get:48 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libpostproc55 amd64 7:4.2.7-0ubuntu0.1 [55.0 kB]
Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 libsamplerate0 amd64 0.1.9-2 [939 kB]
Get:50 http://archive.ubuntu.com/ubuntu focal/universe amd64 librubberband2 amd64 1.8.2-1build1 [89.4 kB]
Get:51 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libswscale5 amd64 7:4.2.7-0ubuntu0.1 [157 kB]
Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 libvidstab1.1 amd64 1.1.0-2 [35.0 kB]
Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 libnorm1 amd64 1.5.8+dfsg2-2build1 [290 kB]
Get:54 http://archive.ubuntu.com/ubuntu focal/universe amd64 libpgm-5.2-0 amd64 5.2.122~dfsg-3ubuntu1 [158 kB]
Get:55 http://archive.ubuntu.com/ubuntu focal/universe amd64 libzmq5 amd64 4.3.2-2ubuntu1 [242 kB]
Get:56 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libavfilter7 amd64 7:4.2.7-0ubuntu0.1 [1085 kB]
Get:57 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libcaca0 amd64 0.99.beta19-2.1ubuntu1.20.04.2 [203 kB]
Get:58 http://archive.ubuntu.com/ubuntu focal/main amd64 libcdio18 amd64 2.0.0-2 [58.6 kB]
Get:59 http://archive.ubuntu.com/ubuntu focal/main amd64 libcdio-cdda2 amd64 10.2+2.0.0-1 [17.6 kB]
Get:60 http://archive.ubuntu.com/ubuntu focal/main amd64 libcdio-paranoia2 amd64 10.2+2.0.0-1 [16.2 kB]
Get:61 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdc1394-22 amd64 2.2.5-2.1 [79.6 kB]
Get:62 http://archive.ubuntu.com/ubuntu focal/main amd64 libiec61883-0 amd64 1.2.0-3 [24.3 kB]
Get:63 http://archive.ubuntu.com/ubuntu focal/main amd64 libjack-jackd2-0 amd64 1.9.12~dfsg-2ubuntu2 [267 kB]
Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 libopenal-data all 1:1.19.1-1 [162 kB]
Get:65 http://archive.ubuntu.com/ubuntu focal/universe amd64 libsndio7.0 amd64 1.5.0-3 [24.5 kB]
Get:66 http://archive.ubuntu.com/ubuntu focal/universe amd64 libopenal1 amd64 1:1.19.1-1 [492 kB]
Get:67 http://archive.ubuntu.com/ubuntu focal/main amd64 libasyncns0 amd64 0.8-6 [12.1 kB]
Get:68 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libflac8 amd64 1.3.3-1ubuntu0.2 [103 kB]
Get:69 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libsndfile1 amd64 1.0.28-7ubuntu0.2 [170 kB]
Get:70 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpulse0 amd64 1:13.99.1-1ubuntu3.13 [262 kB]
Get:71 http://archive.ubuntu.com/ubuntu focal/universe amd64 libsdl2-2.0-0 amd64 2.0.10+dfsg1-3 [407 kB]
Get:72 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libavdevice58 amd64 7:4.2.7-0ubuntu0.1 [74.3 kB]
Get:73 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libavresample4 amd64 7:4.2.7-0ubuntu0.1 [54.2 kB]
Get:74 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 ffmpeg amd64 7:4.2.7-0ubuntu0.1 [1453 kB]
Get:75 http://archive.ubuntu.com/ubuntu focal/universe amd64 libigdgmm11 amd64 20.1.1+ds1-1 [111 kB]
Get:76 http://archive.ubuntu.com/ubuntu focal/universe amd64 intel-media-va-driver amd64 20.1.1+dfsg1-1 [1764 kB]
Get:77 http://archive.ubuntu.com/ubuntu focal/universe amd64 libaacs0 amd64 0.9.0-2 [50.1 kB]
Get:78 http://archive.ubuntu.com/ubuntu focal/universe amd64 libbdplus0 amd64 0.1.2-3 [47.3 kB]
Get:79 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 mesa-va-drivers amd64 21.2.6-0ubuntu0.1~20.04.2 [2970 kB]
Get:80 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 mesa-vdpau-drivers amd64 21.2.6-0ubuntu0.1~20.04.2 [3089 kB]
Get:81 http://archive.ubuntu.com/ubuntu focal/universe amd64 i965-va-driver amd64 2.4.0-0ubuntu1 [924 kB]
Get:82 http://archive.ubuntu.com/ubuntu focal/universe amd64 va-driver-all amd64 2.7.0-2 [4020 B]
Get:83 http://archive.ubuntu.com/ubuntu focal/main amd64 vdpau-driver-all amd64 1.3-1ubuntu2 [4596 B]
Fetched 50.0 MB in 27s (1829 kB/s)                                             
Extracting templates from packages: 100%

78Selecting previously unselected package libslang2:amd64.
(Reading database ... 63384 files and directories currently installed.)
Preparing to unpack .../00-libslang2_2.3.2-4_amd64.deb ...
7Progress: [  0%] [..........................................................] 8Unpacking libslang2:amd64 (2.3.2-4) ...
Selecting previously unselected package libsodium23:amd64.
Preparing to unpack .../01-libsodium23_1.0.18-1_amd64.deb ...
Unpacking libsodium23:amd64 (1.0.18-1) ...
7Progress: [  1%] [..........................................................] 8Selecting previously unselected package libaom0:amd64.
Preparing to unpack .../02-libaom0_1.0.0.errata1-3+deb11u1build0.20.04.1_amd64.deb ...
Unpacking libaom0:amd64 (1.0.0.errata1-3+deb11u1build0.20.04.1) ...
Selecting previously unselected package libva2:amd64.
Preparing to unpack .../03-libva2_2.7.0-2_amd64.deb ...
7Progress: [  2%] [#.........................................................] 8Unpacking libva2:amd64 (2.7.0-2) ...
Selecting previously unselected package libva-drm2:amd64.
Preparing to unpack .../04-libva-drm2_2.7.0-2_amd64.deb ...
Unpacking libva-drm2:amd64 (2.7.0-2) ...
7Progress: [  3%] [#.........................................................] 8Selecting previously unselected package libva-x11-2:amd64.
Preparing to unpack .../05-libva-x11-2_2.7.0-2_amd64.deb ...
Unpacking libva-x11-2:amd64 (2.7.0-2) ...
Selecting previously unselected package libvdpau1:amd64.
Preparing to unpack .../06-libvdpau1_1.3-1ubuntu2_amd64.deb ...
Unpacking libvdpau1:amd64 (1.3-1ubuntu2) ...
7Progress: [  4%] [##........................................................] 8Selecting previously unselected package ocl-icd-libopencl1:amd64.
Preparing to unpack .../07-ocl-icd-libopencl1_2.2.11-1ubuntu1_amd64.deb ...
Unpacking ocl-icd-libopencl1:amd64 (2.2.11-1ubuntu1) ...
Selecting previously unselected package libavutil56:amd64.
Preparing to unpack .../08-libavutil56_7%3a4.2.7-0ubuntu0.1_amd64.deb ...
7Progress: [  5%] [##........................................................] 8Unpacking libavutil56:amd64 (7:4.2.7-0ubuntu0.1) ...
Selecting previously unselected package libcodec2-0.9:amd64.
Preparing to unpack .../09-libcodec2-0.9_0.9.2-2_amd64.deb ...
Unpacking libcodec2-0.9:amd64 (0.9.2-2) ...
Selecting previously unselected package libgsm1:amd64.
Preparing to unpack .../10-libgsm1_1.0.18-2_amd64.deb ...
Unpacking libgsm1:amd64 (1.0.18-2) ...
Selecting previously unselected package libmp3lame0:amd64.
Preparing to unpack .../11-libmp3lame0_3.100-3_amd64.deb ...
Unpacking libmp3lame0:amd64 (3.100-3) ...
7Progress: [  7%] [####......................................................] 8Selecting previously unselected package libopenjp2-7:amd64.
Preparing to unpack .../12-libopenjp2-7_2.3.1-1ubuntu4.20.04.1_amd64.deb ...
Unpacking libopenjp2-7:amd64 (2.3.1-1ubuntu4.20.04.1) ...
Selecting previously unselected package libopus0:amd64.
Preparing to unpack .../13-libopus0_1.3.1-0ubuntu1_amd64.deb ...
7Progress: [  8%] [####......................................................] 8Unpacking libopus0:amd64 (1.3.1-0ubuntu1) ...
Selecting previously unselected package libshine3:amd64.
Preparing to unpack .../14-libshine3_3.1.1-2_amd64.deb ...
Unpacking libshine3:amd64 (3.1.1-2) ...
7Progress: [  9%] [#####.....................................................] 8Selecting previously unselected package libsnappy1v5:amd64.
Preparing to unpack .../15-libsnappy1v5_1.1.8-1build1_amd64.deb ...
Unpacking libsnappy1v5:amd64 (1.1.8-1build1) ...
Selecting previously unselected package libspeex1:amd64.
Preparing to unpack .../16-libspeex1_1.2~rc1.2-1.1ubuntu1.20.04.1_amd64.deb ...
Unpacking libspeex1:amd64 (1.2~rc1.2-1.1ubuntu1.20.04.1) ...
7Progress: [ 10%] [#####.....................................................] 8Selecting previously unselected package libsoxr0:amd64.
Preparing to unpack .../17-libsoxr0_0.1.3-2build1_amd64.deb ...
Unpacking libsoxr0:amd64 (0.1.3-2build1) ...
Selecting previously unselected package libswresample3:amd64.
Preparing to unpack .../18-libswresample3_7%3a4.2.7-0ubuntu0.1_amd64.deb ...
7Progress: [ 11%] [######....................................................] 8Unpacking libswresample3:amd64 (7:4.2.7-0ubuntu0.1) ...
Selecting previously unselected package libtheora0:amd64.
Preparing to unpack .../19-libtheora0_1.1.1+dfsg.1-15ubuntu2_amd64.deb ...
Unpacking libtheora0:amd64 (1.1.1+dfsg.1-15ubuntu2) ...
7Progress: [ 12%] [######....................................................] 8Selecting previously unselected package libtwolame0:amd64.
Preparing to unpack .../20-libtwolame0_0.4.0-2_amd64.deb ...
Unpacking libtwolame0:amd64 (0.4.0-2) ...
Selecting previously unselected package libvorbisenc2:amd64.
Preparing to unpack .../21-libvorbisenc2_1.3.6-2ubuntu1_amd64.deb ...
Unpacking libvorbisenc2:amd64 (1.3.6-2ubuntu1) ...
7Progress: [ 13%] [#######...................................................] 8Selecting previously unselected package libvpx6:amd64.
Preparing to unpack .../22-libvpx6_1.8.2-1ubuntu0.2_amd64.deb ...
Unpacking libvpx6:amd64 (1.8.2-1ubuntu0.2) ...
Selecting previously unselected package libwavpack1:amd64.
Preparing to unpack .../23-libwavpack1_5.2.0-1ubuntu0.1_amd64.deb ...
7Progress: [ 14%] [########..................................................] 8Unpacking libwavpack1:amd64 (5.2.0-1ubuntu0.1) ...
Selecting previously unselected package libx264-155:amd64.
Preparing to unpack .../24-libx264-155_2%3a0.155.2917+git0a84d98-2_amd64.deb ...
Unpacking libx264-155:amd64 (2:0.155.2917+git0a84d98-2) ...
7Progress: [ 15%] [########..................................................] 8Selecting previously unselected package libx265-179:amd64.
Preparing to unpack .../25-libx265-179_3.2.1-1build1_amd64.deb ...
Unpacking libx265-179:amd64 (3.2.1-1build1) ...
Selecting previously unselected package libxvidcore4:amd64.
Preparing to unpack .../26-libxvidcore4_2%3a1.3.7-1_amd64.deb ...
Unpacking libxvidcore4:amd64 (2:1.3.7-1) ...
7Progress: [ 16%] [#########.................................................] 8Selecting previously unselected package libzvbi-common.
Preparing to unpack .../27-libzvbi-common_0.2.35-17_all.deb ...
Unpacking libzvbi-common (0.2.35-17) ...
Selecting previously unselected package libzvbi0:amd64.
Preparing to unpack .../28-libzvbi0_0.2.35-17_amd64.deb ...
7Progress: [ 17%] [#########.................................................] 8Unpacking libzvbi0:amd64 (0.2.35-17) ...
Selecting previously unselected package libavcodec58:amd64.
Preparing to unpack .../29-libavcodec58_7%3a4.2.7-0ubuntu0.1_amd64.deb ...
Unpacking libavcodec58:amd64 (7:4.2.7-0ubuntu0.1) ...
7Progress: [ 18%] [##########................................................] 8Selecting previously unselected package libraw1394-11:amd64.
Preparing to unpack .../30-libraw1394-11_2.1.2-1_amd64.deb ...
Unpacking libraw1394-11:amd64 (2.1.2-1) ...
Selecting previously unselected package libavc1394-0:amd64.
Preparing to unpack .../31-libavc1394-0_0.5.4-5_amd64.deb ...
Unpacking libavc1394-0:amd64 (0.5.4-5) ...
7Progress: [ 19%] [###########...............................................] 8Selecting previously unselected package libass9:amd64.
Preparing to unpack .../32-libass9_1%3a0.14.0-2_amd64.deb ...
Unpacking libass9:amd64 (1:0.14.0-2) ...
Selecting previously unselected package libbluray2:amd64.
Preparing to unpack .../33-libbluray2_1%3a1.2.0-1_amd64.deb ...
7Progress: [ 20%] [###########...............................................] 8Unpacking libbluray2:amd64 (1:1.2.0-1) ...
Selecting previously unselected package libchromaprint1:amd64.
Preparing to unpack .../34-libchromaprint1_1.4.3-3build1_amd64.deb ...
Unpacking libchromaprint1:amd64 (1.4.3-3build1) ...
7Progress: [ 21%] [############..............................................] 8Selecting previously unselected package libgme0:amd64.
Preparing to unpack .../35-libgme0_0.6.2-1build1_amd64.deb ...
Unpacking libgme0:amd64 (0.6.2-1build1) ...
Selecting previously unselected package libmpg123-0:amd64.
Preparing to unpack .../36-libmpg123-0_1.25.13-1_amd64.deb ...
Unpacking libmpg123-0:amd64 (1.25.13-1) ...
7Progress: [ 22%] [############..............................................] 8Selecting previously unselected package libopenmpt0:amd64.
Preparing to unpack .../37-libopenmpt0_0.4.11-1build1_amd64.deb ...
Unpacking libopenmpt0:amd64 (0.4.11-1build1) ...
Selecting previously unselected package libssh-gcrypt-4:amd64.
Preparing to unpack .../38-libssh-gcrypt-4_0.9.3-2ubuntu2.5_amd64.deb ...
7Progress: [ 23%] [#############.............................................] 8Unpacking libssh-gcrypt-4:amd64 (0.9.3-2ubuntu2.5) ...
Selecting previously unselected package libavformat58:amd64.
Preparing to unpack .../39-libavformat58_7%3a4.2.7-0ubuntu0.1_amd64.deb ...
Unpacking libavformat58:amd64 (7:4.2.7-0ubuntu0.1) ...
7Progress: [ 24%] [#############.............................................] 8Selecting previously unselected package libbs2b0:amd64.
Preparing to unpack .../40-libbs2b0_3.1.0+dfsg-2.2build1_amd64.deb ...
Unpacking libbs2b0:amd64 (3.1.0+dfsg-2.2build1) ...
Selecting previously unselected package libflite1:amd64.
Preparing to unpack .../41-libflite1_2.1-release-3_amd64.deb ...
Unpacking libflite1:amd64 (2.1-release-3) ...
7Progress: [ 25%] [##############............................................] 8Selecting previously unselected package libserd-0-0:amd64.
Preparing to unpack .../42-libserd-0-0_0.30.2-1_amd64.deb ...
Unpacking libserd-0-0:amd64 (0.30.2-1) ...
Selecting previously unselected package libsord-0-0:amd64.
Preparing to unpack .../43-libsord-0-0_0.16.4-1_amd64.deb ...
7Progress: [ 26%] [###############...........................................] 8Unpacking libsord-0-0:amd64 (0.16.4-1) ...
Selecting previously unselected package libsratom-0-0:amd64.
Preparing to unpack .../44-libsratom-0-0_0.6.4-1_amd64.deb ...
Unpacking libsratom-0-0:amd64 (0.6.4-1) ...
7Progress: [ 27%] [###############...........................................] 8Selecting previously unselected package liblilv-0-0:amd64.
Preparing to unpack .../45-liblilv-0-0_0.24.6-1ubuntu0.1_amd64.deb ...
Unpacking liblilv-0-0:amd64 (0.24.6-1ubuntu0.1) ...
Selecting previously unselected package libmysofa1:amd64.
Preparing to unpack .../46-libmysofa1_1.0~dfsg0-1_amd64.deb ...
Unpacking libmysofa1:amd64 (1.0~dfsg0-1) ...
7Progress: [ 28%] [################..........................................] 8Selecting previously unselected package libpostproc55:amd64.
Preparing to unpack .../47-libpostproc55_7%3a4.2.7-0ubuntu0.1_amd64.deb ...
Unpacking libpostproc55:amd64 (7:4.2.7-0ubuntu0.1) ...
Selecting previously unselected package libsamplerate0:amd64.
Preparing to unpack .../48-libsamplerate0_0.1.9-2_amd64.deb ...
7Progress: [ 29%] [################..........................................] 8Unpacking libsamplerate0:amd64 (0.1.9-2) ...
Selecting previously unselected package librubberband2:amd64.
Preparing to unpack .../49-librubberband2_1.8.2-1build1_amd64.deb ...
Unpacking librubberband2:amd64 (1.8.2-1build1) ...
7Progress: [ 30%] [#################.........................................] 8Selecting previously unselected package libswscale5:amd64.
Preparing to unpack .../50-libswscale5_7%3a4.2.7-0ubuntu0.1_amd64.deb ...
Unpacking libswscale5:amd64 (7:4.2.7-0ubuntu0.1) ...
Selecting previously unselected package libvidstab1.1:amd64.
Preparing to unpack .../51-libvidstab1.1_1.1.0-2_amd64.deb ...
Unpacking libvidstab1.1:amd64 (1.1.0-2) ...
7Progress: [ 31%] [##################........................................] 8Selecting previously unselected package libnorm1:amd64.
Preparing to unpack .../52-libnorm1_1.5.8+dfsg2-2build1_amd64.deb ...
Unpacking libnorm1:amd64 (1.5.8+dfsg2-2build1) ...
Selecting previously unselected package libpgm-5.2-0:amd64.
Preparing to unpack .../53-libpgm-5.2-0_5.2.122~dfsg-3ubuntu1_amd64.deb ...
7Progress: [ 32%] [##################........................................] 8Unpacking libpgm-5.2-0:amd64 (5.2.122~dfsg-3ubuntu1) ...
Selecting previously unselected package libzmq5:amd64.
Preparing to unpack .../54-libzmq5_4.3.2-2ubuntu1_amd64.deb ...
Unpacking libzmq5:amd64 (4.3.2-2ubuntu1) ...
7Progress: [ 33%] [###################.......................................] 8Selecting previously unselected package libavfilter7:amd64.
Preparing to unpack .../55-libavfilter7_7%3a4.2.7-0ubuntu0.1_amd64.deb ...
Unpacking libavfilter7:amd64 (7:4.2.7-0ubuntu0.1) ...
Selecting previously unselected package libcaca0:amd64.
Preparing to unpack .../56-libcaca0_0.99.beta19-2.1ubuntu1.20.04.2_amd64.deb ...
Unpacking libcaca0:amd64 (0.99.beta19-2.1ubuntu1.20.04.2) ...
7Progress: [ 34%] [###################.......................................] 8Selecting previously unselected package libcdio18:amd64.
Preparing to unpack .../57-libcdio18_2.0.0-2_amd64.deb ...
Unpacking libcdio18:amd64 (2.0.0-2) ...
Selecting previously unselected package libcdio-cdda2:amd64.
Preparing to unpack .../58-libcdio-cdda2_10.2+2.0.0-1_amd64.deb ...
7Progress: [ 35%] [####################......................................] 8Unpacking libcdio-cdda2:amd64 (10.2+2.0.0-1) ...
Selecting previously unselected package libcdio-paranoia2:amd64.
Preparing to unpack .../59-libcdio-paranoia2_10.2+2.0.0-1_amd64.deb ...
Unpacking libcdio-paranoia2:amd64 (10.2+2.0.0-1) ...
7Progress: [ 36%] [####################......................................] 8Selecting previously unselected package libdc1394-22:amd64.
Preparing to unpack .../60-libdc1394-22_2.2.5-2.1_amd64.deb ...
Unpacking libdc1394-22:amd64 (2.2.5-2.1) ...
Selecting previously unselected package libiec61883-0:amd64.
Preparing to unpack .../61-libiec61883-0_1.2.0-3_amd64.deb ...
Unpacking libiec61883-0:amd64 (1.2.0-3) ...
7Progress: [ 37%] [#####################.....................................] 8Selecting previously unselected package libjack-jackd2-0:amd64.
Preparing to unpack .../62-libjack-jackd2-0_1.9.12~dfsg-2ubuntu2_amd64.deb ...
Unpacking libjack-jackd2-0:amd64 (1.9.12~dfsg-2ubuntu2) ...
Selecting previously unselected package libopenal-data.
Preparing to unpack .../63-libopenal-data_1%3a1.19.1-1_all.deb ...
7Progress: [ 38%] [######################....................................] 8Unpacking libopenal-data (1:1.19.1-1) ...
Selecting previously unselected package libsndio7.0:amd64.
Preparing to unpack .../64-libsndio7.0_1.5.0-3_amd64.deb ...
Unpacking libsndio7.0:amd64 (1.5.0-3) ...
7Progress: [ 39%] [######################....................................] 8Selecting previously unselected package libopenal1:amd64.
Preparing to unpack .../65-libopenal1_1%3a1.19.1-1_amd64.deb ...
Unpacking libopenal1:amd64 (1:1.19.1-1) ...
Selecting previously unselected package libasyncns0:amd64.
Preparing to unpack .../66-libasyncns0_0.8-6_amd64.deb ...
Unpacking libasyncns0:amd64 (0.8-6) ...
7Progress: [ 40%] [#######################...................................] 8Selecting previously unselected package libflac8:amd64.
Preparing to unpack .../67-libflac8_1.3.3-1ubuntu0.2_amd64.deb ...
Unpacking libflac8:amd64 (1.3.3-1ubuntu0.2) ...
Selecting previously unselected package libsndfile1:amd64.
Preparing to unpack .../68-libsndfile1_1.0.28-7ubuntu0.2_amd64.deb ...
7Progress: [ 41%] [#######################...................................] 8Unpacking libsndfile1:amd64 (1.0.28-7ubuntu0.2) ...
Selecting previously unselected package libpulse0:amd64.
Preparing to unpack .../69-libpulse0_1%3a13.99.1-1ubuntu3.13_amd64.deb ...
Unpacking libpulse0:amd64 (1:13.99.1-1ubuntu3.13) ...
7Progress: [ 42%] [########################..................................] 8Selecting previously unselected package libsdl2-2.0-0:amd64.
Preparing to unpack .../70-libsdl2-2.0-0_2.0.10+dfsg1-3_amd64.deb ...
Unpacking libsdl2-2.0-0:amd64 (2.0.10+dfsg1-3) ...
Selecting previously unselected package libavdevice58:amd64.
Preparing to unpack .../71-libavdevice58_7%3a4.2.7-0ubuntu0.1_amd64.deb ...
Unpacking libavdevice58:amd64 (7:4.2.7-0ubuntu0.1) ...
7Progress: [ 43%] [#########################.................................] 8Selecting previously unselected package libavresample4:amd64.
Preparing to unpack .../72-libavresample4_7%3a4.2.7-0ubuntu0.1_amd64.deb ...
Unpacking libavresample4:amd64 (7:4.2.7-0ubuntu0.1) ...
Selecting previously unselected package ffmpeg.
Preparing to unpack .../73-ffmpeg_7%3a4.2.7-0ubuntu0.1_amd64.deb ...
7Progress: [ 44%] [#########################.................................] 8Unpacking ffmpeg (7:4.2.7-0ubuntu0.1) ...
Selecting previously unselected package libigdgmm11:amd64.
Preparing to unpack .../74-libigdgmm11_20.1.1+ds1-1_amd64.deb ...
Unpacking libigdgmm11:amd64 (20.1.1+ds1-1) ...
7Progress: [ 45%] [##########################................................] 8Selecting previously unselected package intel-media-va-driver:amd64.
Preparing to unpack .../75-intel-media-va-driver_20.1.1+dfsg1-1_amd64.deb ...
Unpacking intel-media-va-driver:amd64 (20.1.1+dfsg1-1) ...
Selecting previously unselected package libaacs0:amd64.
Preparing to unpack .../76-libaacs0_0.9.0-2_amd64.deb ...
Unpacking libaacs0:amd64 (0.9.0-2) ...
7Progress: [ 46%] [##########################................................] 8Selecting previously unselected package libbdplus0:amd64.
Preparing to unpack .../77-libbdplus0_0.1.2-3_amd64.deb ...
Unpacking libbdplus0:amd64 (0.1.2-3) ...
Selecting previously unselected package mesa-va-drivers:amd64.
Preparing to unpack .../78-mesa-va-drivers_21.2.6-0ubuntu0.1~20.04.2_amd64.deb ...
7Progress: [ 47%] [###########################...............................] 8Unpacking mesa-va-drivers:amd64 (21.2.6-0ubuntu0.1~20.04.2) ...
Selecting previously unselected package mesa-vdpau-drivers:amd64.
Preparing to unpack .../79-mesa-vdpau-drivers_21.2.6-0ubuntu0.1~20.04.2_amd64.deb ...
Unpacking mesa-vdpau-drivers:amd64 (21.2.6-0ubuntu0.1~20.04.2) ...
7Progress: [ 48%] [###########################...............................] 8Selecting previously unselected package i965-va-driver:amd64.
Preparing to unpack .../80-i965-va-driver_2.4.0-0ubuntu1_amd64.deb ...
Unpacking i965-va-driver:amd64 (2.4.0-0ubuntu1) ...
Selecting previously unselected package va-driver-all:amd64.
Preparing to unpack .../81-va-driver-all_2.7.0-2_amd64.deb ...
Unpacking va-driver-all:amd64 (2.7.0-2) ...
7Progress: [ 49%] [############################..............................] 8Selecting previously unselected package vdpau-driver-all:amd64.
Preparing to unpack .../82-vdpau-driver-all_1.3-1ubuntu2_amd64.deb ...
Unpacking vdpau-driver-all:amd64 (1.3-1ubuntu2) ...
Setting up libgme0:amd64 (0.6.2-1build1) ...
7Progress: [ 50%] [#############################.............................] 8Setting up libssh-gcrypt-4:amd64 (0.9.3-2ubuntu2.5) ...
7Progress: [ 51%] [#############################.............................] 8Setting up libraw1394-11:amd64 (2.1.2-1) ...
Setting up libsodium23:amd64 (1.0.18-1) ...
7Progress: [ 52%] [##############################............................] 8Setting up libmpg123-0:amd64 (1.25.13-1) ...
Setting up libspeex1:amd64 (1.2~rc1.2-1.1ubuntu1.20.04.1) ...
7Progress: [ 53%] [##############################............................] 8Setting up libshine3:amd64 (3.1.1-2) ...
7Progress: [ 54%] [###############################...........................] 8Setting up libtwolame0:amd64 (0.4.0-2) ...
Setting up libgsm1:amd64 (1.0.18-2) ...
7Progress: [ 55%] [################################..........................] 8Setting up libx264-155:amd64 (2:0.155.2917+git0a84d98-2) ...
Setting up libx265-179:amd64 (3.2.1-1build1) ...
7Progress: [ 56%] [################################..........................] 8Setting up libsoxr0:amd64 (0.1.3-2build1) ...
7Progress: [ 57%] [#################################.........................] 8Setting up libaom0:amd64 (1.0.0.errata1-3+deb11u1build0.20.04.1) ...
Setting up libnorm1:amd64 (1.5.8+dfsg2-2build1) ...
7Progress: [ 58%] [#################################.........................] 8Setting up libmysofa1:amd64 (1.0~dfsg0-1) ...
Setting up libdc1394-22:amd64 (2.2.5-2.1) ...
7Progress: [ 59%] [##################################........................] 8Setting up libcdio18:amd64 (2.0.0-2) ...
7Progress: [ 60%] [##################################........................] 8Setting up libxvidcore4:amd64 (2:1.3.7-1) ...
Setting up libsnappy1v5:amd64 (1.1.8-1build1) ...
7Progress: [ 61%] [###################################.......................] 8Setting up libflac8:amd64 (1.3.3-1ubuntu0.2) ...
Setting up libass9:amd64 (1:0.14.0-2) ...
7Progress: [ 62%] [####################################......................] 8Setting up libslang2:amd64 (2.3.2-4) ...
7Progress: [ 63%] [####################################......................] 8Setting up libva2:amd64 (2.7.0-2) ...
Setting up libigdgmm11:amd64 (20.1.1+ds1-1) ...
7Progress: [ 64%] [#####################################.....................] 8Setting up libcodec2-0.9:amd64 (0.9.2-2) ...
Setting up libopus0:amd64 (1.3.1-0ubuntu1) ...
7Progress: [ 65%] [#####################################.....................] 8Setting up intel-media-va-driver:amd64 (20.1.1+dfsg1-1) ...
7Progress: [ 66%] [######################################....................] 8Setting up libaacs0:amd64 (0.9.0-2) ...
Setting up libsndio7.0:amd64 (1.5.0-3) ...
7Progress: [ 67%] [#######################################...................] 8Setting up libbdplus0:amd64 (0.1.2-3) ...
Setting up libvidstab1.1:amd64 (1.1.0-2) ...
7Progress: [ 68%] [#######################################...................] 8Setting up libflite1:amd64 (2.1-release-3) ...
7Progress: [ 69%] [########################################..................] 8Setting up libva-drm2:amd64 (2.7.0-2) ...
Setting up ocl-icd-libopencl1:amd64 (2.2.11-1ubuntu1) ...
7Progress: [ 70%] [########################################..................] 8Setting up libasyncns0:amd64 (0.8-6) ...
Setting up libvpx6:amd64 (1.8.2-1ubuntu0.2) ...
7Progress: [ 71%] [#########################################.................] 8Setting up libvdpau1:amd64 (1.3-1ubuntu2) ...
7Progress: [ 72%] [#########################################.................] 8Setting up libwavpack1:amd64 (5.2.0-1ubuntu0.1) ...
Setting up libbs2b0:amd64 (3.1.0+dfsg-2.2build1) ...
7Progress: [ 73%] [##########################################................] 8Setting up libtheora0:amd64 (1.1.1+dfsg.1-15ubuntu2) ...
Setting up libopenjp2-7:amd64 (2.3.1-1ubuntu4.20.04.1) ...
7Progress: [ 74%] [###########################################...............] 8Setting up libopenal-data (1:1.19.1-1) ...
7Progress: [ 75%] [###########################################...............] 8Setting up mesa-va-drivers:amd64 (21.2.6-0ubuntu0.1~20.04.2) ...
Setting up libbluray2:amd64 (1:1.2.0-1) ...
7Progress: [ 76%] [############################################..............] 8Setting up libsamplerate0:amd64 (0.1.9-2) ...
Setting up libva-x11-2:amd64 (2.7.0-2) ...
7Progress: [ 77%] [############################################..............] 8Setting up libopenmpt0:amd64 (0.4.11-1build1) ...
7Progress: [ 78%] [#############################################.............] 8Setting up libzvbi-common (0.2.35-17) ...
Setting up libmp3lame0:amd64 (3.100-3) ...
7Progress: [ 79%] [#############################################.............] 8Setting up i965-va-driver:amd64 (2.4.0-0ubuntu1) ...
Setting up libvorbisenc2:amd64 (1.3.6-2ubuntu1) ...
7Progress: [ 80%] [##############################################............] 8Setting up libpgm-5.2-0:amd64 (5.2.122~dfsg-3ubuntu1) ...
7Progress: [ 81%] [###############################################...........] 8Setting up libiec61883-0:amd64 (1.2.0-3) ...
Setting up libserd-0-0:amd64 (0.30.2-1) ...
7Progress: [ 82%] [###############################################...........] 8Setting up libavc1394-0:amd64 (0.5.4-5) ...
Setting up mesa-vdpau-drivers:amd64 (21.2.6-0ubuntu0.1~20.04.2) ...
7Progress: [ 83%] [################################################..........] 8Setting up libzvbi0:amd64 (0.2.35-17) ...
7Progress: [ 84%] [################################################..........] 8Setting up libzmq5:amd64 (4.3.2-2ubuntu1) ...
Setting up libcaca0:amd64 (0.99.beta19-2.1ubuntu1.20.04.2) ...
7Progress: [ 85%] [#################################################.........] 8Setting up libcdio-cdda2:amd64 (10.2+2.0.0-1) ...
Setting up libcdio-paranoia2:amd64 (10.2+2.0.0-1) ...
7Progress: [ 86%] [#################################################.........] 8Setting up libopenal1:amd64 (1:1.19.1-1) ...
7Progress: [ 87%] [##################################################........] 8Setting up libavutil56:amd64 (7:4.2.7-0ubuntu0.1) ...
Setting up va-driver-all:amd64 (2.7.0-2) ...
7Progress: [ 88%] [###################################################.......] 8Setting up libpostproc55:amd64 (7:4.2.7-0ubuntu0.1) ...
Setting up librubberband2:amd64 (1.8.2-1build1) ...
7Progress: [ 89%] [###################################################.......] 8Setting up libjack-jackd2-0:amd64 (1.9.12~dfsg-2ubuntu2) ...
7Progress: [ 90%] [####################################################......] 8Setting up vdpau-driver-all:amd64 (1.3-1ubuntu2) ...
Setting up libsord-0-0:amd64 (0.16.4-1) ...
7Progress: [ 91%] [####################################################......] 8Setting up libsratom-0-0:amd64 (0.6.4-1) ...
Setting up libswscale5:amd64 (7:4.2.7-0ubuntu0.1) ...
7Progress: [ 92%] [#####################################################.....] 8Setting up libsndfile1:amd64 (1.0.28-7ubuntu0.2) ...
7Progress: [ 93%] [#####################################################.....] 8Setting up liblilv-0-0:amd64 (0.24.6-1ubuntu0.1) ...
Setting up libpulse0:amd64 (1:13.99.1-1ubuntu3.13) ...
7Progress: [ 94%] [######################################################....] 8Setting up libswresample3:amd64 (7:4.2.7-0ubuntu0.1) ...
Setting up libavresample4:amd64 (7:4.2.7-0ubuntu0.1) ...
7Progress: [ 95%] [#######################################################...] 8Setting up libavcodec58:amd64 (7:4.2.7-0ubuntu0.1) ...
7Progress: [ 96%] [#######################################################...] 8Setting up libsdl2-2.0-0:amd64 (2.0.10+dfsg1-3) ...
Setting up libchromaprint1:amd64 (1.4.3-3build1) ...
7Progress: [ 97%] [########################################################..] 8Setting up libavformat58:amd64 (7:4.2.7-0ubuntu0.1) ...
Setting up libavfilter7:amd64 (7:4.2.7-0ubuntu0.1) ...
7Progress: [ 98%] [########################################################..] 8Setting up libavdevice58:amd64 (7:4.2.7-0ubuntu0.1) ...
7Progress: [ 99%] [#########################################################.] 8Setting up ffmpeg (7:4.2.7-0ubuntu0.1) ...
Processing triggers for man-db (2.9.1-1) ...
Processing triggers for libc-bin (2.31-0ubuntu9.9) ...
/sbin/ldconfig.real: File /lib/x86_64-linux-gnu/libnvidia-allocator.so.1 is empty, not checked.
/sbin/ldconfig.real: File /lib/x86_64-linux-gnu/libcuda.so.510.108.03 is empty, not checked.
/sbin/ldconfig.real: File /lib/x86_64-linux-gnu/libcuda.so.1 is empty, not checked.
/sbin/ldconfig.real: File /lib/x86_64-linux-gnu/libnvidia-ml.so.470.82.01 is empty, not checked.
/sbin/ldconfig.real: File /lib/x86_64-linux-gnu/libnvidia-ptxjitcompiler.so.1 is empty, not checked.
/sbin/ldconfig.real: File /lib/x86_64-linux-gnu/libnvidia-opencl.so.1 is empty, not checked.
/sbin/ldconfig.real: File /lib/x86_64-linux-gnu/libnvidia-ptxjitcompiler.so.470.82.01 is empty, not checked.
/sbin/ldconfig.real: File /lib/x86_64-linux-gnu/libcuda.so.470.82.01 is empty, not checked.
/sbin/ldconfig.real: File /lib/x86_64-linux-gnu/libnvidia-cfg.so.470.82.01 is empty, not checked.
/sbin/ldconfig.real: File /lib/x86_64-linux-gnu/libnvidia-cfg.so.1 is empty, not checked.
/sbin/ldconfig.real: File /lib/x86_64-linux-gnu/libcuda.so is empty, not checked.
/sbin/ldconfig.real: File /lib/x86_64-linux-gnu/libnvidia-allocator.so.470.82.01 is empty, not checked.
/sbin/ldconfig.real: File /lib/x86_64-linux-gnu/libnvidia-ptxjitcompiler.so.510.108.03 is empty, not checked.
/sbin/ldconfig.real: File /lib/x86_64-linux-gnu/libnvidia-compiler.so.470.82.01 is empty, not checked.
/sbin/ldconfig.real: File /lib/x86_64-linux-gnu/libnvidia-ml.so.1 is empty, not checked.
/sbin/ldconfig.real: File /lib/x86_64-linux-gnu/libnvidia-opencl.so.470.82.01 is empty, not checked.

78
代码
文本

We get the same translation as with the pipeline example.

代码
文本

Audio classification

代码
文本

Audio classification assigns a class to an audio signal. The Keyword Spotting dataset from the SUPERB benchmark is an example dataset that can be used for audio classification fine-tuning. This dataset contains ten classes of keywords for classification. If you'd like to fine-tune a model for audio classification, take a look at the run_audio_classification.py script or this how-to guide.

The following examples demonstrate how to use a pipeline() and a model and tokenizer for audio classification inference:

代码
文本
[48]
from transformers import pipeline
from datasets import load_dataset
import torch

torch.manual_seed(42)
dataset = load_dataset("hf-internal-testing/librispeech_asr_demo", "clean", split="validation")
dataset = dataset.sort("id")
audio_file = dataset[0]["audio"]["path"]

audio_classifier = pipeline(
task="audio-classification", model="ehcalabres/wav2vec2-lg-xlsr-en-speech-emotion-recognition"
)
predictions = audio_classifier(audio_file)
predictions = [{"score": round(pred["score"], 4), "label": pred["label"]} for pred in predictions]
predictions
Found cached dataset librispeech_asr_demo (/root/.cache/huggingface/datasets/hf-internal-testing___librispeech_asr_demo/clean/2.1.0/d3bc4c2bc2078fcde3ad0f0f635862e4c0fef78ba94c4a34c4c250a097af240b)
/opt/conda/lib/python3.8/site-packages/transformers/configuration_utils.py:379: UserWarning: Passing `gradient_checkpointing` to a config initialization is deprecated and will be removed in v5 Transformers. Using `model.gradient_checkpointing_enable()` instead, or if you are using the `Trainer` API, pass `gradient_checkpointing=True` in your `TrainingArguments`.
  warnings.warn(
Some weights of the model checkpoint at ehcalabres/wav2vec2-lg-xlsr-en-speech-emotion-recognition were not used when initializing Wav2Vec2ForSequenceClassification: ['classifier.dense.bias', 'classifier.output.weight', 'classifier.dense.weight', 'classifier.output.bias']
- This IS expected if you are initializing Wav2Vec2ForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing Wav2Vec2ForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Some weights of Wav2Vec2ForSequenceClassification were not initialized from the model checkpoint at ehcalabres/wav2vec2-lg-xlsr-en-speech-emotion-recognition and are newly initialized: ['projector.weight', 'classifier.bias', 'classifier.weight', 'projector.bias']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.

[{'score': 0.1315, 'label': 'calm'},
 {'score': 0.1307, 'label': 'neutral'},
 {'score': 0.1274, 'label': 'sad'},
 {'score': 0.1261, 'label': 'fearful'},
 {'score': 0.1242, 'label': 'happy'}]
代码
文本

The general process for using a model and feature extractor for audio classification is:

  1. Instantiate a feature extractor and a model from the checkpoint name.
  2. Process the audio signal to be classified with a feature extractor.
  3. Pass the input through the model and take the argmax to retrieve the most likely class.
  4. Convert the class id to a class name with id2label to return an interpretable result.
代码
文本
[49]
from transformers import AutoFeatureExtractor, AutoModelForAudioClassification
from datasets import load_dataset
import torch

dataset = load_dataset("hf-internal-testing/librispeech_asr_demo", "clean", split="validation")
dataset = dataset.sort("id")
sampling_rate = dataset.features["audio"].sampling_rate

feature_extractor = AutoFeatureExtractor.from_pretrained("superb/wav2vec2-base-superb-ks")
model = AutoModelForAudioClassification.from_pretrained("superb/wav2vec2-base-superb-ks")

inputs = feature_extractor(dataset[0]["audio"]["array"], sampling_rate=sampling_rate, return_tensors="pt")

with torch.no_grad():
logits = model(**inputs).logits

predicted_class_ids = torch.argmax(logits, dim=-1).item()
predicted_label = model.config.id2label[predicted_class_ids]
predicted_label
Found cached dataset librispeech_asr_demo (/root/.cache/huggingface/datasets/hf-internal-testing___librispeech_asr_demo/clean/2.1.0/d3bc4c2bc2078fcde3ad0f0f635862e4c0fef78ba94c4a34c4c250a097af240b)
/opt/conda/lib/python3.8/site-packages/transformers/configuration_utils.py:379: UserWarning: Passing `gradient_checkpointing` to a config initialization is deprecated and will be removed in v5 Transformers. Using `model.gradient_checkpointing_enable()` instead, or if you are using the `Trainer` API, pass `gradient_checkpointing=True` in your `TrainingArguments`.
  warnings.warn(

'_unknown_'
代码
文本

Automatic speech recognition

代码
文本

Automatic speech recognition transcribes an audio signal to text. The Common Voice dataset is an example dataset that can be used for automatic speech recognition fine-tuning. It contains an audio file of a speaker and the corresponding sentence. If you'd like to fine-tune a model for automatic speech recognition, take a look at the run_speech_recognition_ctc.py or run_speech_recognition_seq2seq.py scripts or this how-to guide.

The following examples demonstrate how to use a pipeline() and a model and tokenizer for automatic speech recognition inference:

代码
文本
[50]
from transformers import pipeline
from datasets import load_dataset

dataset = load_dataset("hf-internal-testing/librispeech_asr_demo", "clean", split="validation")
dataset = dataset.sort("id")
audio_file = dataset[0]["audio"]["path"]

speech_recognizer = pipeline(task="automatic-speech-recognition", model="facebook/wav2vec2-base-960h")
speech_recognizer(audio_file)
Found cached dataset librispeech_asr_demo (/root/.cache/huggingface/datasets/hf-internal-testing___librispeech_asr_demo/clean/2.1.0/d3bc4c2bc2078fcde3ad0f0f635862e4c0fef78ba94c4a34c4c250a097af240b)
Some weights of Wav2Vec2ForCTC were not initialized from the model checkpoint at facebook/wav2vec2-base-960h and are newly initialized: ['wav2vec2.masked_spec_embed']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.

{'text': 'MISTER QUILTER IS THE APOSTLE OF THE MIDDLE CLASSES AND WE ARE GLAD TO WELCOME HIS GOSPEL'}
代码
文本

The general process for using a model and processor for automatic speech recognition is:

  1. Instantiate a processor (which regroups a feature extractor for input processing and a tokenizer for decoding) and a model from the checkpoint name.
  2. Process the audio signal and text with a processor.
  3. Pass the input through the model and take the argmax to retrieve the predicted text.
  4. Decode the text with a tokenizer to obtain the transcription.
代码
文本
[51]
from transformers import AutoProcessor, AutoModelForCTC
from datasets import load_dataset
import torch

dataset = load_dataset("hf-internal-testing/librispeech_asr_demo", "clean", split="validation")
dataset = dataset.sort("id")
sampling_rate = dataset.features["audio"].sampling_rate

processor = AutoProcessor.from_pretrained("facebook/wav2vec2-base-960h")
model = AutoModelForCTC.from_pretrained("facebook/wav2vec2-base-960h")

inputs = processor(dataset[0]["audio"]["array"], sampling_rate=sampling_rate, return_tensors="pt")
with torch.no_grad():
logits = model(**inputs).logits
predicted_ids = torch.argmax(logits, dim=-1)

transcription = processor.batch_decode(predicted_ids)
transcription[0]
Found cached dataset librispeech_asr_demo (/root/.cache/huggingface/datasets/hf-internal-testing___librispeech_asr_demo/clean/2.1.0/d3bc4c2bc2078fcde3ad0f0f635862e4c0fef78ba94c4a34c4c250a097af240b)
Some weights of Wav2Vec2ForCTC were not initialized from the model checkpoint at facebook/wav2vec2-base-960h and are newly initialized: ['wav2vec2.masked_spec_embed']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.

'MISTER QUILTER IS THE APOSTLE OF THE MIDDLE CLASSES AND WE ARE GLAD TO WELCOME HIS GOSPEL'
代码
文本

Image classification

代码
文本

Like text and audio classification, image classification assigns a class to an image. The CIFAR-100 dataset is an example dataset that can be used for image classification fine-tuning. It contains an image and the corresponding class. If you'd like to fine-tune a model for image classification, take a look at the run_image_classification.py script or this how-to guide.

The following examples demonstrate how to use a pipeline() and a model and tokenizer for image classification inference:

代码
文本
[52]
from transformers import pipeline

vision_classifier = pipeline(task="image-classification")
result = vision_classifier(
images="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/pipeline-cat-chonk.jpeg"
)
print("\n".join([f"Class {d['label']} with score {round(d['score'], 4)}" for d in result]))
No model was supplied, defaulted to google/vit-base-patch16-224 and revision 5dca96d (https://huggingface.co/google/vit-base-patch16-224).
Using a pipeline without specifying a model name and revision in production is not recommended.

Class lynx, catamount with score 0.4403
Class cougar, puma, catamount, mountain lion, painter, panther, Felis concolor with score 0.0343
Class snow leopard, ounce, Panthera uncia with score 0.0321
Class Egyptian cat with score 0.0235
Class tiger cat with score 0.023
代码
文本

The general process for using a model and image processor for image classification is:

  1. Instantiate an image processor and a model from the checkpoint name.
  2. Process the image to be classified with an image processor.
  3. Pass the input through the model and take the argmax to retrieve the predicted class.
  4. Convert the class id to a class name with id2label to return an interpretable result.
代码
文本
[53]
from transformers import AutoImageProcessor, AutoModelForImageClassification
import torch
from datasets import load_dataset

dataset = load_dataset("huggingface/cats-image")
image = dataset["test"]["image"][0]

feature_extractor = AutoImageProcessor.from_pretrained("google/vit-base-patch16-224")
model = AutoModelForImageClassification.from_pretrained("google/vit-base-patch16-224")

inputs = feature_extractor(image, return_tensors="pt")

with torch.no_grad():
logits = model(**inputs).logits

predicted_label = logits.argmax(-1).item()
print(model.config.id2label[predicted_label])
No config specified, defaulting to: cats-image/image
Downloading and preparing dataset cats-image/image to /root/.cache/huggingface/datasets/huggingface___cats-image/image/1.9.0/68fbc793fb10cd165e490867f5d61fa366086ea40c73e549a020103dcb4f597e...

Dataset cats-image downloaded and prepared to /root/.cache/huggingface/datasets/huggingface___cats-image/image/1.9.0/68fbc793fb10cd165e490867f5d61fa366086ea40c73e549a020103dcb4f597e. Subsequent calls will reuse this data.

Egyptian cat
代码
文本
Transformers
Transformers
已赞1
推荐阅读
公开
Hugging Face Tutorials: Quick Tour
Hugging Face
Hugging Face
爱学习的王一博
发布于 2024-05-12
1 赞
公开
Summary of the tasks
Transformers
Transformers
ck
更新于 2024-06-13
1 赞