Interfaces of all functions are the same as mbart model on Huggingface transformers.
Here is an example of using a asian bart model. (ecjk model)
Other language work the same way. change both model and tokenizer's from_pretrained.
English only: from_pretrained("hyunwoongko/asian-bart-en")
Chinese only: from_pretrained("hyunwoongko/asian-bart-zh")
Japanese only: from_pretrained("hyunwoongko/asian-bart-ja")
Korean only: from_pretrained("hyunwoongko/asian-bart-ko")
# import modules>>>importtorch>>>fromasian_bartimportAsianBartTokenizer, AsianBartForConditionalGeneration>>>fromtransformers.models.bart.modeling_bartimportshift_tokens_right# create model and tokenizer>>>tokenizer=AsianBartTokenizer.from_pretrained("hyunwoongko/asian-bart-ecjk")
>>>model=AsianBartForConditionalGeneration.from_pretrained("hyunwoongko/asian-bart-ecjk")
# tokenize texts>>>tokens=tokenizer.prepare_seq2seq_batch(
... src_texts="Kevin is the <mask> man in the world.",
... src_langs="en_XX",
... tgt_texts="Kevin is the most kind man in the world.",
... tgt_langs="en_XX",
... )
>>>input_ids=tokens["input_ids"]
>>>attention_mask=tokens["attention_mask"]
>>>labels=tokens["labels"]
>>>decoder_input_ids=shift_tokens_right(labels, tokenizer.pad_token_id)
# forwarding model for training>>>output=model(
... input_ids=input_ids,
... attention_mask=attention_mask,
... decoder_input_ids=decoder_input_ids,
... )
# compute loss>>>lm_logits=outputs[0]
>>>loss_function=torch.nn.CrossEntropyLoss(
... ignore_index=tokenizer.pad_token_id
... )
>>>loss=loss_function(
... lm_logits.view(-1, lm_logits.shape[-1]),
... labels.view(-1)
... )
# generate text>>>output=model.generate(
... input_ids=input_ids,
... attention_mask=attention_mask,
... decoder_start_token_id=tokenizer.lang_code_to_id["en_XX"],
... )
Downstream tasks
You can train various downstream tasks with asian bart.
All interfaces have the same usage as the Huggingface transformers.
Supported classes:
AsianBartTokenizer
AsianBartModel
AsianBartForCausalLM
AsianBartForQuestionAnswering
AsianBartForConditionalGeneration
AsianBartForSequenceClassification
License
Copyright 2021 Hyunwoong Ko.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
The Tidelift Subscription provides access to a continuously curated stream of human-researched and maintainer-verified data on open source packages and their licenses, releases, vulnerabilities, and development practices.