Skip to content

masked_lm_labels -100 and not -1 #4

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
akhileshgotmare wants to merge 1 commit into
base: crayon
Choose a base branch
from
Open

masked_lm_labels -100 and not -1 #4

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions cbert_utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -197,7 +197,7 @@ def extract_features(tokens_a, tokens_label, max_seq_length, tokenizer):
masked_lm_probs = 0.15
max_predictions_per_seq = 20
rng = random.Random(12345)
original_masked_lm_labels = [-1] * max_seq_length
original_masked_lm_labels = [-100] * max_seq_length
(output_tokens, masked_lm_positions,
masked_lm_labels) = create_masked_lm_predictions(
tokens, masked_lm_probs, original_masked_lm_labels, max_predictions_per_seq, rng, tokenizer)
Expand Down Expand Up @@ -271,4 +271,4 @@ def create_masked_lm_predictions(tokens, masked_lm_probs, masked_lm_labels,
masked_lm_labels[index] = convert_tokens_to_ids([tokens[index]], tokenizer)[0]
output_tokens[index] = masked_token
masked_lm_positions.append(index)
return output_tokens, masked_lm_positions, masked_lm_labels
return output_tokens, masked_lm_positions, masked_lm_labels