关于GCU、沐曦GPGPU、MLU、0卡V100资源4月7日恢复上架的公告>>> 关于共建具身智能开源数据集的倡议>>> 关于云脑任务中统一路径访问方式的公告>>> 关于将启智集群GPU资源迁移至智算集群的公告>>>
shaunbys
/
3-6-JiangShuoRan-bioner_joint_learning
Not watched
1
0
Fork 0
Code
Releases 0 Wiki Activity
Issues 0 Pull Requests 0 Datasets Cloudbrain
You can not select more than 25 topics Topics must start with a chinese character,a letter or number, can include dashes ('-') and can be up to 35 characters long.
2 Commits
1 Branch
181 KiB
12 Download
 
 
 
 
Branch: master
Branches Tags
${ item.name }
Create branch ${ searchTerm }
from 'master'
${ noResults }
3-6-JiangShuoRan-Drug_knowl.../src
SRCHIANG 4d4ee9219a first commit 2 years ago
..
__pycache__ first commit 2 years ago
biocodes first commit 2 years ago
LICENSE first commit 2 years ago
README.md first commit 2 years ago
__init__.py first commit 2 years ago
biocodes.zip first commit 2 years ago
create_pretraining_data.py first commit 2 years ago
extract_features.py first commit 2 years ago
input_pipeline.py first commit 2 years ago
lstm_crf_layer.py first commit 2 years ago
modeling.py first commit 2 years ago
modeling_test.py first commit 2 years ago
optimization.py first commit 2 years ago
optimization_test.py first commit 2 years ago
requirements.txt first commit 2 years ago
run_classifier.py first commit 2 years ago
run_multi_task_ner.py first commit 2 years ago
run_multihead_multitask_ner.py first commit 2 years ago
run_ner.py first commit 2 years ago
run_pretraining.py first commit 2 years ago
run_qa.py first commit 2 years ago
run_re.py first commit 2 years ago
sample_text.txt first commit 2 years ago
tf_metrics.py first commit 2 years ago
tokenization.py first commit 2 years ago
tokenization_test.py first commit 2 years ago
utils.py first commit 2 years ago

README.md

joint learning

python biocodes/ner_detokenize.py
--token_test_path=/cephfs/group/sng-gdt-isd-conversion/dongfangli/AGAC/output/bioner_joint_learning/token_test_0.txt
--label_test_path=/cephfs/group/sng-gdt-isd-conversion/dongfangli/AGAC/output/bioner_joint_learning/test_results_task_0.tsv
--answer_path=/cephfs/group/sng-gdt-isd-conversion/dongfangli/AGAC/AGAC/test.tsv
--output_dir=/cephfs/group/sng-gdt-isd-conversion/dongfangli/AGAC/output/bioner_joint_learning

perl biocodes/conlleval.pl < /cephfs/group/sng-gdt-isd-conversion/dongfangli/AGAC/output/bioner_joint_learning/NER_result_conll.txt

after finetune

python biocodes/ner_detokenize.py
--token_test_path=/cephfs/group/sng-gdt-isd-conversion/dongfangli/AGAC/output/bioner_after_finetune/token_test.txt
--label_test_path=/cephfs/group/sng-gdt-isd-conversion/dongfangli/AGAC/output/bioner_after_finetune/label_test.txt
--answer_path=/cephfs/group/sng-gdt-isd-conversion/dongfangli/AGAC/AGAC/test.tsv
--output_dir=/cephfs/group/sng-gdt-isd-conversion/dongfangli/AGAC/output/bioner_after_finetune

perl biocodes/conlleval.pl < /cephfs/group/sng-gdt-isd-conversion/dongfangli/AGAC/output/bioner_after_finetune/NER_result_conll.txt

python biocodes/re_eval.py --output_path=/cephfs/group/sng-gdt-isd-conversion/dongfangli/AGAC/output/bioner_rel/test_results.tsv --answer_path=/cephfs/group/sng-gdt-isd-conversion/dongfangli/AGAC/REdata/GAD/1/test.tsv

BC5CDR-disease

python biocodes/ner_detokenize.py
--token_test_path=/cephfs/group/sng-gdt-isd-conversion/dongfangli/AGAC/output/bioner_crf/token_test.txt
--label_test_path=/cephfs/group/sng-gdt-isd-conversion/dongfangli/AGAC/output/bioner_crf/label_test.txt
--answer_path=/cephfs/group/sng-gdt-isd-conversion/dongfangli/AGAC/NERdata//test.tsv
--output_dir=/cephfs/group/sng-gdt-isd-conversion/dongfangli/AGAC/output/bioner_crf

perl biocodes/conlleval.pl < /cephfs/group/sng-gdt-isd-conversion/dongfangli/AGAC/output/bioner_crf/NER_result_conll.txt

python biocodes/ner_detokenize.py
--token_test_path=/cephfs/group/sng-gdt-isd-conversion/dongfangli/AGAC/output/bioner_finetune/token_test.txt
--label_test_path=/cephfs/group/sng-gdt-isd-conversion/dongfangli/AGAC/output/bioner_finetune/label_test.txt
--answer_path=/cephfs/group/sng-gdt-isd-conversion/dongfangli/AGAC/NERdata/BC4CHEMD/test.tsv
--output_dir=/cephfs/group/sng-gdt-isd-conversion/dongfangli/AGAC/output/bioner_finetune

perl biocodes/conlleval.pl < /cephfs/group/sng-gdt-isd-conversion/dongfangli/AGAC/output/bioner_finetune/NER_result_conll.txt

train command

--do_train=true
--do_eval=true
--do_predict=true
--max_seq_length=128
--vocab_file=/cephfs/group/sng-gdt-isd-conversion/dongfangli/AGAC/biobert_v1.1_pubmed/vocab.txt
--char_vocab_file=/cephfs/group/sng-gdt-isd-conversion/dongfangli/AGAC/biobert_v1.1_pubmed/char_vocab.txt
--bert_config_file=/cephfs/group/sng-gdt-isd-conversion/dongfangli/AGAC/biobert_v1.1_pubmed/bert_config.json
--init_checkpoint=/cephfs/group/sng-gdt-isd-conversion/dongfangli/AGAC/biobert_v1.1_pubmed/model.ckpt-1000000
--num_train_epochs=15.0
--train_batch_size=5
--learning_rate=2e-5
--other_learning_rate=3e-5
--data_dir=/cephfs/group/sng-gdt-isd-conversion/dongfangli/AGAC/AGAC
--output_dir=/cephfs/group/sng-gdt-isd-conversion/dongfangli/AGAC/output/bioner_crf_biobert

BCDI

--do_train=true
--do_eval=true
--do_predict=true
--do_lower_case=true
--vocab_file=/cephfs/group/sng-gdt-isd-conversion/dongfangli/AGAC/NCBI_BERT_pubmed_mimic_uncased_L-12_H-768_A-12/vocab.txt
--char_vocab_file=/cephfs/group/sng-gdt-isd-conversion/dongfangli/AGAC/NCBI_BERT_pubmed_mimic_uncased_L-12_H-768_A-12/char_vocab.txt
--bert_config_file=/cephfs/group/sng-gdt-isd-conversion/dongfangli/AGAC/NCBI_BERT_pubmed_mimic_uncased_L-12_H-768_A-12/bert_config.json
--init_checkpoint=/cephfs/group/sng-gdt-isd-conversion/dongfangli/AGAC/NCBI_BERT_pubmed_mimic_uncased_L-12_H-768_A-12/bert_model.ckpt
--num_train_epochs=15.0
--learning_rate=2e-5
--train_batch_size=5
--other_learning_rate=3e-5
--data_dir=/cephfs/group/sng-gdt-isd-conversion/dongfangli/AGAC/AGAC
--output_dir=/cephfs/group/sng-gdt-isd-conversion/dongfangli/AGAC/output/bioner_crf_bcdi

offline

--do_train=true
--do_eval=true
--vocab_file=../input/biobert_v1.1_pubmed/vocab.txt
--char_vocab_file=../input/biobert_v1.1_pubmed/char_vocab.txt
--bert_config_file=../input/biobert_v1.1_pubmed/bert_config.json
--init_checkpoint=../input/biobert_v1.1_pubmed/model.ckpt-1000000
--num_train_epochs=10.0
--data_dir=../input/BC4CHEMD
--output_dir=../output/bioner/

joint learning

--do_train=true
--do_eval=true
--do_predict=true
--max_seq_length=128
--vocab_file=/cephfs/group/sng-gdt-isd-conversion/dongfangli/AGAC/biobert_v1.1_pubmed/vocab.txt
--char_vocab_file=/cephfs/group/sng-gdt-isd-conversion/dongfangli/AGAC/biobert_v1.1_pubmed/char_vocab.txt
--bert_config_file=/cephfs/group/sng-gdt-isd-conversion/dongfangli/AGAC/biobert_v1.1_pubmed/bert_config.json
--init_checkpoint=/cephfs/group/sng-gdt-isd-conversion/dongfangli/AGAC/biobert_v1.1_pubmed/model.ckpt-1000000
--num_train_epochs=15.0
--train_batch_size=10
--learning_rate=2e-5
--other_learning_rate=3e-5
--data_dir=/cephfs/group/sng-gdt-isd-conversion/dongfangli/AGAC/AGAC,/cephfs/group/sng-gdt-isd-conversion/dongfangli/AGAC/NERdata/BC2GM,/cephfs/group/sng-gdt-isd-conversion/dongfangli/AGAC/NERdata/BC4CHEMD,/cephfs/group/sng-gdt-isd-conversion/dongfangli/AGAC/NERdata/BC5CDR-chem,/cephfs/group/sng-gdt-isd-conversion/dongfangli/AGAC/NERdata/NCBI-disease,/cephfs/group/sng-gdt-isd-conversion/dongfangli/AGAC/NERdata/BC5CDR-disease,/cephfs/group/sng-gdt-isd-conversion/dongfangli/AGAC/NERdata/JNLPBA,/cephfs/group/sng-gdt-isd-conversion/dongfangli/AGAC/NERdata/linnaeus,/cephfs/group/sng-gdt-isd-conversion/dongfangli/AGAC/NERdata/s800
--output_dir=/cephfs/group/sng-gdt-isd-conversion/dongfangli/AGAC/output/bioner_joint_learning
--task_nums=9
--main_task=0

Delete task

Are you sure you want to delete this task? Once this task is deleted, it cannot be recovered.

cancel
confirm

No Description

Python Perl Text other

Please read the following content carefully:

Dear OpenI User

Thank you for your continuous support to the Openl Qizhi Community AI Collaboration Platform. In order to protect your usage rights and ensure network security, we updated the Openl Qizhi Community AI Collaboration Platform Usage Agreement in January 2024. The updated agreement specifies that users are prohibited from using intranet penetration tools. After you click "Agree and continue", you can continue to use our services. Thank you for your cooperation and understanding.

For more agreement content, please refer to the《Openl Qizhi Community AI Collaboration Platform Usage Agreement》

Agree and continue
Disagree, exit