|
- Unparsed args: ['--lstm_layer', '2']
- fix_gaz_emb : True
- fix_entity_emb : True
- contrast : False
- number_layer : 1
- bilstm_flag : True
- gat_nhidden : 300
- gat_nhead : 4
- gat_layer : 2
- pre_model : BERT
- ner_encode_type : lstm
- strategy : n
- model_type : lstm
- entity : True
- use_bert : True
- attention : True
- tri_fuse : False
- alpha : 0.1
- dropout : 0.6
- droplstm : 0.0
- dropbert : 0.1
- dropgat : 0.0
- gaz_dropout : 0.2
- kg_dropout : 0.2
- use_lstm : False
- dataset_name : meizhou3
- train_file : data/train_ch.txt
- test_file : data/train_ch.txt
- dev_file : data/train_ch.txt
- gaz_file : None
- kg_file : None
- char_embedding_path : None
- data_stored_directory : data/generated_data/
- param_stored_directory : data/model_param/
- entity_type_embdding_path : None
- language : CH
- norm_char_emb : True
- norm_gaz_emb : True
- norm_kg_emb : False
- number_normalized : True
- max_sentence_length : 200
- batch_size : 80
- max_epoch : 8
- lr : 2e-05
- lr_other : 0.0001
- lr_decay : 0.005
- use_clip : True
- clip : 5.0
- gradient_accumulation_steps : 1
- optimizer : Adam
- l2_penalty : 5e-08
- refresh : False
- use_gpu : True
- visible_gpu : 0
- random_seed : 2022
- Gaz file is None, load nothing
- kg file is None, load nothing
- +++++++++++++++++
- 31991
- ********************
- 28791
- +++++++++++++++++
- 31991
- build entity pretrain emb...
- Embedding: None
- pretrain num:0, prefect match:0, case_match:0, oov:2, oov%:1.0
- build entity id...
- DATA SUMMARY START:
- Dataset name: meizhou3
- Tag scheme: BIO
- Max Sentence Length: 200
- Char alphabet size: 2066
- Gaz alphabet size: 2
- Label alphabet size: 18
- Char embedding size: 200
- Gaz embedding size: 100
- Number normalized: True
- Norm char emb: True
- Norm gaz emb: True
- Train instance number: 28791
- Dev instance number: 3200
- Test instance number: 31991
- Train cut number: 0
- Dev cut number: 0
- Test cut number: 0
- DATA SUMMARY END.
- Data setting saved to file: data/generated_data/meizhou3_dataset.dset
- build BLSTM_GCN_CRF model...
- using model lstm......
- Epoch: 0/8
- Learning rate is setted as: 2e-05
- Instance: 2000; Time: 142.03s; loss: 210286.2871
- Instance: 4000; Time: 142.47s; loss: 40851.6465
- Instance: 6000; Time: 142.53s; loss: 16347.6562
- Instance: 8000; Time: 138.52s; loss: 11764.0039
- Instance: 10000; Time: 139.74s; loss: 8832.8613
- Instance: 12000; Time: 141.23s; loss: 7723.6973
- Instance: 14000; Time: 141.00s; loss: 8997.8926
- Instance: 16000; Time: 141.25s; loss: 7468.6719
- Instance: 18000; Time: 143.18s; loss: 6401.1406
- Instance: 20000; Time: 143.20s; loss: 6900.0840
- Instance: 22000; Time: 138.35s; loss: 6353.8281
- Instance: 24000; Time: 142.55s; loss: 5690.2871
- Instance: 26000; Time: 145.78s; loss: 7340.8359
- Instance: 28000; Time: 140.79s; loss: 6447.0430
- Instance: 28791; Time: 56.48s; loss: 2524.0215
- Epoch: 0 training finished. Time: 2039.10s, speed: 14.12st/s, total loss: 353929.95703125
- $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
- 3200
- gold_num = 5318 pred_num = 5430 right_num = 4836
- Dev: time: 174.81s, speed: 18.32st/s; acc: 0.9795, p: 0.8906, r: 0.9094, f: 0.8999
- Exceed previous best f score: -1
- $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
- 31991
- gold_num = 53249 pred_num = 54431 right_num = 48780
- Test: speed: 22.00st/s; acc: 0.9813, p: 0.8962, r: 0.9161, f: 0.9060
- Epoch: 1/8
- Learning rate is setted as: 1.9900000000000003e-05
- Instance: 2000; Time: 148.90s; loss: 5630.1836
- Instance: 4000; Time: 145.79s; loss: 5371.3047
- Instance: 6000; Time: 149.41s; loss: 5135.6523
- Instance: 8000; Time: 146.80s; loss: 6374.0137
- Instance: 10000; Time: 143.93s; loss: 5877.5742
- Instance: 12000; Time: 149.29s; loss: 5859.4883
- Instance: 14000; Time: 147.82s; loss: 5021.8438
- Instance: 16000; Time: 145.41s; loss: 5193.7656
- Instance: 18000; Time: 138.54s; loss: 5229.8086
- Instance: 20000; Time: 144.71s; loss: 5881.3711
- Instance: 22000; Time: 144.54s; loss: 5498.8125
- Instance: 24000; Time: 140.78s; loss: 4763.3438
- Instance: 26000; Time: 147.47s; loss: 5352.1328
- Instance: 28000; Time: 147.34s; loss: 4916.3516
- Instance: 28791; Time: 58.85s; loss: 2230.0312
- Epoch: 1 training finished. Time: 2099.58s, speed: 13.71st/s, total loss: 78335.677734375
- $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
- 3200
- gold_num = 5318 pred_num = 5428 right_num = 4872
- Dev: time: 174.81s, speed: 18.32st/s; acc: 0.9804, p: 0.8976, r: 0.9161, f: 0.9068
- Exceed previous best f score: 0.8998883513211761
- $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
- 31991
- gold_num = 53249 pred_num = 54281 right_num = 49365
- Test: speed: 22.14st/s; acc: 0.9844, p: 0.9094, r: 0.9271, f: 0.9182
- Epoch: 2/8
- Learning rate is setted as: 1.98005e-05
- Instance: 2000; Time: 145.16s; loss: 4414.9102
- Instance: 4000; Time: 146.78s; loss: 4244.4062
- Instance: 6000; Time: 146.00s; loss: 3639.7930
- Instance: 8000; Time: 145.79s; loss: 4392.3320
- Instance: 10000; Time: 143.19s; loss: 4030.9961
- Instance: 12000; Time: 143.44s; loss: 4527.9297
- Instance: 14000; Time: 141.22s; loss: 5370.3047
- Instance: 16000; Time: 146.52s; loss: 4058.3789
- Instance: 18000; Time: 144.57s; loss: 3837.4336
- Instance: 20000; Time: 146.78s; loss: 4675.5781
- Instance: 22000; Time: 143.92s; loss: 4341.8281
- Instance: 24000; Time: 147.82s; loss: 4769.5625
- Instance: 26000; Time: 143.67s; loss: 4359.0352
- Instance: 28000; Time: 146.03s; loss: 4725.9375
- Instance: 28791; Time: 56.77s; loss: 1341.5195
- Epoch: 2 training finished. Time: 2087.68s, speed: 13.79st/s, total loss: 62729.9453125
- $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
- 3200
- gold_num = 5318 pred_num = 5432 right_num = 4914
- Dev: time: 174.01s, speed: 18.40st/s; acc: 0.9807, p: 0.9046, r: 0.9240, f: 0.9142
- Exceed previous best f score: 0.9067560022333891
- $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
- 31991
- gold_num = 53249 pred_num = 54621 right_num = 50308
- Test: speed: 22.15st/s; acc: 0.9861, p: 0.9210, r: 0.9448, f: 0.9328
- Epoch: 3/8
- Learning rate is setted as: 1.97014975e-05
- Instance: 2000; Time: 145.18s; loss: 3789.1953
- Instance: 4000; Time: 145.90s; loss: 3524.3633
- Instance: 6000; Time: 146.81s; loss: 3934.2070
- Instance: 8000; Time: 146.58s; loss: 3316.4414
- Instance: 10000; Time: 148.05s; loss: 4747.9922
- Instance: 12000; Time: 148.32s; loss: 3521.8828
- Instance: 14000; Time: 147.31s; loss: 3595.0234
- Instance: 16000; Time: 148.38s; loss: 4053.2734
- Instance: 18000; Time: 148.01s; loss: 3812.3906
- Instance: 20000; Time: 145.44s; loss: 3615.7617
- Instance: 22000; Time: 135.37s; loss: 4154.0039
- Instance: 24000; Time: 147.69s; loss: 3385.2734
- Instance: 26000; Time: 145.33s; loss: 3749.9688
- Instance: 28000; Time: 144.56s; loss: 4086.7812
- Instance: 28791; Time: 59.00s; loss: 1456.5273
- Epoch: 3 training finished. Time: 2101.93s, speed: 13.70st/s, total loss: 54743.0859375
- $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
- 3200
- gold_num = 5318 pred_num = 5474 right_num = 4938
- Dev: time: 173.68s, speed: 18.44st/s; acc: 0.9800, p: 0.9021, r: 0.9285, f: 0.9151
- Exceed previous best f score: 0.914232558139535
- $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
- 31991
- gold_num = 53249 pred_num = 54696 right_num = 50684
- Test: speed: 22.17st/s; acc: 0.9872, p: 0.9266, r: 0.9518, f: 0.9391
- Epoch: 4/8
- Learning rate is setted as: 1.9602990012500002e-05
- Instance: 2000; Time: 141.19s; loss: 3316.2305
- Instance: 4000; Time: 145.34s; loss: 2795.5312
- Instance: 6000; Time: 146.30s; loss: 3274.4180
- Instance: 8000; Time: 143.61s; loss: 3203.4688
- Instance: 10000; Time: 145.50s; loss: 3489.0273
- Instance: 12000; Time: 144.52s; loss: 3350.3125
- Instance: 14000; Time: 147.17s; loss: 3611.4297
- Instance: 16000; Time: 145.79s; loss: 3505.0898
- Instance: 18000; Time: 138.62s; loss: 3708.7812
- Instance: 20000; Time: 143.67s; loss: 2637.8906
- Instance: 22000; Time: 143.46s; loss: 3692.5391
- Instance: 24000; Time: 145.08s; loss: 3148.5898
- Instance: 26000; Time: 140.94s; loss: 3479.0469
- Instance: 28000; Time: 142.50s; loss: 3849.1133
- Instance: 28791; Time: 58.22s; loss: 1149.3086
- Epoch: 4 training finished. Time: 2071.91s, speed: 13.90st/s, total loss: 48210.77734375
- $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
- 3200
- gold_num = 5318 pred_num = 5471 right_num = 4934
- Dev: time: 172.20s, speed: 18.59st/s; acc: 0.9807, p: 0.9018, r: 0.9278, f: 0.9146
- Epoch: 5/8
- Learning rate is setted as: 1.95049750624375e-05
- Instance: 2000; Time: 146.79s; loss: 3487.0156
- Instance: 4000; Time: 149.59s; loss: 2794.5703
- Instance: 6000; Time: 145.22s; loss: 2796.6641
- Instance: 8000; Time: 149.57s; loss: 2962.7461
- Instance: 10000; Time: 148.11s; loss: 3248.1367
- Instance: 12000; Time: 149.41s; loss: 2707.7852
- Instance: 14000; Time: 143.29s; loss: 3788.0781
- Instance: 16000; Time: 149.61s; loss: 3232.1367
- Instance: 18000; Time: 147.60s; loss: 3051.0273
- Instance: 20000; Time: 141.69s; loss: 3005.7578
- Instance: 22000; Time: 141.36s; loss: 2794.3828
- Instance: 24000; Time: 147.84s; loss: 2825.8711
- Instance: 26000; Time: 146.46s; loss: 2950.2773
- Instance: 28000; Time: 147.45s; loss: 2910.8789
- Instance: 28791; Time: 56.68s; loss: 1359.0664
- Epoch: 5 training finished. Time: 2110.66s, speed: 13.64st/s, total loss: 43914.39453125
- $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
- 3200
- gold_num = 5318 pred_num = 5479 right_num = 4948
- Dev: time: 174.85s, speed: 18.31st/s; acc: 0.9803, p: 0.9031, r: 0.9304, f: 0.9166
- Exceed previous best f score: 0.9151223128243143
- $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
- 31991
- gold_num = 53249 pred_num = 54571 right_num = 50987
- Test: speed: 21.91st/s; acc: 0.9891, p: 0.9343, r: 0.9575, f: 0.9458
- Epoch: 6/8
- Learning rate is setted as: 1.9407450187125313e-05
- Instance: 2000; Time: 148.38s; loss: 2600.9141
- Instance: 4000; Time: 144.04s; loss: 2764.4883
- Instance: 6000; Time: 145.39s; loss: 2807.6367
- Instance: 8000; Time: 144.38s; loss: 2890.7031
- Instance: 10000; Time: 148.22s; loss: 2421.9336
- Instance: 12000; Time: 147.78s; loss: 2518.5938
- Instance: 14000; Time: 140.71s; loss: 3113.4219
- Instance: 16000; Time: 143.03s; loss: 2638.6445
- Instance: 18000; Time: 146.65s; loss: 2555.5547
- Instance: 20000; Time: 142.51s; loss: 2916.3789
- Instance: 22000; Time: 143.40s; loss: 2510.6641
- Instance: 24000; Time: 144.08s; loss: 3016.9375
- Instance: 26000; Time: 142.41s; loss: 3498.6445
- Instance: 28000; Time: 146.15s; loss: 2688.6055
- Instance: 28791; Time: 59.01s; loss: 1112.8984
- Epoch: 6 training finished. Time: 2086.15s, speed: 13.80st/s, total loss: 40056.01953125
- $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
- 3200
- gold_num = 5318 pred_num = 5472 right_num = 4934
- Dev: time: 173.71s, speed: 18.43st/s; acc: 0.9802, p: 0.9017, r: 0.9278, f: 0.9146
- Epoch: 7/8
- Learning rate is setted as: 1.9310412936189687e-05
- Instance: 2000; Time: 146.56s; loss: 2360.9531
- Instance: 4000; Time: 146.79s; loss: 2986.2109
- Instance: 6000; Time: 148.62s; loss: 2391.6367
- Instance: 8000; Time: 141.56s; loss: 2972.7578
- Instance: 10000; Time: 139.57s; loss: 2428.3125
- Instance: 12000; Time: 144.43s; loss: 2546.8008
- Instance: 14000; Time: 148.38s; loss: 2822.0078
- Instance: 16000; Time: 145.78s; loss: 2605.3711
- Instance: 18000; Time: 140.83s; loss: 2300.3789
- Instance: 20000; Time: 138.16s; loss: 2320.9648
- Instance: 22000; Time: 145.98s; loss: 2867.5195
- Instance: 24000; Time: 141.41s; loss: 2740.9023
- Instance: 26000; Time: 145.95s; loss: 2181.5156
- Instance: 28000; Time: 146.55s; loss: 3013.6797
- Instance: 28791; Time: 58.46s; loss: 904.2891
- Epoch: 7 training finished. Time: 2079.02s, speed: 13.85st/s, total loss: 37443.30078125
- $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
- 3200
- gold_num = 5318 pred_num = 5470 right_num = 4939
- Dev: time: 173.91s, speed: 18.41st/s; acc: 0.9804, p: 0.9029, r: 0.9287, f: 0.9156
|