|
- Unparsed args: ['--lstm_layer', '2']
- fix_gaz_emb : True
- fix_entity_emb : True
- contrast : False
- number_layer : 1
- bilstm_flag : True
- gat_nhidden : 300
- gat_nhead : 4
- gat_layer : 2
- pre_model : BERT
- ner_encode_type : lstm
- strategy : n
- model_type : lstm
- entity : True
- use_bert : True
- attention : True
- tri_fuse : False
- alpha : 0.1
- dropout : 0.6
- droplstm : 0.0
- dropbert : 0.1
- dropgat : 0.0
- gaz_dropout : 0.2
- kg_dropout : 0.2
- use_lstm : False
- dataset_name : meizhou
- train_file : data/train_ch.txt
- test_file : data/train_ch.txt
- dev_file : data/train_ch.txt
- gaz_file : None
- kg_file : None
- char_embedding_path : None
- data_stored_directory : data/generated_data/
- param_stored_directory : data/model_param/
- entity_type_embdding_path : None
- language : CH
- norm_char_emb : True
- norm_gaz_emb : True
- norm_kg_emb : False
- number_normalized : True
- max_sentence_length : 200
- batch_size : 80
- max_epoch : 10
- lr : 2e-05
- lr_other : 0.0001
- lr_decay : 0.005
- use_clip : True
- clip : 5.0
- gradient_accumulation_steps : 1
- optimizer : Adam
- l2_penalty : 5e-08
- refresh : False
- use_gpu : True
- visible_gpu : 0
- random_seed : 1234
- Gaz file is None, load nothing
- kg file is None, load nothing
- +++++++++++++++++
- 31991
- ********************
- 28791
- +++++++++++++++++
- 31991
- build entity pretrain emb...
- Embedding: None
- pretrain num:0, prefect match:0, case_match:0, oov:2, oov%:1.0
- build entity id...
- DATA SUMMARY START:
- Dataset name: meizhou
- Tag scheme: BIO
- Max Sentence Length: 200
- Char alphabet size: 2066
- Gaz alphabet size: 2
- Label alphabet size: 18
- Char embedding size: 200
- Gaz embedding size: 100
- Number normalized: True
- Norm char emb: True
- Norm gaz emb: True
- Train instance number: 28791
- Dev instance number: 3200
- Test instance number: 31991
- Train cut number: 0
- Dev cut number: 0
- Test cut number: 0
- DATA SUMMARY END.
- Data setting saved to file: data/generated_data/meizhou_dataset.dset
- build BLSTM_GCN_CRF model...
- using model lstm......
- Epoch: 0/10
- Learning rate is setted as: 2e-05
- Instance: 2000; Time: 142.58s; loss: 171302.4873
- Instance: 4000; Time: 142.88s; loss: 44153.7988
- Instance: 6000; Time: 142.38s; loss: 18209.8379
- Instance: 8000; Time: 138.45s; loss: 11931.5684
- Instance: 10000; Time: 140.09s; loss: 9244.4395
- Instance: 12000; Time: 141.15s; loss: 7582.6777
- Instance: 14000; Time: 140.76s; loss: 9183.5273
- Instance: 16000; Time: 141.27s; loss: 7506.4688
- Instance: 18000; Time: 143.13s; loss: 6382.6328
- Instance: 20000; Time: 143.37s; loss: 6727.5078
- Instance: 22000; Time: 138.71s; loss: 6478.2598
- Instance: 24000; Time: 140.59s; loss: 6246.2891
- Instance: 26000; Time: 143.03s; loss: 7434.1055
- Instance: 28000; Time: 139.19s; loss: 6686.4922
- Instance: 28791; Time: 55.22s; loss: 2634.4883
- Epoch: 0 training finished. Time: 2032.82s, speed: 14.16st/s, total loss: 321704.5810546875
- $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
- 3200
- gold_num = 5318 pred_num = 5425 right_num = 4799
- Dev: time: 172.63s, speed: 18.55st/s; acc: 0.9776, p: 0.8846, r: 0.9024, f: 0.8934
- Exceed previous best f score: -1
- $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
- 31991
- gold_num = 53249 pred_num = 54441 right_num = 48310
- Test: speed: 22.23st/s; acc: 0.9798, p: 0.8874, r: 0.9072, f: 0.8972
- Epoch: 1/10
- Learning rate is setted as: 1.9900000000000003e-05
- Instance: 2000; Time: 143.80s; loss: 5943.3125
- Instance: 4000; Time: 139.82s; loss: 5789.2949
- Instance: 6000; Time: 144.86s; loss: 5537.9453
- Instance: 8000; Time: 144.94s; loss: 6946.1152
- Instance: 10000; Time: 142.85s; loss: 6228.5137
- Instance: 12000; Time: 147.25s; loss: 5957.9375
- Instance: 14000; Time: 146.40s; loss: 5393.2656
- Instance: 16000; Time: 144.49s; loss: 5702.0742
- Instance: 18000; Time: 133.84s; loss: 5511.9082
- Instance: 20000; Time: 139.61s; loss: 6026.8594
- Instance: 22000; Time: 140.30s; loss: 5835.2676
- Instance: 24000; Time: 139.31s; loss: 4976.8320
- Instance: 26000; Time: 146.03s; loss: 5763.1680
- Instance: 28000; Time: 145.43s; loss: 5224.8867
- Instance: 28791; Time: 58.38s; loss: 2331.9141
- Epoch: 1 training finished. Time: 2057.30s, speed: 13.99st/s, total loss: 83169.294921875
- $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
- 3200
- gold_num = 5318 pred_num = 5381 right_num = 4833
- Dev: time: 171.19s, speed: 18.70st/s; acc: 0.9801, p: 0.8982, r: 0.9088, f: 0.9034
- Exceed previous best f score: 0.8934189704924138
- $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
- 31991
- gold_num = 53249 pred_num = 53753 right_num = 48946
- Test: speed: 22.35st/s; acc: 0.9835, p: 0.9106, r: 0.9192, f: 0.9149
- Epoch: 2/10
- Learning rate is setted as: 1.98005e-05
- Instance: 2000; Time: 145.34s; loss: 4732.1953
- Instance: 4000; Time: 144.20s; loss: 4506.8789
- Instance: 6000; Time: 141.02s; loss: 4119.4414
- Instance: 8000; Time: 141.10s; loss: 4677.8398
- Instance: 10000; Time: 141.74s; loss: 4271.2266
- Instance: 12000; Time: 142.83s; loss: 4796.4805
- Instance: 14000; Time: 140.65s; loss: 5380.6797
- Instance: 16000; Time: 146.08s; loss: 4325.3594
- Instance: 18000; Time: 143.97s; loss: 4054.3984
- Instance: 20000; Time: 144.83s; loss: 4843.8398
- Instance: 22000; Time: 139.42s; loss: 4592.3555
- Instance: 24000; Time: 143.11s; loss: 5002.5508
- Instance: 26000; Time: 141.34s; loss: 4684.6055
- Instance: 28000; Time: 144.77s; loss: 4982.7383
- Instance: 28791; Time: 56.89s; loss: 1429.5781
- Epoch: 2 training finished. Time: 2057.30s, speed: 13.99st/s, total loss: 66400.16796875
- $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
- 3200
- gold_num = 5318 pred_num = 5440 right_num = 4903
- Dev: time: 171.67s, speed: 18.65st/s; acc: 0.9801, p: 0.9013, r: 0.9220, f: 0.9115
- Exceed previous best f score: 0.903448920459856
- $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
- 31991
- gold_num = 53249 pred_num = 54669 right_num = 50101
- Test: speed: 22.25st/s; acc: 0.9853, p: 0.9164, r: 0.9409, f: 0.9285
- Epoch: 3/10
- Learning rate is setted as: 1.97014975e-05
- Instance: 2000; Time: 143.85s; loss: 3828.3086
- Instance: 4000; Time: 145.54s; loss: 3871.9844
- Instance: 6000; Time: 146.22s; loss: 4142.3398
- Instance: 8000; Time: 143.58s; loss: 3691.2930
- Instance: 10000; Time: 144.66s; loss: 4965.6992
- Instance: 12000; Time: 147.70s; loss: 3689.2109
- Instance: 14000; Time: 146.36s; loss: 3978.6172
- Instance: 16000; Time: 147.51s; loss: 4234.1172
- Instance: 18000; Time: 147.61s; loss: 4010.0664
- Instance: 20000; Time: 144.97s; loss: 3831.7266
- Instance: 22000; Time: 135.06s; loss: 4274.9102
- Instance: 24000; Time: 145.39s; loss: 3619.6562
- Instance: 26000; Time: 141.57s; loss: 3963.7422
- Instance: 28000; Time: 143.75s; loss: 4395.2539
- Instance: 28791; Time: 58.49s; loss: 1548.7734
- Epoch: 3 training finished. Time: 2082.26s, speed: 13.83st/s, total loss: 58045.69921875
- $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
- 3200
- gold_num = 5318 pred_num = 5482 right_num = 4949
- Dev: time: 172.11s, speed: 18.60st/s; acc: 0.9800, p: 0.9028, r: 0.9306, f: 0.9165
- Exceed previous best f score: 0.9115077151886968
- $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
- 31991
- gold_num = 53249 pred_num = 54687 right_num = 50544
- Test: speed: 22.35st/s; acc: 0.9866, p: 0.9242, r: 0.9492, f: 0.9366
- Epoch: 4/10
- Learning rate is setted as: 1.9602990012500002e-05
- Instance: 2000; Time: 143.17s; loss: 3440.5781
- Instance: 4000; Time: 147.51s; loss: 2959.6992
- Instance: 6000; Time: 148.10s; loss: 3428.9141
- Instance: 8000; Time: 145.06s; loss: 3388.5430
- Instance: 10000; Time: 145.94s; loss: 3700.2539
- Instance: 12000; Time: 143.41s; loss: 3594.1250
- Instance: 14000; Time: 145.90s; loss: 3864.4258
- Instance: 16000; Time: 146.98s; loss: 3772.0273
- Instance: 18000; Time: 139.80s; loss: 3753.7773
- Instance: 20000; Time: 143.80s; loss: 2921.6914
- Instance: 22000; Time: 144.31s; loss: 3989.4180
- Instance: 24000; Time: 145.78s; loss: 3356.3242
- Instance: 26000; Time: 142.31s; loss: 3695.9883
- Instance: 28000; Time: 142.34s; loss: 4018.6641
- Instance: 28791; Time: 56.99s; loss: 1194.1016
- Epoch: 4 training finished. Time: 2081.41s, speed: 13.83st/s, total loss: 51078.53125
- $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
- 3200
- gold_num = 5318 pred_num = 5444 right_num = 4926
- Dev: time: 171.42s, speed: 18.68st/s; acc: 0.9804, p: 0.9048, r: 0.9263, f: 0.9154
- Epoch: 5/10
- Learning rate is setted as: 1.95049750624375e-05
- Instance: 2000; Time: 143.80s; loss: 3534.4531
- Instance: 4000; Time: 147.36s; loss: 2896.7969
- Instance: 6000; Time: 142.68s; loss: 2933.7461
- Instance: 8000; Time: 147.75s; loss: 3138.9609
- Instance: 10000; Time: 146.18s; loss: 3364.2266
- Instance: 12000; Time: 147.34s; loss: 2860.0195
- Instance: 14000; Time: 142.28s; loss: 3880.6758
- Instance: 16000; Time: 144.91s; loss: 3440.1133
- Instance: 18000; Time: 143.78s; loss: 3172.7500
- Instance: 20000; Time: 139.80s; loss: 3086.7109
- Instance: 22000; Time: 139.63s; loss: 2921.7305
- Instance: 24000; Time: 145.49s; loss: 3038.1602
- Instance: 26000; Time: 144.17s; loss: 3256.7422
- Instance: 28000; Time: 145.54s; loss: 3184.8828
- Instance: 28791; Time: 55.36s; loss: 1421.6992
- Epoch: 5 training finished. Time: 2076.07s, speed: 13.87st/s, total loss: 46131.66796875
- $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
- 3200
- gold_num = 5318 pred_num = 5382 right_num = 4896
- Dev: time: 169.97s, speed: 18.84st/s; acc: 0.9804, p: 0.9097, r: 0.9206, f: 0.9151
- Epoch: 6/10
- Learning rate is setted as: 1.9407450187125313e-05
- Instance: 2000; Time: 147.88s; loss: 2721.9219
- Instance: 4000; Time: 142.36s; loss: 2858.7266
- Instance: 6000; Time: 146.94s; loss: 3038.6211
- Instance: 8000; Time: 146.19s; loss: 3013.9297
- Instance: 10000; Time: 147.12s; loss: 2544.1953
- Instance: 12000; Time: 147.23s; loss: 2690.6836
- Instance: 14000; Time: 140.35s; loss: 3250.0195
- Instance: 16000; Time: 142.75s; loss: 2631.9180
- Instance: 18000; Time: 146.43s; loss: 2840.4883
- Instance: 20000; Time: 141.14s; loss: 3097.7305
- Instance: 22000; Time: 144.29s; loss: 2669.6914
- Instance: 24000; Time: 147.01s; loss: 3369.8203
- Instance: 26000; Time: 180.36s; loss: 3570.5625
- Instance: 28000; Time: 251.09s; loss: 2721.0664
- Instance: 28791; Time: 99.14s; loss: 1167.7969
- Epoch: 6 training finished. Time: 2270.28s, speed: 12.68st/s, total loss: 42187.171875
- $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
- 3200
- gold_num = 5318 pred_num = 5419 right_num = 4916
- Dev: time: 184.23s, speed: 17.38st/s; acc: 0.9806, p: 0.9072, r: 0.9244, f: 0.9157
- Epoch: 7/10
- Learning rate is setted as: 1.9310412936189687e-05
- Instance: 2000; Time: 248.87s; loss: 2374.7109
- Instance: 4000; Time: 235.63s; loss: 2990.2539
- Instance: 6000; Time: 148.18s; loss: 2553.2344
- Instance: 8000; Time: 140.12s; loss: 2982.5820
- Instance: 10000; Time: 138.28s; loss: 2517.8867
- Instance: 12000; Time: 143.16s; loss: 2598.0117
- Instance: 14000; Time: 147.05s; loss: 2915.6523
- Instance: 16000; Time: 145.60s; loss: 2699.1406
- Instance: 18000; Time: 140.82s; loss: 2276.5273
- Instance: 20000; Time: 137.72s; loss: 2581.9727
- Instance: 22000; Time: 145.82s; loss: 3094.3164
- Instance: 24000; Time: 139.66s; loss: 2763.2773
- Instance: 26000; Time: 144.28s; loss: 2444.8008
- Instance: 28000; Time: 145.76s; loss: 3226.3633
- Instance: 28791; Time: 57.62s; loss: 951.1797
- Epoch: 7 training finished. Time: 2258.56s, speed: 12.75st/s, total loss: 38969.91015625
- $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
- 3200
- gold_num = 5318 pred_num = 5471 right_num = 4920
- Dev: time: 170.77s, speed: 18.75st/s; acc: 0.9796, p: 0.8993, r: 0.9252, f: 0.9120
- Epoch: 8/10
- Learning rate is setted as: 1.921386087150874e-05
- Instance: 2000; Time: 144.34s; loss: 2340.8320
- Instance: 4000; Time: 146.56s; loss: 2354.9258
- Instance: 6000; Time: 145.05s; loss: 2696.7930
- Instance: 8000; Time: 146.20s; loss: 2558.7031
- Instance: 10000; Time: 146.94s; loss: 2227.5352
- Instance: 12000; Time: 139.47s; loss: 2740.9258
- Instance: 14000; Time: 139.84s; loss: 2578.0938
- Instance: 16000; Time: 138.76s; loss: 2569.6992
- Instance: 18000; Time: 142.95s; loss: 2673.1875
- Instance: 20000; Time: 148.06s; loss: 2649.2344
- Instance: 22000; Time: 142.22s; loss: 2867.9492
- Instance: 24000; Time: 147.79s; loss: 2510.2812
- Instance: 26000; Time: 145.26s; loss: 2388.5664
- Instance: 28000; Time: 144.68s; loss: 2399.1523
- Instance: 28791; Time: 57.50s; loss: 899.7734
- Epoch: 8 training finished. Time: 2075.61s, speed: 13.87st/s, total loss: 36455.65234375
- $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
- 3200
- gold_num = 5318 pred_num = 5441 right_num = 4913
- Dev: time: 172.24s, speed: 18.59st/s; acc: 0.9798, p: 0.9030, r: 0.9238, f: 0.9133
- Epoch: 9/10
- Learning rate is setted as: 1.9117791567151195e-05
- Instance: 2000; Time: 144.49s; loss: 2654.0117
- Instance: 4000; Time: 148.08s; loss: 2221.6992
- Instance: 6000; Time: 142.95s; loss: 2678.3945
- Instance: 8000; Time: 144.58s; loss: 2261.1680
- Instance: 10000; Time: 143.02s; loss: 2101.1914
- Instance: 12000; Time: 144.87s; loss: 2470.3516
- Instance: 14000; Time: 141.43s; loss: 2026.7070
- Instance: 16000; Time: 142.66s; loss: 2273.5703
- Instance: 18000; Time: 147.59s; loss: 2474.6367
- Instance: 20000; Time: 142.77s; loss: 2491.7109
- Instance: 22000; Time: 148.05s; loss: 2696.8281
- Instance: 24000; Time: 146.62s; loss: 2300.0273
- Instance: 26000; Time: 147.74s; loss: 2605.1055
- Instance: 28000; Time: 138.43s; loss: 2226.3789
- Instance: 28791; Time: 53.81s; loss: 922.8242
- Epoch: 9 training finished. Time: 2077.09s, speed: 13.86st/s, total loss: 34404.60546875
- $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
- 3200
- gold_num = 5318 pred_num = 5440 right_num = 4905
- Dev: time: 172.30s, speed: 18.58st/s; acc: 0.9795, p: 0.9017, r: 0.9223, f: 0.9119
|