|
- Unparsed args: ['--lstm_layer', '2']
- fix_gaz_emb : True
- fix_entity_emb : True
- contrast : False
- number_layer : 1
- bilstm_flag : True
- gat_nhidden : 300
- gat_nhead : 4
- gat_layer : 2
- pre_model : BERT
- ner_encode_type : lstm
- strategy : n
- model_type : lstm
- entity : True
- use_bert : True
- attention : True
- tri_fuse : False
- alpha : 0.1
- dropout : 0.6
- droplstm : 0.0
- dropbert : 0.1
- dropgat : 0.0
- gaz_dropout : 0.2
- kg_dropout : 0.2
- use_lstm : False
- dataset_name : meizhou5
- train_file : data/train_ch.txt
- test_file : data/train_ch.txt
- dev_file : data/train_ch.txt
- gaz_file : None
- kg_file : None
- char_embedding_path : None
- data_stored_directory : data/generated_data/
- param_stored_directory : data/model_param/
- entity_type_embdding_path : None
- language : CH
- norm_char_emb : True
- norm_gaz_emb : True
- norm_kg_emb : False
- number_normalized : True
- max_sentence_length : 200
- batch_size : 80
- max_epoch : 8
- lr : 2e-05
- lr_other : 0.0001
- lr_decay : 0.005
- use_clip : True
- clip : 5.0
- gradient_accumulation_steps : 1
- optimizer : Adam
- l2_penalty : 5e-08
- refresh : False
- use_gpu : True
- visible_gpu : 0
- random_seed : 5555
- Gaz file is None, load nothing
- kg file is None, load nothing
- +++++++++++++++++
- 31991
- ********************
- 28791
- +++++++++++++++++
- 31991
- build entity pretrain emb...
- Embedding: None
- pretrain num:0, prefect match:0, case_match:0, oov:2, oov%:1.0
- build entity id...
- DATA SUMMARY START:
- Dataset name: meizhou5
- Tag scheme: BIO
- Max Sentence Length: 200
- Char alphabet size: 2066
- Gaz alphabet size: 2
- Label alphabet size: 18
- Char embedding size: 200
- Gaz embedding size: 100
- Number normalized: True
- Norm char emb: True
- Norm gaz emb: True
- Train instance number: 28791
- Dev instance number: 3200
- Test instance number: 31991
- Train cut number: 0
- Dev cut number: 0
- Test cut number: 0
- DATA SUMMARY END.
- Data setting saved to file: data/generated_data/meizhou5_dataset.dset
- build BLSTM_GCN_CRF model...
- using model lstm......
- Epoch: 0/8
- Learning rate is setted as: 2e-05
- Instance: 2000; Time: 143.90s; loss: 162425.5391
- Instance: 4000; Time: 145.02s; loss: 37894.7969
- Instance: 6000; Time: 144.21s; loss: 15145.8965
- Instance: 8000; Time: 140.71s; loss: 11221.2363
- Instance: 10000; Time: 141.98s; loss: 8720.3008
- Instance: 12000; Time: 143.23s; loss: 7552.8867
- Instance: 14000; Time: 143.06s; loss: 8751.7148
- Instance: 16000; Time: 143.31s; loss: 7383.5879
- Instance: 18000; Time: 144.88s; loss: 6340.6230
- Instance: 20000; Time: 144.89s; loss: 6761.2988
- Instance: 22000; Time: 140.41s; loss: 6356.2500
- Instance: 24000; Time: 144.29s; loss: 5677.1426
- Instance: 26000; Time: 147.27s; loss: 7309.5664
- Instance: 28000; Time: 142.30s; loss: 6489.9844
- Instance: 28791; Time: 56.92s; loss: 2562.0312
- Epoch: 0 training finished. Time: 2066.37s, speed: 13.93st/s, total loss: 300592.85546875
- $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
- 3200
- gold_num = 5318 pred_num = 5377 right_num = 4830
- Dev: time: 176.07s, speed: 18.18st/s; acc: 0.9778, p: 0.8983, r: 0.9082, f: 0.9032
- Exceed previous best f score: -1
- $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
- 31991
- gold_num = 53249 pred_num = 54085 right_num = 48775
- Test: speed: 21.58st/s; acc: 0.9800, p: 0.9018, r: 0.9160, f: 0.9088
- Epoch: 1/8
- Learning rate is setted as: 1.9900000000000003e-05
- Instance: 2000; Time: 148.60s; loss: 5519.0039
- Instance: 4000; Time: 144.67s; loss: 5389.3359
- Instance: 6000; Time: 148.44s; loss: 5076.1562
- Instance: 8000; Time: 145.93s; loss: 6308.5586
- Instance: 10000; Time: 143.70s; loss: 5647.9453
- Instance: 12000; Time: 148.68s; loss: 5637.3438
- Instance: 14000; Time: 147.28s; loss: 4857.1719
- Instance: 16000; Time: 145.62s; loss: 5261.3633
- Instance: 18000; Time: 137.93s; loss: 5246.2734
- Instance: 20000; Time: 144.99s; loss: 5650.4844
- Instance: 22000; Time: 144.52s; loss: 5499.1895
- Instance: 24000; Time: 140.30s; loss: 4655.5820
- Instance: 26000; Time: 146.83s; loss: 5461.6953
- Instance: 28000; Time: 146.97s; loss: 4737.2148
- Instance: 28791; Time: 58.97s; loss: 2235.0391
- Epoch: 1 training finished. Time: 2093.42s, speed: 13.75st/s, total loss: 77182.357421875
- $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
- 3200
- gold_num = 5318 pred_num = 5399 right_num = 4877
- Dev: time: 176.39s, speed: 18.15st/s; acc: 0.9803, p: 0.9033, r: 0.9171, f: 0.9101
- Exceed previous best f score: 0.9032258064516128
- $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
- 31991
- gold_num = 53249 pred_num = 54111 right_num = 49515
- Test: speed: 21.81st/s; acc: 0.9844, p: 0.9151, r: 0.9299, f: 0.9224
- Epoch: 2/8
- Learning rate is setted as: 1.98005e-05
- Instance: 2000; Time: 146.70s; loss: 4354.2773
- Instance: 4000; Time: 147.11s; loss: 4171.7695
- Instance: 6000; Time: 146.55s; loss: 3654.5781
- Instance: 8000; Time: 146.35s; loss: 4419.8867
- Instance: 10000; Time: 143.97s; loss: 4007.7305
- Instance: 12000; Time: 143.56s; loss: 4596.8594
- Instance: 14000; Time: 142.15s; loss: 5087.9297
- Instance: 16000; Time: 146.95s; loss: 4110.7227
- Instance: 18000; Time: 145.73s; loss: 3762.1016
- Instance: 20000; Time: 147.26s; loss: 4643.8633
- Instance: 22000; Time: 144.58s; loss: 4200.7070
- Instance: 24000; Time: 149.13s; loss: 4789.3398
- Instance: 26000; Time: 144.49s; loss: 4335.8672
- Instance: 28000; Time: 147.06s; loss: 4613.7266
- Instance: 28791; Time: 56.89s; loss: 1362.7578
- Epoch: 2 training finished. Time: 2098.47s, speed: 13.72st/s, total loss: 62112.1171875
- $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
- 3200
- gold_num = 5318 pred_num = 5451 right_num = 4918
- Dev: time: 177.10s, speed: 18.08st/s; acc: 0.9804, p: 0.9022, r: 0.9248, f: 0.9134
- Exceed previous best f score: 0.9101427638331622
- $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
- 31991
- gold_num = 53249 pred_num = 54670 right_num = 50381
- Test: speed: 21.88st/s; acc: 0.9860, p: 0.9215, r: 0.9461, f: 0.9337
- Epoch: 3/8
- Learning rate is setted as: 1.97014975e-05
- Instance: 2000; Time: 146.14s; loss: 3608.9414
- Instance: 4000; Time: 146.27s; loss: 3428.9531
- Instance: 6000; Time: 147.40s; loss: 3929.1602
- Instance: 8000; Time: 147.34s; loss: 3368.2109
- Instance: 10000; Time: 149.09s; loss: 4812.7891
- Instance: 12000; Time: 149.23s; loss: 3448.6562
- Instance: 14000; Time: 148.29s; loss: 3627.2539
- Instance: 16000; Time: 149.53s; loss: 3897.1211
- Instance: 18000; Time: 148.98s; loss: 3800.2812
- Instance: 20000; Time: 146.12s; loss: 3575.6406
- Instance: 22000; Time: 136.15s; loss: 4180.4961
- Instance: 24000; Time: 148.41s; loss: 3321.2852
- Instance: 26000; Time: 146.87s; loss: 3718.7578
- Instance: 28000; Time: 145.05s; loss: 4209.9141
- Instance: 28791; Time: 59.24s; loss: 1452.4219
- Epoch: 3 training finished. Time: 2114.11s, speed: 13.62st/s, total loss: 54379.8828125
- $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
- 3200
- gold_num = 5318 pred_num = 5480 right_num = 4935
- Dev: time: 177.55s, speed: 18.03st/s; acc: 0.9798, p: 0.9005, r: 0.9280, f: 0.9141
- Exceed previous best f score: 0.9133624291949114
- $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
- 31991
- gold_num = 53249 pred_num = 54694 right_num = 50658
- Test: speed: 21.79st/s; acc: 0.9871, p: 0.9262, r: 0.9513, f: 0.9386
- Epoch: 4/8
- Learning rate is setted as: 1.9602990012500002e-05
- Instance: 2000; Time: 143.89s; loss: 3273.8242
- Instance: 4000; Time: 148.13s; loss: 2741.9648
- Instance: 6000; Time: 148.69s; loss: 3335.4961
- Instance: 8000; Time: 145.63s; loss: 3187.5586
- Instance: 10000; Time: 147.30s; loss: 3590.1641
- Instance: 12000; Time: 146.67s; loss: 3292.2109
- Instance: 14000; Time: 148.80s; loss: 3548.0781
- Instance: 16000; Time: 147.66s; loss: 3465.6992
- Instance: 18000; Time: 140.75s; loss: 3697.1602
- Instance: 20000; Time: 145.11s; loss: 2628.9688
- Instance: 22000; Time: 145.56s; loss: 3746.4688
- Instance: 24000; Time: 147.37s; loss: 3193.6641
- Instance: 26000; Time: 143.37s; loss: 3510.3789
- Instance: 28000; Time: 144.68s; loss: 3864.9961
- Instance: 28791; Time: 59.00s; loss: 1120.7148
- Epoch: 4 training finished. Time: 2102.59s, speed: 13.69st/s, total loss: 48197.34765625
- $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
- 3200
- gold_num = 5318 pred_num = 5465 right_num = 4933
- Dev: time: 177.69s, speed: 18.02st/s; acc: 0.9806, p: 0.9027, r: 0.9276, f: 0.9150
- Exceed previous best f score: 0.9140581589183181
- $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
- 31991
- gold_num = 53249 pred_num = 54679 right_num = 50869
- Test: speed: 21.74st/s; acc: 0.9885, p: 0.9303, r: 0.9553, f: 0.9426
- Epoch: 5/8
- Learning rate is setted as: 1.95049750624375e-05
- Instance: 2000; Time: 146.18s; loss: 3387.5703
- Instance: 4000; Time: 150.12s; loss: 2773.3398
- Instance: 6000; Time: 144.64s; loss: 2800.8438
- Instance: 8000; Time: 149.31s; loss: 2960.8398
- Instance: 10000; Time: 148.43s; loss: 3144.8320
- Instance: 12000; Time: 149.78s; loss: 2630.9844
- Instance: 14000; Time: 143.57s; loss: 3655.4727
- Instance: 16000; Time: 149.73s; loss: 3274.9219
- Instance: 18000; Time: 147.64s; loss: 2955.9180
- Instance: 20000; Time: 141.59s; loss: 2908.0977
- Instance: 22000; Time: 142.19s; loss: 2818.3398
- Instance: 24000; Time: 147.47s; loss: 2930.6602
- Instance: 26000; Time: 146.50s; loss: 2939.9492
- Instance: 28000; Time: 147.41s; loss: 2861.3047
- Instance: 28791; Time: 56.27s; loss: 1335.8984
- Epoch: 5 training finished. Time: 2110.83s, speed: 13.64st/s, total loss: 43378.97265625
- $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
- 3200
- gold_num = 5318 pred_num = 5412 right_num = 4902
- Dev: time: 177.37s, speed: 18.05st/s; acc: 0.9806, p: 0.9058, r: 0.9218, f: 0.9137
- Epoch: 6/8
- Learning rate is setted as: 1.9407450187125313e-05
- Instance: 2000; Time: 145.61s; loss: 2578.9297
- Instance: 4000; Time: 142.76s; loss: 2655.6719
- Instance: 6000; Time: 143.75s; loss: 2767.4336
- Instance: 8000; Time: 146.44s; loss: 2896.5977
- Instance: 10000; Time: 147.70s; loss: 2439.3867
- Instance: 12000; Time: 147.42s; loss: 2497.6367
- Instance: 14000; Time: 140.05s; loss: 3012.9766
- Instance: 16000; Time: 142.54s; loss: 2565.6172
- Instance: 18000; Time: 145.99s; loss: 2567.1602
- Instance: 20000; Time: 144.20s; loss: 2938.5742
- Instance: 22000; Time: 145.38s; loss: 2523.3125
- Instance: 24000; Time: 146.09s; loss: 2949.3789
- Instance: 26000; Time: 142.28s; loss: 3419.7930
- Instance: 28000; Time: 145.24s; loss: 2493.3984
- Instance: 28791; Time: 58.52s; loss: 1123.0977
- Epoch: 6 training finished. Time: 2083.97s, speed: 13.82st/s, total loss: 39428.96484375
- $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
- 3200
- gold_num = 5318 pred_num = 5445 right_num = 4913
- Dev: time: 175.65s, speed: 18.23st/s; acc: 0.9803, p: 0.9023, r: 0.9238, f: 0.9129
- Epoch: 7/8
- Learning rate is setted as: 1.9310412936189687e-05
- Instance: 2000; Time: 147.35s; loss: 2321.7930
- Instance: 4000; Time: 147.31s; loss: 2829.9023
- Instance: 6000; Time: 148.73s; loss: 2320.7500
- Instance: 8000; Time: 144.58s; loss: 2949.8359
- Instance: 10000; Time: 140.07s; loss: 2489.4375
- Instance: 12000; Time: 144.28s; loss: 2436.8594
- Instance: 14000; Time: 148.11s; loss: 2768.7812
- Instance: 16000; Time: 146.42s; loss: 2466.5664
- Instance: 18000; Time: 141.97s; loss: 2323.8672
- Instance: 20000; Time: 139.65s; loss: 2306.9023
- Instance: 22000; Time: 147.51s; loss: 2851.7930
- Instance: 24000; Time: 145.07s; loss: 2736.7461
- Instance: 26000; Time: 146.50s; loss: 2170.4922
- Instance: 28000; Time: 146.72s; loss: 3013.7773
- Instance: 28791; Time: 58.71s; loss: 862.8320
- Epoch: 7 training finished. Time: 2092.98s, speed: 13.76st/s, total loss: 36850.3359375
- $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
- 3200
- gold_num = 5318 pred_num = 5439 right_num = 4915
- Dev: time: 176.47s, speed: 18.14st/s; acc: 0.9802, p: 0.9037, r: 0.9242, f: 0.9138
|