é¢åç产ç¯å¢çå€è¯ç§èªç¶è¯èšå€çå·¥å ·å ïŒåºäºPyTorchåTensorFlow 2.xååŒæïŒç®æ æ¯æ®åèœå°æå沿çNLPææ¯ãHanLPå ·å€åèœå®åã粟床åç¡®ãæ§èœé«æãè¯ææ¶æ°ãæ¶ææž æ°ãå¯èªå®ä¹çç¹ç¹ã
åå©äžçäžæ倧çå€è¯ç§è¯æåºïŒHanLP2.1æ¯æå æ¬ç®ç¹äžè±æ¥ä¿æ³åŸ·åšå ç130ç§è¯èšäžç10ç§èåä»»å¡ä»¥åå€ç§åä»»å¡ãHanLPé¢è®ç»äºåå ç§ä»»å¡äžçæ°å䞪暡å并äžæ£åšæç»è¿ä»£è¯æåºäžæš¡åïŒ
- è¯å¹²æåãè¯æ³è¯æ³ç¹åŸæå请åèè±ææçšïŒè¯åéåå®åœ¢å¡«ç©ºè¯·åèçžåºææ¡£ã
- ç®ç¹èœ¬æ¢ãæŒé³ãæ°è¯åç°ãææ¬è类请åè1.xæçšã
éäœè£è¡£ïŒHanLPæäŸRESTfulånative䞀ç§APIïŒåå«é¢å蜻é级åæµ·é级䞀ç§åºæ¯ãæ 论äœç§APIäœç§è¯èšïŒHanLPæ¥å£åšè¯ä¹äžä¿æäžèŽïŒåšä»£ç äžåæåŒæºãåŠææšåšç 究äžäœ¿çšäºHanLPïŒè¯·åŒçšæ们çEMNLP论æã
ä»
æ°KBïŒéåææ·åŒåã移åšAPPçåºæ¯ãç®åæçšïŒæ éGPUé
ç¯å¢ïŒç§éå®è£
ãè¯ææŽå€ãæš¡åæŽå€§ã粟床æŽé«ïŒåŒºçæšèãæå¡åšGPUç®åæéïŒå¿åçšæ·é
é¢èŸå°ïŒå»ºè®®ç³è¯·å
莹å
¬çAPIç§é¥auth
ã
pip install hanlp_restful
å建客æ·ç«¯ïŒå¡«å ¥æå¡åšå°ååç§é¥ïŒ
from hanlp_restful import HanLPClient
HanLP = HanLPClient('https://www.hanlp.com/api', auth=None, language='zh') # authäžå¡«åå¿åïŒzhäžæïŒmulå€è¯ç§
å®è£
go get -u github.com/hankcs/gohanlp@main
ïŒå建客æ·ç«¯ïŒå¡«å
¥æå¡åšå°ååç§é¥ïŒ
HanLP := hanlp.HanLPClient(hanlp.WithAuth(""),hanlp.WithLanguage("zh")) // authäžå¡«åå¿åïŒzhäžæïŒmulå€è¯ç§
åšpom.xml
äžæ·»å äŸèµïŒ
<dependency>
<groupId>com.hankcs.hanlp.restful</groupId>
<artifactId>hanlp-restful</artifactId>
<version>0.0.12</version>
</dependency>
å建客æ·ç«¯ïŒå¡«å ¥æå¡åšå°ååç§é¥ïŒ
HanLPClient HanLP = new HanLPClient("https://www.hanlp.com/api", null, "zh"); // authäžå¡«åå¿åïŒzhäžæïŒmulå€è¯ç§
æ 论äœç§åŒåè¯èšïŒè°çšparse
æ¥å£ïŒäŒ å
¥äžç¯æç« ïŒåŸå°HanLP粟åçåæç»æã
HanLP.parse("2021幎HanLPv2.1䞺ç产ç¯å¢åžŠæ¥æ¬¡äžä»£æå
è¿çå€è¯ç§NLPææ¯ãé¿å©äž»æ¥å°å京ç«æ¹åºåè§èªç¶è¯ä¹ç§æå
¬åžã")
æŽå€åèœå æ¬è¯ä¹çžäŒŒåºŠãé£æ Œèœ¬æ¢ãæ代æ¶è§£çïŒè¯·åèææ¡£åæµè¯çšäŸã
äŸèµPyTorchãTensorFlowç深床åŠä¹ ææ¯ïŒéåäžäžNLPå·¥çšåžãç 究è 以åæ¬å°æµ·éæ°æ®åºæ¯ãèŠæ±Python 3.6è³3.10ïŒæ¯æWindowsïŒæšè*nixãå¯ä»¥åšCPUäžè¿è¡ïŒæšèGPU/TPUãå®è£ PyTorchçïŒ
pip install hanlp
- HanLPæ¯æ¬¡ååžéœéè¿äºLinuxãmacOSåWindowsäžPython3.6è³3.10çåå æµè¯ïŒäžååšå®è£ é®é¢ã
HanLPååžçæš¡åå䞺å€ä»»å¡ååä»»å¡äž€ç§ïŒå€ä»»å¡é床快çæŸåïŒåä»»å¡ç²ŸåºŠé«æŽçµæŽ»ã
HanLPçå·¥äœæµçšäžºå 蜜暡åç¶åå°å ¶åœäœåœæ°è°çšïŒäŸåŠäžåèåå€ä»»å¡æš¡åïŒ
import hanlp
HanLP = hanlp.load(hanlp.pretrained.mtl.CLOSE_TOK_POS_NER_SRL_DEP_SDP_CON_ELECTRA_SMALL_ZH) # äžçæ倧äžæè¯æåº
HanLP(['2021幎HanLPv2.1䞺ç产ç¯å¢åžŠæ¥æ¬¡äžä»£æå
è¿çå€è¯ç§NLPææ¯ã', 'é¿å©äž»æ¥å°å京ç«æ¹åºåè§èªç¶è¯ä¹ç§æå
¬åžã'])
Native APIçèŸå ¥åäœäžºå¥åïŒé䜿çšå€è¯ç§åå¥æš¡åæåºäºè§åçåå¥åœæ°å è¡åå¥ãRESTfulånative䞀ç§APIçè¯ä¹è®Ÿè®¡å®å šäžèŽïŒçšæ·å¯ä»¥æ çŒäºæ¢ãç®æŽçæ¥å£ä¹æ¯æçµæŽ»çåæ°ïŒåžžçšçæå·§æïŒ
- çµæŽ»ç
tasks
ä»»å¡è°åºŠïŒä»»å¡è¶å°ïŒé床è¶å¿«ïŒè¯Šè§æçšãåšå åæéçåºæ¯äžïŒçšæ·è¿å¯ä»¥å é€äžéèŠçä»»å¡èŸŸå°æš¡åçŠèº«çææã - é«æçtrieæ èªå®ä¹è¯å žïŒä»¥å区å¶ãå并ãæ ¡æ£3ç§è§åïŒè¯·åèdemoåææ¡£ãè§åç³»ç»çææå°æ çŒåºçšå°åç»ç»è®¡æš¡åïŒä»èå¿«ééåºæ°é¢åã
æ ¹æ®æ们çææ°ç 究ïŒå€ä»»å¡åŠä¹ çäŒå¿åšäºé床åæŸåïŒç¶è粟床åŸåŸäžåŠåä»»å¡æš¡åãæ以ïŒHanLPé¢è®ç»äºè®žå€åä»»å¡æš¡å并讟计äºäŒé çæµæ°Žçº¿æš¡åŒå°å ¶ç»è£ èµ·æ¥ã
import hanlp
HanLP = hanlp.pipeline() \
.append(hanlp.utils.rules.split_sentence, output_key='sentences') \
.append(hanlp.load('FINE_ELECTRA_SMALL_ZH'), output_key='tok') \
.append(hanlp.load('CTB9_POS_ELECTRA_SMALL'), output_key='pos') \
.append(hanlp.load('MSRA_NER_ELECTRA_SMALL_ZH'), output_key='ner', input_key='tok') \
.append(hanlp.load('CTB9_DEP_ELECTRA_SMALL', conll=0), output_key='dep', input_key='tok')\
.append(hanlp.load('CTB9_CON_ELECTRA_SMALL'), output_key='con', input_key='tok')
HanLP('2021幎HanLPv2.1䞺ç产ç¯å¢åžŠæ¥æ¬¡äžä»£æå
è¿çå€è¯ç§NLPææ¯ãé¿å©äž»æ¥å°å京ç«æ¹åºåè§èªç¶è¯ä¹ç§æå
¬åžã')
æŽå€åèœïŒè¯·åèdemoåææ¡£äºè§£æŽå€æš¡åäžçšæ³ã
æ 论äœç§APIäœç§åŒåè¯èšäœç§èªç¶è¯èšïŒHanLPçèŸåºç»äžäžºjson
æ ŒåŒå
Œå®¹dict
çDocument
:
{
"tok/fine": [
["2021幎", "HanLPv2.1", "䞺", "ç产", "ç¯å¢", "垊æ¥", "次", "äžä»£", "æ", "å
è¿", "ç", "å€", "è¯ç§", "NLP", "ææ¯", "ã"],
["é¿å©äž»", "æ¥å°", "å京", "ç«æ¹åº", "åè§", "èªç¶", "è¯ä¹", "ç§æ", "å
¬åž", "ã"]
],
"tok/coarse": [
["2021幎", "HanLPv2.1", "䞺", "ç产", "ç¯å¢", "垊æ¥", "次äžä»£", "æ", "å
è¿", "ç", "å€è¯ç§", "NLP", "ææ¯", "ã"],
["é¿å©äž»", "æ¥å°", "å京ç«æ¹åº", "åè§", "èªç¶è¯ä¹ç§æå
¬åž", "ã"]
],
"pos/ctb": [
["NT", "NR", "P", "NN", "NN", "VV", "JJ", "NN", "AD", "JJ", "DEG", "CD", "NN", "NR", "NN", "PU"],
["NN", "VV", "NR", "NR", "VV", "NN", "NN", "NN", "NN", "PU"]
],
"pos/pku": [
["t", "nx", "p", "vn", "n", "v", "b", "n", "d", "a", "u", "a", "n", "nx", "n", "w"],
["n", "v", "ns", "ns", "v", "n", "n", "n", "n", "w"]
],
"pos/863": [
["nt", "w", "p", "v", "n", "v", "a", "nt", "d", "a", "u", "a", "n", "ws", "n", "w"],
["n", "v", "ns", "n", "v", "n", "n", "n", "n", "w"]
],
"ner/pku": [
[],
[["å京ç«æ¹åº", "ns", 2, 4], ["èªç¶è¯ä¹ç§æå
¬åž", "nt", 5, 9]]
],
"ner/msra": [
[["2021幎", "DATE", 0, 1], ["HanLPv2.1", "ORGANIZATION", 1, 2]],
[["å京", "LOCATION", 2, 3], ["ç«æ¹åº", "LOCATION", 3, 4], ["èªç¶è¯ä¹ç§æå
¬åž", "ORGANIZATION", 5, 9]]
],
"ner/ontonotes": [
[["2021幎", "DATE", 0, 1], ["HanLPv2.1", "ORG", 1, 2]],
[["å京ç«æ¹åº", "FAC", 2, 4], ["èªç¶è¯ä¹ç§æå
¬åž", "ORG", 5, 9]]
],
"srl": [
[[["2021幎", "ARGM-TMP", 0, 1], ["HanLPv2.1", "ARG0", 1, 2], ["䞺ç产ç¯å¢", "ARG2", 2, 5], ["垊æ¥", "PRED", 5, 6], ["次äžä»£æå
è¿çå€è¯ç§NLPææ¯", "ARG1", 6, 15]], [["æ", "ARGM-ADV", 8, 9], ["å
è¿", "PRED", 9, 10], ["ææ¯", "ARG0", 14, 15]]],
[[["é¿å©äž»", "ARG0", 0, 1], ["æ¥å°", "PRED", 1, 2], ["å京ç«æ¹åº", "ARG1", 2, 4]], [["é¿å©äž»", "ARG0", 0, 1], ["åè§", "PRED", 4, 5], ["èªç¶è¯ä¹ç§æå
¬åž", "ARG1", 5, 9]]]
],
"dep": [
[[6, "tmod"], [6, "nsubj"], [6, "prep"], [5, "nn"], [3, "pobj"], [0, "root"], [8, "amod"], [15, "nn"], [10, "advmod"], [15, "rcmod"], [10, "assm"], [13, "nummod"], [15, "nn"], [15, "nn"], [6, "dobj"], [6, "punct"]],
[[2, "nsubj"], [0, "root"], [4, "nn"], [2, "dobj"], [2, "conj"], [9, "nn"], [9, "nn"], [9, "nn"], [5, "dobj"], [2, "punct"]]
],
"sdp": [
[[[6, "Time"]], [[6, "Exp"]], [[5, "mPrep"]], [[5, "Desc"]], [[6, "Datv"]], [[13, "dDesc"]], [[0, "Root"], [8, "Desc"], [13, "Desc"]], [[15, "Time"]], [[10, "mDegr"]], [[15, "Desc"]], [[10, "mAux"]], [[8, "Quan"], [13, "Quan"]], [[15, "Desc"]], [[15, "Nmod"]], [[6, "Pat"]], [[6, "mPunc"]]],
[[[2, "Agt"], [5, "Agt"]], [[0, "Root"]], [[4, "Loc"]], [[2, "Lfin"]], [[2, "ePurp"]], [[8, "Nmod"]], [[9, "Nmod"]], [[9, "Nmod"]], [[5, "Datv"]], [[5, "mPunc"]]]
],
"con": [
["TOP", [["IP", [["NP", [["NT", ["2021幎"]]]], ["NP", [["NR", ["HanLPv2.1"]]]], ["VP", [["PP", [["P", ["䞺"]], ["NP", [["NN", ["ç产"]], ["NN", ["ç¯å¢"]]]]]], ["VP", [["VV", ["垊æ¥"]], ["NP", [["ADJP", [["NP", [["ADJP", [["JJ", ["次"]]]], ["NP", [["NN", ["äžä»£"]]]]]], ["ADVP", [["AD", ["æ"]]]], ["VP", [["JJ", ["å
è¿"]]]]]], ["DEG", ["ç"]], ["NP", [["QP", [["CD", ["å€"]]]], ["NP", [["NN", ["è¯ç§"]]]]]], ["NP", [["NR", ["NLP"]], ["NN", ["ææ¯"]]]]]]]]]], ["PU", ["ã"]]]]]],
["TOP", [["IP", [["NP", [["NN", ["é¿å©äž»"]]]], ["VP", [["VP", [["VV", ["æ¥å°"]], ["NP", [["NR", ["å京"]], ["NR", ["ç«æ¹åº"]]]]]], ["VP", [["VV", ["åè§"]], ["NP", [["NN", ["èªç¶"]], ["NN", ["è¯ä¹"]], ["NN", ["ç§æ"]], ["NN", ["å
¬åž"]]]]]]]], ["PU", ["ã"]]]]]]
]
}
ç¹å«å°ïŒPython RESTfulånative APIæ¯æåºäºç宜åäœçå¯è§åïŒèœå€çŽæ¥å°è¯èšåŠç»æåšæ§å¶å°å å¯è§ååºæ¥ïŒ
HanLP(['2021幎HanLPv2.1䞺ç产ç¯å¢åžŠæ¥æ¬¡äžä»£æå
è¿çå€è¯ç§NLPææ¯ã', 'é¿å©äž»æ¥å°å京ç«æ¹åºåè§èªç¶è¯ä¹ç§æå
¬åžã']).pretty_print()
Dep Tree Token Relati PoS Tok NER Type Tok SRL PA1 Tok SRL PA2 Tok PoS 3 4 5 6 7 8 9
ââââââââââââ âââââââââ ââââââ âââ âââââââââ ââââââââââââââââ âââââââââ ââââââââââââ âââââââââ ââââââââââââ âââââââââ âââââââââââââââââââââââââââââââââââââââââââââââââââââââââ
ââââââââââ⺠2021幎 tmod NT 2021幎 ââââºDATE 2021幎 ââââºARGM-TMP 2021幎 2021幎 NT ââââââââââââââââââââââââââââââââââââââââââââºNP ââââ
ââââââââââ⺠HanLPv2.1 nsubj NR HanLPv2.1 ââââºORGANIZATION HanLPv2.1 ââââºARG0 HanLPv2.1 HanLPv2.1 NR ââââââââââââââââââââââââââââââââââââââââââââºNPâââââ€
âââââºââââââ 䞺 prep P 䞺 䞺 âââ 䞺 䞺 P ââââââââââââ â
âââ â ââ⺠ç产 nn NN ç产 ç产 ââºARG2 ç产 ç产 NN âââ ââââââââââââââââââââââââââºPP ââââ â
âââ âââºâââ ç¯å¢ pobj NN ç¯å¢ ç¯å¢ âââ ç¯å¢ ç¯å¢ NN âââŽâºNP ââââ â â
ââŒâŽâŽââââââââ åžŠæ¥ root VV åžŠæ¥ åžŠæ¥ ââââºPRED åžŠæ¥ åžŠæ¥ VV âââââââââââââââââââââââââââââââââââ â â
ââ ââ⺠次 amod JJ 次 次 âââ 次 次 JJ ââââºADJPâââ â ââºVPâââââ€
ââ âââââºâââ äžä»£ nn NN äžä»£ äžä»£ â äžä»£ äžä»£ NN ââââºNP ââââŽâºNP ââââ â â â
ââ â ââ⺠æ advmod AD æ æ â æ ââââºARGM-ADV æ AD ââââââââââââºADVPâââŒâºADJPâââ ââºVP ââââ ââºIP
ââ âââââºâââ å
è¿ rcmod JJ å
è¿ å
è¿ â å
è¿ ââââºPRED å
è¿ JJ ââââââââââââºVP ââââ â â â
ââ ââ ââ⺠ç assm DEG ç ç ââºARG1 ç ç DEGââââââââââââââââââââââââââ†â â
ââ ââ âââº å€ nummod CD å€ å€ â å€ å€ CD ââââºQP ââââ ââºNP ââââ â
ââ âââââºâââ è¯ç§ nn NN è¯ç§ è¯ç§ â è¯ç§ è¯ç§ NN ââââºNP ââââŽâââââââââºNPââââ†â
ââ âââ ââ⺠NLP nn NR NLP NLP â NLP NLP NR âââ â â
ââââºââŽâŽâââŽââ ææ¯ dobj NN ææ¯ ææ¯ âââ ææ¯ ââââºARG0 ææ¯ NN âââŽâââââââââââââââââºNP ââââ â
âââââââââââ⺠ã punct PU ã ã ã ã PU âââââââââââââââââââââââââââââââââââââââââââââââââââ
Dep Tree Tok Relat Po Tok NER Type Tok SRL PA1 Tok SRL PA2 Tok Po 3 4 5 6
ââââââââââââ âââ âââââ ââ âââ ââââââââââââââââ âââ ââââââââ âââ ââââââââ âââ ââââââââââââââââââââââââââââââââ
ââ⺠é¿å©äž» nsubj NN é¿å©äž» é¿å©äž» ââââºARG0 é¿å©äž» ââââºARG0 é¿å©äž» NNââââââââââââââââââââºNP ââââ
ââ¬âââââ¬âââŽââ æ¥å° root VV æ¥å° æ¥å° ââââºPRED æ¥å° æ¥å° VVâââââââââââ â
ââ â ââ⺠å京 nn NR å京 ââââºLOCATION å京 âââ å京 å京 NRâââ ââºVP ââââ â
ââ âââºâââ ç«æ¹åº dobj NR ç«æ¹åº ââââºLOCATION ç«æ¹åº âââŽâºARG1 ç«æ¹åº ç«æ¹åº NRâââŽâºNP ââââ â â
ââââºââââââââ åè§ conj VV åè§ åè§ åè§ ââââºPRED åè§ VVâââââââââââ ââºVPâââââ€
â â ââââ⺠èªç¶ nn NN èªç¶ âââ èªç¶ èªç¶ âââ èªç¶ NNâââ â â ââºIP
â â ââââ⺠è¯ä¹ nn NN è¯ä¹ â è¯ä¹ è¯ä¹ â è¯ä¹ NN â ââºVP ââââ â
â â ââââ⺠ç§æ nn NN ç§æ ââºORGANIZATION ç§æ ç§æ ââºARG1 ç§æ NN ââºNP ââââ â
â âââºââŽâŽââ å
¬åž dobj NN å
¬åž âââ å
¬åž å
¬åž âââ å
¬åž NNâââ â
âââââââââââ⺠ã punct PU ã ã ã ã PUâââââââââââââââââââââââââââ
å ³äºæ 泚éå«ä¹ïŒè¯·åèãè¯èšåŠæ 泚è§èãåãæ ŒåŒè§èããæ们èŽä¹°ãæ 泚æéçšäºäžçäžé级æ倧ãç§ç±»æå€çè¯æåºçšäºèåå€è¯ç§å€ä»»å¡åŠä¹ ïŒæ以HanLPçæ 泚éä¹æ¯èŠçé¢æ广çã
å深床åŠä¹ æš¡åäžç¹éœäžéŸïŒéŸçæ¯å€ç°èŸé«çåç¡®çãäžå代ç å±ç€ºäºåŠäœåšsighan2005 PKUè¯æåºäžè±6åéè®ç»äžäžªè¶ è¶åŠæ¯çstate-of-the-artçäžæåè¯æš¡åã
tokenizer = TransformerTaggingTokenizer()
save_dir = 'data/model/cws/sighan2005_pku_bert_base_96.73'
tokenizer.fit(
SIGHAN2005_PKU_TRAIN_ALL,
SIGHAN2005_PKU_TEST, # Conventionally, no devset is used. See Tian et al. (2020).
save_dir,
'bert-base-chinese',
max_seq_len=300,
char_level=True,
hard_constraint=True,
sampler_builder=SortingSamplerBuilder(batch_size=32),
epochs=3,
adam_epsilon=1e-6,
warmup_steps=0.1,
weight_decay=0.01,
word_dropout=0.1,
seed=1660853059,
)
tokenizer.evaluate(SIGHAN2005_PKU_TEST, save_dir)
å
¶äžïŒç±äºæå®äºéæºæ°ç§åïŒç»æäžå®æ¯96.73
ãäžåäºé£äºèåå®£äŒ çåŠæ¯è®ºææåäžé¡¹ç®ïŒHanLPä¿è¯ææç»æå¯å€ç°ãåŠæäœ æä»»äœèŽšçïŒæ们å°åœäœæé«äŒå
级çèŽåœæ§bug第äžæ¶éŽææ¥é®é¢ã
请åèdemoäºè§£æŽå€è®ç»èæ¬ã
lang | corpora | model | tok | pos | ner | dep | con | srl | sdp | lem | fea | amr | |||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
fine | coarse | ctb | pku | 863 | ud | pku | msra | ontonotes | SemEval16 | DM | PAS | PSD | |||||||||
mul | UD2.7 OntoNotes5 |
small | 98.62 | - | - | - | - | 93.23 | - | - | 74.42 | 79.10 | 76.85 | 70.63 | - | 91.19 | 93.67 | 85.34 | 87.71 | 84.51 | - |
base | 98.97 | - | - | - | - | 90.32 | - | - | 80.32 | 78.74 | 71.23 | 73.63 | - | 92.60 | 96.04 | 81.19 | 85.08 | 82.13 | - | ||
zh | open | small | 97.25 | - | 96.66 | - | - | - | - | - | 95.00 | 84.57 | 87.62 | 73.40 | 84.57 | - | - | - | - | - | - |
base | 97.50 | - | 97.07 | - | - | - | - | - | 96.04 | 87.11 | 89.84 | 77.78 | 87.11 | - | - | - | - | - | - | ||
close | small | 96.70 | 95.93 | 96.87 | 97.56 | 95.05 | - | 96.22 | 95.74 | 76.79 | 84.44 | 88.13 | 75.81 | 74.28 | - | - | - | - | - | - | |
base | 97.52 | 96.44 | 96.99 | 97.59 | 95.29 | - | 96.48 | 95.72 | 77.77 | 85.29 | 88.57 | 76.52 | 73.76 | - | - | - | - | - | - | ||
ernie | 96.95 | 97.29 | 96.76 | 97.64 | 95.22 | - | 97.31 | 96.47 | 77.95 | 85.67 | 89.17 | 78.51 | 74.10 | - | - | - | - | - | - |
- æ ¹æ®æ们çææ°ç 究ïŒåä»»å¡åŠä¹ çæ§èœåŸåŸäŒäºå€ä»»å¡åŠä¹ ãåšä¹ç²ŸåºŠçäºé床çè¯ïŒå»ºè®®äœ¿çšåä»»å¡æš¡åã
HanLPéçšçæ°æ®é¢å€çäžæåæ¯äŸäžæµè¡æ¹æ³æªå¿ çžåïŒæ¯åŠHanLPéçšäºå®æŽççMSRAåœåå®äœè¯å«è¯æïŒèé倧äŒäœ¿çšçéå²çïŒHanLP䜿çšäºè¯æ³èŠçæŽå¹¿çStanford Dependenciesæ åïŒèéåŠæ¯ç沿çšçZhang and Clark (2008)æ åïŒHanLPæåºäºåååå²CTBçæ¹æ³ïŒèäžéçšåŠæ¯çäžååäžéæŒäº51䞪é»éæ件çæ¹æ³ãHanLPåŒæºäºäžæŽå¥è¯æé¢å€çèæ¬äžçžåºè¯æåºïŒååŸæšåšäžæNLPçéæåã
æ»ä¹ïŒHanLPåªåæ们讀䞺æ£ç¡®ãå è¿çäºæ ïŒèäžäžå®æ¯æµè¡ãæåšçäºæ ã
åŠæäœ åšç 究äžäœ¿çšäºHanLPïŒè¯·æåŠäžæ ŒåŒåŒçšïŒ
@inproceedings{he-choi-2021-stem,
title = "The Stem Cell Hypothesis: Dilemma behind Multi-Task Learning with Transformer Encoders",
author = "He, Han and Choi, Jinho D.",
booktitle = "Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing",
month = nov,
year = "2021",
address = "Online and Punta Cana, Dominican Republic",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2021.emnlp-main.451",
pages = "5555--5577",
abstract = "Multi-task learning with transformer encoders (MTL) has emerged as a powerful technique to improve performance on closely-related tasks for both accuracy and efficiency while a question still remains whether or not it would perform as well on tasks that are distinct in nature. We first present MTL results on five NLP tasks, POS, NER, DEP, CON, and SRL, and depict its deficiency over single-task learning. We then conduct an extensive pruning analysis to show that a certain set of attention heads get claimed by most tasks during MTL, who interfere with one another to fine-tune those heads for their own objectives. Based on this finding, we propose the Stem Cell Hypothesis to reveal the existence of attention heads naturally talented for many tasks that cannot be jointly trained to create adequate embeddings for all of those tasks. Finally, we design novel parameter-free probes to justify our hypothesis and demonstrate how attention heads are transformed across the five tasks during MTL through label analysis.",
}
HanLPæºä»£ç çææå议䞺 Apache License 2.0ïŒå¯å 莹çšååäžçšéã请åšäº§å诎æäžéå HanLPçéŸæ¥åææåè®®ãHanLPåçææ³ä¿æ€ïŒäŸµæå¿ ç©¶ã
HanLPä»v1.7çèµ·ç¬ç«è¿äœïŒç±èªç¶è¯ä¹ïŒéå²ïŒç§ææéå ¬åžäœäžºé¡¹ç®äž»äœïŒäž»å¯Œåç»çæ¬çåŒåïŒå¹¶æ¥æåç»çæ¬ççæã
HanLP v1.3~v1.65çç±å€§å¿«æ玢䞻富åŒåïŒç»§ç»å®å šåŒæºïŒå€§å¿«æ玢æ¥æçžå ³çæã
HanLP æ©æåŸå°äºäžæµ·æåå ¬åžç倧åæ¯æïŒå¹¶æ¥æ1.28åååºçæ¬ççæïŒçžå ³çæ¬ä¹æŸåšäžæµ·æåå ¬åžçœç«ååžã
æºåšåŠä¹ æš¡åçææåšæ³åŸäžæ²¡æå®è®ºïŒäœæ¬çå°éåŒæºè¯æåºåå§ææç粟ç¥ïŒåŠäžç¹å«è¯ŽæïŒHanLPçå€è¯ç§æš¡åææ沿çšCC BY-NC-SA 4.0ïŒäžææš¡åææäžºä» äŸç 究äžæåŠäœ¿çšã