python – nltk：如何防止专有名词的堵塞

我正在尝试使用Stanford POS标记器和NER编写关键字提取程序.对于关键字提取,我只对专有名词感兴趣.这是基本方法

>通过删除除字母之外的任何内容来清理数据
>删除停用词
>干每个字
>确定每个单词的POS标签
>如果POS标签是名词,则将其提供给NER
>然后,NER将确定该单词是个人,组织还是位置

示例代码

docText="'Jack Frost works for Boeing Company. He manages 5 aircraft and their crew in London"

words = re.split("W+",docText) 

stops = set(stopwords.words("english"))

#remove stop words from the list
words = [w for w in words if w not in stops and len(w) > 2]

# Stemming
pstem = PorterStemmer()

words = [pstem.stem(w) for w in words]    

nounsWeWant = set(['NN','NNS','NNP','NNPS'])

finalWords = []

stn = StanfordNERTagger('english.all.3class.distsim.crf.ser.gz') 
stp = StanfordPOSTagger('english-bidirectional-distsim.tagger') 

for w in words:
    if stp.tag([w.lower()])[0][1] not in nounsWeWant:
        finalWords.append(w.lower())
    else:
        finalWords.append(w)

finalString = " ".join(finalWords)
print finalString

tagged = stn.tag(finalWords)
print tagged

这给了我

Jack Frost work Boe Compani manag aircraft crew London
[(u'Jack',u'PERSON'),(u'Frost',(u'work',u'O'),(u'Boe',(u'Compani',(u'manag',(u'aircraft',(u'crew',(u'London',u'LOCATION')]

很明显,我不希望波音被阻止.也不是公司.因为我的输入可能包含像Performing这样的术语,所以我需要阻止这些词语.我已经看到像NING这样的词会被NER选为专有名词,因此可以归类为组织.因此,首先我阻止所有单词并转换为小写.然后我检查这个单词的POS标签是否是名词.如果是这样,我保持原样.如果没有,我将单词转换为小写并将其添加到将传递给NER的最终单词列表中.

关于如何避免扼杀专有名词的任何想法？

wget http://nlp.stanford.edu/software/stanford-corenlp-full-2015-12-09.zip
unzip http://nlp.stanford.edu/software/stanford-corenlp-full-2015-12-09.zip
cd stanford-corenlp-full-2015-12-09
echo "Jack Frost works for Boeing Company. He manages 5 aircraft and their crew in London" > test.txt
java -cp "*" -Xmx2g edu.stanford.nlp.pipeline.StanfordCoreNLP -annotators tokenize,ssplit,pos,lemma,ner,parse,dcoref -file test.txt
cat test.txt.out 

[OUT]：

java -cp "*" -Xmx2g edu.stanford.nlp.pipeline.StanfordCoreNLP -annotators tokenize,dcoref -file test.txt -outputFormat json