专利名称：TWO-PASS HASH EXTRACTION OF TEXT STRINGS
发明人：POUZIN, DOMINIC
申请号：US2008074586
申请日：20080828
公开号：WO2009045668A3
公开日：
20090528
专利内容由知识产权出版社提供
字段字符串去重复
摘要：Data compression and key word recognition may be provided. A first pass may walk a text string, generate terms, and calculate a hash value for each generated term. For each hash value, a hash bucket may be created where an associated occurrence count may be maintained. The hash buckets may be sorted by occurrence count and a few top buckets may be kept. Once those top buckets are known, a second pass may walk the text string, generate terms, and calculate a hash value for each term. If the hash values of terms match hash values of one of the kept buckets, then the term may be considered a frequent term. Consequently, the term may be added to a dictionary along with a corresponding frequency count. Then, the dictionary may be examined to remove terms that may not be frequent, but appeared due to hash collisions.
申请人：MICROSOFT CORPORATION
更多信息请下载全文后查看

版权声明：本站内容均来自互联网，仅供演示用，请勿用于商业和其他非法用途。如果侵犯了您的权益请与我们联系QQ:729038198，我们将在24小时内删除。