- Teks
- Sejarah
FIELD OF THE INVENTIONThe present i
FIELD OF THE INVENTION
The present invention relates to an information retrieval system for indexing, searching, and classifying documents in a large scale corpus, such as the Internet.
BACKGROUND OF THE INVENTION
Information retrieval systems, generally called search engines, are now an essential tool for finding information in large scale, diverse, and growing corpuses such as the Internet. Generally, search engines create an index that relates documents (or “pages”) to the individual words present in each document. A document is retrieved in response to a query containing a number of query terms, typically based on having some number of query terms present in the document. The retrieved documents are then ranked according to other statistical measures, such as frequency of occurrence of the query terms, host domain, link analysis, and the like. The retrieved documents are then presented to the user, typically in their ranked order, and without any further grouping or imposed hierarchy. In some cases, a selected portion of a text of a document is presented to provide the user with a glimpse of the document's content.
Direct “Boolean” matching of query terms has well known limitations, and in particular does not identify documents that do not have the query terms, but have related words. For example, in a typical Boolean system, a search on “Australian Shepherds” would not return documents about other herding dogs such as Border Collies that do not have the exact query terms. Rather, such a system is likely to also retrieve and highly rank documents that are about Australia (and have nothing to do with dogs), and documents about “shepherds” generally.
The problem here is that conventional systems index documents based on individual terms, than on concepts. Concepts are often expressed in phrases, such as “Australian Shepherd,” “President of the United States,” or “Sundance Film Festival”. At best, some prior systems will index documents with respect to a predetermined and very limited set of ‘known’ phrases, which are typically selected by a human operator. Indexing of phrases is typically avoided because of the perceived computational and memory requirements to identify all possible phrases of say three, four, or five or more words. For example, on the assumption that any five words could constitute a phrase, and a large corpus would have at least 200,000 unique terms, there would approximately 3.2×1026 possible phrases, clearly more than any existing system could store in memory or otherwise programmatically manipulate. A further problem is that phrases continually enter and leave the lexicon in terms of their usage, much more frequently than new individual words are invented. New phrases are always being generated, from sources such technology, arts, world events, and law. Other phrases will decline in usage over time.
Some existing information retrieval systems attempt to provide retrieval of concepts by using co-occurrence patterns of individual words. In these systems a search on one word, such as “President” will also retrieve documents that have other words that frequently appear with “President”, such as “White” and “House.” While this approach may produce search results having documents that are conceptually related at the level of individual words, it does not typically capture topical relationships that inhere between co-occurring phrases.
Accordingly, there is a need for an information retrieval system and methodology that can comprehensively identify phrases in a large scale corpus, index documents according to phrases, search and rank documents in accordance with their phrases. Additionally, there is a need in such a system to allow users to provide additional phrase information to the system and to capture and integrate the resulting semantic knowledge.
SUMMARY OF THE INVENTION
An information retrieval system and methodology uses phrases to index, search, rank, and describe documents in the document collection. The system is adapted to identify phrases that have sufficiently frequent and/or distinguished usage in the document collection to indicate that they are “valid” or “good” phrases. In this manner multiple word phrases, for example phrases of four, five, or more terms, can be identified. This avoids the problem of having to identify and index every possible phrase resulting from all of the possible sequences of a given number of words.
The system is further adapted to identify phrases that are related to each other, based on a phrase's ability to predict the presence of other phrases in a document. More specifically, a prediction measure is used that relates the actual co-occurrence rate of two phrases to an expected co-occurrence rate of the two phrases. Information gain, as the ratio of actual co-occurrence rate to expected co-occurrence rate, is one such prediction measure. Two phrases are related where the prediction measure exceeds a predetermined threshold. In that case, the second phrase has significant information gain with respect to the first phrase. Semantically, related phrases will be those that are commonly used to discuss or describe a given topic or concept, such as “President of the United States” and “White House.” For a given phrase, the related phrases can be ordered according to their relevance or significance based on their respective prediction measures.
An information retrieval system indexes documents in the document collection by the valid or good phrases. For each phrase, a posting list identifies the documents that contain the phrase. In addition, for a given phrase, a second list, vector, or other structure is used to store data indicating which of the related phrases of the given phrase are also present in each document containing the given phrase. In this manner, the system can readily identify not only which documents contain which phrases in response to a search query, but which documents also contain phrases that are related to query phrases, and thus more likely to be specifically about the topics or concepts expressed in the query phrases.
The use of phrases and related phrases further provides for the creation and use of clusters of related phrases, which represent semantically meaningful groupings of phrases. Clusters are identified from related phrases that have very high prediction measure between all of the phrases in the cluster. Clusters can be used to organize the results of a search, including selecting which documents to include in the search results and their order, as well as eliminating documents from the search results.
Websites typically have anywhere from a few pages to potentially hundreds or thousands of pages. Thus, phrase information generated by the information retrieval system can be used to determine a list of top phrases for each website, such as the most representative phrases for the website. This can be done by examining the related phrase information for the phrases that appear in documents on the website. Further, phrase information may be later supplemented and refined by capturing changes made to the top phrase list by administrators or other authorized users and integrating the resulting semantic knowledge into the phrase information already contained within the system. An administrator can associate additional related phrases with any of the top phrases for the website. The related phrase information for the top phrases for which additional related phrases have been received is then updated to include information pertaining to the additional related phrases, and the additional related phrases are also updated to include information from the top phrases. This operates to treat the additional phrases as if they were present in the website. In addition, the additional related phrases can be updated to use the related phrase information for the top phrases.
The present invention has further embodiments in system and software architectures, computer program products and computer implemented methods, and computer generated user interfaces and presentations.
The foregoing are just some of the features of an information retrieval system and methodology based on phrases. Those of skill in the art of information retrieval will appreciate the flexibility of generality of the phrase information allows for a large variety of uses and applications in indexing, document annotation, searching, ranking, and other areas of document analysis and processing.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 is block diagram of the software architecture of one embodiment of the present invention.
FIG. 2 illustrates a method of identifying phrases in documents.
FIG. 3 illustrates a document with a phrase window and a secondary window.
FIG. 4 illustrates a method of identifying related phrases.
FIG. 5 illustrates a method of indexing documents for related phrases.
FIG. 6 illustrates a method of retrieving documents based on phrases.
FIGS. 7 a and 7 b illustrate relationships between referencing and referenced documents.
FIG. 8 illustrates a method of obtaining and integrating phrase information input from users.
FIG. 9 illustrates a sample user interface for displaying top phrases and allowing users to input changes.
The figures depict a preferred embodiment of the present invention for purposes of illustration only. One skilled in the art will readily recognize from the following discussion that alternative embodiments of the structures and methods illustrated herein may be employed without departing from the principles of the invention described herein.
The present invention relates to an information retrieval system for indexing, searching, and classifying documents in a large scale corpus, such as the Internet.
BACKGROUND OF THE INVENTION
Information retrieval systems, generally called search engines, are now an essential tool for finding information in large scale, diverse, and growing corpuses such as the Internet. Generally, search engines create an index that relates documents (or “pages”) to the individual words present in each document. A document is retrieved in response to a query containing a number of query terms, typically based on having some number of query terms present in the document. The retrieved documents are then ranked according to other statistical measures, such as frequency of occurrence of the query terms, host domain, link analysis, and the like. The retrieved documents are then presented to the user, typically in their ranked order, and without any further grouping or imposed hierarchy. In some cases, a selected portion of a text of a document is presented to provide the user with a glimpse of the document's content.
Direct “Boolean” matching of query terms has well known limitations, and in particular does not identify documents that do not have the query terms, but have related words. For example, in a typical Boolean system, a search on “Australian Shepherds” would not return documents about other herding dogs such as Border Collies that do not have the exact query terms. Rather, such a system is likely to also retrieve and highly rank documents that are about Australia (and have nothing to do with dogs), and documents about “shepherds” generally.
The problem here is that conventional systems index documents based on individual terms, than on concepts. Concepts are often expressed in phrases, such as “Australian Shepherd,” “President of the United States,” or “Sundance Film Festival”. At best, some prior systems will index documents with respect to a predetermined and very limited set of ‘known’ phrases, which are typically selected by a human operator. Indexing of phrases is typically avoided because of the perceived computational and memory requirements to identify all possible phrases of say three, four, or five or more words. For example, on the assumption that any five words could constitute a phrase, and a large corpus would have at least 200,000 unique terms, there would approximately 3.2×1026 possible phrases, clearly more than any existing system could store in memory or otherwise programmatically manipulate. A further problem is that phrases continually enter and leave the lexicon in terms of their usage, much more frequently than new individual words are invented. New phrases are always being generated, from sources such technology, arts, world events, and law. Other phrases will decline in usage over time.
Some existing information retrieval systems attempt to provide retrieval of concepts by using co-occurrence patterns of individual words. In these systems a search on one word, such as “President” will also retrieve documents that have other words that frequently appear with “President”, such as “White” and “House.” While this approach may produce search results having documents that are conceptually related at the level of individual words, it does not typically capture topical relationships that inhere between co-occurring phrases.
Accordingly, there is a need for an information retrieval system and methodology that can comprehensively identify phrases in a large scale corpus, index documents according to phrases, search and rank documents in accordance with their phrases. Additionally, there is a need in such a system to allow users to provide additional phrase information to the system and to capture and integrate the resulting semantic knowledge.
SUMMARY OF THE INVENTION
An information retrieval system and methodology uses phrases to index, search, rank, and describe documents in the document collection. The system is adapted to identify phrases that have sufficiently frequent and/or distinguished usage in the document collection to indicate that they are “valid” or “good” phrases. In this manner multiple word phrases, for example phrases of four, five, or more terms, can be identified. This avoids the problem of having to identify and index every possible phrase resulting from all of the possible sequences of a given number of words.
The system is further adapted to identify phrases that are related to each other, based on a phrase's ability to predict the presence of other phrases in a document. More specifically, a prediction measure is used that relates the actual co-occurrence rate of two phrases to an expected co-occurrence rate of the two phrases. Information gain, as the ratio of actual co-occurrence rate to expected co-occurrence rate, is one such prediction measure. Two phrases are related where the prediction measure exceeds a predetermined threshold. In that case, the second phrase has significant information gain with respect to the first phrase. Semantically, related phrases will be those that are commonly used to discuss or describe a given topic or concept, such as “President of the United States” and “White House.” For a given phrase, the related phrases can be ordered according to their relevance or significance based on their respective prediction measures.
An information retrieval system indexes documents in the document collection by the valid or good phrases. For each phrase, a posting list identifies the documents that contain the phrase. In addition, for a given phrase, a second list, vector, or other structure is used to store data indicating which of the related phrases of the given phrase are also present in each document containing the given phrase. In this manner, the system can readily identify not only which documents contain which phrases in response to a search query, but which documents also contain phrases that are related to query phrases, and thus more likely to be specifically about the topics or concepts expressed in the query phrases.
The use of phrases and related phrases further provides for the creation and use of clusters of related phrases, which represent semantically meaningful groupings of phrases. Clusters are identified from related phrases that have very high prediction measure between all of the phrases in the cluster. Clusters can be used to organize the results of a search, including selecting which documents to include in the search results and their order, as well as eliminating documents from the search results.
Websites typically have anywhere from a few pages to potentially hundreds or thousands of pages. Thus, phrase information generated by the information retrieval system can be used to determine a list of top phrases for each website, such as the most representative phrases for the website. This can be done by examining the related phrase information for the phrases that appear in documents on the website. Further, phrase information may be later supplemented and refined by capturing changes made to the top phrase list by administrators or other authorized users and integrating the resulting semantic knowledge into the phrase information already contained within the system. An administrator can associate additional related phrases with any of the top phrases for the website. The related phrase information for the top phrases for which additional related phrases have been received is then updated to include information pertaining to the additional related phrases, and the additional related phrases are also updated to include information from the top phrases. This operates to treat the additional phrases as if they were present in the website. In addition, the additional related phrases can be updated to use the related phrase information for the top phrases.
The present invention has further embodiments in system and software architectures, computer program products and computer implemented methods, and computer generated user interfaces and presentations.
The foregoing are just some of the features of an information retrieval system and methodology based on phrases. Those of skill in the art of information retrieval will appreciate the flexibility of generality of the phrase information allows for a large variety of uses and applications in indexing, document annotation, searching, ranking, and other areas of document analysis and processing.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 is block diagram of the software architecture of one embodiment of the present invention.
FIG. 2 illustrates a method of identifying phrases in documents.
FIG. 3 illustrates a document with a phrase window and a secondary window.
FIG. 4 illustrates a method of identifying related phrases.
FIG. 5 illustrates a method of indexing documents for related phrases.
FIG. 6 illustrates a method of retrieving documents based on phrases.
FIGS. 7 a and 7 b illustrate relationships between referencing and referenced documents.
FIG. 8 illustrates a method of obtaining and integrating phrase information input from users.
FIG. 9 illustrates a sample user interface for displaying top phrases and allowing users to input changes.
The figures depict a preferred embodiment of the present invention for purposes of illustration only. One skilled in the art will readily recognize from the following discussion that alternative embodiments of the structures and methods illustrated herein may be employed without departing from the principles of the invention described herein.
0/5000
FIELD OF THE INVENTIONThe present invention relates to an information retrieval system for indexing, searching, and classifying documents in a large scale corpus, such as the Internet.BACKGROUND OF THE INVENTIONInformation retrieval systems, generally called search engines, are now an essential tool for finding information in large scale, diverse, and growing corpuses such as the Internet. Generally, search engines create an index that relates documents (or “pages”) to the individual words present in each document. A document is retrieved in response to a query containing a number of query terms, typically based on having some number of query terms present in the document. The retrieved documents are then ranked according to other statistical measures, such as frequency of occurrence of the query terms, host domain, link analysis, and the like. The retrieved documents are then presented to the user, typically in their ranked order, and without any further grouping or imposed hierarchy. In some cases, a selected portion of a text of a document is presented to provide the user with a glimpse of the document's content.Direct “Boolean” matching of query terms has well known limitations, and in particular does not identify documents that do not have the query terms, but have related words. For example, in a typical Boolean system, a search on “Australian Shepherds” would not return documents about other herding dogs such as Border Collies that do not have the exact query terms. Rather, such a system is likely to also retrieve and highly rank documents that are about Australia (and have nothing to do with dogs), and documents about “shepherds” generally.The problem here is that conventional systems index documents based on individual terms, than on concepts. Concepts are often expressed in phrases, such as “Australian Shepherd,” “President of the United States,” or “Sundance Film Festival”. At best, some prior systems will index documents with respect to a predetermined and very limited set of ‘known’ phrases, which are typically selected by a human operator. Indexing of phrases is typically avoided because of the perceived computational and memory requirements to identify all possible phrases of say three, four, or five or more words. For example, on the assumption that any five words could constitute a phrase, and a large corpus would have at least 200,000 unique terms, there would approximately 3.2×1026 possible phrases, clearly more than any existing system could store in memory or otherwise programmatically manipulate. A further problem is that phrases continually enter and leave the lexicon in terms of their usage, much more frequently than new individual words are invented. New phrases are always being generated, from sources such technology, arts, world events, and law. Other phrases will decline in usage over time.
Some existing information retrieval systems attempt to provide retrieval of concepts by using co-occurrence patterns of individual words. In these systems a search on one word, such as “President” will also retrieve documents that have other words that frequently appear with “President”, such as “White” and “House.” While this approach may produce search results having documents that are conceptually related at the level of individual words, it does not typically capture topical relationships that inhere between co-occurring phrases.
Accordingly, there is a need for an information retrieval system and methodology that can comprehensively identify phrases in a large scale corpus, index documents according to phrases, search and rank documents in accordance with their phrases. Additionally, there is a need in such a system to allow users to provide additional phrase information to the system and to capture and integrate the resulting semantic knowledge.
SUMMARY OF THE INVENTION
An information retrieval system and methodology uses phrases to index, search, rank, and describe documents in the document collection. The system is adapted to identify phrases that have sufficiently frequent and/or distinguished usage in the document collection to indicate that they are “valid” or “good” phrases. In this manner multiple word phrases, for example phrases of four, five, or more terms, can be identified. This avoids the problem of having to identify and index every possible phrase resulting from all of the possible sequences of a given number of words.
The system is further adapted to identify phrases that are related to each other, based on a phrase's ability to predict the presence of other phrases in a document. More specifically, a prediction measure is used that relates the actual co-occurrence rate of two phrases to an expected co-occurrence rate of the two phrases. Information gain, as the ratio of actual co-occurrence rate to expected co-occurrence rate, is one such prediction measure. Two phrases are related where the prediction measure exceeds a predetermined threshold. In that case, the second phrase has significant information gain with respect to the first phrase. Semantically, related phrases will be those that are commonly used to discuss or describe a given topic or concept, such as “President of the United States” and “White House.” For a given phrase, the related phrases can be ordered according to their relevance or significance based on their respective prediction measures.
An information retrieval system indexes documents in the document collection by the valid or good phrases. For each phrase, a posting list identifies the documents that contain the phrase. In addition, for a given phrase, a second list, vector, or other structure is used to store data indicating which of the related phrases of the given phrase are also present in each document containing the given phrase. In this manner, the system can readily identify not only which documents contain which phrases in response to a search query, but which documents also contain phrases that are related to query phrases, and thus more likely to be specifically about the topics or concepts expressed in the query phrases.
The use of phrases and related phrases further provides for the creation and use of clusters of related phrases, which represent semantically meaningful groupings of phrases. Clusters are identified from related phrases that have very high prediction measure between all of the phrases in the cluster. Clusters can be used to organize the results of a search, including selecting which documents to include in the search results and their order, as well as eliminating documents from the search results.
Websites typically have anywhere from a few pages to potentially hundreds or thousands of pages. Thus, phrase information generated by the information retrieval system can be used to determine a list of top phrases for each website, such as the most representative phrases for the website. This can be done by examining the related phrase information for the phrases that appear in documents on the website. Further, phrase information may be later supplemented and refined by capturing changes made to the top phrase list by administrators or other authorized users and integrating the resulting semantic knowledge into the phrase information already contained within the system. An administrator can associate additional related phrases with any of the top phrases for the website. The related phrase information for the top phrases for which additional related phrases have been received is then updated to include information pertaining to the additional related phrases, and the additional related phrases are also updated to include information from the top phrases. This operates to treat the additional phrases as if they were present in the website. In addition, the additional related phrases can be updated to use the related phrase information for the top phrases.
The present invention has further embodiments in system and software architectures, computer program products and computer implemented methods, and computer generated user interfaces and presentations.
The foregoing are just some of the features of an information retrieval system and methodology based on phrases. Those of skill in the art of information retrieval will appreciate the flexibility of generality of the phrase information allows for a large variety of uses and applications in indexing, document annotation, searching, ranking, and other areas of document analysis and processing.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 is block diagram of the software architecture of one embodiment of the present invention.
FIG. 2 illustrates a method of identifying phrases in documents.
FIG. 3 illustrates a document with a phrase window and a secondary window.
FIG. 4 illustrates a method of identifying related phrases.
FIG. 5 illustrates a method of indexing documents for related phrases.
FIG. 6 illustrates a method of retrieving documents based on phrases.
FIGS. 7 a and 7 b illustrate relationships between referencing and referenced documents.
FIG. 8 illustrates a method of obtaining and integrating phrase information input from users.
FIG. 9 illustrates a sample user interface for displaying top phrases and allowing users to input changes.
The figures depict a preferred embodiment of the present invention for purposes of illustration only. One skilled in the art will readily recognize from the following discussion that alternative embodiments of the structures and methods illustrated herein may be employed without departing from the principles of the invention described herein.
Some existing information retrieval systems attempt to provide retrieval of concepts by using co-occurrence patterns of individual words. In these systems a search on one word, such as “President” will also retrieve documents that have other words that frequently appear with “President”, such as “White” and “House.” While this approach may produce search results having documents that are conceptually related at the level of individual words, it does not typically capture topical relationships that inhere between co-occurring phrases.
Accordingly, there is a need for an information retrieval system and methodology that can comprehensively identify phrases in a large scale corpus, index documents according to phrases, search and rank documents in accordance with their phrases. Additionally, there is a need in such a system to allow users to provide additional phrase information to the system and to capture and integrate the resulting semantic knowledge.
SUMMARY OF THE INVENTION
An information retrieval system and methodology uses phrases to index, search, rank, and describe documents in the document collection. The system is adapted to identify phrases that have sufficiently frequent and/or distinguished usage in the document collection to indicate that they are “valid” or “good” phrases. In this manner multiple word phrases, for example phrases of four, five, or more terms, can be identified. This avoids the problem of having to identify and index every possible phrase resulting from all of the possible sequences of a given number of words.
The system is further adapted to identify phrases that are related to each other, based on a phrase's ability to predict the presence of other phrases in a document. More specifically, a prediction measure is used that relates the actual co-occurrence rate of two phrases to an expected co-occurrence rate of the two phrases. Information gain, as the ratio of actual co-occurrence rate to expected co-occurrence rate, is one such prediction measure. Two phrases are related where the prediction measure exceeds a predetermined threshold. In that case, the second phrase has significant information gain with respect to the first phrase. Semantically, related phrases will be those that are commonly used to discuss or describe a given topic or concept, such as “President of the United States” and “White House.” For a given phrase, the related phrases can be ordered according to their relevance or significance based on their respective prediction measures.
An information retrieval system indexes documents in the document collection by the valid or good phrases. For each phrase, a posting list identifies the documents that contain the phrase. In addition, for a given phrase, a second list, vector, or other structure is used to store data indicating which of the related phrases of the given phrase are also present in each document containing the given phrase. In this manner, the system can readily identify not only which documents contain which phrases in response to a search query, but which documents also contain phrases that are related to query phrases, and thus more likely to be specifically about the topics or concepts expressed in the query phrases.
The use of phrases and related phrases further provides for the creation and use of clusters of related phrases, which represent semantically meaningful groupings of phrases. Clusters are identified from related phrases that have very high prediction measure between all of the phrases in the cluster. Clusters can be used to organize the results of a search, including selecting which documents to include in the search results and their order, as well as eliminating documents from the search results.
Websites typically have anywhere from a few pages to potentially hundreds or thousands of pages. Thus, phrase information generated by the information retrieval system can be used to determine a list of top phrases for each website, such as the most representative phrases for the website. This can be done by examining the related phrase information for the phrases that appear in documents on the website. Further, phrase information may be later supplemented and refined by capturing changes made to the top phrase list by administrators or other authorized users and integrating the resulting semantic knowledge into the phrase information already contained within the system. An administrator can associate additional related phrases with any of the top phrases for the website. The related phrase information for the top phrases for which additional related phrases have been received is then updated to include information pertaining to the additional related phrases, and the additional related phrases are also updated to include information from the top phrases. This operates to treat the additional phrases as if they were present in the website. In addition, the additional related phrases can be updated to use the related phrase information for the top phrases.
The present invention has further embodiments in system and software architectures, computer program products and computer implemented methods, and computer generated user interfaces and presentations.
The foregoing are just some of the features of an information retrieval system and methodology based on phrases. Those of skill in the art of information retrieval will appreciate the flexibility of generality of the phrase information allows for a large variety of uses and applications in indexing, document annotation, searching, ranking, and other areas of document analysis and processing.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 is block diagram of the software architecture of one embodiment of the present invention.
FIG. 2 illustrates a method of identifying phrases in documents.
FIG. 3 illustrates a document with a phrase window and a secondary window.
FIG. 4 illustrates a method of identifying related phrases.
FIG. 5 illustrates a method of indexing documents for related phrases.
FIG. 6 illustrates a method of retrieving documents based on phrases.
FIGS. 7 a and 7 b illustrate relationships between referencing and referenced documents.
FIG. 8 illustrates a method of obtaining and integrating phrase information input from users.
FIG. 9 illustrates a sample user interface for displaying top phrases and allowing users to input changes.
The figures depict a preferred embodiment of the present invention for purposes of illustration only. One skilled in the art will readily recognize from the following discussion that alternative embodiments of the structures and methods illustrated herein may be employed without departing from the principles of the invention described herein.
Sedang diterjemahkan, harap tunggu..
Bidang Teknik Penemuan
Penemuan ini berhubungan dengan sistem pengambilan informasi untuk mengindeks, mencari, dan mengklasifikasikan dokumen dalam corpus skala besar, seperti Internet. Latar Belakang Penemuan sistem pengambilan informasi, umumnya disebut mesin pencari, sekarang alat penting untuk menemukan informasi dalam skala besar, beragam, dan corpuses berkembang seperti Internet. Umumnya, mesin pencari membuat indeks yang berhubungan dokumen (atau "halaman") dengan kata-kata individu hadir dalam setiap dokumen. Sebuah dokumen yang diambil dalam menanggapi permintaan yang berisi sejumlah istilah permintaan, biasanya didasarkan pada memiliki beberapa jumlah istilah permintaan hadir dalam dokumen. Dokumen yang diambil kemudian peringkat menurut ukuran statistik lainnya, seperti frekuensi terjadinya istilah permintaan, host domain, analisa link, dan sejenisnya. Dokumen yang diambil kemudian disajikan kepada pengguna, biasanya dalam rangka mereka peringkat, dan tanpa pengelompokan lebih lanjut atau hierarki dikenakan. Dalam beberapa kasus, sebagian dipilih dari teks dokumen disajikan untuk memberikan pengguna dengan sekilas isi dokumen. Langsung "Boolean" pencocokan istilah permintaan telah dikenal keterbatasan, dan khususnya tidak mengidentifikasi dokumen yang tidak memiliki istilah permintaan, tetapi terkait kata-kata. Misalnya, dalam sistem Boolean khas, pencarian di "Shepherds Australia" tidak akan kembali dokumen tentang anjing menggiring lain seperti Border Collies yang tidak memiliki istilah permintaan yang tepat. Sebaliknya, sistem seperti ini cenderung juga mengambil dan sangat peringkat dokumen yang sekitar Australia (dan tidak ada hubungannya dengan anjing), dan dokumen tentang "gembala" umumnya. Masalahnya di sini adalah bahwa dokumen Indeks sistem konvensional berdasarkan pada istilah individu, dari pada konsep. Konsep yang sering dinyatakan dalam ungkapan, seperti "Australia Gembala", "Presiden Amerika Serikat," atau "Sundance Film Festival". Paling-paling, beberapa sistem sebelumnya akan mengindeks dokumen sehubungan dengan satu set yang telah ditentukan dan sangat terbatas 'dikenal' frase, yang biasanya dipilih oleh operator manusia. Pengindeksan frase biasanya dihindari karena kebutuhan komputasi dan memori yang dirasakan untuk mengidentifikasi semua kemungkinan frase dari mengatakan tiga, empat, atau lima atau lebih kata-kata. Misalnya, pada asumsi bahwa setiap lima kata bisa merupakan frase, dan korpus besar akan memiliki setidaknya 200.000 istilah yang unik, ada akan sekitar 3,2 × 1026 frase mungkin, jelas lebih dari sistem yang ada bisa menyimpan dalam memori atau pemrograman memanipulasi . Masalah selanjutnya adalah bahwa frase terus masuk dan meninggalkan leksikon dalam hal penggunaan mereka, jauh lebih sering daripada kata-kata individu baru yang diciptakan. Frase baru selalu dihasilkan, dari sumber-sumber teknologi tersebut, seni, peristiwa dunia, dan hukum. Frase lain akan menurun dalam penggunaan dari waktu ke waktu. Beberapa sistem pengambilan informasi yang ada berusaha untuk menyediakan pengambilan konsep dengan menggunakan pola co-terjadinya kata-kata individu. Dalam sistem ini pencarian di satu kata, seperti "Presiden" juga akan mengambil dokumen yang memiliki kata-kata lain yang sering muncul dengan "Presiden", seperti "White" dan "Rumah." Meskipun pendekatan ini dapat menghasilkan hasil pencarian yang memiliki dokumen yang yang secara konseptual terkait pada tingkat kata-kata individu, tidak biasanya menangkap hubungan topikal yang inheren antara frase co-terjadi. Oleh karena itu, ada kebutuhan untuk sistem pencarian informasi dan metodologi yang komprehensif dapat mengidentifikasi frasa dalam skala besar corpus, indeks dokumen sesuai dengan frase, mencari dan dokumen peringkat sesuai dengan frase mereka. Selain itu, ada kebutuhan dalam sistem tersebut untuk memungkinkan pengguna untuk memberikan informasi frase tambahan ke sistem dan untuk menangkap dan mengintegrasikan pengetahuan semantik yang dihasilkan. Ringkasan Penemuan Suatu sistem pencarian informasi dan metodologi menggunakan frase untuk indeks, pencarian, peringkat, dan menjelaskan dokumen dalam koleksi dokumen. Sistem ini disesuaikan dengan mengidentifikasi frase yang memiliki dan / atau penggunaan yg cukup sering dalam koleksi dokumen untuk menunjukkan bahwa mereka adalah "sah" atau "baik" frase. Dengan cara ini beberapa frase kata, misalnya frase empat, lima, atau lebih istilah, dapat diidentifikasi. Hal ini untuk menghindari masalah harus mengidentifikasi dan indeks setiap frase yang mungkin dihasilkan dari semua kemungkinan urutan dari angka yang diberikan kata-kata. Sistem ini lebih disesuaikan dengan mengidentifikasi frase yang terkait satu sama lain, berdasarkan kemampuan frase untuk memprediksi Kehadiran frasa lain dalam dokumen. Lebih khusus, ukuran prediksi digunakan yang berhubungan dengan tingkat co-kejadian yang sebenarnya dari dua frase untuk tingkat co-kejadian yang diharapkan dari dua frase. Gain informasi, sebagai rasio tingkat co-kejadian yang sebenarnya dengan yang diharapkan tingkat co-kejadian, merupakan salah satu ukuran prediksi tersebut. Dua frase yang terkait di mana ukuran prediksi melebihi ambang batas yang telah ditentukan. Dalam hal ini, kalimat kedua memiliki gain informasi yang signifikan sehubungan dengan kalimat pertama. Semantik, frase terkait akan menjadi orang yang umum digunakan untuk membahas atau menjelaskan suatu topik tertentu atau konsep, seperti "Presiden Amerika Serikat" dan "White House." Untuk frase tertentu, frase terkait dapat dipesan sesuai dengan mereka relevansi atau makna berdasarkan langkah-langkah prediksi masing-masing. Sebuah dokumen indeks sistem pencarian informasi dalam koleksi dokumen dengan frasa valid atau baik. Untuk setiap frase, daftar postingan mengidentifikasi dokumen yang berisi frase. Selain itu, untuk frase tertentu, daftar kedua, vektor, atau struktur lain digunakan untuk menyimpan data yang menunjukkan dari frase terkait kalimat yang diberikan juga hadir dalam setiap dokumen yang berisi frase diberikan. Dengan cara ini, sistem dapat dengan mudah mengidentifikasi tidak hanya yang mengandung dokumen yang frase dalam menanggapi permintaan pencarian, tetapi yang Dokumen-dokumen juga mengandung frase yang terkait dengan permintaan frase, dan dengan demikian lebih mungkin untuk secara khusus tentang topik atau konsep yang dinyatakan dalam frase query. Penggunaan frasa dan frasa terkait lanjut menyediakan untuk penciptaan dan penggunaan cluster frase terkait, yang mewakili kelompok semantik bermakna frase. Cluster diidentifikasi dari frase terkait yang memiliki ukuran prediksi yang sangat tinggi antara semua frasa di cluster. Cluster dapat digunakan untuk mengatur hasil pencarian, termasuk memilih yang dokumen dimasukkan ke dalam hasil pencarian dan ketertiban mereka, serta dokumen menghilangkan dari hasil pencarian. Website biasanya memiliki mana saja dari beberapa halaman untuk berpotensi ratusan atau ribuan halaman. Dengan demikian, informasi kalimat yang dihasilkan oleh sistem pengambilan informasi dapat digunakan untuk menentukan daftar frasa atas untuk setiap situs web, seperti frase yang paling representatif untuk website. Hal ini dapat dilakukan dengan memeriksa informasi frase terkait untuk frasa yang muncul dalam dokumen di website. Selanjutnya, informasi frase dapat kemudian dilengkapi dan disempurnakan dengan menangkap perubahan yang dibuat ke daftar frase atas oleh administrator atau pengguna lain yang berwenang dan mengintegrasikan pengetahuan semantik yang dihasilkan menjadi informasi frase sudah terkandung di dalam sistem. Administrator dapat mengaitkan frasa terkait tambahan dengan salah satu frase atas untuk website. Informasi frase terkait untuk frasa atas untuk frase terkait tambahan telah diterima kemudian diperbarui untuk menyertakan informasi yang berkaitan dengan frase terkait tambahan, dan frase yang terkait tambahan juga diperbarui untuk menyertakan informasi dari frasa atas. Ini beroperasi untuk mengobati frase tambahan seolah-olah mereka hadir di website. Selain itu, frase terkait tambahan dapat diperbarui untuk menggunakan informasi frase terkait untuk frasa atas. Penemuan ini memiliki perwujudan lebih lanjut dalam sistem dan perangkat lunak arsitektur, produk program komputer dan metode komputer dilaksanakan, dan komputer yang dihasilkan antarmuka pengguna dan presentasi. The sebelumnya hanya beberapa fitur dari sistem pencarian informasi dan metodologi berdasarkan frase. Orang yang ahli dalam seni pencarian informasi akan menghargai fleksibilitas umum dari informasi frase memungkinkan untuk berbagai macam kegunaan dan aplikasi dalam pengindeksan, dokumen penjelasan, mencari, peringkat, dan daerah lain analisis dokumen dan pengolahan. Uraian Singkat Gambar Gambar. 1 adalah diagram blok dari arsitektur perangkat lunak dari salah satu perwujudan dari penemuan ini. Gambar. 2 menggambarkan metode untuk mengidentifikasi frasa dalam dokumen. Gambar. 3 menggambarkan dokumen dengan jendela frase dan jendela sekunder. Gambar. 4 menggambarkan metode untuk mengidentifikasi frase terkait. Gambar. 5 menggambarkan metode dokumen pengindeksan untuk frase terkait. Gambar. 6 menggambarkan metode mengambil dokumen berdasarkan frase. Gambar. 7 dan 7 b menggambarkan hubungan antara referensi dan dokumen yang dirujuk. Gambar. 8 menggambarkan metode memperoleh dan mengintegrasikan frase masukan informasi dari pengguna. Gambar. 9 menggambarkan sampel antarmuka pengguna untuk menampilkan frase atas dan memungkinkan pengguna untuk perubahan masukan. Angka-angka menggambarkan perwujudan dari penemuan ini untuk tujuan ilustrasi saja. Salah satu ahli dalam bidang ini akan mudah mengenali dari pembahasan berikut yang perwujudan alternatif struktur dan metode diilustrasikan disini dapat digunakan tanpa menyimpang dari prinsip-prinsip dari penemuan yang dijelaskan di sini.
Penemuan ini berhubungan dengan sistem pengambilan informasi untuk mengindeks, mencari, dan mengklasifikasikan dokumen dalam corpus skala besar, seperti Internet. Latar Belakang Penemuan sistem pengambilan informasi, umumnya disebut mesin pencari, sekarang alat penting untuk menemukan informasi dalam skala besar, beragam, dan corpuses berkembang seperti Internet. Umumnya, mesin pencari membuat indeks yang berhubungan dokumen (atau "halaman") dengan kata-kata individu hadir dalam setiap dokumen. Sebuah dokumen yang diambil dalam menanggapi permintaan yang berisi sejumlah istilah permintaan, biasanya didasarkan pada memiliki beberapa jumlah istilah permintaan hadir dalam dokumen. Dokumen yang diambil kemudian peringkat menurut ukuran statistik lainnya, seperti frekuensi terjadinya istilah permintaan, host domain, analisa link, dan sejenisnya. Dokumen yang diambil kemudian disajikan kepada pengguna, biasanya dalam rangka mereka peringkat, dan tanpa pengelompokan lebih lanjut atau hierarki dikenakan. Dalam beberapa kasus, sebagian dipilih dari teks dokumen disajikan untuk memberikan pengguna dengan sekilas isi dokumen. Langsung "Boolean" pencocokan istilah permintaan telah dikenal keterbatasan, dan khususnya tidak mengidentifikasi dokumen yang tidak memiliki istilah permintaan, tetapi terkait kata-kata. Misalnya, dalam sistem Boolean khas, pencarian di "Shepherds Australia" tidak akan kembali dokumen tentang anjing menggiring lain seperti Border Collies yang tidak memiliki istilah permintaan yang tepat. Sebaliknya, sistem seperti ini cenderung juga mengambil dan sangat peringkat dokumen yang sekitar Australia (dan tidak ada hubungannya dengan anjing), dan dokumen tentang "gembala" umumnya. Masalahnya di sini adalah bahwa dokumen Indeks sistem konvensional berdasarkan pada istilah individu, dari pada konsep. Konsep yang sering dinyatakan dalam ungkapan, seperti "Australia Gembala", "Presiden Amerika Serikat," atau "Sundance Film Festival". Paling-paling, beberapa sistem sebelumnya akan mengindeks dokumen sehubungan dengan satu set yang telah ditentukan dan sangat terbatas 'dikenal' frase, yang biasanya dipilih oleh operator manusia. Pengindeksan frase biasanya dihindari karena kebutuhan komputasi dan memori yang dirasakan untuk mengidentifikasi semua kemungkinan frase dari mengatakan tiga, empat, atau lima atau lebih kata-kata. Misalnya, pada asumsi bahwa setiap lima kata bisa merupakan frase, dan korpus besar akan memiliki setidaknya 200.000 istilah yang unik, ada akan sekitar 3,2 × 1026 frase mungkin, jelas lebih dari sistem yang ada bisa menyimpan dalam memori atau pemrograman memanipulasi . Masalah selanjutnya adalah bahwa frase terus masuk dan meninggalkan leksikon dalam hal penggunaan mereka, jauh lebih sering daripada kata-kata individu baru yang diciptakan. Frase baru selalu dihasilkan, dari sumber-sumber teknologi tersebut, seni, peristiwa dunia, dan hukum. Frase lain akan menurun dalam penggunaan dari waktu ke waktu. Beberapa sistem pengambilan informasi yang ada berusaha untuk menyediakan pengambilan konsep dengan menggunakan pola co-terjadinya kata-kata individu. Dalam sistem ini pencarian di satu kata, seperti "Presiden" juga akan mengambil dokumen yang memiliki kata-kata lain yang sering muncul dengan "Presiden", seperti "White" dan "Rumah." Meskipun pendekatan ini dapat menghasilkan hasil pencarian yang memiliki dokumen yang yang secara konseptual terkait pada tingkat kata-kata individu, tidak biasanya menangkap hubungan topikal yang inheren antara frase co-terjadi. Oleh karena itu, ada kebutuhan untuk sistem pencarian informasi dan metodologi yang komprehensif dapat mengidentifikasi frasa dalam skala besar corpus, indeks dokumen sesuai dengan frase, mencari dan dokumen peringkat sesuai dengan frase mereka. Selain itu, ada kebutuhan dalam sistem tersebut untuk memungkinkan pengguna untuk memberikan informasi frase tambahan ke sistem dan untuk menangkap dan mengintegrasikan pengetahuan semantik yang dihasilkan. Ringkasan Penemuan Suatu sistem pencarian informasi dan metodologi menggunakan frase untuk indeks, pencarian, peringkat, dan menjelaskan dokumen dalam koleksi dokumen. Sistem ini disesuaikan dengan mengidentifikasi frase yang memiliki dan / atau penggunaan yg cukup sering dalam koleksi dokumen untuk menunjukkan bahwa mereka adalah "sah" atau "baik" frase. Dengan cara ini beberapa frase kata, misalnya frase empat, lima, atau lebih istilah, dapat diidentifikasi. Hal ini untuk menghindari masalah harus mengidentifikasi dan indeks setiap frase yang mungkin dihasilkan dari semua kemungkinan urutan dari angka yang diberikan kata-kata. Sistem ini lebih disesuaikan dengan mengidentifikasi frase yang terkait satu sama lain, berdasarkan kemampuan frase untuk memprediksi Kehadiran frasa lain dalam dokumen. Lebih khusus, ukuran prediksi digunakan yang berhubungan dengan tingkat co-kejadian yang sebenarnya dari dua frase untuk tingkat co-kejadian yang diharapkan dari dua frase. Gain informasi, sebagai rasio tingkat co-kejadian yang sebenarnya dengan yang diharapkan tingkat co-kejadian, merupakan salah satu ukuran prediksi tersebut. Dua frase yang terkait di mana ukuran prediksi melebihi ambang batas yang telah ditentukan. Dalam hal ini, kalimat kedua memiliki gain informasi yang signifikan sehubungan dengan kalimat pertama. Semantik, frase terkait akan menjadi orang yang umum digunakan untuk membahas atau menjelaskan suatu topik tertentu atau konsep, seperti "Presiden Amerika Serikat" dan "White House." Untuk frase tertentu, frase terkait dapat dipesan sesuai dengan mereka relevansi atau makna berdasarkan langkah-langkah prediksi masing-masing. Sebuah dokumen indeks sistem pencarian informasi dalam koleksi dokumen dengan frasa valid atau baik. Untuk setiap frase, daftar postingan mengidentifikasi dokumen yang berisi frase. Selain itu, untuk frase tertentu, daftar kedua, vektor, atau struktur lain digunakan untuk menyimpan data yang menunjukkan dari frase terkait kalimat yang diberikan juga hadir dalam setiap dokumen yang berisi frase diberikan. Dengan cara ini, sistem dapat dengan mudah mengidentifikasi tidak hanya yang mengandung dokumen yang frase dalam menanggapi permintaan pencarian, tetapi yang Dokumen-dokumen juga mengandung frase yang terkait dengan permintaan frase, dan dengan demikian lebih mungkin untuk secara khusus tentang topik atau konsep yang dinyatakan dalam frase query. Penggunaan frasa dan frasa terkait lanjut menyediakan untuk penciptaan dan penggunaan cluster frase terkait, yang mewakili kelompok semantik bermakna frase. Cluster diidentifikasi dari frase terkait yang memiliki ukuran prediksi yang sangat tinggi antara semua frasa di cluster. Cluster dapat digunakan untuk mengatur hasil pencarian, termasuk memilih yang dokumen dimasukkan ke dalam hasil pencarian dan ketertiban mereka, serta dokumen menghilangkan dari hasil pencarian. Website biasanya memiliki mana saja dari beberapa halaman untuk berpotensi ratusan atau ribuan halaman. Dengan demikian, informasi kalimat yang dihasilkan oleh sistem pengambilan informasi dapat digunakan untuk menentukan daftar frasa atas untuk setiap situs web, seperti frase yang paling representatif untuk website. Hal ini dapat dilakukan dengan memeriksa informasi frase terkait untuk frasa yang muncul dalam dokumen di website. Selanjutnya, informasi frase dapat kemudian dilengkapi dan disempurnakan dengan menangkap perubahan yang dibuat ke daftar frase atas oleh administrator atau pengguna lain yang berwenang dan mengintegrasikan pengetahuan semantik yang dihasilkan menjadi informasi frase sudah terkandung di dalam sistem. Administrator dapat mengaitkan frasa terkait tambahan dengan salah satu frase atas untuk website. Informasi frase terkait untuk frasa atas untuk frase terkait tambahan telah diterima kemudian diperbarui untuk menyertakan informasi yang berkaitan dengan frase terkait tambahan, dan frase yang terkait tambahan juga diperbarui untuk menyertakan informasi dari frasa atas. Ini beroperasi untuk mengobati frase tambahan seolah-olah mereka hadir di website. Selain itu, frase terkait tambahan dapat diperbarui untuk menggunakan informasi frase terkait untuk frasa atas. Penemuan ini memiliki perwujudan lebih lanjut dalam sistem dan perangkat lunak arsitektur, produk program komputer dan metode komputer dilaksanakan, dan komputer yang dihasilkan antarmuka pengguna dan presentasi. The sebelumnya hanya beberapa fitur dari sistem pencarian informasi dan metodologi berdasarkan frase. Orang yang ahli dalam seni pencarian informasi akan menghargai fleksibilitas umum dari informasi frase memungkinkan untuk berbagai macam kegunaan dan aplikasi dalam pengindeksan, dokumen penjelasan, mencari, peringkat, dan daerah lain analisis dokumen dan pengolahan. Uraian Singkat Gambar Gambar. 1 adalah diagram blok dari arsitektur perangkat lunak dari salah satu perwujudan dari penemuan ini. Gambar. 2 menggambarkan metode untuk mengidentifikasi frasa dalam dokumen. Gambar. 3 menggambarkan dokumen dengan jendela frase dan jendela sekunder. Gambar. 4 menggambarkan metode untuk mengidentifikasi frase terkait. Gambar. 5 menggambarkan metode dokumen pengindeksan untuk frase terkait. Gambar. 6 menggambarkan metode mengambil dokumen berdasarkan frase. Gambar. 7 dan 7 b menggambarkan hubungan antara referensi dan dokumen yang dirujuk. Gambar. 8 menggambarkan metode memperoleh dan mengintegrasikan frase masukan informasi dari pengguna. Gambar. 9 menggambarkan sampel antarmuka pengguna untuk menampilkan frase atas dan memungkinkan pengguna untuk perubahan masukan. Angka-angka menggambarkan perwujudan dari penemuan ini untuk tujuan ilustrasi saja. Salah satu ahli dalam bidang ini akan mudah mengenali dari pembahasan berikut yang perwujudan alternatif struktur dan metode diilustrasikan disini dapat digunakan tanpa menyimpang dari prinsip-prinsip dari penemuan yang dijelaskan di sini.
Sedang diterjemahkan, harap tunggu..
Bahasa lainnya
Dukungan alat penerjemahan: Afrikans, Albania, Amhara, Arab, Armenia, Azerbaijan, Bahasa Indonesia, Basque, Belanda, Belarussia, Bengali, Bosnia, Bulgaria, Burma, Cebuano, Ceko, Chichewa, China, Cina Tradisional, Denmark, Deteksi bahasa, Esperanto, Estonia, Farsi, Finlandia, Frisia, Gaelig, Gaelik Skotlandia, Galisia, Georgia, Gujarati, Hausa, Hawaii, Hindi, Hmong, Ibrani, Igbo, Inggris, Islan, Italia, Jawa, Jepang, Jerman, Kannada, Katala, Kazak, Khmer, Kinyarwanda, Kirghiz, Klingon, Korea, Korsika, Kreol Haiti, Kroat, Kurdi, Laos, Latin, Latvia, Lituania, Luksemburg, Magyar, Makedonia, Malagasi, Malayalam, Malta, Maori, Marathi, Melayu, Mongol, Nepal, Norsk, Odia (Oriya), Pashto, Polandia, Portugis, Prancis, Punjabi, Rumania, Rusia, Samoa, Serb, Sesotho, Shona, Sindhi, Sinhala, Slovakia, Slovenia, Somali, Spanyol, Sunda, Swahili, Swensk, Tagalog, Tajik, Tamil, Tatar, Telugu, Thai, Turki, Turkmen, Ukraina, Urdu, Uyghur, Uzbek, Vietnam, Wales, Xhosa, Yiddi, Yoruba, Yunani, Zulu, Bahasa terjemahan.
- saya sayang nyonya,kalau saya sehat past
- Hijack the system
- segar
- rain can you help me? please to tell him
- Hey! You have a blog.You can change the
- Aku kangen dan kamu tidak pernah peduli
- Hey! You have a blog.You can change the
- you know? I miss you so badly dear :*
- segar
- Takdir setiap manusia memang telah diten
- dear
- segar
- segar
- La maison
- Aku sayang dan mencintaimu selamanya
- segar
- segar
- Viola une grande place
- segar
- Ajarkan saya bahasa thailand
- segar
- Tak ada kata kata yang banyak untuk aku
- segar
- Would you like me ?
