Returned Results Could Slightly Differ From

Embed Doc
Copy Link
Readcast
Collections

CommentGo Back

Download

Gin for PostgreSQL==================Gin was sponsored by jfg://networks (http://www.jfg-networks.com/)Gin stands for Generalized Inverted Index and should be considered as a genie,not a drink.Generalized means that the index does not know which operation it accelerates.It instead works with custom strategies, defined for specific data types (read"Index Method Strategies" in the PostgreSQL documentation). In that sense, Ginis similar to GiST and differs from btree indices, which have predefined,comparison-based operations.An inverted index is an index structure storing a set of (key, posting list)pairs, where 'posting list' is a set of documents in which the key occurs.(A text document would usually contain many keys.) The primary goal ofGin indices is support for highly scalable, full-text search in PostgreSQL.Gin consists of a B-tree index constructed over entries (ET, entries tree),where each entry is an element of the indexed value (element of array, lexemefor tsvector) and where each tuple in a leaf page is either a pointer to aB-tree over item pointers (PT, posting tree), or a list of item pointers(PL, posting list) if the tuple is small enough.Note: There is no delete operation for ET. The reason for this is that fromour experience, a set of unique words over a large collection change veryrarely. This greatly simplifies the code and concurrency algorithms.Gin comes with built-in support for one-dimensional arrays (eg. integer[],text[]), but no support for NULL elements. The following operations areavailable:* contains: value_array @ query_array* overlap: value_array && query_array* contained: value_array ~ query_arraySynopsis--------=# create index txt_idx on aa using gin(a);Features--------* Concurrency* Write-Ahead Logging (WAL). (Recoverability from crashes.)* User-defined opclasses. (The scheme is similar to GiST.)* Optimized index creation (Makes use of maintenance_work_mem to accumulatepostings in memory.)* Tsearch2 support via an opclass* Soft upper limit on the returned results set using a GUC variable:gin_fuzzy_search_limitGin Fuzzy Limit---------------There are often situations when a full-text search returns a very large set of

results. Since reading tuples from the disk and sorting them could take alot of time, this is unacceptable for production. (Note that the searchitself is very fast.)Such queries usually contain very frequent lexemes, so the results are notvery helpful. To facilitate execution of such queries Gin has a configurablesoft upper limit of the size of the returned set, determined by the'gin_fuzzy_search_limit' GUC variable. This is set to 0 by default (nolimit).If a non-zero search limit is set, then the returned set is a subset of thewhole result set, chosen at random."Soft" means that the actual number of returned results could slightly differfrom the specified limit, depending on the query and the quality of thesystem's random number generator.From experience, a value of 'gin_fuzzy_search_limit' in the thousands(eg. 5000-20000) works well. This means that 'gin_fuzzy_search_limit' willhave no effect for queries returning a result set with less tuples than thisnumber.Limitations-----------* No support for multicolumn indices* Gin doesn't uses scan->kill_prior_tuple & scan->ignore_killed_tuples* Gin searches entries only by equality matching. This may be improved infuture.* Gin doesn't support full scans of indices.* Gin doesn't index NULL values.Gin Interface-------------Opclass interface pseudocode. An example for a Gin opclass can be found inginarayproc.c.Datum* extractValue(Datum inputValue, uint32* nentries)Returns an array of Datum of entries of the value to be indexed. nentriesshould contain the number of returned entries. int compareEntry(Datum a, Datum b)Compares two entries (not the indexing values) Datum* extractQuery(Datum query, uint32* nentries, StrategyNumber n)Returns an array of Datum of entries of the query to be executed.n contains the strategy number of the operation. bool consistent(bool[] check, StrategyNumber n, Datum query)The size of the check array is the same as sizeof of the array returned byextractQuery. Each element of the check array is true if the indexed valuehas a corresponding entry in the query. i.e. if (check[i] == TRUE) thenthe i-th entry of the query is present in the indexed value. The Function

of 00

Leave a Comment